Re: [Pharo-users] Fwd: Fwd: Debris' Quuve - Technologies used
Yanni, please find my answers below. On Thu, Apr 16, 2015 at 10:57 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
On Wed, Apr 15, 2015 at 9:53 PM, Yanni Chiu <yanni@rogers.com> wrote:
On Apr 15, 2015, at 8:01 PM, Cameron Sanders via Pharo-users < pharo-users@lists.pharo.org> wrote:
Sure, I am happy to share. We are to the point that we need to get
people interested in this platform; one thing we offer is a source license and therefore we need to document things that we have not yet documented.
So please, fire away any questions.
Are there any instances of Magritte framework that are not an instances of a custom subclass? (Did you have to subclass everything?)
No, we did not have to subclass everything. But quite some. For example, the components, yes, we have to subclass them all. The main reasons are more or less these:
1) We want everything to be ajaxfied (so every single component needed to be rewritten for AJAX) 2) We want all our components to use Bootstrap layout 3) We want components/descriptions/renderes/etc that better fit our own needs 4) We needed more hooks
The only part that we did subclass everything is the components. Then we have subclasses some descriptions but not all. We also have some special descriptions.
In addition, we have subclassed:
- Memento: to provide extra hooks.... - StringWriters: mostly to change the way we print floats, dateandtime, dates, etc. - TableRenderer: because we have 2-columns and 1-column type of form (and to layout these forms using Bootstrap) - FormDecoration and ValidationDecoration: to provide ajax and bootstrap - DescribedColumn, CommandColumn, etc: we have lots of subclasses to specify and improve our magritte reports - other...
What pushed you toward custom subclasses? Was the properties dictionary not sufficient for extending the object state variables, or was it a speed/performance issue? Would refactoring the Magritte classes to have various hook methods allowed you to avoid making custom subclasses? (I found that it was Twitter Bootstrap which drove the need for subclassing the framework, whereas native HTML would have been adaptable to Magritteâs needs).
We DO extend some of Magritte superclasses using extension methods that use the properties. We have quite some properties extended that way. However, many other places we need to change the way things are displayed, or arranges, or provide more hooks, to better fit our need etc. In that case, just using properties is not enough. Yes, if we would have had more hooks, we likely would have need to subclass less. But not everything. We think Magritte was easy enough to extend as is. Another thing we found is that sometimes is a bit hard to agree on what is needed for all magritte users. So fully refactoring Magritte may affect other developers or other use-cases of Magritte.
Did you use the OneToOne and OneToMany relationship descriptions by subclassing, or did you create entirely new ones?
We have subclasses those. Why? Because we found out that it asks the #optionsAndLabels quite a lot of times when it is not really necessary. For example, to view an object, it needs to send #optionsAndLabels just to then search the label for an specific option.... So we have created our own "lazy" subclass of those 2 were #optionsAndLabels are computed as much lazy/late as possible AND we can also plug a #getLabelClosure which avoids getting #optionsAndLabels just for that...
In addition to this, we have some special components (for example one that uses autocomplete) and so the combination of lazy descriptions + autocomplete component is very useful for long lists.
What are some statistics on your model - e.g. number to classes? average (mean/median/mode) number of attributes/methods per class? average depth of the class hierarchy?
We'll have to get back to you on that one -- if we ever count them...
Do you translate/map the âSmalltalkâ query blocks for GemStone to get
better speed, or is this unnecessary?
Yes. But only in a few places. We have currently 3 or 4 places where we have indexes in GemStone collections. So what we did is that we have a QueryBuilder strategy, with 2 subclasses, one for Pharo and one for GemStone. And a singleton #current. Each class knows how to answer the query closure for a particular query. It simply answers the closure we then pass to the #select:. GemStone one will answer the { } one.
Do you use Magritte descriptions to tag things for indexing on GemStone? Or, do you just manually create indexes on GemStone as necessary (e.g. via install code that encapsulates knowledge of where to index, or does the system automatically raise flags when indexing is needed).
The latter. We do not use Magritte here.
From various postings, I gather you have a lot of data to process. Do you have any numbers like: - total number of objects in the system
This is tricky, because it really depends on the advisory firm or the researcher. Quuve is deployed for each customer. So the size really really varies.
- biggest collection, typical collection size
Right now for example we are dealing with storage of fundamental data of companies (for research) , and prices... some collections have a few million objects. What is important is that you must analyze which collection class to use. Best (from our point of view) is one that supports both, RC and Indexes. Just in case you need any of those later.
-- Mariano http://marianopeck.wordpress.com
On Apr 16, 2015, at 10:33 AM, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
Yanni, please find my answers below.
Mariano, thanks for your detailed answers. Your experience building a CRUD framework using Magritte and Bootstrap was very similar to mine. My stuff is on a back burner for now, but I hope to get back to it sometime soon. I hope Quuve is successful for you guys.
On 16/04/15 16:33, Mariano Martinez Peck wrote:
1) We want everything to be ajaxfied (so every single component needed to be rewritten for AJAX) 2) We want all our components to use Bootstrap layout
In QCMagritte we managed to keep most, using an Ajax-Memento and the chained visitors.
3) We want components/descriptions/renderes/etc that better fit our own needs 4) We needed more hooks
Definitely.
We think Magritte was easy enough to extend as is. Another thing we found is that sometimes is a bit hard to agree on what is needed for all magritte users. So fully refactoring Magritte may affect other developers or other use-cases of Magritte.
Yep, fully agree. Stephan
Just a remark I found strange that not a single little improvements of Magritte was necessary for the Qude project. Now if heavy users of open-source libraries do not enhance these open-source libraries and keep their extensions under close source then the open-source libraries will never make progress and I will immensely sad. You know that I like business and that people can make money but the fight is against other community and competetitor and not within Pharo. Stef Le 17/4/15 19:43, Stephan Eggermont a écrit :
On 16/04/15 16:33, Mariano Martinez Peck wrote:
1) We want everything to be ajaxfied (so every single component needed to be rewritten for AJAX) 2) We want all our components to use Bootstrap layout
In QCMagritte we managed to keep most, using an Ajax-Memento and the chained visitors.
3) We want components/descriptions/renderes/etc that better fit our own needs 4) We needed more hooks
Definitely.
We think Magritte was easy enough to extend as is. Another thing we found is that sometimes is a bit hard to agree on what is needed for all magritte users. So fully refactoring Magritte may affect other developers or other use-cases of Magritte.
Yep, fully agree.
Stephan
On Apr 18, 2015, at 2:18 AM, stepharo <stepharo@free.fr> wrote:
I found strange that not a single little improvements of Magritte was necessary for the Qude project.
Itâs not strange to me, because when working with Magritte 2, it seemed that almost everything I wanted to do had a hook method to override, or an obvious place to subclass or extend. So much so, that when I could not find the extension point, I thought it was my fault. What was missing was instance-based descriptions, which was added by the community in Magritte 3. Given how well-factored Magritte 2 was, my deduction was that a lot of effort had already been done to extract out an open-source artefact, from whatever system drove itâs development. So, no surprise that few improvements to the core Magritte were needed.
Now if heavy users of open-source libraries do not enhance these open-source libraries and keep their extensions under close source then the open-source libraries will never make progress and I will immensely sad.
Given that Magritte is already well-factored for extensions, what tends to be written is exactly the custom code which you would not be releasing to open-source. However, there might be a case for add-ons, such as Twitter Bootstrap support. As mentioned in another post, changing the Magritte API/interfaces at this point is tricky, because it would affect current users - thatâs the conundrum: success and wider adoption will constrain the evolution. The addition of instance-based descriptions was worth the (minor) pain of the transition, but what level of pain would be tolerable to add Bootstrap support, especially if youâre not using Bootstrap.
Le 18/4/15 20:04, Yanni Chiu a écrit :
On Apr 18, 2015, at 2:18 AM, stepharo <stepharo@free.fr> wrote:
I found strange that not a single little improvements of Magritte was necessary for the Qude project. Itâs not strange to me, because when working with Magritte 2, it seemed that almost everything I wanted to do had a hook method to override, or an obvious place to subclass or extend. So much so, that when I could not find the extension point, I thought it was my fault. What was missing was instance-based descriptions, which was added by the community in Magritte 3.
:)
Given how well-factored Magritte 2 was, my deduction was that a lot of effort had already been done to extract out an open-source artefact, from whatever system drove itâs development. So, no surprise that few improvements to the core Magritte were needed.
Ok so we have perfect software. Good to know.
Now if heavy users of open-source libraries do not enhance these open-source libraries and keep their extensions under close source then the open-source libraries will never make progress and I will immensely sad. Given that Magritte is already well-factored for extensions, what tends to be written is exactly the custom code which you would not be releasing to open-source.
but you see I did not use magritte since long time. It would be great to have example of custom-code. For example are relationships any useful.
However, there might be a case for add-ons, such as Twitter Bootstrap support. As mentioned in another post, changing the Magritte API/interfaces at this point is tricky, because it would affect current users - thatâs the conundrum: success and wider adoption will constrain the evolution.
We have versions to help there.
The addition of instance-based descriptions was worth the (minor) pain of the transition, but what level of pain would be tolerable to add Bootstrap support, especially if youâre not using Bootstrap. But we are talking about SeasideMagritte here? I do not know because I do not have experience. I was talking about magritte.
And BTW I would really like to see how we can do crude with magritte-described objects. Le 18/4/15 21:37, stepharo a écrit :
Le 18/4/15 20:04, Yanni Chiu a écrit :
On Apr 18, 2015, at 2:18 AM, stepharo <stepharo@free.fr> wrote:
I found strange that not a single little improvements of Magritte was necessary for the Qude project. Itâs not strange to me, because when working with Magritte 2, it seemed that almost everything I wanted to do had a hook method to override, or an obvious place to subclass or extend. So much so, that when I could not find the extension point, I thought it was my fault. What was missing was instance-based descriptions, which was added by the community in Magritte 3.
:)
Given how well-factored Magritte 2 was, my deduction was that a lot of effort had already been done to extract out an open-source artefact, from whatever system drove itâs development. So, no surprise that few improvements to the core Magritte were needed.
Ok so we have perfect software. Good to know.
Now if heavy users of open-source libraries do not enhance these open-source libraries and keep their extensions under close source then the open-source libraries will never make progress and I will immensely sad. Given that Magritte is already well-factored for extensions, what tends to be written is exactly the custom code which you would not be releasing to open-source.
but you see I did not use magritte since long time. It would be great to have example of custom-code. For example are relationships any useful.
However, there might be a case for add-ons, such as Twitter Bootstrap support. As mentioned in another post, changing the Magritte API/interfaces at this point is tricky, because it would affect current users - thatâs the conundrum: success and wider adoption will constrain the evolution.
We have versions to help there.
The addition of instance-based descriptions was worth the (minor) pain of the transition, but what level of pain would be tolerable to add Bootstrap support, especially if youâre not using Bootstrap. But we are talking about SeasideMagritte here? I do not know because I do not have experience. I was talking about magritte.
Hi Stef, Similar to what Yanni said, 99% of our extensions are to Magritte-Seaside, not to magritte core itself. I honestly see Magritte as a very small and super extensible framework from which you can build your own on top of it. Like QCMagritte. Or likely what Yanni did, or likely what we did. That is...a set of subclasses/extensions to Magritte Seaside. This doesn't mean we shouldn't make this open-source, but I am just saying all our extensions should not be Magritte core code but kind of a framework build on top of Magritte core. Just to give you an idea of the extensibility of Magritte... is that with all our framework build on top of Magritte, we had NO OVERRIDE and our classes/extensions load cleanly after loading Magritte. Cheers, On Sat, Apr 18, 2015 at 4:55 PM, stepharo <stepharo@free.fr> wrote:
And BTW I would really like to see how we can do crude with magritte-described objects.
Le 18/4/15 21:37, stepharo a écrit :
Le 18/4/15 20:04, Yanni Chiu a écrit :
On Apr 18, 2015, at 2:18 AM, stepharo <stepharo@free.fr> wrote:
I found strange that not a single little improvements of Magritte was
necessary for the Qude project.
Itâs not strange to me, because when working with Magritte 2, it seemed that almost everything I wanted to do had a hook method to override, or an obvious place to subclass or extend. So much so, that when I could not find the extension point, I thought it was my fault. What was missing was instance-based descriptions, which was added by the community in Magritte 3.
:)
Given how well-factored Magritte 2 was, my deduction was that a lot of effort had already been done to extract out an open-source artefact, from whatever system drove itâs development. So, no surprise that few improvements to the core Magritte were needed.
Ok so we have perfect software. Good to know.
Now if heavy users of open-source libraries do not enhance these
open-source libraries and keep their extensions under close source then the open-source libraries will never make progress and I will immensely sad.
Given that Magritte is already well-factored for extensions, what tends to be written is exactly the custom code which you would not be releasing to open-source.
but you see I did not use magritte since long time. It would be great to have example of custom-code. For example are relationships any useful.
However, there might be a case for add-ons, such as Twitter Bootstrap support. As mentioned in another post, changing the Magritte API/interfaces at this point is tricky, because it would affect current users - thatâs the conundrum: success and wider adoption will constrain the evolution.
We have versions to help there.
The addition of instance-based descriptions was worth the (minor) pain
of the transition, but what level of pain would be tolerable to add Bootstrap support, especially if youâre not using Bootstrap.
But we are talking about SeasideMagritte here? I do not know because I do not have experience. I was talking about magritte.
-- Mariano http://marianopeck.wordpress.com
Le 21/4/15 18:26, Mariano Martinez Peck a écrit :
Hi Stef,
Similar to what Yanni said, 99% of our extensions are to Magritte-Seaside, not to magritte core itself. I honestly see Magritte as a very small and super extensible framework from which you can build your own on top of it. Like QCMagritte. Or likely what Yanni did, or likely what we did. That is...a set of subclasses/extensions to Magritte Seaside. This doesn't mean we shouldn't make this open-source, but I am just saying all our extensions should not be Magritte core code but kind of a framework build on top of Magritte core.
Just to give you an idea of the extensibility of Magritte... is that with all our framework build on top of Magritte, we had NO OVERRIDE and our classes/extensions load cleanly after loading Magritte.
Ok I trust you. So this is good. Now I do not know how to do crud applications with magritte.
Cheers,
On Sat, Apr 18, 2015 at 4:55 PM, stepharo <stepharo@free.fr <mailto:stepharo@free.fr>> wrote:
And BTW I would really like to see how we can do crude with magritte-described objects.
Le 18/4/15 21:37, stepharo a écrit :
Le 18/4/15 20:04, Yanni Chiu a écrit :
On Apr 18, 2015, at 2:18 AM, stepharo <stepharo@free.fr <mailto:stepharo@free.fr>> wrote:
I found strange that not a single little improvements of Magritte was necessary for the Qude project.
Itâs not strange to me, because when working with Magritte 2, it seemed that almost everything I wanted to do had a hook method to override, or an obvious place to subclass or extend. So much so, that when I could not find the extension point, I thought it was my fault. What was missing was instance-based descriptions, which was added by the community in Magritte 3.
:)
Given how well-factored Magritte 2 was, my deduction was that a lot of effort had already been done to extract out an open-source artefact, from whatever system drove itâs development. So, no surprise that few improvements to the core Magritte were needed.
Ok so we have perfect software. Good to know.
Now if heavy users of open-source libraries do not enhance these open-source libraries and keep their extensions under close source then the open-source libraries will never make progress and I will immensely sad.
Given that Magritte is already well-factored for extensions, what tends to be written is exactly the custom code which you would not be releasing to open-source.
but you see I did not use magritte since long time. It would be great to have example of custom-code. For example are relationships any useful.
However, there might be a case for add-ons, such as Twitter Bootstrap support. As mentioned in another post, changing the Magritte API/interfaces at this point is tricky, because it would affect current users - thatâs the conundrum: success and wider adoption will constrain the evolution.
We have versions to help there.
The addition of instance-based descriptions was worth the (minor) pain of the transition, but what level of pain would be tolerable to add Bootstrap support, especially if youâre not using Bootstrap.
But we are talking about SeasideMagritte here? I do not know because I do not have experience. I was talking about magritte.
-- Mariano http://marianopeck.wordpress.com
Magritte could be improved to allow a context of usage to be specified e.g. #tableReport, or #editor, and then allow class designers to specify the pairs of keys for CSS style to be written for the context. We'll follow up on this point soon. Seaside could accept more renderingBlocks to be more customizable. And/or Magritte descriptions should allow these to be passed down. But where does one draw the line? Like the above paragraph on CSS, different contexts demand different rendering blocks. So in fact, we pull together our Magritte descriptions in non-standard ways for our most important view: the portfolio browser. Many are created dynamically from accounting terms in a DB, because there are thousands of possible rules/equations, and they cannot all be hand-coded for this one class. - In our application, Magritte provides a mechanism to describe the attributes (or computed terms) in order to paint an editor, a read-only view, and/or table presentation/edit dialogs. For complex specification and controller classes, we often still need to write dedicated Components. We do not use Magritte for persistence, instead we use a variant of DebrisDB that works on Gemstone & Pharo. So our CRUD solution involves our own tables (seaside subclasses) that allow us to install blocks to implement various behaviors such as "update table from DB". We also allow application-level "plugin" rules to be presented as buttons and controls, but these are very specific to our whole framework. The launching of these tables is handled through various of our own constructs, specifically our ApplicationContext, which knows where to put new panes/components, for example. Much of the speed with which we can define a new class, have it persistent (no time using DebrisDB), and have editors/presenters (perhaps 1 to 15 minutes per instVar for simple classes), is due to the culmination of various tools in our environment and our experience with such. -Cam On Tue, Apr 21, 2015 at 2:55 PM, stepharo <stepharo@free.fr> wrote:
Le 21/4/15 18:26, Mariano Martinez Peck a écrit :
Hi Stef,
Similar to what Yanni said, 99% of our extensions are to Magritte-Seaside, not to magritte core itself. I honestly see Magritte as a very small and super extensible framework from which you can build your own on top of it. Like QCMagritte. Or likely what Yanni did, or likely what we did. That is...a set of subclasses/extensions to Magritte Seaside. This doesn't mean we shouldn't make this open-source, but I am just saying all our extensions should not be Magritte core code but kind of a framework build on top of Magritte core.
Just to give you an idea of the extensibility of Magritte... is that with all our framework build on top of Magritte, we had NO OVERRIDE and our classes/extensions load cleanly after loading Magritte.
Ok I trust you. So this is good.
Now I do not know how to do crud applications with magritte.
Cheers,
On Sat, Apr 18, 2015 at 4:55 PM, stepharo <stepharo@free.fr> wrote:
And BTW I would really like to see how we can do crude with magritte-described objects.
Le 18/4/15 21:37, stepharo a écrit :
Le 18/4/15 20:04, Yanni Chiu a écrit :
On Apr 18, 2015, at 2:18 AM, stepharo <stepharo@free.fr> wrote:
I found strange that not a single little improvements of Magritte was
necessary for the Qude project.
Itâs not strange to me, because when working with Magritte 2, it seemed that almost everything I wanted to do had a hook method to override, or an obvious place to subclass or extend. So much so, that when I could not find the extension point, I thought it was my fault. What was missing was instance-based descriptions, which was added by the community in Magritte 3.
:)
Given how well-factored Magritte 2 was, my deduction was that a lot of effort had already been done to extract out an open-source artefact, from whatever system drove itâs development. So, no surprise that few improvements to the core Magritte were needed.
Ok so we have perfect software. Good to know.
Now if heavy users of open-source libraries do not enhance these
open-source libraries and keep their extensions under close source then the open-source libraries will never make progress and I will immensely sad.
Given that Magritte is already well-factored for extensions, what tends to be written is exactly the custom code which you would not be releasing to open-source.
but you see I did not use magritte since long time. It would be great to have example of custom-code. For example are relationships any useful.
However, there might be a case for add-ons, such as Twitter Bootstrap support. As mentioned in another post, changing the Magritte API/interfaces at this point is tricky, because it would affect current users - thatâs the conundrum: success and wider adoption will constrain the evolution.
We have versions to help there.
The addition of instance-based descriptions was worth the (minor) pain
of the transition, but what level of pain would be tolerable to add Bootstrap support, especially if youâre not using Bootstrap.
But we are talking about SeasideMagritte here? I do not know because I do not have experience. I was talking about magritte.
-- Mariano http://marianopeck.wordpress.com
participants (5)
-
Cameron Sanders -
Mariano Martinez Peck -
Stephan Eggermont -
stepharo -
Yanni Chiu