April 2014 - Pharo-users - lists.pharo.org

Re: [Pharo-users] glorp

by Sven Van Caekenberghe

On 25 Apr 2014, at 22:20, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: > So coming back to Pharo realm, this is why I think ORM is really > important for us, and I'm really happy this discussion finally took > place. But one of his points was that ORM layers made SQL obsolete by hiding much of its power, the ability to do (query and other) processing on the DB. BTW, I don't think there is such a thing as transparent persistency.

April 25, 2014

Re: [Pharo-users] glorp

by Esteban A. Maringolo

2014-04-25 17:00 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > Here is another nice opinion: > > http://use-the-index-luke.com/blog/2013-04/whats-left-of-nosql Very interesting article Sven. "How many companies really need to cope with data at the scale of Google, Facebook or Twitter? Incidentally three companies that grew up on the open source database MySQL." And related with that quote, one of things that puzzled me a few weeks ago was WebScaleSQL (http://webscalesql.org/) And if you look at its FAQs (http://webscalesql.org/faq.html) Google, Facebook and Twitter are supporters of this project. Also, Google provides SQL (MySQL) in its Google Cloud services, so there is certainly a big demand for this. So coming back to Pharo realm, this is why I think ORM is really important for us, and I'm really happy this discussion finally took place. Best regards! Esteban A. Maringolo

April 25, 2014

Re: [Pharo-users] glorp

by Sven Van Caekenberghe

Here is another nice opinion: http://use-the-index-luke.com/blog/2013-04/whats-left-of-nosql On 25 Apr 2014, at 11:52, Franz Josef Konrad <franzl(a)fjkonrad.de> wrote: > Slightly off topic but on the other hand it fits perfectly into the discussion: IMHO a very enlightening podcast about RDBMS, NoSQL, NewSQL databases from one of the old database gurus Michael Stonebraker (<http://www.se-radio.net/2013/12/episode-199-michael-stonebraker/>).

April 25, 2014

Re: [Pharo-users] glorp

by Esteban A. Maringolo

Seems like Couchbase got SQL for its documents, with joins and many other nice features: http://blog.couchbase.com/n1ql-it-makes-cents I followed the online tutorial and it is pretty impressive. Esteban A. Maringolo 2014-04-25 10:39 GMT-03:00 Esteban A. Maringolo <emaringolo(a)gmail.com>: > 2014-04-25 6:30 GMT-03:00 Norbert Hartl <norbert(a)hartl.name>: > >> MongoDB is good at indexing geo >> spatial stuff. So mongo + voyage is a good fit for this. I need to do >> explicit commits and have to do additional house keeping both of them you >> donât need to do in GemStone. But I get the indexing capability I need. We >> also do statistical aggregation of data. I used to do it with mongo and >> map-reduce jobs at night but it is cumbersome. Now I use elasticsearch for >> that because I can put in JSON and I get rich and fast query cababilities. >> So my choices of persistence are always driven by the use case (any my >> laziness of course) > > JSON is the new lingua franca of databases (and almost everything > else), as SQL was before. > > Aside from pros/cons of ORM and NoSQL, there is a real explosion of > NoSQL ecosystem things like ElasticSearch, Lucene, etc, and everybody > speaks JSON. > > At the infrastructure level most of the benefits of NoSQL come from > solving the Availability and Partitioning from the CAP Theorem. Truth > is... most of us don't have issues dealing with partitioning. I ran a > system with millions of rows which were perfectly handled by a single > database server, and even using replicas for off-site quering or > backup. > > Making silly analogies, NoSQL is to RDBMS what JSON is to XML. Both > have their place, but most of the times the latter is > overkill/convoluted. > > Regards,

April 25, 2014

Re: [Pharo-users] glorp

by Esteban A. Maringolo

2014-04-25 6:30 GMT-03:00 Norbert Hartl <norbert(a)hartl.name>: > MongoDB is good at indexing geo > spatial stuff. So mongo + voyage is a good fit for this. I need to do > explicit commits and have to do additional house keeping both of them you > donât need to do in GemStone. But I get the indexing capability I need. We > also do statistical aggregation of data. I used to do it with mongo and > map-reduce jobs at night but it is cumbersome. Now I use elasticsearch for > that because I can put in JSON and I get rich and fast query cababilities. > So my choices of persistence are always driven by the use case (any my > laziness of course) JSON is the new lingua franca of databases (and almost everything else), as SQL was before. Aside from pros/cons of ORM and NoSQL, there is a real explosion of NoSQL ecosystem things like ElasticSearch, Lucene, etc, and everybody speaks JSON. At the infrastructure level most of the benefits of NoSQL come from solving the Availability and Partitioning from the CAP Theorem. Truth is... most of us don't have issues dealing with partitioning. I ran a system with millions of rows which were perfectly handled by a single database server, and even using replicas for off-site quering or backup. Making silly analogies, NoSQL is to RDBMS what JSON is to XML. Both have their place, but most of the times the latter is overkill/convoluted. Regards,

April 25, 2014

Re: [Pharo-users] glorp

by Esteban A. Maringolo

Interesting discussion. I'm on the verge of NoSQL databases, I prototyped a system with Voyage but didn't feel confident enough as to continue with it to a production system. One of the things that held me back was the lack of tooling[*] and how hard would it be to change something from "outside" of Voyage. I used ORM in several fashions for over a decade, and wrote one in my last job, so I rely more on SQL than anything else. [*] Have tried running a "complex" query using MongoDB? It's JSON syntax is awful. Fortunately MongoTalk does most of the heavylifting, but as said before, if for any reason I need to run a query from outside (using any other mongo client) it is not going to be easy. To me SQL is concise. I know need a mindset change, which I look forward to get. But today I still have to deal with the utter-verbose definition of GLORP's DescriptionSystem Regards, Esteban A. Maringolo 2014-04-25 6:30 GMT-03:00 Norbert Hartl <norbert(a)hartl.name>: > Joachim, > > Am 25.04.2014 um 10:58 schrieb jtuchel(a)objektfabrik.de: > > Hi Norbert, > > I am way too old for holy wars ;-) > > So in essence, we are both saying that hammering objects into either an SQL > or NoSQL database can be hard and both approaches have negative drawbacks as > well as plusses. > > > Yes. > > I didn't come up with Transactions because they also can make things hard. > See the atomic counter question from a few days ago.IN ORMs you usually end > up with both image-side and database-side implementations of Transactions, > and this can be helpful and complicated at the same time. Sometimes you'd > just like to save one object without saving lots of associated ones from the > same unit of work. > > Does this all lead us to the object database grounds? Or is it all just a > debate of taste and faith? I am still wondering if my life would be easier > if I used an object database (Gemstone comes to mind, but also Magma). And > still I cannot find the final answer. > > > The choice for a technology is not a choice about having more or less > problems but about which problems you like to have and which ones to avoid. > > The problem we have is that main memory is persistent. The separation > between to the kinds of memories is pain. OODBMs are the closest to mitigate > the gap. Theoretical it is hard to decide that. It depends one what you do. > I donât use SQL databases because I like to work with objects. I use > GemStone for quite some years now which is fantastic in certain cases. But I > have projects where I need geo spatial and fulltext indexes (and fast > hierarchical queries). As GemStone does not provide them (the hierarchical > fast query you get by using gemstone specific stuff) and I donât want to > implement them I need something else. MongoDB is good at indexing geo > spatial stuff. So mongo + voyage is a good fit for this. I need to do > explicit commits and have to do additional house keeping both of them you > donât need to do in GemStone. But I get the indexing capability I need. We > also do statistical aggregation of data. I used to do it with mongo and > map-reduce jobs at night but it is cumbersome. Now I use elasticsearch for > that because I can put in JSON and I get rich and fast query cababilities. > So my choices of persistence are always driven by the use case (any my > laziness of course) > > > Maybe that is because I have some knowledge with ORM, but very little with > OODBMS. > > No matter what, my initial point should have been that I think it is naive > to think that NoSQL DBs solve a lot of the problems you have with RDBMS. > They just don't address them, and if you don't see som eof these problems in > your domain, I guess you're best off using NoSQL. > > Agreed. > > ObjectID and stuff in mongo still means you have to make design decisions > and implement something to assign/retrieve those ids and store them in > places where you want to reference your objects. So you do a subset of the > bookkeeping that an RDBMS can do for you by hand. Is that a fair way to put > it? > > You use Glorp and I use Voyage. It is basically the same in this regard. > > You made a good point and I tend to agree to most of what you say. > > Thanks! > > Norbert > > Joachim > > > > > Am 25.04.14 10:35, schrieb Norbert Hartl: > > Joachim, > > thanks for your explanation. I appreciate that. I was thinking if it is a > good idea to write my mail. Usually this ends in a holy war which I donât > want. Comments inline. > > Am 25.04.2014 um 09:02 schrieb jtuchel(a)objektfabrik.de: > > Norbert, > > you are right, I should have given an example of what I mean. > > So here is one: > > If you serialize an object graph to, say, json and store it in a NoSQL DB, > you need to decide how "deep" you serialize your graph into one djson > document and how to build up segments of your graph that need to be > serialized separately, like for an equivalent of an 1:n relationship. > Exanple: an Order contains OrderItems, each of which reference a Product. > But the Product can be referenced from many OrderItems. So you need to "cut > off" parts of your model to serialize them separately. And you need a way to > save the "referenceâ. > > > You can do that in e.g. mongo as well. You just use on ObjectId as a field > value or you use a DBRef. Btw. what you describe is not 1:n but m:n. An > OrderItems can have n products and a Product can be in m OrderItems. > > > One of the next questions then is "what if I delete a Product?". What > happens to the OrderItems or InvoiceItems and so on? > > > If you just delete it the collections will have a stale reference. I think > there is no universal answer to that even if it seems the removal in the > collection is that universal answer. If it is about integrity you need one > way to make it happen. Reestablishing of integrity can happen on write time > or on read time. > What will happen in a SQL context? You canât delete an object that is > pointed to by a foreign key. What does it help then? Not taking your > business model into account you couldnât do anything more to find out where > that reference is. That is probably the only point I wanted to make > questioning your last mail. If we take that scenario you can only solve it > if you take the problem one level higher (well, if you have cascading > deletes you may ignore it). So these problems tend to end in the application > logic. And that is what my experience tells me. Database ensured integrity > isnât much of a help in many cases. So you solve the problems in the > application logic (knowing which things reference what). Being there I see > no big differences to using a NoSQL database. In NoSQL those features are > just not there per se. You have to model it regarding your use case. > > None of these problems are unsolvable, but these problems need to be > addressed either in your persistence framework on top of a NoSQl DB or in > your application code. In Relational DBs, they've built solutions for these > problems before you and I were born ;-) I am talking of foreign keys, > referential integrities and normalization. To my knowledge, these have not > yet found their standardized counterparts in NoSQL DBs. So NoSQL can be a > good solution for many problems, but they can also be bad for many others. > > > Iâm questioning the use of each of those. As I said above I doubt there are > many use cases where foreign keys are the best way to go. Btw. if you ever > administrate a database and you have to recover after a crash then you might > have a different view on foreign keys because they are able to make it close > to impossible to load the data back. Referential integrity is either done > by the database by foreign keys or in the application logic as I said above. > The need for normalization is heavily use case depend. It is nothing good > per se. So these are not good examples IMHO. Iâm wondering you didnât bring > up the only good reason for SQL databases (for most). For me this is having > atomic operations using transactions. This is the one case that can drive > you nuts if try to model something with a NoSQL database to achieve it. > > > I am not saying anything new here. This debate has been going on for decades > already, and much more clever people than me have made good points for both > sides over the years now. > > > Sure. But from time to time it is good to refresh the memory. And for me you > are clever enough ;) > > > You also asked for examples for problems you get from using Glorp. > > One of my biggest concerns is that it can be hard to model an m:n > relationship where one or both sides of the relationship are abstract > superclasses (or even concrete ones with multiple subclasses). It gets even > harder when you want to be able to associate an object (like a Category or > Tag) to "anything". This really is hard or involves some additional coding > on top of the persistence framework. > > Glorp does miss one feature painfully: The ability to map classes with > subclasses that spread over three or more "subtables" for the subclasses. > This one is hard to explain: if you have a superclass "Vehicle" with two > subclasses "FlyingVehicle" and "DrivingVehicle", both of which have > subclasses, there is no support for a model which has tables on the > "FlyingVehicle" or "DrivingVehicle" level. You must model your DB in a way > that there are all attributes in a "Vehicle" table or in the "Car", "Train", > "Spaceship" and "Kite" tables. This really is a pity and I don't think > anybody is working on changing this any time soon. > > Yep. Dealing with these cases I learned that Objects and SQL databases donât > fit together and why ORM cannot work. I mean Glorp is really good at that by > providing your use case with subclasses via virtual lookups. I donâ remember > how it works. It was like a descriptor with candidate classes which lead to > multiple lookups. I figured that out while talking to Alan 8 years ago. I > think I managed it by using a three column m:n mapping table where you have > the two ids and an identifier that gives a hint which subclass/table it is > to look into. Anyway even if Glorp supported sublasses lookups it was a can > of worms to open. I once changed a single thing in my glorp descriptions and > then the login the web site took 1,5 minutes just because all the magic in > Glorp produced 432 database lookups initiate by a single button click. You > just try to force a schema (OO) onto a technology (SQL) that does not > support this case at all. It is an entity relationship model where the > entities are flat. Inheritance is not part of the solution. > > So the summary would be: A lot of people making big efforts to forcing an > object model into a technology that doesnât support it just because they > want to have features like integrity you can establish in other ways. This > is why I donât use SQL databases anymore until I have a use case that does > benefit from it. > > > Does this explain what I mean? > > Yes and I hope I made point also very clear :) > > Norbert > > Joachim > > > > > > > > Am 25.04.14 08:41, schrieb Norbert Hartl: > > > > Am 25.04.2014 um 06:51 schrieb Joachim Tuchel <jtuchel(a)objektfabrik.de>: > > Hi, > > We're using Glorp on VA ST for kontolino.de. It is an active project in the > sense of somebody is taking care of bugs. The lead developer(s) work(s) for > Cincom - and Cincom uses Glorp as supported component of CST. Instantiations > also provides Glorp with VA ST and supports it. Glorp is very stable and is > not moving fast, which is not a disadvantage for a production system. > Features are being added and bugs are fixed. > > Re: Pharoers do NoSQL: judging from the discussions on this list and others, > you have to be careful what you need. To me it seems many developers want > features from relational DBs and hammer them into their applications with > brute force. The first wow! soon gives room to a lot of problems that have > been solved in relational DBs three decades ago. > > I think such a statement should be based on an example. > > But it is true that o/r mapping is not always fun. It can force you to make > decisions about your model that look strange from the object perspective. > > Agreed. And especially in the context of O/R I'm curious what will be your > example to prove your point above. > > Couldn't resist, > > Norbert > > I'd either go full objects (gemstone, magma) or relational for my typical > projects (business apps). > > Hth > > Joachim > > ClÃ©ment Bera <bera.clement(a)gmail.com> schrieb: > > Hello, > > I think Glorp is used with DBXTalk for relational databases by multiple > people. > > Usually, with Pharo, people use as a persistance layer either MongoDB with > the frameworks MongoTalk/Voyage. > > Gemstone is also used quite often for large scale application but Gemstone > is not free. > > Regards. > > > 2014-04-23 20:34 GMT-07:00 Pablo R. Digonzelli <pdigonzelli(a)gmail.com>: >> >> Hi all, can someone tell me if glorp is active there ar is using glorp or >> other persistence framework in production projects? >> >> Tia >> >> ________________________________ >> Ing. Pablo Digonzelli >> Software Solutions >> IP-Solutiones SRL >> Metrotec SRL >> 25 de Mayo 521 >> Email: pdigonzelli(a)softsargentina.com >> pdigonzelli(a)gmail.com >> Cel: 5493815982714 >> > > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > > > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 > >

April 25, 2014

Re: [Pharo-users] glorp

by Franz Josef Konrad

Slightly off topic but on the other hand it fits perfectly into the discussion: IMHO a very enlightening podcast about RDBMS, NoSQL, NewSQL databases from one of the old database gurus Michael Stonebraker (<http://www.se-radio.net/2013/12/episode-199-michael-stonebraker/>). Franz Josef Am 25.04.2014 10:58, schrieb jtuchel(a)objektfabrik.de: > Hi Norbert, > > I am way too old for holy wars ;-)

April 25, 2014

Re: [Pharo-users] glorp

by Norbert Hartl

Joachim, Am 25.04.2014 um 10:58 schrieb jtuchel(a)objektfabrik.de: > Hi Norbert, > > I am way too old for holy wars ;-) > > So in essence, we are both saying that hammering objects into either an SQL or NoSQL database can be hard and both approaches have negative drawbacks as well as plusses. Yes. > I didn't come up with Transactions because they also can make things hard. See the atomic counter question from a few days ago.IN ORMs you usually end up with both image-side and database-side implementations of Transactions, and this can be helpful and complicated at the same time. Sometimes you'd just like to save one object without saving lots of associated ones from the same unit of work. > > Does this all lead us to the object database grounds? Or is it all just a debate of taste and faith? I am still wondering if my life would be easier if I used an object database (Gemstone comes to mind, but also Magma). And still I cannot find the final answer. The choice for a technology is not a choice about having more or less problems but about which problems you like to have and which ones to avoid. The problem we have is that main memory is persistent. The separation between to the kinds of memories is pain. OODBMs are the closest to mitigate the gap. Theoretical it is hard to decide that. It depends one what you do. I donât use SQL databases because I like to work with objects. I use GemStone for quite some years now which is fantastic in certain cases. But I have projects where I need geo spatial and fulltext indexes (and fast hierarchical queries). As GemStone does not provide them (the hierarchical fast query you get by using gemstone specific stuff) and I donât want to implement them I need something else. MongoDB is good at indexing geo spatial stuff. So mongo + voyage is a good fit for this. I need to do explicit commits and have to do additional house keeping both of them you donât need to do in GemStone. But I get the indexing capability I need. We also do statistical aggregation of data. I used to do it with mongo and map-reduce jobs at night but it is cumbersome. Now I use elasticsearch for that because I can put in JSON and I get rich and fast query cababilities. So my choices of persistence are always driven by the use case (any my laziness of course) > > Maybe that is because I have some knowledge with ORM, but very little with OODBMS. > > No matter what, my initial point should have been that I think it is naive to think that NoSQL DBs solve a lot of the problems you have with RDBMS. They just don't address them, and if you don't see som eof these problems in your domain, I guess you're best off using NoSQL. > Agreed. > ObjectID and stuff in mongo still means you have to make design decisions and implement something to assign/retrieve those ids and store them in places where you want to reference your objects. So you do a subset of the bookkeeping that an RDBMS can do for you by hand. Is that a fair way to put it? > You use Glorp and I use Voyage. It is basically the same in this regard. > You made a good point and I tend to agree to most of what you say. > Thanks! Norbert > Joachim > > > > > Am 25.04.14 10:35, schrieb Norbert Hartl: >> Joachim, >> >> thanks for your explanation. I appreciate that. I was thinking if it is a good idea to write my mail. Usually this ends in a holy war which I donât want. Comments inline. >> >> Am 25.04.2014 um 09:02 schrieb jtuchel(a)objektfabrik.de: >> >>> Norbert, >>> >>> you are right, I should have given an example of what I mean. >>> >>> So here is one: >>> >>> If you serialize an object graph to, say, json and store it in a NoSQL DB, you need to decide how "deep" you serialize your graph into one djson document and how to build up segments of your graph that need to be serialized separately, like for an equivalent of an 1:n relationship. Exanple: an Order contains OrderItems, each of which reference a Product. But the Product can be referenced from many OrderItems. So you need to "cut off" parts of your model to serialize them separately. And you need a way to save the "referenceâ. >> >> You can do that in e.g. mongo as well. You just use on ObjectId as a field value or you use a DBRef. Btw. what you describe is not 1:n but m:n. An OrderItems can have n products and a Product can be in m OrderItems. >>> >>> One of the next questions then is "what if I delete a Product?". What happens to the OrderItems or InvoiceItems and so on? >> >> If you just delete it the collections will have a stale reference. I think there is no universal answer to that even if it seems the removal in the collection is that universal answer. If it is about integrity you need one way to make it happen. Reestablishing of integrity can happen on write time or on read time. >> What will happen in a SQL context? You canât delete an object that is pointed to by a foreign key. What does it help then? Not taking your business model into account you couldnât do anything more to find out where that reference is. That is probably the only point I wanted to make questioning your last mail. If we take that scenario you can only solve it if you take the problem one level higher (well, if you have cascading deletes you may ignore it). So these problems tend to end in the application logic. And that is what my experience tells me. Database ensured integrity isnât much of a help in many cases. So you solve the problems in the application logic (knowing which things reference what). Being there I see no big differences to using a NoSQL database. In NoSQL those features are just not there per se. You have to model it regarding your use case. >> >>> None of these problems are unsolvable, but these problems need to be addressed either in your persistence framework on top of a NoSQl DB or in your application code. In Relational DBs, they've built solutions for these problems before you and I were born ;-) I am talking of foreign keys, referential integrities and normalization. To my knowledge, these have not yet found their standardized counterparts in NoSQL DBs. So NoSQL can be a good solution for many problems, but they can also be bad for many others. >> >> Iâm questioning the use of each of those. As I said above I doubt there are many use cases where foreign keys are the best way to go. Btw. if you ever administrate a database and you have to recover after a crash then you might have a different view on foreign keys because they are able to make it close to impossible to load the data back. Referential integrity is either done by the database by foreign keys or in the application logic as I said above. The need for normalization is heavily use case depend. It is nothing good per se. So these are not good examples IMHO. Iâm wondering you didnât bring up the only good reason for SQL databases (for most). For me this is having atomic operations using transactions. This is the one case that can drive you nuts if try to model something with a NoSQL database to achieve it. >>> >>> I am not saying anything new here. This debate has been going on for decades already, and much more clever people than me have made good points for both sides over the years now. >> >> Sure. But from time to time it is good to refresh the memory. And for me you are clever enough ;) >>> >>> You also asked for examples for problems you get from using Glorp. >>> >>> One of my biggest concerns is that it can be hard to model an m:n relationship where one or both sides of the relationship are abstract superclasses (or even concrete ones with multiple subclasses). It gets even harder when you want to be able to associate an object (like a Category or Tag) to "anything". This really is hard or involves some additional coding on top of the persistence framework. >>> >>> Glorp does miss one feature painfully: The ability to map classes with subclasses that spread over three or more "subtables" for the subclasses. This one is hard to explain: if you have a superclass "Vehicle" with two subclasses "FlyingVehicle" and "DrivingVehicle", both of which have subclasses, there is no support for a model which has tables on the "FlyingVehicle" or "DrivingVehicle" level. You must model your DB in a way that there are all attributes in a "Vehicle" table or in the "Car", "Train", "Spaceship" and "Kite" tables. This really is a pity and I don't think anybody is working on changing this any time soon. >>> >> Yep. Dealing with these cases I learned that Objects and SQL databases donât fit together and why ORM cannot work. I mean Glorp is really good at that by providing your use case with subclasses via virtual lookups. I donâ remember how it works. It was like a descriptor with candidate classes which lead to multiple lookups. I figured that out while talking to Alan 8 years ago. I think I managed it by using a three column m:n mapping table where you have the two ids and an identifier that gives a hint which subclass/table it is to look into. Anyway even if Glorp supported sublasses lookups it was a can of worms to open. I once changed a single thing in my glorp descriptions and then the login the web site took 1,5 minutes just because all the magic in Glorp produced 432 database lookups initiate by a single button click. You just try to force a schema (OO) onto a technology (SQL) that does not support this case at all. It is an entity relationship model where the entities are flat. Inheritance is not part of the solution. >> >> So the summary would be: A lot of people making big efforts to forcing an object model into a technology that doesnât support it just because they want to have features like integrity you can establish in other ways. This is why I donât use SQL databases anymore until I have a use case that does benefit from it. >> >>> Does this explain what I mean? >>> >> Yes and I hope I made point also very clear :) >> >> Norbert >> >>> Joachim >>> >>> >>> >>> >>> >>> >>> >>> Am 25.04.14 08:41, schrieb Norbert Hartl: >>>> >>>> >>>> Am 25.04.2014 um 06:51 schrieb Joachim Tuchel <jtuchel(a)objektfabrik.de>: >>>> >>>>> Hi, >>>>> >>>>> We're using Glorp on VA ST for kontolino.de. It is an active project in the sense of somebody is taking care of bugs. The lead developer(s) work(s) for Cincom - and Cincom uses Glorp as supported component of CST. Instantiations also provides Glorp with VA ST and supports it. Glorp is very stable and is not moving fast, which is not a disadvantage for a production system. Features are being added and bugs are fixed. >>>>> >>>>> Re: Pharoers do NoSQL: judging from the discussions on this list and others, you have to be careful what you need. To me it seems many developers want features from relational DBs and hammer them into their applications with brute force. The first wow! soon gives room to a lot of problems that have been solved in relational DBs three decades ago. >>>>> >>>> I think such a statement should be based on an example. >>>> >>>>> But it is true that o/r mapping is not always fun. It can force you to make decisions about your model that look strange from the object perspective. >>>>> >>>> Agreed. And especially in the context of O/R I'm curious what will be your example to prove your point above. >>>> >>>> Couldn't resist, >>>> >>>> Norbert >>>>> I'd either go full objects (gemstone, magma) or relational for my typical projects (business apps). >>>>> >>>>> Hth >>>>> >>>>> Joachim >>>>> >>>>> ClÃ©ment Bera <bera.clement(a)gmail.com> schrieb: >>>>> >>>>> Hello, >>>>> >>>>> I think Glorp is used with DBXTalk for relational databases by multiple people. >>>>> >>>>> Usually, with Pharo, people use as a persistance layer either MongoDB with the frameworks MongoTalk/Voyage. >>>>> >>>>> Gemstone is also used quite often for large scale application but Gemstone is not free. >>>>> >>>>> Regards. >>>>> >>>>> >>>>> 2014-04-23 20:34 GMT-07:00 Pablo R. Digonzelli <pdigonzelli(a)gmail.com>: >>>>> Hi all, can someone tell me if glorp is active there ar is using glorp or other persistence framework in production projects? >>>>> >>>>> Tia >>>>> >>>>> Ing. Pablo Digonzelli >>>>> Software Solutions >>>>> IP-Solutiones SRL >>>>> Metrotec SRL >>>>> 25 de Mayo 521 >>>>> Email: pdigonzelli(a)softsargentina.com >>>>> pdigonzelli(a)gmail.com >>>>> Cel: 5493815982714 >>>>> >>>>> >>> >>> >>> -- >>> ----------------------------------------------------------------------- >>> Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de >>> Fliederweg 1 http://www.objektfabrik.de >>> D-71640 Ludwigsburg http://joachimtuchel.wordpress.com >>> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >>> >> > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >

April 25, 2014

Re: [Pharo-users] glorp

by Esteban Lorenzano

On 25 Apr 2014, at 10:58, jtuchel(a)objektfabrik.de wrote: > Hi Norbert, > > I am way too old for holy wars ;-) > > So in essence, we are both saying that hammering objects into either an SQL or NoSQL database can be hard and both approaches have negative drawbacks as well as plusses. > I didn't come up with Transactions because they also can make things hard. See the atomic counter question from a few days ago.IN ORMs you usually end up with both image-side and database-side implementations of Transactions, and this can be helpful and complicated at the same time. Sometimes you'd just like to save one object without saving lots of associated ones from the same unit of work. > > Does this all lead us to the object database grounds? Or is it all just a debate of taste and faith? I am still wondering if my life would be easier if I used an object database (Gemstone comes to mind, but also Magma). And still I cannot find the final answer. > > Maybe that is because I have some knowledge with ORM, but very little with OODBMS. > > No matter what, my initial point should have been that I think it is naive to think that NoSQL DBs solve a lot of the problems you have with RDBMS. They just don't address them, and if you don't see som eof these problems in your domain, I guess you're best off using NoSQL. > > ObjectID and stuff in mongo still means you have to make design decisions and implement something to assign/retrieve those ids and store them in places where you want to reference your objects. So you do a subset of the bookkeeping that an RDBMS can do for you by hand. Is that a fair way to put it? it is called Voyage and it does all the painful referencing for you. You just need to declare a class as root and voyage takes care of all the work (not for deleting objects) in general (not saying that you do it, but since you call it a âholy warââ¦), I think (and Iâm also old enough to have battle *a lot* in all areas), I think most people uses relational databases for a lot of wrong reasonsâ¦ then they use wrong tool for their problem and in consequence they end taking wrong design decisions just to make their model fit into a wrong solution pattern. Then, they repeat same problem once and again until what become hammered is their own head: they end thinking that the only possible solution is what they decided a priori, without any thinking on the real problem to solve. As a conclusionâ¦ how many people do you know that believes that relational is the âde factoâ way to go instead actually think in their problem? In my case, while I admit there are contexts that are better for a relational database, I think that in object oriented programming it should be the last resource: in general, the amount of concessions you have to do in your model to properly fit in a tabular design is just too much. So the gain should be too much also to make it valuable (and that of course talking in general: concrete problems can have other concrete solutions) :) Esteban > > You made a good point and I tend to agree to most of what you say. > > Joachim > > > > > Am 25.04.14 10:35, schrieb Norbert Hartl: >> Joachim, >> >> thanks for your explanation. I appreciate that. I was thinking if it is a good idea to write my mail. Usually this ends in a holy war which I donât want. Comments inline. >> >> Am 25.04.2014 um 09:02 schrieb jtuchel(a)objektfabrik.de: >> >>> Norbert, >>> >>> you are right, I should have given an example of what I mean. >>> >>> So here is one: >>> >>> If you serialize an object graph to, say, json and store it in a NoSQL DB, you need to decide how "deep" you serialize your graph into one djson document and how to build up segments of your graph that need to be serialized separately, like for an equivalent of an 1:n relationship. Exanple: an Order contains OrderItems, each of which reference a Product. But the Product can be referenced from many OrderItems. So you need to "cut off" parts of your model to serialize them separately. And you need a way to save the "referenceâ. >> >> You can do that in e.g. mongo as well. You just use on ObjectId as a field value or you use a DBRef. Btw. what you describe is not 1:n but m:n. An OrderItems can have n products and a Product can be in m OrderItems. >>> >>> One of the next questions then is "what if I delete a Product?". What happens to the OrderItems or InvoiceItems and so on? >> >> If you just delete it the collections will have a stale reference. I think there is no universal answer to that even if it seems the removal in the collection is that universal answer. If it is about integrity you need one way to make it happen. Reestablishing of integrity can happen on write time or on read time. >> What will happen in a SQL context? You canât delete an object that is pointed to by a foreign key. What does it help then? Not taking your business model into account you couldnât do anything more to find out where that reference is. That is probably the only point I wanted to make questioning your last mail. If we take that scenario you can only solve it if you take the problem one level higher (well, if you have cascading deletes you may ignore it). So these problems tend to end in the application logic. And that is what my experience tells me. Database ensured integrity isnât much of a help in many cases. So you solve the problems in the application logic (knowing which things reference what). Being there I see no big differences to using a NoSQL database. In NoSQL those features are just not there per se. You have to model it regarding your use case. >> >>> None of these problems are unsolvable, but these problems need to be addressed either in your persistence framework on top of a NoSQl DB or in your application code. In Relational DBs, they've built solutions for these problems before you and I were born ;-) I am talking of foreign keys, referential integrities and normalization. To my knowledge, these have not yet found their standardized counterparts in NoSQL DBs. So NoSQL can be a good solution for many problems, but they can also be bad for many others. >> >> Iâm questioning the use of each of those. As I said above I doubt there are many use cases where foreign keys are the best way to go. Btw. if you ever administrate a database and you have to recover after a crash then you might have a different view on foreign keys because they are able to make it close to impossible to load the data back. Referential integrity is either done by the database by foreign keys or in the application logic as I said above. The need for normalization is heavily use case depend. It is nothing good per se. So these are not good examples IMHO. Iâm wondering you didnât bring up the only good reason for SQL databases (for most). For me this is having atomic operations using transactions. This is the one case that can drive you nuts if try to model something with a NoSQL database to achieve it. >>> >>> I am not saying anything new here. This debate has been going on for decades already, and much more clever people than me have made good points for both sides over the years now. >> >> Sure. But from time to time it is good to refresh the memory. And for me you are clever enough ;) >>> >>> You also asked for examples for problems you get from using Glorp. >>> >>> One of my biggest concerns is that it can be hard to model an m:n relationship where one or both sides of the relationship are abstract superclasses (or even concrete ones with multiple subclasses). It gets even harder when you want to be able to associate an object (like a Category or Tag) to "anything". This really is hard or involves some additional coding on top of the persistence framework. >>> >>> Glorp does miss one feature painfully: The ability to map classes with subclasses that spread over three or more "subtables" for the subclasses. This one is hard to explain: if you have a superclass "Vehicle" with two subclasses "FlyingVehicle" and "DrivingVehicle", both of which have subclasses, there is no support for a model which has tables on the "FlyingVehicle" or "DrivingVehicle" level. You must model your DB in a way that there are all attributes in a "Vehicle" table or in the "Car", "Train", "Spaceship" and "Kite" tables. This really is a pity and I don't think anybody is working on changing this any time soon. >>> >> Yep. Dealing with these cases I learned that Objects and SQL databases donât fit together and why ORM cannot work. I mean Glorp is really good at that by providing your use case with subclasses via virtual lookups. I donâ remember how it works. It was like a descriptor with candidate classes which lead to multiple lookups. I figured that out while talking to Alan 8 years ago. I think I managed it by using a three column m:n mapping table where you have the two ids and an identifier that gives a hint which subclass/table it is to look into. Anyway even if Glorp supported sublasses lookups it was a can of worms to open. I once changed a single thing in my glorp descriptions and then the login the web site took 1,5 minutes just because all the magic in Glorp produced 432 database lookups initiate by a single button click. You just try to force a schema (OO) onto a technology (SQL) that does not support this case at all. It is an entity relationship model where the entities are flat. Inheritance is not part of the solution. >> >> So the summary would be: A lot of people making big efforts to forcing an object model into a technology that doesnât support it just because they want to have features like integrity you can establish in other ways. This is why I donât use SQL databases anymore until I have a use case that does benefit from it. >> >>> Does this explain what I mean? >>> >> Yes and I hope I made point also very clear :) >> >> Norbert >> >>> Joachim >>> >>> >>> >>> >>> >>> >>> >>> Am 25.04.14 08:41, schrieb Norbert Hartl: >>>> >>>> >>>> Am 25.04.2014 um 06:51 schrieb Joachim Tuchel <jtuchel(a)objektfabrik.de>: >>>> >>>>> Hi, >>>>> >>>>> We're using Glorp on VA ST for kontolino.de. It is an active project in the sense of somebody is taking care of bugs. The lead developer(s) work(s) for Cincom - and Cincom uses Glorp as supported component of CST. Instantiations also provides Glorp with VA ST and supports it. Glorp is very stable and is not moving fast, which is not a disadvantage for a production system. Features are being added and bugs are fixed. >>>>> >>>>> Re: Pharoers do NoSQL: judging from the discussions on this list and others, you have to be careful what you need. To me it seems many developers want features from relational DBs and hammer them into their applications with brute force. The first wow! soon gives room to a lot of problems that have been solved in relational DBs three decades ago. >>>>> >>>> I think such a statement should be based on an example. >>>> >>>>> But it is true that o/r mapping is not always fun. It can force you to make decisions about your model that look strange from the object perspective. >>>>> >>>> Agreed. And especially in the context of O/R I'm curious what will be your example to prove your point above. >>>> >>>> Couldn't resist, >>>> >>>> Norbert >>>>> I'd either go full objects (gemstone, magma) or relational for my typical projects (business apps). >>>>> >>>>> Hth >>>>> >>>>> Joachim >>>>> >>>>> ClÃ©ment Bera <bera.clement(a)gmail.com> schrieb: >>>>> >>>>> Hello, >>>>> >>>>> I think Glorp is used with DBXTalk for relational databases by multiple people. >>>>> >>>>> Usually, with Pharo, people use as a persistance layer either MongoDB with the frameworks MongoTalk/Voyage. >>>>> >>>>> Gemstone is also used quite often for large scale application but Gemstone is not free. >>>>> >>>>> Regards. >>>>> >>>>> >>>>> 2014-04-23 20:34 GMT-07:00 Pablo R. Digonzelli <pdigonzelli(a)gmail.com>: >>>>> Hi all, can someone tell me if glorp is active there ar is using glorp or other persistence framework in production projects? >>>>> >>>>> Tia >>>>> >>>>> Ing. Pablo Digonzelli >>>>> Software Solutions >>>>> IP-Solutiones SRL >>>>> Metrotec SRL >>>>> 25 de Mayo 521 >>>>> Email: pdigonzelli(a)softsargentina.com >>>>> pdigonzelli(a)gmail.com >>>>> Cel: 5493815982714 >>>>> >>>>> >>> >>> >>> -- >>> ----------------------------------------------------------------------- >>> Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de >>> Fliederweg 1 http://www.objektfabrik.de >>> D-71640 Ludwigsburg http://joachimtuchel.wordpress.com >>> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >>> >> > > > -- > ----------------------------------------------------------------------- > Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de > Fliederweg 1 http://www.objektfabrik.de > D-71640 Ludwigsburg http://joachimtuchel.wordpress.com > Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >

April 25, 2014

Re: [Pharo-users] glorp

by jtuchel＠objektfabrik.de

Hi Norbert, I am way too old for holy wars ;-) So in essence, we are both saying that hammering objects into either an SQL or NoSQL database can be hard and both approaches have negative drawbacks as well as plusses. I didn't come up with Transactions because they also can make things hard. See the atomic counter question from a few days ago.IN ORMs you usually end up with both image-side and database-side implementations of Transactions, and this can be helpful and complicated at the same time. Sometimes you'd just like to save one object without saving lots of associated ones from the same unit of work. Does this all lead us to the object database grounds? Or is it all just a debate of taste and faith? I am still wondering if my life would be easier if I used an object database (Gemstone comes to mind, but also Magma). And still I cannot find the final answer. Maybe that is because I have some knowledge with ORM, but very little with OODBMS. No matter what, my initial point should have been that I think it is naive to think that NoSQL DBs solve a lot of the problems you have with RDBMS. They just don't address them, and if you don't see som eof these problems in your domain, I guess you're best off using NoSQL. ObjectID and stuff in mongo still means you have to make design decisions and implement something to assign/retrieve those ids and store them in places where you want to reference your objects. So you do a subset of the bookkeeping that an RDBMS can do for you by hand. Is that a fair way to put it? You made a good point and I tend to agree to most of what you say. Joachim Am 25.04.14 10:35, schrieb Norbert Hartl: > Joachim, > > thanks for your explanation. I appreciate that. I was thinking if it > is a good idea to write my mail. Usually this ends in a holy war which > I donât want. Comments inline. > > Am 25.04.2014 um 09:02 schrieb jtuchel(a)objektfabrik.de > <mailto:jtuchel@objektfabrik.de>: > >> Norbert, >> >> you are right, I should have given an example of what I mean. >> >> So here is one: >> >> If you serialize an object graph to, say, json and store it in a >> NoSQL DB, you need to decide how "deep" you serialize your graph into >> one djson document and how to build up segments of your graph that >> need to be serialized separately, like for an equivalent of an 1:n >> relationship. Exanple: an Order contains OrderItems, each of which >> reference a Product. But the Product can be referenced from many >> OrderItems. So you need to "cut off" parts of your model to serialize >> them separately. And you need a way to save the "referenceâ. > > You can do that in e.g. mongo as well. You just use on ObjectId as a > field value or you use a DBRef. Btw. what you describe is not 1:n but > m:n. An OrderItems can have n products and a Product can be in m > OrderItems. >> >> One of the next questions then is "what if I delete a Product?". What >> happens to the OrderItems or InvoiceItems and so on? > > If you just delete it the collections will have a stale reference. I > think there is no universal answer to that even if it seems the > removal in the collection is that universal answer. If it is about > integrity you need one way to make it happen. Reestablishing of > integrity can happen on write time or on read time. > What will happen in a SQL context? You canât delete an object that is > pointed to by a foreign key. What does it help then? Not taking your > business model into account you couldnât do anything more to find out > where that reference is. That is probably the only point I wanted to > make questioning your last mail. If we take that scenario you can only > solve it if you take the problem one level higher (well, if you have > cascading deletes you may ignore it). So these problems tend to end in > the application logic. And that is what my experience tells me. > Database ensured integrity isnât much of a help in many cases. So you > solve the problems in the application logic (knowing which things > reference what). Being there I see no big differences to using a NoSQL > database. In NoSQL those features are just not there per se. You have > to model it regarding your use case. > >> None of these problems are unsolvable, but these problems need to be >> addressed either in your persistence framework on top of a NoSQl DB >> or in your application code. In Relational DBs, they've built >> solutions for these problems before you and I were born ;-) I am >> talking of foreign keys, referential integrities and normalization. >> To my knowledge, these have not yet found their standardized >> counterparts in NoSQL DBs. So NoSQL can be a good solution for many >> problems, but they can also be bad for many others. > > Iâm questioning the use of each of those. As I said above I doubt > there are many use cases where foreign keys are the best way to go. > Btw. if you ever administrate a database and you have to recover after > a crash then you might have a different view on foreign keys because > they are able to make it close to impossible to load the data back. > Referential integrity is either done by the database by foreign keys > or in the application logic as I said above. The need for > normalization is heavily use case depend. It is nothing good per se. > So these are not good examples IMHO. Iâm wondering you didnât bring up > the only good reason for SQL databases (for most). For me this is > having atomic operations using transactions. This is the one case that > can drive you nuts if try to model something with a NoSQL database to > achieve it. >> >> I am not saying anything new here. This debate has been going on for >> decades already, and much more clever people than me have made good >> points for both sides over the years now. > > Sure. But from time to time it is good to refresh the memory. And for > me you are clever enough ;) >> >> You also asked for examples for problems you get from using Glorp. >> >> One of my biggest concerns is that it can be hard to model an m:n >> relationship where one or both sides of the relationship are abstract >> superclasses (or even concrete ones with multiple subclasses). It >> gets even harder when you want to be able to associate an object >> (like a Category or Tag) to "anything". This really is hard or >> involves some additional coding on top of the persistence framework. >> >> Glorp does miss one feature painfully: The ability to map classes >> with subclasses that spread over three or more "subtables" for the >> subclasses. This one is hard to explain: if you have a superclass >> "Vehicle" with two subclasses "FlyingVehicle" and "DrivingVehicle", >> both of which have subclasses, there is no support for a model which >> has tables on the "FlyingVehicle" or "DrivingVehicle" level. You must >> model your DB in a way that there are all attributes in a "Vehicle" >> table or in the "Car", "Train", "Spaceship" and "Kite" tables. This >> really is a pity and I don't think anybody is working on changing >> this any time soon. >> > Yep. Dealing with these cases I learned that Objects and SQL databases > donât fit together and why ORM cannot work. I mean Glorp is really > good at that by providing your use case with subclasses via virtual > lookups. I donâ remember how it works. It was like a descriptor with > candidate classes which lead to multiple lookups. I figured that out > while talking to Alan 8 years ago. I think I managed it by using a > three column m:n mapping table where you have the two ids and an > identifier that gives a hint which subclass/table it is to look into. > Anyway even if Glorp supported sublasses lookups it was a can of worms > to open. I once changed a single thing in my glorp descriptions and > then the login the web site took 1,5 minutes just because all the > magic in Glorp produced 432 database lookups initiate by a single > button click. You just try to force a schema (OO) onto a technology > (SQL) that does not support this case at all. It is an entity > relationship model where the entities are flat. Inheritance is not > part of the solution. > > So the summary would be: A lot of people making big efforts to forcing > an object model into a technology that doesnât support it just because > they want to have features like integrity you can establish in other > ways. This is why I donât use SQL databases anymore until I have a use > case that does benefit from it. > >> Does this explain what I mean? >> > Yes and I hope I made point also very clear :) > > Norbert > >> Joachim >> >> >> >> >> >> >> >> Am 25.04.14 08:41, schrieb Norbert Hartl: >>> >>> >>> Am 25.04.2014 um 06:51 schrieb Joachim Tuchel >>> <jtuchel(a)objektfabrik.de <mailto:jtuchel@objektfabrik.de>>: >>> >>>> Hi, >>>> >>>> We're using Glorp on VA ST for kontolino.de <http://kontolino.de/>. >>>> It is an active project in the sense of somebody is taking care of >>>> bugs. The lead developer(s) work(s) for Cincom - and Cincom uses >>>> Glorp as supported component of CST. Instantiations also provides >>>> Glorp with VA ST and supports it. Glorp is very stable and is not >>>> moving fast, which is not a disadvantage for a production system. >>>> Features are being added and bugs are fixed. >>>> >>>> Re: Pharoers do NoSQL: judging from the discussions on this list >>>> and others, you have to be careful what you need. To me it seems >>>> many developers want features from relational DBs and hammer them >>>> into their applications with brute force. The first wow! soon gives >>>> room to a lot of problems that have been solved in relational DBs >>>> three decades ago. >>>> >>> I think such a statement should be based on an example. >>> >>>> But it is true that o/r mapping is not always fun. It can force you >>>> to make decisions about your model that look strange from the >>>> object perspective. >>>> >>> Agreed. And especially in the context of O/R I'm curious what will >>> be your example to prove your point above. >>> >>> Couldn't resist, >>> >>> Norbert >>>> I'd either go full objects (gemstone, magma) or relational for my >>>> typical projects (business apps). >>>> >>>> Hth >>>> >>>> Joachim >>>> >>>> ClÃ©ment Bera <bera.clement(a)gmail.com >>>> <mailto:bera.clement@gmail.com>> schrieb: >>>> >>>> Hello, >>>> >>>> I think Glorp is used with DBXTalk for relational databases by >>>> multiple people. >>>> >>>> Usually, with Pharo, people use as a persistance layer either >>>> MongoDB with the frameworks MongoTalk/Voyage. >>>> >>>> Gemstone is also used quite often for large scale application but >>>> Gemstone is not free. >>>> >>>> Regards. >>>> >>>> >>>> 2014-04-23 20:34 GMT-07:00 Pablo R. Digonzelli >>>> <pdigonzelli(a)gmail.com <mailto:pdigonzelli@gmail.com>>: >>>> >>>> Hi all, can someone tell me if glorp is active there ar is >>>> using glorp or other persistence framework in production projects? >>>> >>>> Tia >>>> >>>> ** >>>> ------------------------------------------------------------------------ >>>> *Ing. Pablo Digonzelli* >>>> Software Solutions >>>> IP-Solutiones SRL >>>> Metrotec SRL >>>> 25 de Mayo 521 >>>> Email: pdigonzelli(a)softsargentina.com >>>> <mailto:pdigonzelli@softsargentina.com> >>>> pdigonzelli(a)gmail.com <mailto:pdigonzelli@gmail.com> >>>> Cel: 5493815982714 >>>> >>>> >> >> >> -- >> ----------------------------------------------------------------------- >> Objektfabrik Joachim Tuchelmailto:jtuchel@objektfabrik.de >> Fliederweg 1http://www.objektfabrik.de >> D-71640 Ludwigsburghttp://joachimtuchel.wordpress.com >> Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1 >> > -- ----------------------------------------------------------------------- Objektfabrik Joachim Tuchel mailto:jtuchel@objektfabrik.de Fliederweg 1 http://www.objektfabrik.de D-71640 Ludwigsburg http://joachimtuchel.wordpress.com Telefon: +49 7141 56 10 86 0 Fax: +49 7141 56 10 86 1

April 25, 2014