Voyage: how to delete references to objects.
Hi, I'm using Voyage in a Pharo application. I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection. How can I remove properly a book and the reference to the book ? Best regards Olivier ;-)
Am 17.04.2014 um 13:53 schrieb olivier <olivier.auverlot@gmail.com>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted. If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time. hope that helps, Norbert
thanks Robert for the explications. I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection. But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ? Olivier ;-) 2014-04-17 14:10 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 17.04.2014 um 13:53 schrieb olivier <olivier.auverlot@gmail.com>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
Am 17.04.2014 um 15:05 schrieb olivier auverlot <olivier.auverlot@gmail.com>:
thanks Robert for the explications.
Itâs Norbert btw. :)
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
You need to explicitly save it as we donât have something like write barriers in the image. So aComicCollection removeBook: aBook; save is needed. Norbert
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 17.04.2014 um 13:53 schrieb olivier <olivier.auverlot@gmail.com>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
Norbert, Sorry for my mistake ;-) 2014-04-17 15:18 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 17.04.2014 um 15:05 schrieb olivier auverlot < olivier.auverlot@gmail.com>:
thanks Robert for the explications.
Itâs Norbert btw. :)
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
You need to explicitly save it as we donât have something like write barriers in the image. So
aComicCollection removeBook: aBook; save
is needed.
Norbert
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 17.04.2014 um 13:53 schrieb olivier <olivier.auverlot@gmail.com>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
On 17 Apr 2014, at 15:18, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.04.2014 um 15:05 schrieb olivier auverlot <olivier.auverlot@gmail.com>:
thanks Robert for the explications.
Itâs Norbert btw. :)
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
You need to explicitly save it as we donât have something like write barriers in the image. So
aComicCollection removeBook: aBook; save
is needed.
yes, and also (if you want to be clean): aBook remove. (assuming aBook is also persistent). But as Sabina points: #enableMissingContent in CommicCollection will help you to simplify things, then you just need: "aBook remove", and next time you read aComicCollection it will âcleanâ the invalid reference. I call that âeventual integrityâ :P but beware of it⦠is considered âpower voyage programmingâ :) Esteban
Norbert
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 17.04.2014 um 13:53 schrieb olivier <olivier.auverlot@gmail.com>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
esteban you should do a pass on the voyage documentation chapter and add this kind of infomration. We can pair write if you want. stef On 17/4/14 17:56, Esteban Lorenzano wrote:
On 17 Apr 2014, at 15:18, Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote:
Am 17.04.2014 um 15:05 schrieb olivier auverlot <olivier.auverlot@gmail.com <mailto:olivier.auverlot@gmail.com>>:
thanks Robert for the explications.
Itâs Norbert btw. :)
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
You need to explicitly save it as we donât have something like write barriers in the image. So
aComicCollection removeBook: aBook; save
is needed.
yes, and also (if you want to be clean):
aBook remove.
(assuming aBook is also persistent).
But as Sabina points: #enableMissingContent in CommicCollection will help you to simplify things, then you just need: "aBook remove", and next time you read aComicCollection it will âcleanâ the invalid reference. I call that âeventual integrityâ :P but beware of it⦠is considered âpower voyage programmingâ :)
Esteban
Norbert
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>>:
Am 17.04.2014 um 13:53 schrieb olivier <olivier.auverlot@gmail.com <mailto:olivier.auverlot@gmail.com>>:
> Hi, > > I'm using Voyage in a Pharo application. > > I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection. > > How can I remove properly a book and the reference to the book ? > From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
Hi, This morning, I arrived early at office because I was impatient to test the proposed solutions. Thank for your help, my problem is solved and I understand better the usage of Voyage and Mongo. We learn something each day and it's cool ! Best regards Olivier ;-) 2014-04-17 21:56 GMT+02:00 pharo4Stef@free.fr <pharo4Stef@free.fr>:
esteban
you should do a pass on the voyage documentation chapter and add this kind of infomration. We can pair write if you want.
stef
On 17/4/14 17:56, Esteban Lorenzano wrote:
On 17 Apr 2014, at 15:18, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.04.2014 um 15:05 schrieb olivier auverlot < olivier.auverlot@gmail.com>:
thanks Robert for the explications.
Itâs Norbert btw. :)
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
You need to explicitly save it as we donât have something like write barriers in the image. So
aComicCollection removeBook: aBook; save
is needed.
yes, and also (if you want to be clean):
aBook remove.
(assuming aBook is also persistent).
But as Sabina points: #enableMissingContent in CommicCollection will help you to simplify things, then you just need: "aBook remove", and next time you read aComicCollection it will âcleanâ the invalid reference. I call that âeventual integrityâ :P but beware of it⦠is considered âpower voyage programmingâ :)
Esteban
Norbert
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <norbert@hartl.name>:
Am 17.04.2014 um 13:53 schrieb olivier <olivier.auverlot@gmail.com>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
Hi Olivier, possibly the option VOMongoContainer>>enableMissingContent; is useful to avoid http://forum.world.st/Voyage-image-freeze-VOMongoError-Lazy-reference-not-fo... Here is an example: http://esug.org/data/ESUG2013/4-Thu/03-ESUG2013%20-%20VoyageTutorial.pdf Regards Sabine On Thu, Apr 17, 2014 at 3:07 PM, Olivier Auverlot [via Smalltalk] < ml-node+s1294792n4755092h47@n4.nabble.com> wrote:
thanks Robert for the explications.
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <[hidden email]<http://user/SendEmail.jtp?type=node&node=4755092&i=0>
:
Am 17.04.2014 um 13:53 schrieb olivier <[hidden email]<http://user/SendEmail.jtp?type=node&node=4755092&i=1>
:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
------------------------------ If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email ml-node+s1294792n1310670h65@n4.nabble.com To unsubscribe from Pharo Smalltalk Users, click here<http://forum.world.st/template/NamlServlet.jtp?macro=unsubscribe_by_code&nod...> . NAML<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant...>
-- View this message in context: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Sabine, I read the case now the first time. Just one note. Having references in both directions is probably not the best idea if using a non-object store. For navigating the graph it might be good but for persisting it is not. The serialization is a lot more complex if you do and the gain is nothing. For most databases with proper indexes you can request both directions easily and fast. So having one reference pointing from A to B you get the opposite direction for free. Norbert Am 17.04.2014 um 15:22 schrieb Sabine Knöfel <sabine.knoefel@gmail.com>:
Hi Olivier,
possibly the option VOMongoContainer>>enableMissingContent; is useful to avoid http://forum.world.st/Voyage-image-freeze-VOMongoError-Lazy-reference-not-fo...
Here is an example: http://esug.org/data/ESUG2013/4-Thu/03-ESUG2013%20-%20VoyageTutorial.pdf
Regards Sabine
On Thu, Apr 17, 2014 at 3:07 PM, Olivier Auverlot [via Smalltalk] <[hidden email]> wrote: thanks Robert for the explications.
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <[hidden email]>:
Am 17.04.2014 um 13:53 schrieb olivier <[hidden email]>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
If you reply to this email, your message will be added to the discussion below: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email [hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML
View this message in context: Re: Voyage: how to delete references to objects. Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Hi Norbert, for me the gain within smalltalk is huge because I want to navigate from person to trip and from trip to person by the attributes without asking mongo for the correspondent object via indexes. Don't you think that this is a question of design of the object model and should not be decided by database arguments? Regards Sabine On Thu, Apr 17, 2014 at 3:40 PM, NorbertHartl [via Smalltalk] < ml-node+s1294792n4755102h57@n4.nabble.com> wrote:
Sabine,
I read the case now the first time. Just one note. Having references in both directions is probably not the best idea if using a non-object store. For navigating the graph it might be good but for persisting it is not. The serialization is a lot more complex if you do and the gain is nothing. For most databases with proper indexes you can request both directions easily and fast. So having one reference pointing from A to B you get the opposite direction for free.
Norbert
Am 17.04.2014 um 15:22 schrieb Sabine Knöfel <[hidden email]<http://user/SendEmail.jtp?type=node&node=4755102&i=0>
:
Hi Olivier,
possibly the option VOMongoContainer>>enableMissingContent; is useful to avoid
http://forum.world.st/Voyage-image-freeze-VOMongoError-Lazy-reference-not-fo...
Here is an example: http://esug.org/data/ESUG2013/4-Thu/03-ESUG2013%20-%20VoyageTutorial.pdf
Regards Sabine
On Thu, Apr 17, 2014 at 3:07 PM, Olivier Auverlot [via Smalltalk] <<a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:
thanks Robert for the explications.
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <[hidden email]<http://user/SendEmail.jtp?type=node&node=4755092&i=0>
:
Am 17.04.2014 um 13:53 schrieb olivier <[hidden email]<http://user/SendEmail.jtp?type=node&node=4755092&i=1>
:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
------------------------------ If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email <a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=1" target="_top" rel="nofollow" link="external">[hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant...>
------------------------------ View this message in context: Re: Voyage: how to delete references to objects.<http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4...> Sent from the Pharo Smalltalk Users mailing list archive<http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html>at Nabble.com.
------------------------------ If you reply to this email, your message will be added to the discussion below:
http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email ml-node+s1294792n1310670h65@n4.nabble.com To unsubscribe from Pharo Smalltalk Users, click here<http://forum.world.st/template/NamlServlet.jtp?macro=unsubscribe_by_code&nod...> . NAML<http://forum.world.st/template/NamlServlet.jtp?macro=macro_viewer&id=instant...>
-- View this message in context: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Am 17.04.2014 um 15:52 schrieb Sabine Knöfel <sabine.knoefel@gmail.com>:
Hi Norbert,
for me the gain within smalltalk is huge because I want to navigate from person to trip and from trip to person by the attributes without asking mongo for the correspondent object via indexes.
Don't you think that this is a question of design of the object model and should not be decided by database arguments?
I think there isnât only one _or_ the other. Of course it is a lot more beautiful to have set up proper objects and collections and do everything in a smalltalkish way. That does only count as long as you stay on the island (smalltalk image). As soon as you start to play with the outer world you have to change your rules. If you agree it isnât perfectly beautiful if you have a proper object model that freezes your image, right? :) IMHO the beauty comes from a mixture of both that fit them together without restricting one or the other. So I would try to avoid storing both directions. But which one to skip? If I assume that a person can have a lot of trips than I would skip the collection inside Person. The more trips a Person has the bigger the serialized document becomes. This makes it first slower to operate on the database and will fail if you reach a document size of 16MB because that is the maximum Mongo supports. Most of the time this will be a non-issue but for now I take it into account. So the data model will have a class Trip with an instVar pointing to Person. And Person will have an instVar trips that is transient (see VOMongoTransientDescription). The implementation for Person then goes Person>>#trips ^ trips ifNil: [ trips := self fetchTrips ] Person>>#fetchTrips ^ Trips selectMany: [: each| ⦠] Person>>#addTrip: aTrip aTrip person: self. aTrip save So the Trip will have a reference to Person in the database but Person will have no reference to Trip. Person will retrieve its trips as soon as they are used in a really smalltalkish way :). Person acts as a smalltalk object and as a database object cache at the same time. So this an implementation of Person that on the inside takes the database nature in regard but on the outside the interface is pure smalltalk. Still not good? Norbert
Regards Sabine
On Thu, Apr 17, 2014 at 3:40 PM, NorbertHartl [via Smalltalk] <[hidden email]> wrote: Sabine,
I read the case now the first time. Just one note. Having references in both directions is probably not the best idea if using a non-object store. For navigating the graph it might be good but for persisting it is not. The serialization is a lot more complex if you do and the gain is nothing. For most databases with proper indexes you can request both directions easily and fast. So having one reference pointing from A to B you get the opposite direction for free.
Norbert
Am 17.04.2014 um 15:22 schrieb Sabine Knöfel <[hidden email]>:
Hi Olivier,
possibly the option VOMongoContainer>>enableMissingContent; is useful to avoid http://forum.world.st/Voyage-image-freeze-VOMongoError-Lazy-reference-not-fo...
Here is an example: http://esug.org/data/ESUG2013/4-Thu/03-ESUG2013%20-%20VoyageTutorial.pdf
Regards Sabine
On Thu, Apr 17, 2014 at 3:07 PM, Olivier Auverlot [via Smalltalk] <<a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote: thanks Robert for the explications.
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <[hidden email]>:
Am 17.04.2014 um 13:53 schrieb olivier <[hidden email]>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
If you reply to this email, your message will be added to the discussion below: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email <a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=1" target="_top" rel="nofollow" link="external">[hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML
View this message in context: Re: Voyage: how to delete references to objects. Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email [hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML
View this message in context: Re: Voyage: how to delete references to objects. Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
On 17 Apr 2014, at 16:26, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.04.2014 um 15:52 schrieb Sabine Knöfel <sabine.knoefel@gmail.com>:
Hi Norbert,
for me the gain within smalltalk is huge because I want to navigate from person to trip and from trip to person by the attributes without asking mongo for the correspondent object via indexes.
Don't you think that this is a question of design of the object model and should not be decided by database arguments?
I think there isnât only one _or_ the other. Of course it is a lot more beautiful to have set up proper objects and collections and do everything in a smalltalkish way. That does only count as long as you stay on the island (smalltalk image). As soon as you start to play with the outer world you have to change your rules. If you agree it isnât perfectly beautiful if you have a proper object model that freezes your image, right? :) IMHO the beauty comes from a mixture of both that fit them together without restricting one or the other.
So I would try to avoid storing both directions. But which one to skip? If I assume that a person can have a lot of trips than I would skip the collection inside Person. The more trips a Person has the bigger the serialized document becomes. This makes it first slower to operate on the database and will fail if you reach a document size of 16MB because that is the maximum Mongo supports. Most of the time this will be a non-issue but for now I take it into account. So the data model will have a class Trip with an instVar pointing to Person. And Person will have an instVar trips that is transient (see VOMongoTransientDescription). The implementation for Person then goes
Person>>#trips ^ trips ifNil: [ trips := self fetchTrips ]
Person>>#fetchTrips ^ Trips selectMany: [: each| ⦠]
Person>>#addTrip: aTrip aTrip person: self. aTrip save
So the Trip will have a reference to Person in the database but Person will have no reference to Trip. Person will retrieve its trips as soon as they are used in a really smalltalkish way :). Person acts as a smalltalk object and as a database object cache at the same time. So this an implementation of Person that on the inside takes the database nature in regard but on the outside the interface is pure smalltalk.
Still not good?
yes, no good :) I mean: I understand there are times when you need it, but I would prefer a framework to solve that for me (then I can think just in my objects). And if the framework does not do a good job, then is a bad framework (even if made by me :P) and it needs to be improved. having to think in persistence is a cancer that needs to be removed :P Esteban
Norbert
Regards Sabine
On Thu, Apr 17, 2014 at 3:40 PM, NorbertHartl [via Smalltalk] <[hidden email]> wrote: Sabine,
I read the case now the first time. Just one note. Having references in both directions is probably not the best idea if using a non-object store. For navigating the graph it might be good but for persisting it is not. The serialization is a lot more complex if you do and the gain is nothing. For most databases with proper indexes you can request both directions easily and fast. So having one reference pointing from A to B you get the opposite direction for free.
Norbert
Am 17.04.2014 um 15:22 schrieb Sabine Knöfel <[hidden email]>:
Hi Olivier,
possibly the option VOMongoContainer>>enableMissingContent; is useful to avoid http://forum.world.st/Voyage-image-freeze-VOMongoError-Lazy-reference-not-fo...
Here is an example: http://esug.org/data/ESUG2013/4-Thu/03-ESUG2013%20-%20VoyageTutorial.pdf
Regards Sabine
On Thu, Apr 17, 2014 at 3:07 PM, Olivier Auverlot [via Smalltalk] <<a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote: thanks Robert for the explications.
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <[hidden email]>:
Am 17.04.2014 um 13:53 schrieb olivier <[hidden email]>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
If you reply to this email, your message will be added to the discussion below: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email <a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=1" target="_top" rel="nofollow" link="external">[hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML
View this message in context: Re: Voyage: how to delete references to objects. Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email [hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML
View this message in context: Re: Voyage: how to delete references to objects. Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
Am 17.04.2014 um 18:05 schrieb Esteban Lorenzano <estebanlm@gmail.com>:
On 17 Apr 2014, at 16:26, Norbert Hartl <norbert@hartl.name> wrote:
Am 17.04.2014 um 15:52 schrieb Sabine Knöfel <sabine.knoefel@gmail.com>:
Hi Norbert,
for me the gain within smalltalk is huge because I want to navigate from person to trip and from trip to person by the attributes without asking mongo for the correspondent object via indexes.
Don't you think that this is a question of design of the object model and should not be decided by database arguments?
I think there isnât only one _or_ the other. Of course it is a lot more beautiful to have set up proper objects and collections and do everything in a smalltalkish way. That does only count as long as you stay on the island (smalltalk image). As soon as you start to play with the outer world you have to change your rules. If you agree it isnât perfectly beautiful if you have a proper object model that freezes your image, right? :) IMHO the beauty comes from a mixture of both that fit them together without restricting one or the other.
So I would try to avoid storing both directions. But which one to skip? If I assume that a person can have a lot of trips than I would skip the collection inside Person. The more trips a Person has the bigger the serialized document becomes. This makes it first slower to operate on the database and will fail if you reach a document size of 16MB because that is the maximum Mongo supports. Most of the time this will be a non-issue but for now I take it into account. So the data model will have a class Trip with an instVar pointing to Person. And Person will have an instVar trips that is transient (see VOMongoTransientDescription). The implementation for Person then goes
Person>>#trips ^ trips ifNil: [ trips := self fetchTrips ]
Person>>#fetchTrips ^ Trips selectMany: [: each| ⦠]
Person>>#addTrip: aTrip aTrip person: self. aTrip save
So the Trip will have a reference to Person in the database but Person will have no reference to Trip. Person will retrieve its trips as soon as they are used in a really smalltalkish way :). Person acts as a smalltalk object and as a database object cache at the same time. So this an implementation of Person that on the inside takes the database nature in regard but on the outside the interface is pure smalltalk.
Still not good?
yes, no good :) I mean: I understand there are times when you need it, but I would prefer a framework to solve that for me (then I can think just in my objects). And if the framework does not do a good job, then is a bad framework (even if made by me :P) and it needs to be improved.
Sure. I was talking about _real_ stuff you can do _now_. I like to think just in objects, too. But we have a separation in hard disc and memory so we need to take care about persistence. I would also prefer to think just in programming and not have to use a stupid device like a keyboard and a mouse just in order to type silly characters that when interpreted probably come close to what I meant. But for the time being I need to stick to it, I guess.
having to think in persistence is a cancer that needs to be removed :P
Yep, we just need a common memory space where we can waste Zetabytes of memory to do little programs. Ah, and did I mention it needs to be infinitely fast because I have a certain use case and I donât like to optimize it because I think that is cancer, too? Very academic view, well done! :) Norbert
Esteban
Norbert
Regards Sabine
On Thu, Apr 17, 2014 at 3:40 PM, NorbertHartl [via Smalltalk] <[hidden email]> wrote: Sabine,
I read the case now the first time. Just one note. Having references in both directions is probably not the best idea if using a non-object store. For navigating the graph it might be good but for persisting it is not. The serialization is a lot more complex if you do and the gain is nothing. For most databases with proper indexes you can request both directions easily and fast. So having one reference pointing from A to B you get the opposite direction for free.
Norbert
Am 17.04.2014 um 15:22 schrieb Sabine Knöfel <[hidden email]>:
Hi Olivier,
possibly the option VOMongoContainer>>enableMissingContent; is useful to avoid http://forum.world.st/Voyage-image-freeze-VOMongoError-Lazy-reference-not-fo...
Here is an example: http://esug.org/data/ESUG2013/4-Thu/03-ESUG2013%20-%20VoyageTutorial.pdf
Regards Sabine
On Thu, Apr 17, 2014 at 3:07 PM, Olivier Auverlot [via Smalltalk] <<a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote: thanks Robert for the explications.
I agree with you that's the best way to remove a book is to delete the reference in ComicsCollection.
But how to do that ? Must I simply remove the reference in the ordered collection ? Voyage will syncronize automatically the data in memory with the content of the database ? Must I force the save of the data after removing the reference ?
Olivier ;-)
2014-04-17 14:10 GMT+02:00 Norbert Hartl <[hidden email]>:
Am 17.04.2014 um 13:53 schrieb olivier <[hidden email]>:
Hi,
I'm using Voyage in a Pharo application.
I have two MongoDB collections which are ComicsCollection and ComicsBook. Each book is attached to a instance of ComicsCollection. The reference of each book is stored in an ordered collection (in the instance of ComicsCollection). The problem is that if I remove a book, the reference to the book is not deleted from ComicsCollection.
How can I remove properly a book and the reference to the book ?
From the smalltalk image view the problem is usually exactly the opposite: You donât delete objects but you remove references to them. As voyage maps objects it is a good idea to stay in the object realm. So you should rather remove the book from the collection. This way you wonât get errors just garbage. The ârealâ problem that arises then is that the book would still be in the database. Just like it is in the image but there is no garbage collector for mongo. To decide that from the image is not possible. You load only a sub graph from the database into image memory. So you donât know if there are other objects referencing the book.
If your records fit all in memory you could load all collections and books and build the difference between all referenced books in the collections and the total amount of books. The difference will be the set of objects not being referenced by a collection and those can be deleted.
If the records do not fit in memory an alternative strategy would be needed. I started to research if it is possible to build a map/reduce based garbage collector for common cases but got distracted. But I have the same problem so I will need to pick it up some time.
hope that helps,
Norbert
If you reply to this email, your message will be added to the discussion below: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email <a href="x-msg://56/user/SendEmail.jtp?type=node&node=4755097&i=1" target="_top" rel="nofollow" link="external">[hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML
View this message in context: Re: Voyage: how to delete references to objects. Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
If you reply to this email, your message will be added to the discussion below: http://forum.world.st/Voyage-how-to-delete-references-to-objects-tp4755088p4... To start a new topic under Pharo Smalltalk Users, email [hidden email] To unsubscribe from Pharo Smalltalk Users, click here. NAML
View this message in context: Re: Voyage: how to delete references to objects. Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.
participants (6)
-
Esteban Lorenzano -
Norbert Hartl -
olivier -
olivier auverlot
-
pharo4Stef＠free.fr -
Sabine Knöfel