[Pharo-project] GlorpDBX PostgreSQL Patch
Hi, I have a patch for Glorp(DBX) PostgreSQL to support the newer bytea hex format (see file:///usr/local/pgsql/share/doc/html/datatype-binary.html#AEN5037). I can't write to the repository, so here it is: PostgreSQLPlatform>>#convertSQLStringToByteArray: aString for: aType aString isNil ifTrue: [ ^ nil ]. ^ (aString beginsWith: '\x') ifTrue: [ self convertHexSQLStringToByteArray: aString ] ifFalse: [ self convertEscapeSQLStringToByteArray: aString ] PostgreSQLPlatform>>#convertEscapeSQLStringToByteArray: aString | aStream str | aStream := (ByteArray new: aString size // 4) writeStream. str := aString readStream. [str atEnd] whileFalse: [ |nextChar| nextChar := str next. aStream nextPut: (nextChar = $\ ifTrue: [str peek = $\ ifTrue: [str next asInteger] ifFalse: [Number readFrom: (str next: 3) base: 8]] ifFalse: [nextChar asInteger]) ]. aStream close. ^aStream contents PostgreSQLPlatform>>#convertHexSQLStringToByteArray: aString ^ ByteArray new: (aString size // 2) - 1 streamContents: [ :out | | in | (in := aString readStream) skip: 2. [ in atEnd ] whileFalse: [ out nextPut: (Number readFrom: (in next: 2) base: 16) ] ] #convertEscapeSQLStringToByteArray: is the old code, unmodified; #convertHexSQLStringToByteArray: is new. This might be of use upstream as well, I don't really know. If it would help, I could save this as an MC package somewhere. Regards, Sven
Thanks Sven. We let the repo as global read because we usually receive unintentional commits. Do you think it is worth it to have a kind of DBXTalkInbox ? Alan, do you think this is useful also for Glorp trunk ? Thanks! On Tue, Nov 8, 2011 at 5:59 AM, Sven Van Caekenberghe <sven@beta9.be> wrote:
Hi,
I have a patch for Glorp(DBX) PostgreSQL to support the newer bytea hex format (see file:///usr/local/pgsql/share/doc/html/datatype-binary.html#AEN5037). I can't write to the repository, so here it is:
PostgreSQLPlatform>>#convertSQLStringToByteArray: aString for: aType aString isNil ifTrue: [ ^ nil ]. ^ (aString beginsWith: '\x') ifTrue: [ self convertHexSQLStringToByteArray: aString ] ifFalse: [ self convertEscapeSQLStringToByteArray: aString ]
PostgreSQLPlatform>>#convertEscapeSQLStringToByteArray: aString | aStream str | aStream := (ByteArray new: aString size // 4) writeStream. str := aString readStream. [str atEnd] whileFalse: [ |nextChar| nextChar := str next. aStream nextPut: (nextChar = $\ ifTrue: [str peek = $\
ifTrue: [str next asInteger]
ifFalse: [Number readFrom: (str next: 3) base: 8]] ifFalse: [nextChar asInteger]) ]. aStream close. ^aStream contents
PostgreSQLPlatform>>#convertHexSQLStringToByteArray: aString ^ ByteArray new: (aString size // 2) - 1 streamContents: [ :out | | in | (in := aString readStream) skip: 2. [ in atEnd ] whileFalse: [ out nextPut: (Number readFrom: (in next: 2) base: 16) ] ]
#convertEscapeSQLStringToByteArray: is the old code, unmodified; #convertHexSQLStringToByteArray: is new. This might be of use upstream as well, I don't really know.
If it would help, I could save this as an MC package somewhere.
Regards,
Sven
-- Mariano http://marianopeck.wordpress.com
My inclination would be to think that this is something that the database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable.
------------------------------------------------------------------------
Mariano Martinez Peck <mailto:marianopeck@gmail.com> 9 November, 2011 8:59 AM
Thanks Sven. We let the repo as global read because we usually receive unintentional commits. Do you think it is worth it to have a kind of DBXTalkInbox ?
Alan, do you think this is useful also for Glorp trunk ?
Thanks!
-- Mariano http://marianopeck.wordpress.com
------------------------------------------------------------------------
Sven Van Caekenberghe <mailto:sven@beta9.be> 8 November, 2011 3:59 AM
Hi,
I have a patch for Glorp(DBX) PostgreSQL to support the newer bytea hex format (see file:///usr/local/pgsql/share/doc/html/datatype-binary.html#AEN5037). I can't write to the repository, so here it is:
PostgreSQLPlatform>>#convertSQLStringToByteArray: aString for: aType aString isNil ifTrue: [ ^ nil ]. ^ (aString beginsWith: '\x') ifTrue: [ self convertHexSQLStringToByteArray: aString ] ifFalse: [ self convertEscapeSQLStringToByteArray: aString ]
PostgreSQLPlatform>>#convertEscapeSQLStringToByteArray: aString | aStream str | aStream := (ByteArray new: aString size // 4) writeStream. str := aString readStream. [str atEnd] whileFalse: [ |nextChar| nextChar := str next. aStream nextPut: (nextChar = $\ ifTrue: [str peek = $\ ifTrue: [str next asInteger] ifFalse: [Number readFrom: (str next: 3) base: 8]] ifFalse: [nextChar asInteger]) ]. aStream close. ^aStream contents
PostgreSQLPlatform>>#convertHexSQLStringToByteArray: aString ^ ByteArray new: (aString size // 2) - 1 streamContents: [ :out | | in | (in := aString readStream) skip: 2. [ in atEnd ] whileFalse: [ out nextPut: (Number readFrom: (in next: 2) base: 16) ] ]
#convertEscapeSQLStringToByteArray: is the old code, unmodified; #convertHexSQLStringToByteArray: is new. This might be of use upstream as well, I don't really know.
If it would help, I could save this as an MC package somewhere.
Regards,
Sven
I like such things, like visualizing class dependencies with girvan newman algorithm: http://igraph.sourceforge.net/screenshots2.html or here: http://www.little-idiot.de/teambuilding/apache-structure.pdf Should be nice to see, how Pharo has changed over time. Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) Tnx, Guido Stepken
As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed: What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision? -- Cesar Rabak Em 09/11/2011 19:15, Guido Stepken < gstepken@googlemail.com > escreveu: I like such things, like visualizing class dependencies with girvan newman algorithm: http://igraph.sourceforge.net/screenshots2.html or here: http://www.little-idiot.de/teambuilding/apache-structure.pdf Should be nice to see, how Pharo has changed over time. Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...) Tnx, Guido Stepken
Project managers directly can *see*, how much brainpower each programmer is investing and in how far he really cleans up code, defines clear and usable (and used by other coprogrammers) interfaces, or he is messing up the code, producing even more work and costs for others. You know - "lines not written" -> but they directly show up in the graphs! Should be standard everywhere, IMHO. Have fun! Guido Stepken Am 10.11.2011 um 21:25 schrieb csrabak@bol.com.br:
As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed:
What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision?
-- Cesar Rabak
Em 09/11/2011 19:15, Guido Stepken < gstepken@googlemail.com > escreveu: I like such things, like visualizing class dependencies with girvan newman algorithm:
http://igraph.sourceforge.net/screenshots2.html
or here:
http://www.little-idiot.de/teambuilding/apache-structure.pdf
Should be nice to see, how Pharo has changed over time.
Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...)
Tnx, Guido Stepken
On Nov 10, 2011, at 9:44 PM, Guido Stepken wrote:
Project managers directly can *see*, how much brainpower each programmer is investing and in how far he really cleans up code, defines clear and usable (and used by other coprogrammers) interfaces, or he is messing up the code, producing even more work and costs for others.
You know - "lines not written" -> but they directly show up in the graphs!
do you have an example? A web site because I'm interested for Moose. Stef
Should be standard everywhere, IMHO.
Have fun!
Guido Stepken
Am 10.11.2011 um 21:25 schrieb csrabak@bol.com.br:
As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed:
What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision?
-- Cesar Rabak
Em 09/11/2011 19:15, Guido Stepken < gstepken@googlemail.com > escreveu: I like such things, like visualizing class dependencies with girvan newman algorithm:
http://igraph.sourceforge.net/screenshots2.html
or here:
http://www.little-idiot.de/teambuilding/apache-structure.pdf
Should be nice to see, how Pharo has changed over time.
Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...)
Tnx, Guido Stepken
Guido, Notwithstanding how useful would be to track these attributes you just mentioned, I cannot see how the graphs of the clusters shown in the earlier post can be either a representation of them nor even a summary of what you brought now in this comment. . . Regards, -- Cesar Rabak Em 10/11/2011 18:44, Guido Stepken < gstepken@googlemail.com > escreveu: Project managers directly can *see*, how much brainpower each programmer is investing and in how far he really cleans up code, defines clear and usable (and used by other coprogrammers) interfaces, or he is messing up the code, producing even more work and costs for others. You know - "lines not written" -> but they directly show up in the graphs! Should be standard everywhere, IMHO. Have fun! Guido Stepken Am 10.11.2011 um 21:25 schrieb csrabak@bol.com.br:
As the saying says: "Every person to its taste!", I will not delve in the liking or not of the graphs, but I think the question must be posed:
What practical measures can be extracted of such representation and how can they be used to [help] fulfill the Pharo vision?
-- Cesar Rabak
Em 09/11/2011 19:15, Guido Stepken < gstepken@googlemail.com > escreveu: I like such things, like visualizing class dependencies with girvan newman algorithm:
http://igraph.sourceforge.net/screenshots2.html
or here:
http://www.little-idiot.de/teambuilding/apache-structure.pdf
Should be nice to see, how Pharo has changed over time.
Are there such "reflective tools" available in Pharo? (Should be pretty easy to handover a list of Pharo class dependencies to igraph ...)
Tnx, Guido Stepken
Alan, On 09 Nov 2011, at 21:20, Alan Knight wrote:
My inclination would be to think that this is something that the database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable.
Well, I just extended the previous conversion that was already located there. I did not consider whether or not that kind of platform specific code was in its right place, architecture wise. So, will it be integrated somewhere ? If it is only integrated in the Pharo/Squeak port that is fine for me too, but it would be frustrating if it got lost on the next upstream integration. I would just love to be able to load the standard release and have it working for me (i.e. loading blobs from a recent PostgreSQL). Thx for the reply, Sven
On 10.11.2011 14:29, Sven Van Caekenberghe wrote:
Alan,
On 09 Nov 2011, at 21:20, Alan Knight wrote:
My inclination would be to think that this is something that the database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable. Well, I just extended the previous conversion that was already located there. I did not consider whether or not that kind of platform specific code was in its right place, architecture wise.
So, will it be integrated somewhere ? If it is only integrated in the Pharo/Squeak port that is fine for me too, but it would be frustrating if it got lost on the next upstream integration.
I would just love to be able to load the standard release and have it working for me (i.e. loading blobs from a recent PostgreSQL).
Thx for the reply,
Sven
I agree with Alan, it should be in the Postgres-specific code of OpenDBX-driver. Glorp can't be the only user who wants to read blobs, can it? ;) Cheers, Henry
On 10 Nov 2011, at 15:15, Henrik Sperre Johansen wrote:
I agree with Alan, it should be in the Postgres-specific code of OpenDBX-driver. Glorp can't be the only user who wants to read blobs, can it? ;)
I kind of agree as well, but these are two separate things: (1) - I submit a patch to existing code, extending a conversion that was already there (2) - a discussion about architecture and a possible refactoring / improvement It feels like a no brainer to accept (1) and move (2) to the todo/issues list. Sven
Hi guys. I am trying to understand how to integrate this in the OpenDBX driver, but I am a little bit lost. First, I don't understand when this #byteArrayToSQLStringConverter is used. What is the bytearray and what is the string? one is the database type and the other one is the inst var of one you your mapping class? Second, I don't know WHERE to put such conversion in the OpenDBX driver. Why it should be there if #convertSQLStringToByteArray: aString for: aType will continue to exist in Glorp. So it means I should also modify Glorp? How does the EXDI driver of VW work here? Thanks! On Thu, Nov 10, 2011 at 11:15 AM, Henrik Sperre Johansen < henrik.s.johansen@veloxit.no> wrote:
On 10.11.2011 14:29, Sven Van Caekenberghe wrote:
Alan,
On 09 Nov 2011, at 21:20, Alan Knight wrote:
My inclination would be to think that this is something that the
database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable.
Well, I just extended the previous conversion that was already located there. I did not consider whether or not that kind of platform specific code was in its right place, architecture wise.
So, will it be integrated somewhere ? If it is only integrated in the Pharo/Squeak port that is fine for me too, but it would be frustrating if it got lost on the next upstream integration.
I would just love to be able to load the standard release and have it working for me (i.e. loading blobs from a recent PostgreSQL).
Thx for the reply,
Sven
I agree with Alan, it should be in the Postgres-specific code of OpenDBX-driver. Glorp can't be the only user who wants to read blobs, can it? ;)
Cheers, Henry
-- Mariano http://marianopeck.wordpress.com
On 11.11.2011 12:16, Mariano Martinez Peck wrote:
Hi guys. I am trying to understand how to integrate this in the OpenDBX driver, but I am a little bit lost. First, I don't understand when this #byteArrayToSQLStringConverter is used. What is the bytearray and what is the string? one is the database type and the other one is the inst var of one you your mapping class?
Yes. Blobs in Postgres-SQL is basically byteStrings, where a single byte can be encoded in one of two ways: - Raw with some escaped - As a hex string So the method should be used by the driver when asked to store a (smalltalk) byteArray into a (postgres) blob column.
Second, I don't know WHERE to put such conversion in the OpenDBX driver. Why it should be there if #convertSQLStringToByteArray: aString for: aType will continue to exist in Glorp. So it means I should also modify Glorp? Yes, it should be removed from Glorp. If you want to store bytearrays to blobs in different backend databases (or same backend, but different drivers), converting from smalltalk object to what the database stores has to be the job of the driver. Otherwise, you might end up double-encoding in some cases, if both Glorp and Driver does the conversion, and if not in driver, duplicate the conversion in any users of the driver if they are to work correctly.
How does the EXDI driver of VW work here?
Driver takes care of conversion. Does not currently support hex-string format though. Cheers, Henry
On Fri, Nov 11, 2011 at 8:16 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
Hi guys. I am trying to understand how to integrate this in the OpenDBX driver, but I am a little bit lost. First, I don't understand when this #byteArrayToSQLStringConverter is used. What is the bytearray and what is the string? one is the database type and the other one is the inst var of one you your mapping class?
This is because the conversion is being done right now in glorp... We are already performing conversions in OpenDBX, but they are not platform specific conversions, since we handle only ANSI conversions ;). Maybe we can build a PostgreSQLOpenDBXDriver (?)
Second, I don't know WHERE to put such conversion in the OpenDBX driver. Why it should be there if #convertSQLStringToByteArray: aString for: aType will continue to exist in Glorp. So it means I should also modify Glorp?
How does the EXDI driver of VW work here?
Thanks!
On Thu, Nov 10, 2011 at 11:15 AM, Henrik Sperre Johansen < henrik.s.johansen@veloxit.no> wrote:
On 10.11.2011 14:29, Sven Van Caekenberghe wrote:
Alan,
On 09 Nov 2011, at 21:20, Alan Knight wrote:
My inclination would be to think that this is something that the
database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable.
Well, I just extended the previous conversion that was already located there. I did not consider whether or not that kind of platform specific code was in its right place, architecture wise.
So, will it be integrated somewhere ? If it is only integrated in the Pharo/Squeak port that is fine for me too, but it would be frustrating if it got lost on the next upstream integration.
I would just love to be able to load the standard release and have it working for me (i.e. loading blobs from a recent PostgreSQL).
Thx for the reply,
Sven
I agree with Alan, it should be in the Postgres-specific code of OpenDBX-driver. Glorp can't be the only user who wants to read blobs, can it? ;)
Cheers, Henry
-- Mariano http://marianopeck.wordpress.com
-- You received this message because you are subscribed to the Google Groups "glorp-group" group. To post to this group, send email to glorp-group@googlegroups.com. To unsubscribe from this group, send email to glorp-group+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/glorp-group?hl=en.
There are always two possible mechanisms for combining the data type with the SQL statement. We can bind it, in which case the driver is responsible for any conversion, and quite possibly it's being converted into some sort of binary representation. Or we can write it inline in the query as a string. So the VisualWorks drivers would be responsible for doing any conversion if we do binding, but it's true that we need to be responsible if we want to do it as a string. In Postgresql, at least in VisualWorks, it's a bit confusing, because the Postgresql driver basically doesn't support binding - or at least what it does for binding is insert the literal into the string. That's more expensive, and just ends up the same, so normally I wouldn't use binding on Postgresql. But the bottom line is that yes, we do need Glorp to be able to do this at its level, and the OpenDBX driver would only be involved if someone was using binding. On Fri, Nov 11, 2011 at 7:33 AM, Guillermo Polito <guillermopolito@gmail.com
wrote:
On Fri, Nov 11, 2011 at 8:16 AM, Mariano Martinez Peck < marianopeck@gmail.com> wrote:
Hi guys. I am trying to understand how to integrate this in the OpenDBX driver, but I am a little bit lost. First, I don't understand when this #byteArrayToSQLStringConverter is used. What is the bytearray and what is the string? one is the database type and the other one is the inst var of one you your mapping class?
This is because the conversion is being done right now in glorp... We are already performing conversions in OpenDBX, but they are not platform specific conversions, since we handle only ANSI conversions ;).
Maybe we can build a PostgreSQLOpenDBXDriver (?)
Second, I don't know WHERE to put such conversion in the OpenDBX driver. Why it should be there if #convertSQLStringToByteArray: aString for: aType will continue to exist in Glorp. So it means I should also modify Glorp?
How does the EXDI driver of VW work here?
Thanks!
On Thu, Nov 10, 2011 at 11:15 AM, Henrik Sperre Johansen < henrik.s.johansen@veloxit.no> wrote:
On 10.11.2011 14:29, Sven Van Caekenberghe wrote:
Alan,
On 09 Nov 2011, at 21:20, Alan Knight wrote:
My inclination would be to think that this is something that the
database driver ought to be handling rather than having it in Glorp at all. But, given that it's in there, I suppose it's not unreasonable.
Well, I just extended the previous conversion that was already located there. I did not consider whether or not that kind of platform specific code was in its right place, architecture wise.
So, will it be integrated somewhere ? If it is only integrated in the Pharo/Squeak port that is fine for me too, but it would be frustrating if it got lost on the next upstream integration.
I would just love to be able to load the standard release and have it working for me (i.e. loading blobs from a recent PostgreSQL).
Thx for the reply,
Sven
I agree with Alan, it should be in the Postgres-specific code of OpenDBX-driver. Glorp can't be the only user who wants to read blobs, can it? ;)
Cheers, Henry
-- Mariano http://marianopeck.wordpress.com
-- You received this message because you are subscribed to the Google Groups "glorp-group" group. To post to this group, send email to glorp-group@googlegroups.com. To unsubscribe from this group, send email to glorp-group+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/glorp-group?hl=en.
-- You received this message because you are subscribed to the Google Groups "glorp-group" group. To post to this group, send email to glorp-group@googlegroups.com. To unsubscribe from this group, send email to glorp-group+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/glorp-group?hl=en.
Hi, There seems to be a problem with #isPrime in Pharo 1.4. The following example runs apparently forever: 100000000000000000000000000039 isPrime. Unfortunately I used #isPrime to demonstrate multiprocessing with worker images with RemoteTask. RemoteTask seems to work fine on Pharo, but the examples run "forever" and thus appear to fail due to the #isPrime problem. RemoteTask is described here <http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-November/162167....> Dave
Thanks dave http://code.google.com/p/pharo/issues/detail?id=4997 Stef On Nov 13, 2011, at 4:58 PM, David T. Lewis wrote:
Hi,
There seems to be a problem with #isPrime in Pharo 1.4. The following example runs apparently forever:
100000000000000000000000000039 isPrime.
Unfortunately I used #isPrime to demonstrate multiprocessing with worker images with RemoteTask. RemoteTask seems to work fine on Pharo, but the examples run "forever" and thus appear to fail due to the #isPrime problem.
RemoteTask is described here <http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-November/162167....>
Dave
I think this may be just an incomplete update from Squeak trunk. I added a comment to the issue to explain. Dave On Sun, Nov 13, 2011 at 06:21:50PM +0100, St?phane Ducasse wrote:
Thanks dave
http://code.google.com/p/pharo/issues/detail?id=4997
Stef
On Nov 13, 2011, at 4:58 PM, David T. Lewis wrote:
Hi,
There seems to be a problem with #isPrime in Pharo 1.4. The following example runs apparently forever:
100000000000000000000000000039 isPrime.
Unfortunately I used #isPrime to demonstrate multiprocessing with worker images with RemoteTask. RemoteTask seems to work fine on Pharo, but the examples run "forever" and thus appear to fail due to the #isPrime problem.
RemoteTask is described here <http://lists.squeakfoundation.org/pipermail/squeak-dev/2011-November/162167....>
Dave
participants (10)
-
Alan Knight -
Alan Knight -
csrabak@bol.com.br -
David T. Lewis -
Guido Stepken -
Guillermo Polito -
Henrik Sperre Johansen -
Mariano Martinez Peck -
Stéphane Ducasse -
Sven Van Caekenberghe