- Pharo-users - lists.pharo.org

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL

by Markus Fritsche

On 2014-11-17 23:14, Brad wrote: > I think it's "(Smalltalk at: #NBPharoOpenDBX) installAsCurrent. That sounds familiar. But since I didn't read a "hooray", I guess that didn't do the trick either? Off for some sleep

Nov. 17, 2014

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL

by Brad

I think it's "(Smalltalk at: #NBPharoOpenDBX) installAsCurrent. Brad Selfridge 913-269-2385 > On Nov 17, 2014, at 3:35 PM, Markus Fritsche <mfritsche(a)reauktion.de> wrote: > > Hi, > > So as far as I can see, you have all dependencies... > > I remember that I had to reinitialize the OpenDBX libraries to change the adaptor to the native library calls (from FFI to NBPharoOpenDBX or similar). Unfortuanetly, I don't have the image at hand. > > >

Nov. 17, 2014

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL

by Brad

How would one re-initialize OpenDBX libraries? Brad Selfridge 913-269-2385 > On Nov 17, 2014, at 3:35 PM, Markus Fritsche <mfritsche(a)reauktion.de> wrote: > > Hi, > > So as far as I can see, you have all dependencies... > > I remember that I had to reinitialize the OpenDBX libraries to change the adaptor to the native library calls (from FFI to NBPharoOpenDBX or similar). Unfortuanetly, I don't have the image at hand. > > >

Nov. 17, 2014

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL

by Markus Fritsche

Hi, So as far as I can see, you have all dependencies... I remember that I had to reinitialize the OpenDBX libraries to change the adaptor to the native library calls (from FFI to NBPharoOpenDBX or similar). Unfortuanetly, I don't have the image at hand.

Nov. 17, 2014

Re: [Pharo-users] running out of memory while processing a 220MB csv file with NeoCSVReader - tips?

by Alain Rastoul

our referential world is very restricted, whatever area we are talking about Le 17/11/2014 21:04, Alain Rastoul a Ã©crit : > you are saying that zip ratio is somewhat related to normalized data, > interesting view, and certainly true :) > And right, this somewhat normalize all fields, a technique used in > specialized columnstore databases (monetdb and others), often BI > databases with id representing values (that were my experiments). > About DateTimes, I think this is not different than with other values, > using a pointer to an interned value should be equivalent to using an > int, as it would be a 32 bits pointer, and with this approach, using > compact records should not make a big difference too if there is not a > lot of different values. > The key I mentioned here is that in real life, this "normalizing ratio" > is very high for almost every kind of data and that's what puzzles me > (not the technique). > > Regards, > Alain > > Le 17/11/2014 10:47, Stephan Eggermont a Ã©crit : >> Open package contents on your vm, >> open Contents, >> take a look at the info.plist >> >> <key>SqueakMaxHeapSize</key> >> <integer>541065216</integer> >> >> That value needs to be increased to be able to use more than ~512 MB. >> >> Alain wrote: >>> Let say it's your current requirement, and you want to do it like that, >>> a trick that may help you : during personal experiments about loading >>> data in memory and statistics from databases, I found that most often 70 >>> to 80 % of real data is the same. >> >> It is easy to confirm if this is the case in your data: just zip the >> csv file. >> Reasonably structured relational database output often reduces to 10% >> of size. >> With explicitly denormalized data I've seen 99% reduction. >> >> In addition, DateAndTime has a rather wasteful representation for your >> purpose. >> Just reduce to one SmallInt, or with Pharo4, use slots to get a more >> compact >> record representation. >> >> Stephan >> > > > >

Nov. 17, 2014

Re: [Pharo-users] running out of memory while processing a 220MB csv file with NeoCSVReader - tips?

by Alain Rastoul

you are saying that zip ratio is somewhat related to normalized data, interesting view, and certainly true :) And right, this somewhat normalize all fields, a technique used in specialized columnstore databases (monetdb and others), often BI databases with id representing values (that were my experiments). About DateTimes, I think this is not different than with other values, using a pointer to an interned value should be equivalent to using an int, as it would be a 32 bits pointer, and with this approach, using compact records should not make a big difference too if there is not a lot of different values. The key I mentioned here is that in real life, this "normalizing ratio" is very high for almost every kind of data and that's what puzzles me (not the technique). Regards, Alain Le 17/11/2014 10:47, Stephan Eggermont a Ã©crit : > Open package contents on your vm, > open Contents, > take a look at the info.plist > > <key>SqueakMaxHeapSize</key> > <integer>541065216</integer> > > That value needs to be increased to be able to use more than ~512 MB. > > Alain wrote: >> Let say it's your current requirement, and you want to do it like that, >> a trick that may help you : during personal experiments about loading >> data in memory and statistics from databases, I found that most often 70 >> to 80 % of real data is the same. > > It is easy to confirm if this is the case in your data: just zip the csv file. > Reasonably structured relational database output often reduces to 10% of size. > With explicitly denormalized data I've seen 99% reduction. > > In addition, DateAndTime has a rather wasteful representation for your purpose. > Just reduce to one SmallInt, or with Pharo4, use slots to get a more compact > record representation. > > Stephan >

Nov. 17, 2014

Re: [Pharo-users] Opentalk or remoting libraries

by Alain Rastoul

For a machine to machine connection, there will be no direct memory reference but under the hood some socket connection with a tcp server and client, and some marshalling of arguments that is to say some serialization. I'm not sure it would be really faster than websocket and Fuel, because it will be a generic api that have to be able to deal with complex cases. I don't see xml and webservice here ? Le 17/11/2014 11:05, Annick Fron a Ã©crit : > Thank you, > My need is on a local network from machine to machine, in real time is possible, so no security involved. > I would have preferred something more performant than web services or XMLRPC, since both serialization and XML serialization are slow. > Annick > Le 13 nov. 2014 Ã 20:17, Sven Van Caekenberghe <sven(a)stfx.eu> a Ã©crit : > >> Nice. >> >> It is of course also important to note the security risks involved: the client can execute absolutely anything. >> >> One partial solution is to bind the server only to the localhost. >> >>> On 13 Nov 2014, at 19:17, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: >>> >>> Hi, >>> CORBA main focus is about interoperability between systems, languages (don't know about opentalk). >>> If you want smalltalk only remote execution, you can very easily do your own on Pharo with Zinc http components and Fuel serializer: a small server that reads smalltalk blocks, evaluates them and return the result (see below an example of server and client). >>> I don't know if Zinc and Fuel run on Squeak, but I guess if not, they are equivalents ones or you could do the same. >>> Beware that Fuel builds a graph of the objects touched by the object it serializes (the block) and serialize them along the way, that may be a problem if you have references to other big objects of your image. >>> >>> " ================================= RPC server" >>> " start Zn server in background" >>> ZnServer startDefaultOn: 1701. >>> ZnServer default delegate: (ZnWebSocketDelegate handler: >>> ([ :webSocket | >>> [ [ | serializedRequest request response serializedResponse | >>> serializedRequest := webSocket readMessage. >>> request := FLMaterializer materializeFromByteArray: serializedRequest. >>> "execute block and it's value to response byte array" >>> response := [ request value ] >>> on: Exception do: [ :ex | ex resume: 'Exception ',ex class name , ' - ', ex messageText ]. >>> serializedResponse := FLSerializer serializeToByteArray: response . >>> webSocket sendMessage: serializedResponse . >>> ] repeat. >>> ] >>> on: ConnectionClosed, ConnectionTimedOut >>> do: [ self crLog: 'Ignoring connection or timeout' ] >>> ]) ). >>> "=========================================" >>> >>> " to stop the server" >>> ZnServer stopDefault . >>> >>> >>> >>> "========================================= RPC CLIENT" >>> >>> " Client call to server: calculate factorial 100 on the other side" >>> | webSocket | >>> webSocket := ZnWebSocket to: 'ws://127.0.0.1:1701'. >>> [ | request serializedRequest serializedResponse response | >>> request := [ 100 factorial ] . " the request is the block to evaluate" >>> serializedRequest := FLSerializer serializeToByteArray: request . "serialize the block" >>> webSocket sendMessage: serializedRequest . "send it" >>> serializedResponse := webSocket readMessage . " read response" >>> response := FLMaterializer materializeFromByteArray: serializedResponse. "deserialize to object" >>> Transcript show: response asString ; cr ] >>> on: ConnectionClosed >>> do: [ self crLog: 'Ignoring connection close, done' ] >>> >>> >>> regards, >>> >>> Alain >>> >>> >>> Le 12/11/2014 18:17, Annick Fron a Ã©crit : >>>> Hi, >>>> >>>> Are there some libraries in pharo to do remote calls like in Opentalk or Corba ? >>>> >>>> Annick >>>> >>> >>> >>> >> >> > > >

Nov. 17, 2014

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL

by Brad Selfridge

On 11/17/2014 11:37 AM, Markus Fritsche wrote: > sudo apt-get install libopendbx1:i386 libopendbx1-mysql:i386 brad@brads-linux-laptop:~$ sudo apt-get install libopendbx1:i386 libopendbx1-mysql:i386 [sudo] password for brad: Reading package lists... Done Building dependency tree Reading state information... Done libopendbx1:i386 is already the newest version. libopendbx1:i386 set to manually installed. libopendbx1-mysql:i386 is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded. brad@brads-linux-laptop:~$

Nov. 17, 2014

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL

by Markus Fritsche

Hello Brad, as far as I can tell, you have the 32 bit mysql client libraries installed (don't worry, the 64 and 32 bit version can coexist) - but I am not sure about opendbx. Please try sudo apt-get install libopendbx1:i386 libopendbx1-mysql:i386 Best regards, Markus

Nov. 17, 2014

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL

by bsselfridge＠gmail.com

The "libmysqlclient18:amd64" looks suspicious. brad@brads-linux-laptop:~$ sudo dpkg -l | grep mysql | awk ' { print $2 } ' libdbd-mysql-perl libmysqlclient18:amd64 libmysqlclient18:i386 libmysqlcppconn7 libopendbx1-mysql libqt4-sql-mysql:i386 mysql-client mysql-client-5.5 mysql-client-core-5.5 mysql-common mysql-server mysql-server-5.5 mysql-server-core-5.5 mysql-utilities mysql-workbench mysql-workbench-data python-mysql.connector brad@brads-linux-laptop:~$ brad@brads-linux-laptop:~$ sudo dpkg -l | grep dbx | awk ' { print $2 } ' libopendbx1 libopendbx1-mysql brad@brads-linux-laptop:~$ ----- Brad Selfridge -- View this message in context: http://forum.world.st/function-unavailable-when-calling-OpenDBXDriver-for-M… Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Nov. 17, 2014