November 2014 - Pharo-users

Re: [Pharo-users] running out of memory while processing a 220MB csv file with NeoCSVReader - tips?
by Stephan Eggermont Nov. 18, 2014

Nov. 18, 2014

Alain wrote: >you are saying that zip ratio is somewhat related to normalized data, >interesting view, and certainly true :) I find it a nice heuristic to help me get started. Just sort the tables on size, start compressing them and start with ones compressing best. >About DateTimes, I think this is not different than with other values, >using a pointer to an interned value should be equivalent to using an >int, as it would be a 32 bits pointer, and with this approach, using >compact records should not make a big difference too if there is not a >lot of different values. Combining multiple booleans in one word still helps a lot, and introducing extra objects for highly correlated fields. >The key I mentioned here is that in real life, this "normalizing ratio" >is very high for almost every kind of data and that's what puzzles me >(not the technique). My impression is that the a lot of the design decisions for relational databases are cargo cult based on the time where most database did not fit into ram and the query optimizers were not good at dealing with lots of joins. Stephan

1 0

[ANN]: NewExternalWebBrowser
by Sean P. DeNigris Nov. 18, 2014

Nov. 18, 2014

Not too long ago, someone requested the ability to open a URL in an external browser. Now that Pharo has NativeBoost, it seems we could implement ExternalWebBrowser much more simply. Here's a first commit for Mac OS X. Gofer it smalltalkhubUser: 'SeanDeNigris' project: 'SeansPlayground'; package: 'NewExternalWebBrowser'; load. 'http://google.com' asUrl open. ----- Cheers, Sean -- View this message in context: http://forum.world.st/ANN-NewExternalWebBrowser-tp4788403.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

4 12

Re: [Pharo-users] 'function unavailable' when calling OpenDBXDriver for MySQL
by Stephan Eggermont Nov. 17, 2014

Nov. 17, 2014

Hi Brad, You don't tell us if your MySQL is 32 bit? Stephan

5 10

Re: [Pharo-users] running out of memory while processing a 220MB csv file with NeoCSVReader - tips?
by Stephan Eggermont Nov. 17, 2014

Nov. 17, 2014

Open package contents on your vm, open Contents, take a look at the info.plist <key>SqueakMaxHeapSize</key> <integer>541065216</integer> That value needs to be increased to be able to use more than ~512 MB. Alain wrote: >Let say it's your current requirement, and you want to do it like that, >a trick that may help you : during personal experiments about loading >data in memory and statistics from databases, I found that most often 70 >to 80 % of real data is the same. It is easy to confirm if this is the case in your data: just zip the csv file. Reasonably structured relational database output often reduces to 10% of size. With explicitly denormalized data I've seen 99% reduction. In addition, DateAndTime has a rather wasteful representation for your purpose. Just reduce to one SmallInt, or with Pharo4, use slots to get a more compact record representation. Stephan

2 2

Opentalk or remoting libraries
by Annick Fron Nov. 17, 2014

Nov. 17, 2014

Hi, Are there some libraries in pharo to do remote calls like in Opentalk or Corba ? Annick

6 8

'function unavailable' when calling OpenDBXDriver for MySQL
by bsselfridge＠gmail.com Nov. 17, 2014

Nov. 17, 2014

I've used the instructions defined on the "http://dbxtalk.smallworks.eu/" website to setup the OpenDBXDriver for a MySQL database. When I try to make a connection to the database, I'm getting the "function unavailable". I have this working on a Ubuntu 14.04 32bit OS. I'm wondering if my problem is my 64bit Ubuntu or do I have other problems. My setup: Ubuntu 14.04 64bit. Pharo 3.0 libopendbx.so.1.2.0 i386 version Brad Selfridge ----- Brad Selfridge -- View this message in context: http://forum.world.st/function-unavailable-when-calling-OpenDBXDriver-for-M… Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

2 2

MultiByteFileStream problem
by JosÃ© ComesaÃ±a Nov. 17, 2014

Nov. 17, 2014

There is an annoying error in MultiByteFileStream, reading back when you have a unicode character. It is also the cause of some FileOut errors. Your can reproduce it this way: testString := 'abcdÃ©'. filename := 'test.txt'. filename asFileReference ensureDelete. filename asFileReference writeStreamDo: [ :stream | stream nextPutAll: testString ]. f := 'test.txt' asFileReference readStream. f setToEnd. f skip: -1. f peek. Any ideas how to solve it? Avoiding reading back is not an option ;). Maybe making skip to go back one more byte to check if there is a unicode character around?. Thanks a lot.

3 4

Re: [Pharo-users] pharo consultants
by Stephan Eggermont Nov. 17, 2014

Nov. 17, 2014

Norbert wrote: > I forgot how people can add themselves. I remember a smalltalk snippet that one can send around to update the list. That doesn't run automatically. There are already some updates waiting. Stephan The script looks like: PharoConsultant new name: âStephan Eggermont'; location: 'Den Bosch, The Netherlands'; email: 'stephan(a)legacycode.nl'; website: 'http://www.legacycode.nl'; languages: 'Dutch (native), English, German'; industryExperienceSince: 1994; smalltalkExperienceSince: 2005; pharoExperienceSince: 2008; areasOfExpertise: 'Legacy Code Refactoring & Migrations, Small Business Applications, Improving Software Development'; yourself

2 1

Polygons in woden
by Nicolas Lusa Nov. 17, 2014

Nov. 17, 2014

Hi everyone, I am working on some 3d representation and I am trying to draw 3D polygons with woden. I actually manage to make "the walls" but I still have to implement top and bottom parts which are not so simple since the shape can be really different from case to case. How I am actually doing it is with: WDGeometryBuilder adding vertices with addP: N: C: TC: messages and with addQuadI1: i2: i3: i4: Furthermore I have some issues with the shadows which actually don't appear. You can see a current polygon created with the code that I made in the attached image. (also notice the shadows are missing as well as the bottom and top part of the polygon). [cid:E6905B6F-F5CA-447D-A42D-BA1604AC7F5A@mobile.usilu.net] Now my question is: is there already something that makes those polygons or does anyone have any hint on how to make the top part with woden? Thannks in advance. Cheers, Nicolas

2 2

running out of memory while processing a 220MB csv file with NeoCSVReader - tips?
by Paul DeBruicker Nov. 16, 2014

Nov. 16, 2014

Hi - I'm processing a 9 GBs of CSV files (the biggest file is 220MB or so). I'm not sure if its because of the size of the files or the code I've written to keep track of the domain objects I'm interested in, but I'm getting out of memory errors & crashes in Pharo 3 on Mac with the latest VM. I haven't checked other vms. I'm going to profile my own code and attempt to split the files manually for now to see what else it could be. Right now I'm doing something similar to |file reader| file:= '/path/to/file/myfile.csv' asFileReference readStream. reader: NeoCSVReader on: file reader recordClass: MyClass; skipHeader; addField: #myField:; .... reader do:[:eachRecord | self seeIfRecordIsInterestingAndIfSoKeepIt: eachRecord]. file close. Is there a facility in NeoCSVReader to read a file in batches (e.g. 1000 lines at a time) or an easy way to do that ? Thanks Paul

5 14