November 2014 - Pharo-users

Re: [Pharo-users] Opentalk or remoting libraries

by Norbert Hartl

Using fuel should be way faster than plain textual serialization. A direct copy of memory would be theoretically faster but you would need to manage references anyway so it is not clear what can be gained. Norbert > Am 17.11.2014 um 11:05 schrieb Annick Fron <list(a)afceurope.com>: > > Thank you, > My need is on a local network from machine to machine, in real time is possible, so no security involved. > I would have preferred something more performant than web services or XMLRPC, since both serialization and XML serialization are slow. > Annick > Le 13 nov. 2014 Ã 20:17, Sven Van Caekenberghe <sven(a)stfx.eu> a Ã©crit : > >> Nice. >> >> It is of course also important to note the security risks involved: the client can execute absolutely anything. >> >> One partial solution is to bind the server only to the localhost. >> >>> On 13 Nov 2014, at 19:17, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: >>> >>> Hi, >>> CORBA main focus is about interoperability between systems, languages (don't know about opentalk). >>> If you want smalltalk only remote execution, you can very easily do your own on Pharo with Zinc http components and Fuel serializer: a small server that reads smalltalk blocks, evaluates them and return the result (see below an example of server and client). >>> I don't know if Zinc and Fuel run on Squeak, but I guess if not, they are equivalents ones or you could do the same. >>> Beware that Fuel builds a graph of the objects touched by the object it serializes (the block) and serialize them along the way, that may be a problem if you have references to other big objects of your image. >>> >>> " ================================= RPC server" >>> " start Zn server in background " >>> ZnServer startDefaultOn: 1701. >>> ZnServer default delegate: (ZnWebSocketDelegate handler: >>> ([ :webSocket | >>> [ [ | serializedRequest request response serializedResponse | >>> serializedRequest := webSocket readMessage. >>> request := FLMaterializer materializeFromByteArray: serializedRequest. >>> "execute block and it's value to response byte array" >>> response := [ request value ] >>> on: Exception do: [ :ex | ex resume: 'Exception ',ex class name , ' - ', ex messageText ]. >>> serializedResponse := FLSerializer serializeToByteArray: response . >>> webSocket sendMessage: serializedResponse . >>> ] repeat. >>> ] >>> on: ConnectionClosed, ConnectionTimedOut >>> do: [ self crLog: 'Ignoring connection or timeout' ] >>> ]) ). >>> "=========================================" >>> >>> " to stop the server" >>> ZnServer stopDefault . >>> >>> >>> >>> "========================================= RPC CLIENT" >>> >>> " Client call to server: calculate factorial 100 on the other side" >>> | webSocket | >>> webSocket := ZnWebSocket to: 'ws://127.0.0.1:1701'. >>> [ | request serializedRequest serializedResponse response | >>> request := [ 100 factorial ] . " the request is the block to evaluate" >>> serializedRequest := FLSerializer serializeToByteArray: request . "serialize the block" >>> webSocket sendMessage: serializedRequest . "send it" >>> serializedResponse := webSocket readMessage . " read response" >>> response := FLMaterializer materializeFromByteArray: serializedResponse. "deserialize to object" >>> Transcript show: response asString ; cr ] >>> on: ConnectionClosed >>> do: [ self crLog: 'Ignoring connection close, done' ] >>> >>> >>> regards, >>> >>> Alain >>> >>> >>> Le 12/11/2014 18:17, Annick Fron a Ã©crit : >>>> Hi, >>>> >>>> Are there some libraries in pharo to do remote calls like in Opentalk or Corba ? >>>> >>>> Annick >>>> >>> >>> >>> >> >> > >

Nov. 17, 2014

Re: [Pharo-users] Opentalk or remoting libraries

by Annick Fron

Thank you, My need is on a local network from machine to machine, in real time is possible, so no security involved. I would have preferred something more performant than web services or XMLRPC, since both serialization and XML serialization are slow. Annick Le 13 nov. 2014 Ã 20:17, Sven Van Caekenberghe <sven(a)stfx.eu> a Ã©crit : > Nice. > > It is of course also important to note the security risks involved: the client can execute absolutely anything. > > One partial solution is to bind the server only to the localhost. > >> On 13 Nov 2014, at 19:17, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: >> >> Hi, >> CORBA main focus is about interoperability between systems, languages (don't know about opentalk). >> If you want smalltalk only remote execution, you can very easily do your own on Pharo with Zinc http components and Fuel serializer: a small server that reads smalltalk blocks, evaluates them and return the result (see below an example of server and client). >> I don't know if Zinc and Fuel run on Squeak, but I guess if not, they are equivalents ones or you could do the same. >> Beware that Fuel builds a graph of the objects touched by the object it serializes (the block) and serialize them along the way, that may be a problem if you have references to other big objects of your image. >> >> " ================================= RPC server" >> " start Zn server in background " >> ZnServer startDefaultOn: 1701. >> ZnServer default delegate: (ZnWebSocketDelegate handler: >> ([ :webSocket | >> [ [ | serializedRequest request response serializedResponse | >> serializedRequest := webSocket readMessage. >> request := FLMaterializer materializeFromByteArray: serializedRequest. >> "execute block and it's value to response byte array" >> response := [ request value ] >> on: Exception do: [ :ex | ex resume: 'Exception ',ex class name , ' - ', ex messageText ]. >> serializedResponse := FLSerializer serializeToByteArray: response . >> webSocket sendMessage: serializedResponse . >> ] repeat. >> ] >> on: ConnectionClosed, ConnectionTimedOut >> do: [ self crLog: 'Ignoring connection or timeout' ] >> ]) ). >> "=========================================" >> >> " to stop the server" >> ZnServer stopDefault . >> >> >> >> "========================================= RPC CLIENT" >> >> " Client call to server: calculate factorial 100 on the other side" >> | webSocket | >> webSocket := ZnWebSocket to: 'ws://127.0.0.1:1701'. >> [ | request serializedRequest serializedResponse response | >> request := [ 100 factorial ] . " the request is the block to evaluate" >> serializedRequest := FLSerializer serializeToByteArray: request . "serialize the block" >> webSocket sendMessage: serializedRequest . "send it" >> serializedResponse := webSocket readMessage . " read response" >> response := FLMaterializer materializeFromByteArray: serializedResponse. "deserialize to object" >> Transcript show: response asString ; cr ] >> on: ConnectionClosed >> do: [ self crLog: 'Ignoring connection close, done' ] >> >> >> regards, >> >> Alain >> >> >> Le 12/11/2014 18:17, Annick Fron a Ã©crit : >>> Hi, >>> >>> Are there some libraries in pharo to do remote calls like in Opentalk or Corba ? >>> >>> Annick >>> >> >> >> > >

Nov. 17, 2014

Re: [Pharo-users] running out of memory while processing a 220MB csv file with NeoCSVReader - tips?

by Stephan Eggermont

Open package contents on your vm, open Contents, take a look at the info.plist <key>SqueakMaxHeapSize</key> <integer>541065216</integer> That value needs to be increased to be able to use more than ~512 MB. Alain wrote: >Let say it's your current requirement, and you want to do it like that, >a trick that may help you : during personal experiments about loading >data in memory and statistics from databases, I found that most often 70 >to 80 % of real data is the same. It is easy to confirm if this is the case in your data: just zip the csv file. Reasonably structured relational database output often reduces to 10% of size. With explicitly denormalized data I've seen 99% reduction. In addition, DateAndTime has a rather wasteful representation for your purpose. Just reduce to one SmallInt, or with Pharo4, use slots to get a more compact record representation. Stephan

Nov. 17, 2014

Re: [Pharo-users] pharo consultants

by Sven Van Caekenberghe

OK, I will redo that page - please send updates. > On 17 Nov 2014, at 10:24, Stephan Eggermont <stephan(a)stack.nl> wrote: > > Norbert wrote: >> I forgot how people can add themselves. I remember a smalltalk snippet that one can send around to update the list. > > That doesn't run automatically. There are already some updates waiting. > > Stephan > > The script looks like: > > PharoConsultant new > name: âStephan Eggermont'; > location: 'Den Bosch, The Netherlands'; > email: 'stephan(a)legacycode.nl'; > website: 'http://www.legacycode.nl'; > languages: 'Dutch (native), English, German'; > industryExperienceSince: 1994; > smalltalkExperienceSince: 2005; > pharoExperienceSince: 2008; > areasOfExpertise: 'Legacy Code Refactoring & Migrations, Small Business Applications, Improving Software Development'; > yourself

Nov. 17, 2014

Re: [Pharo-users] pharo consultants

by Stephan Eggermont

Norbert wrote: > I forgot how people can add themselves. I remember a smalltalk snippet that one can send around to update the list. That doesn't run automatically. There are already some updates waiting. Stephan The script looks like: PharoConsultant new name: âStephan Eggermont'; location: 'Den Bosch, The Netherlands'; email: 'stephan(a)legacycode.nl'; website: 'http://www.legacycode.nl'; languages: 'Dutch (native), English, German'; industryExperienceSince: 1994; smalltalkExperienceSince: 2005; pharoExperienceSince: 2008; areasOfExpertise: 'Legacy Code Refactoring & Migrations, Small Business Applications, Improving Software Development'; yourself

Nov. 17, 2014

Re: [Pharo-users] MultiByteFileStream problem

by JosÃ© ComesaÃ±a

Thank you Sven. I an trying to read a veeeery long log file, and extract the last 5 messages for the user to see them. Reading the whole file from the beginning is too expensive. So maybe I could try reading backwards from the end using binary mode and look for a lf... I have to think about it a little bit, and study your proposal, of course. 2014-11-17 9:26 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > Hi JosÃ©, > > > On 17 Nov 2014, at 02:06, JosÃ© ComesaÃ±a <jose.comesana(a)gmail.com> wrote: > > > > There is an annoying error in MultiByteFileStream, reading back when you > have a unicode character. It is also the cause of some FileOut errors. Your > can reproduce it this way: > > > > testString := 'abcdÃ©'. > > filename := 'test.txt'. > > filename asFileReference ensureDelete. > > filename asFileReference > > writeStreamDo: [ :stream | > > stream > > nextPutAll: testString ]. > > > > f := 'test.txt' asFileReference readStream. > > > > f setToEnd. > > > > f skip: -1. > > > > f peek. > > > > > > Any ideas how to solve it? Avoiding reading back is not an option ;). > Maybe making skip to go back one more byte to check if there is a unicode > character around?. > > > > Thanks a lot. > > This is a known problem/issue/limitation of MultiByteFileStream and the > TextConverters. > > The more modern/logical ZnCharacterEncoders can do this reliably (provided > the underlying stream can itself move backwards and the current position is > between whole characters). > > 'test.txt' asFileReference readStreamDo: [ :in | > in binary; setToEnd. > ZnUTF8Encoder new backOnStream: in; nextFromStream: in ] > > IMHO, the ability to move backwards and to play with positions without > restrictions on streams only makes sense with in-memory streams, not with > file or network streams. It plays badly with buffering too. > > I know this doesn't solve your issue but I hope it gives you a better idea > of the situation. > > Where exactly do you have this issue ? > > Sven > > >

Nov. 17, 2014

Re: [Pharo-users] MultiByteFileStream problem

by Sven Van Caekenberghe

Hi JosÃ©, > On 17 Nov 2014, at 02:06, JosÃ© ComesaÃ±a <jose.comesana(a)gmail.com> wrote: > > There is an annoying error in MultiByteFileStream, reading back when you have a unicode character. It is also the cause of some FileOut errors. Your can reproduce it this way: > > testString := 'abcdÃ©'. > filename := 'test.txt'. > filename asFileReference ensureDelete. > filename asFileReference > writeStreamDo: [ :stream | > stream > nextPutAll: testString ]. > > f := 'test.txt' asFileReference readStream. > > f setToEnd. > > f skip: -1. > > f peek. > > > Any ideas how to solve it? Avoiding reading back is not an option ;). Maybe making skip to go back one more byte to check if there is a unicode character around?. > > Thanks a lot. This is a known problem/issue/limitation of MultiByteFileStream and the TextConverters. The more modern/logical ZnCharacterEncoders can do this reliably (provided the underlying stream can itself move backwards and the current position is between whole characters). 'test.txt' asFileReference readStreamDo: [ :in | in binary; setToEnd. ZnUTF8Encoder new backOnStream: in; nextFromStream: in ] IMHO, the ability to move backwards and to play with positions without restrictions on streams only makes sense with in-memory streams, not with file or network streams. It plays badly with buffering too. I know this doesn't solve your issue but I hope it gives you a better idea of the situation. Where exactly do you have this issue ? Sven

Nov. 17, 2014

Re: [Pharo-users] Polygons in woden

by Nicolas Lusa

Hi, sorry for the late response. Thanks a lot for the hints. Though I didn't explain me well enough for what concern the shades. What I actually meant is how light gets reflected from the object. If I build a cube with: addCubeWithWidth I get a nicely dark effect on the walls of the object when the light is not pointing to it and a shiny effect when the light is pointing towards to it. While if I build a object on my own with addQl the walls are not considering the light at all and I don't get that effect anymore. Does anyone know how to solve this? Moreover do you think it is possible to use some how GLPolygon to draw arbitrary polygons since woden is based on Open GL? Cheers, Nicolas. On Nov 9, 2014, at 11:12 PM, Ronie Salgado <roniesalg(a)gmail.com<mailto:roniesalg@gmail.com>> wrote: Hi Nicola, Sorry for not answering before. I have to improve my mail filters. As for the shadows, currently only the spotlights can cast shadows. I have yet to implement shadow mapping for directional lights. As for point light, I won't be implementing them in the near future, because they are very expensive. Currently you can simulate a point light casting shadows by using 6 spotlights with different orientations but in the same point. Also, you have to use deferred shading if you need shadows (WDFPSSimpleExample8 >> #initializeSceneRenderer) As for the polygons, Woden cannot render arbitrary polygons. Only points, lines, triangles and quads because that is what the graphics cards support. For an arbitrary polygon, you have to perform some kind of triangulation. If the polygon is convex, this is trivial, but if the polygon is concave this is hard (the algorithm missing is a Restricted Delaunay Triangulation). For drawing a convex polygon, you have to do something like this: positions := "Some Collection" .. builder newTrianglesScope. "Add the positions". positions do: [:i | builder addP: .... ]. "Add the indices" 3 to: positions size do: [:i | builder addI1: 1 i2: i - 1 i3: i ]. What this code does is to generate a triangle fan ( http://en.wikipedia.org/wiki/Triangle_fan ) which is a simple way to triangulate a convex polygon. I think that I should add a method to the geometry builder. Greetings, Ronie 2014-10-28 11:33 GMT-03:00 Nicolas Lusa <nicolas.lusa(a)usi.ch<mailto:nicolas.lusa@usi.ch>>: Hi everyone, I am working on some 3d representation and I am trying to draw 3D polygons with woden. I actually manage to make "the walls" but I still have to implement top and bottom parts which are not so simple since the shape can be really different from case to case. How I am actually doing it is with: WDGeometryBuilder adding vertices with addP: N: C: TC: messages and with addQuadI1: i2: i3: i4: Furthermore I have some issues with the shadows which actually don't appear. You can see a current polygon created with the code that I made in the attached image. (also notice the shadows are missing as well as the bottom and top part of the polygon). [cid:E6905B6F-F5CA-447D-A42D-BA1604AC7F5A@mobile.usilu.net<mailto:cid%3AE6905B6F-F5CA-447D-A42D-BA1604AC7F5A@mobile.usilu.net>] Now my question is: is there already something that makes those polygons or does anyone have any hint on how to make the top part with woden? Thannks in advance. Cheers, Nicolas

Nov. 17, 2014

MultiByteFileStream problem

by JosÃ© ComesaÃ±a

There is an annoying error in MultiByteFileStream, reading back when you have a unicode character. It is also the cause of some FileOut errors. Your can reproduce it this way: testString := 'abcdÃ©'. filename := 'test.txt'. filename asFileReference ensureDelete. filename asFileReference writeStreamDo: [ :stream | stream nextPutAll: testString ]. f := 'test.txt' asFileReference readStream. f setToEnd. f skip: -1. f peek. Any ideas how to solve it? Avoiding reading back is not an option ;). Maybe making skip to go back one more byte to check if there is a unicode character around?. Thanks a lot.

Nov. 17, 2014

Re: [Pharo-users] running out of memory while processing a 220MB csv file with NeoCSVReader - tips?

by Igor Stasenko

Never ending memory consumption problem. Hopefully with 64-bit version of VM we'll have a way more space to waste and it could take more effort to put system on its knees. -- Best regards, Igor Stasenko.

Nov. 16, 2014