October 2014 - Pharo-users - lists.pharo.org

Re: [Pharo-users] About Zinc http components

by Sven Van Caekenberghe

Hi Alain, Any network protocol between two processes has overhead, even the one you would write yourself. It does make sense to use something that already exists, uses open standards and is used and battle tested by others. There is a fundamental difference between HTTP and WebSockets: HTTP is request/response where the initiative lies with the client, WebSockets is fully bidirectional once the connection is setup. There is also less overhead with WebSockets. In both cases you should take care to do your data conversions on/off the wire as efficient as possible. I would say: go for it. Sven On 07 Oct 2014, at 01:10, Alain Rastoul <alf.mmm.cat(a)gmail.com> wrote: > Hi, > I played a bit with ZnServer and other zinc components and have > a question I can't answer myself (googling a bit didn't help neither), > and I'm seeking for advice: does it makes sense to use a ZnServer/ZnWebSocket > as a mechanism to transfer data between two pharo processes - in my case 8k ByteArray blocks ? > Or is it a total non sense ? is it reliable ? > First tests looks good : 18k blocks / second (145 Mb/s) on a laptop > core i5 2.6Ghz. > > All comments and suggestions welcome > > Thanks in advance, > > Alain

Oct. 7, 2014

Re: [Pharo-users] Smalltalk based unikernels

by stepharo

> Hello, > > I'm playing with the unikernels idea [1]. Specifically I'm using > Mirage OS [2] to probe a minimal HTTP server running on AWS directly > over Xen, without any OS between the application and the hypervisor. Yes yes yes we would love to have a Pharo solution for that. > > Mirage OS is based on OCaml. Is there any Smalltalk solution to build > this kind of unikernels? The idea of Smalltalk all the way down (at > least to the virtual devices of the hypervisor) is very appealing to > me. Indeed I think this idea was proposed for the first time by Dan > Ingalls: > > ''An operating system is a collection of things that don't fit into a > language. There shouldn't be one.'' > > I have found SqueakNOS [3], but I don't know if it is active or if it > works with Pharo. It was working. > > Thank you, > > Rafa > > [1] http://queue.acm.org/detail.cfm?id=2566628 > [2] http://www.openmirage.org/ > [3] http://squeaknos.blogspot.com.es/

Oct. 7, 2014

About Zinc http components

by Alain Rastoul

Hi, I played a bit with ZnServer and other zinc components and have a question I can't answer myself (googling a bit didn't help neither), and I'm seeking for advice: does it makes sense to use a ZnServer/ZnWebSocket as a mechanism to transfer data between two pharo processes - in my case 8k ByteArray blocks ? Or is it a total non sense ? is it reliable ? First tests looks good : 18k blocks / second (145 Mb/s) on a laptop core i5 2.6Ghz. All comments and suggestions welcome Thanks in advance, Alain

Oct. 6, 2014

Re: [Pharo-users] BLOG: Block Translators - parsing magic

by Udo Schneider

On 06.10.14 11:46, Damien Cassou wrote: > no such guide exist. Do your best and we will do another pass on it afterwards. Ok - good to know. I checked in a version which is IMHO reasonably "clean". I'll check again in two weeks or so - it's sometimes surprising how different something looks when you didn't take a look at it for a while. CU, Udo

Oct. 6, 2014

Smalltalk based unikernels

by Rafael Luque

Hello, I'm playing with the unikernels idea [1]. Specifically I'm using Mirage OS [2] to probe a minimal HTTP server running on AWS directly over Xen, without any OS between the application and the hypervisor. Mirage OS is based on OCaml. Is there any Smalltalk solution to build this kind of unikernels? The idea of Smalltalk all the way down (at least to the virtual devices of the hypervisor) is very appealing to me. Indeed I think this idea was proposed for the first time by Dan Ingalls: ''An operating system is a collection of things that don't fit into a language. There shouldn't be one.'' I have found SqueakNOS [3], but I don't know if it is active or if it works with Pharo. Thank you, Rafa [1] http://queue.acm.org/detail.cfm?id=2566628 [2] http://www.openmirage.org/ [3] http://squeaknos.blogspot.com.es/

Oct. 6, 2014

Re: [Pharo-users] NeoCSVReader skipEmptyLines?

by Esteban A. Maringolo

Excellent! And your implementation is better than simply skipping empty lines :) Thank you Sven! Esteban A. Maringolo 2014-10-06 12:52 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > In #bleedingEdge: > > === > Name: Neo-CSV-Core-SvenVanCaekenberghe.21 > Author: SvenVanCaekenberghe > Time: 6 October 2014, 5:49:01.691696 pm > UUID: a6faeef3-aaa7-460f-a815-8e77b62f9906 > Ancestors: Neo-CSV-Core-SvenVanCaekenberghe.20 > > Added NeoCSVReader>>#addIgnoredFields: > Added NeoCSVReader>>#select:[thenDo:] to skip empty or other records > Added NeoCSVReaderTests>>#testSkippingEmptyRecords > === > Name: Neo-CSV-Tests-SvenVanCaekenberghe.18 > Author: SvenVanCaekenberghe > Time: 6 October 2014, 5:49:36.206743 pm > UUID: 95d87278-4398-41d3-96ed-83705d5488ce > Ancestors: Neo-CSV-Tests-SvenVanCaekenberghe.17 > > Added NeoCSVReader>>#addIgnoredFields: > Added NeoCSVReader>>#select:[thenDo:] to skip empty or other records > Added NeoCSVReaderTests>>#testSkippingEmptyRecords > === > > Here is the test: > > testSkippingEmptyRecords > | input output | > input := '1,2,3\\4,5,6\,,\7,8,9' withCRs. > output := (NeoCSVReader on: input readStream) > select: [ :each | each notEmpty and: [ (each allSatisfy: #isNil) not ] ]. > self assert: output equals: #(#('1' '2' '3') #('4' '5' '6') #('7' '8' '9')). > output := (NeoCSVReader on: input readStream) > emptyFieldValue: ''; > select: [ :each | each notEmpty and: [ (each allSatisfy: #isEmpty) not ] ]. > self assert: output equals: #(#('1' '2' '3') #('4' '5' '6') #('7' '8' '9')) > > Regards, > > Sven > > On 05 Oct 2014, at 02:39, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: > >> 2014-10-04 10:47 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >>> Esteban, >>> >>> On 04 Oct 2014, at 02:42, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: >>> >>>> Sven, >>>> >>>> Is it possible to skip blank/empty lines in NeoCSVReader? >>>> >>>> So if a line is empty or contains ",,,,,,,," it would be skipped from >>>> the iteration. >>>> >>>> Does it already provider such feature? (I couldn't find any setting >>>> related with this). >>>> >>>> Esteban A. Maringolo >>> >>> Could you elaborate a bit on the use case ? >> >> Yes, users upload a CSV file with the proper columns (sometimes they >> don't, but that's another story), and because they export/copy the CSV >> from Excel most of >> the times there are *lots* (might be hundreds) of empty lines at the >> end, it is, lines >> with separators only, or even blank lines. >> >>> My first reaction would be that you have to deal with these yourself afterwards. >> >> When dealing with "array based" records, it is, with no #recordClass >> defined it would be easier. Because you can check whether all the >> values of the array are empty or nil. So on each iteration (#next) you >> could return the value array if it isn't empty or send #next again >> until you get a record with any data. >> >> When working with a record class I think you can't do this, because it >> seems you instantiate the record before iterating through each field >> of the record. >> >>> What would be the definition of an empty record ? What about empty quoted fields ? >> >> An empty record of a four columns CSV file would look like: >> ,,, >> or (if some columns are quoted) >> ,"","", >> >> I don't see how a record like that could be of any use. >> >> Best regards! >> >> Esteban A. Maringolo >> >> >>> PS: I got your other contribution, I think it is OK to add, thanks. >> >> Great! > >

Oct. 6, 2014

Re: [Pharo-users] Glorp + NBSQLite3

by Esteban A. Maringolo

2014-10-06 14:11 GMT-03:00 Sanjay-M <sm(a)planage.com>: > what I was saying was supposing we write a server that alone only accesses > the SQLite database. Users behave like clients accessing the server over > TCP/IP. > and the text i pointed to in my earlier mail detailed how such a scenario > could be implemented. > If someone could write a small sample with say one class then it could be > easier to understand / implement But that has nothing to do with SQLite nor any other persistence solution. In fact most web applications work that way, where the messaging runs over HTTP (and hence TCP/IP) and the server responds to them. NBSQLite3 is a good solution for self contained applications, where you can distribute everything without requiring the installation/configuration of any third party DB server. If you can't have a web server in your network environment (i.e. intranet/LAN) how will you have any other TCP/IP based server? I mean, if you want to run a Pharo image as your server tier, unless you have certain port/policies restrictions, it will make no difference whether you run an HTTP server or your [put your custom protocol here] server. Regards!

Oct. 6, 2014

Re: [Pharo-users] Glorp + NBSQLite3

by Sanjay-M

what I was saying was supposing we write a server that alone only accesses the SQLite database. Users behave like clients accessing the server over TCP/IP. and the text i pointed to in my earlier mail detailed how such a scenario could be implemented. If someone could write a small sample with say one class then it could be easier to understand / implement ----- --- Regards, Sanjay -- View this message in context: http://forum.world.st/Glorp-NBSQLite3-tp4779996p4783001.html Sent from the Pharo Smalltalk Users mailing list archive at Nabble.com.

Oct. 6, 2014

Re: [Pharo-users] NeoCSVReader skipEmptyLines?

by Sven Van Caekenberghe

In #bleedingEdge: === Name: Neo-CSV-Core-SvenVanCaekenberghe.21 Author: SvenVanCaekenberghe Time: 6 October 2014, 5:49:01.691696 pm UUID: a6faeef3-aaa7-460f-a815-8e77b62f9906 Ancestors: Neo-CSV-Core-SvenVanCaekenberghe.20 Added NeoCSVReader>>#addIgnoredFields: Added NeoCSVReader>>#select:[thenDo:] to skip empty or other records Added NeoCSVReaderTests>>#testSkippingEmptyRecords === Name: Neo-CSV-Tests-SvenVanCaekenberghe.18 Author: SvenVanCaekenberghe Time: 6 October 2014, 5:49:36.206743 pm UUID: 95d87278-4398-41d3-96ed-83705d5488ce Ancestors: Neo-CSV-Tests-SvenVanCaekenberghe.17 Added NeoCSVReader>>#addIgnoredFields: Added NeoCSVReader>>#select:[thenDo:] to skip empty or other records Added NeoCSVReaderTests>>#testSkippingEmptyRecords === Here is the test: testSkippingEmptyRecords | input output | input := '1,2,3\\4,5,6\,,\7,8,9' withCRs. output := (NeoCSVReader on: input readStream) select: [ :each | each notEmpty and: [ (each allSatisfy: #isNil) not ] ]. self assert: output equals: #(#('1' '2' '3') #('4' '5' '6') #('7' '8' '9')). output := (NeoCSVReader on: input readStream) emptyFieldValue: ''; select: [ :each | each notEmpty and: [ (each allSatisfy: #isEmpty) not ] ]. self assert: output equals: #(#('1' '2' '3') #('4' '5' '6') #('7' '8' '9')) Regards, Sven On 05 Oct 2014, at 02:39, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: > 2014-10-04 10:47 GMT-03:00 Sven Van Caekenberghe <sven(a)stfx.eu>: >> Esteban, >> >> On 04 Oct 2014, at 02:42, Esteban A. Maringolo <emaringolo(a)gmail.com> wrote: >> >>> Sven, >>> >>> Is it possible to skip blank/empty lines in NeoCSVReader? >>> >>> So if a line is empty or contains ",,,,,,,," it would be skipped from >>> the iteration. >>> >>> Does it already provider such feature? (I couldn't find any setting >>> related with this). >>> >>> Esteban A. Maringolo >> >> Could you elaborate a bit on the use case ? > > Yes, users upload a CSV file with the proper columns (sometimes they > don't, but that's another story), and because they export/copy the CSV > from Excel most of > the times there are *lots* (might be hundreds) of empty lines at the > end, it is, lines > with separators only, or even blank lines. > >> My first reaction would be that you have to deal with these yourself afterwards. > > When dealing with "array based" records, it is, with no #recordClass > defined it would be easier. Because you can check whether all the > values of the array are empty or nil. So on each iteration (#next) you > could return the value array if it isn't empty or send #next again > until you get a record with any data. > > When working with a record class I think you can't do this, because it > seems you instantiate the record before iterating through each field > of the record. > >> What would be the definition of an empty record ? What about empty quoted fields ? > > An empty record of a four columns CSV file would look like: > ,,, > or (if some columns are quoted) > ,"","", > > I don't see how a record like that could be of any use. > > Best regards! > > Esteban A. Maringolo > > >> PS: I got your other contribution, I think it is OK to add, thanks. > > Great!

Oct. 6, 2014

Re: [Pharo-users] Glorp + NBSQLite3

by Esteban A. Maringolo

Pierce, Thank you for the answers. Is the wrapper thread safe? I understan the library is thread-safe per se (at least in Android+Java it is), so using the wrapper in Pharo (single-threaded at the OS level) wouldn't cause issues. But I don't know how NativeBoost plays here. Regards! Esteban A. Maringolo 2014-09-25 13:40 GMT-03:00 Pierce Ng <pierce(a)samadhiweb.com>: > On Wed, Sep 24, 2014 at 01:33:02PM -0300, Esteban A. Maringolo wrote: >> How does SQLite scale in terms of table size and so on? > > According to https://www.sqlite.org/whentouse.html: > > An SQLite database is limited in size to 140 terabytes (2^47 bytes, 128 > tibibytes). And even if it could handle larger databases, SQLite stores the > entire database in a single disk file and many filesystems limit the maximum > size of files to something less than this. > > SQLite will normally work fine as the database backend to a website. But if > you website is so busy that you are thinking of splitting the database > component off onto a separate machine, then you should definitely consider > using an enterprise-class client/server database engine instead of SQLite. > > >> I was surprised to know it is based on an old version of PostgreSQL >> according to this presentation: >> http://www.pgcon.org/2014/schedule/events/736.en.html > > That is a very interesting talk. As it says, SQLite is a replacement for > fopen(), not a replacement for PostgreSQL. In the context of my writing tools > and applications in Smalltalk, I'd like to enjoy SQLite's robustness, hence > my interest in having/building a good Smalltalk wrapper. > > Pierce > >

Oct. 6, 2014