- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] monkey error

by Nicolai Hess

2016-01-25 20:31 GMT+01:00 Esteban Lorenzano <estebanlm(a)gmail.com>: > no ideaâ¦ I think is a sthub problem, I need to take a look at it but as > always, Iâm running with other stuffâ¦ but I will try to add a retry when > downloading (thatâs the stupid timeout). > > ah, then there are a couple of random glamour and other tests that fails. > Just a couple. I vote to fix them or disable them :) > It is not only the timeout or randomly failing tests, sometimes the monkey reports a successful validation but the validation report shows "Finished: FAILURE" (there aren't many people fixing/reviewing bugs - much to few for this amount of bug reports, and in the current state, most time is just wasted because you can do the resolving, checking and reviewing again and again and again). I think , fixing this bugtracker validation errors should have the highest priority, so if anyone can help with this, please do. > > Esteban > > > On 25 Jan 2016, at 20:12, Nicolai Hess <nicolaihess(a)gmail.com> wrote: > > > > Fixing bugs (resolve/review/check monkey result) is > > extremly frustrating at the moment. > > > > Anything we can do ? > > >

Jan. 26, 2016

Re: [Pharo-dev] [ANN] FFICHeaderExtractor first milestone (for early code reviewers) [WAS] Can OSProcess functionality be implemented using FFI instead of plugin?

by stepharo

super! Stef Le 25/1/16 23:27, Mariano Martinez Peck a Ã©crit : > Hi guys, > > OK, I have a first working version and so I wanted to share it with you. > > I have not yet the time to start writing the doc since I just finished > the first pass on the code. Tomorrow I will start with the doc. But I > thought some of you may be interested in taking a look even without > formal "doc" (and some feedback/iteration may avoid re-writing docs..). > > If you have no clue what I am talking about, then this summary is for you: > / > / > /----------/ > /When we use FFI to call a certain library it's quite common that we > need to pass as argument certain constants (for example, SIGKILL to > kill()). These constants are defined in C header files and can even > change it's value in different paltforms. / > /These constants also are sometimes defined by the C preprocessor and > so there is not way to get those values from FFI. If you don't have > the value of those constants, you cannot make the FFI call. / > /----------/ > > I have tested the tool in OSX and CentOS using latest Pharo 5.0. It > won't work in Windows right now. As usual, all classes and methods > have comments and there are enough tests. > > At the end, I decided the C program will output a very naive Smalltalk > literal array kind of thingy. The tool then parses that output and > directly creates a init method (which is compiled into the SharedPool > class) for that platform which is then called automatically at startup > (only if initialization is needed). > > As for real examples, I started to write constants for libc: signal.h > (to use kill()) , wait.h (to use wait() famility), fcntl.h (to use ... > xxx()) , and errno.h. You can take a look to the > package 'FFICHeaderExtractor-LibC'. > > Note that for running the tests you need 'cc' findable by path in OSX > and 'gcc' in Unix. > > To load the code in a latest Pharo 5.0, execute: > > Metacello new > baseline: 'FFICHeaderExtractor'; > repository: > 'github://marianopeck/FFICHeaderExtractor:master/repository'; > load. > Any feedback is appreciated. > I will start writing the doc now. > BTW: Big thanks to Eliot Miranda which helped me answering noob > questions and providing useful code and guidelines. > Best, > > > On Sat, Jan 23, 2016 at 1:12 PM, Eliot Miranda > <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>> wrote: > > > Hi Denis, > > On Jan 23, 2016, at 7:30 AM, Denis Kudriashov > <dionisiydk(a)gmail.com <mailto:dionisiydk@gmail.com>> wrote: > >> >> 2016-01-22 22:35 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com >> <mailto:eliot.miranda@gmail.com>>: >> >> Let's measure this. Let's say we have 8 platforms (that's an >> underestimate, because different Linux distributions may have >> different values for certain constants), but 8, which is 4 >> basic platforms times 32- & 64-bits. We have Mac x86 32-bit, >> Mac x64 64-bit, Windows x86 32-bit, Windows x64 64-bit, Linux >> x86 32-bit, Linux ARM 32-bit, Linux x64 64-bit, and soon >> enough there will be more. Further, there may be different >> versions over time. >> >> So each of those initialization methods has >> - 1 slot for the global variable to be assigned >> - 1 slot for the literal value to assign to it >> - 3 bytes of bytecode per initialization for small methods, 4 >> for large methods. Let's say 4. >> >> So the overhead in 32-bits is 12 bytes per constant, and in >> 64-bits is 20 bytes. So the overhead per constant for all >> platforms is 96 bytes per constant in 32-bits and 160 bytes >> per constant for 64-bits. A full system with sockets, files, >> a database connexion etc could easily exceed 100 constants. >> I think it would be nearer 1000. So the overheads are in the >> 10- to 100-k byte range (100k ~= 0.5% of the image) on >> 32-bits. That's low but it's also pure overhead. Every GC >> has to visit them. Every senders and implementors has to >> visit them, but they offer nothing of value. Whereas the >> small parser for whatever notation is used to store the >> constants externally (if they are needed in a given >> deployment) has a small constant overhead; its simple code. >> >> Further, you still need the machinery to export the constants >> to be able to generate these initialization methods. If >> you've got the machinery and you don't need the methods why >> bother to generate the methods? >> >> As the Scots say, many a mickle makes a muckle. >> >> >> Thank's Eliot for such detailed explanation. It makes sense. >> But personally I prefer Smalltalk solution although Smalltalk >> itself is pure overhead comparing to C. > > I can see the draw of the pure Smalltalk. Simplicity and brows > ability. But imagine a tiny headless image deployed on > containers, say 2mb. Now 100kb of initialization code doesn't > look so good :-). Anyway I'm beating a dead horse. Mariano is > generating initialization methods. > > >> My question was raised by Mariano idea to save ston files in >> methods. I think it can reduce problems which you described. >> But then literal array syntax can be more suitable than ston. > > I just want to be clear, I'm neutral about the notation used to > export info from the C file. Liberal array syntax, chunk source > format, ston, xml. It doesn't matter as long as it's convenient > at expressing an attribute dictionary from names to attributes > such as value, size, offset. Don't get hung up on the specific > notation. If one were to go with the external file the only real > requirements are that it be reasonably compact and quick to > parse. That might kill xml but leave plenty of other candidates. > > > _,,,^..^,,,_ (phone) > > > > > -- > Mariano > http://marianopeck.wordpress.com

Jan. 26, 2016

Re: [Pharo-dev] [ANN] FFICHeaderExtractor first milestone (for early code reviewers) [WAS] Can OSProcess functionality be implemented using FFI instead of plugin?

by Max Leske

Yay! > On 25 Jan 2016, at 23:27, Mariano Martinez Peck <marianopeck(a)gmail.com> wrote: > > Hi guys, > > OK, I have a first working version and so I wanted to share it with you. > > I have not yet the time to start writing the doc since I just finished the first pass on the code. Tomorrow I will start with the doc. But I thought some of you may be interested in taking a look even without formal "doc" (and some feedback/iteration may avoid re-writing docs..). > > If you have no clue what I am talking about, then this summary is for you: > > ---------- > When we use FFI to call a certain library it's quite common that we need to pass as argument certain constants (for example, SIGKILL to kill()). These constants are defined in C header files and can even change it's value in different paltforms. > These constants also are sometimes defined by the C preprocessor and so there is not way to get those values from FFI. If you don't have the value of those constants, you cannot make the FFI call. > ---------- > > I have tested the tool in OSX and CentOS using latest Pharo 5.0. It won't work in Windows right now. As usual, all classes and methods have comments and there are enough tests. > > At the end, I decided the C program will output a very naive Smalltalk literal array kind of thingy. The tool then parses that output and directly creates a init method (which is compiled into the SharedPool class) for that platform which is then called automatically at startup (only if initialization is needed). > > As for real examples, I started to write constants for libc: signal.h (to use kill()) , wait.h (to use wait() famility), fcntl.h (to use ... xxx()) , and errno.h. You can take a look to the package 'FFICHeaderExtractor-LibC'. > > Note that for running the tests you need 'cc' findable by path in OSX and 'gcc' in Unix. > > To load the code in a latest Pharo 5.0, execute: > > Metacello new > baseline: 'FFICHeaderExtractor'; > repository: 'github://marianopeck/FFICHeaderExtractor:master/repository'; > load. > > Any feedback is appreciated. > > I will start writing the doc now. > > BTW: Big thanks to Eliot Miranda which helped me answering noob questions and providing useful code and guidelines. > > Best, > > > > > > > > On Sat, Jan 23, 2016 at 1:12 PM, Eliot Miranda <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>> wrote: > > Hi Denis, > > On Jan 23, 2016, at 7:30 AM, Denis Kudriashov <dionisiydk(a)gmail.com <mailto:dionisiydk@gmail.com>> wrote: > >> >> 2016-01-22 22:35 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>>: >> Let's measure this. Let's say we have 8 platforms (that's an underestimate, because different Linux distributions may have different values for certain constants), but 8, which is 4 basic platforms times 32- & 64-bits. We have Mac x86 32-bit, Mac x64 64-bit, Windows x86 32-bit, Windows x64 64-bit, Linux x86 32-bit, Linux ARM 32-bit, Linux x64 64-bit, and soon enough there will be more. Further, there may be different versions over time. >> >> So each of those initialization methods has >> - 1 slot for the global variable to be assigned >> - 1 slot for the literal value to assign to it >> - 3 bytes of bytecode per initialization for small methods, 4 for large methods. Let's say 4. >> >> So the overhead in 32-bits is 12 bytes per constant, and in 64-bits is 20 bytes. So the overhead per constant for all platforms is 96 bytes per constant in 32-bits and 160 bytes per constant for 64-bits. A full system with sockets, files, a database connexion etc could easily exceed 100 constants. I think it would be nearer 1000. So the overheads are in the 10- to 100-k byte range (100k ~= 0.5% of the image) on 32-bits. That's low but it's also pure overhead. Every GC has to visit them. Every senders and implementors has to visit them, but they offer nothing of value. Whereas the small parser for whatever notation is used to store the constants externally (if they are needed in a given deployment) has a small constant overhead; its simple code. >> >> Further, you still need the machinery to export the constants to be able to generate these initialization methods. If you've got the machinery and you don't need the methods why bother to generate the methods? >> >> As the Scots say, many a mickle makes a muckle. >> >> Thank's Eliot for such detailed explanation. It makes sense. >> But personally I prefer Smalltalk solution although Smalltalk itself is pure overhead comparing to C. > > I can see the draw of the pure Smalltalk. Simplicity and brows ability. But imagine a tiny headless image deployed on containers, say 2mb. Now 100kb of initialization code doesn't look so good :-). Anyway I'm beating a dead horse. Mariano is generating initialization methods. > > >> My question was raised by Mariano idea to save ston files in methods. I think it can reduce problems which you described. >> But then literal array syntax can be more suitable than ston. > > I just want to be clear, I'm neutral about the notation used to export info from the C file. Liberal array syntax, chunk source format, ston, xml. It doesn't matter as long as it's convenient at expressing an attribute dictionary from names to attributes such as value, size, offset. Don't get hung up on the specific notation. If one were to go with the external file the only real requirements are that it be reasonably compact and quick to parse. That might kill xml but leave plenty of other candidates. > > > _,,,^..^,,,_ (phone) > > > > > -- > Mariano > http://marianopeck.wordpress.com <http://marianopeck.wordpress.com/>

Jan. 26, 2016

[ANN] FFICHeaderExtractor first milestone (for early code reviewers) [WAS] Can OSProcess functionality be implemented using FFI instead of plugin?

by Mariano Martinez Peck

Hi guys, OK, I have a first working version and so I wanted to share it with you. I have not yet the time to start writing the doc since I just finished the first pass on the code. Tomorrow I will start with the doc. But I thought some of you may be interested in taking a look even without formal "doc" (and some feedback/iteration may avoid re-writing docs..). If you have no clue what I am talking about, then this summary is for you: *----------* *When we use FFI to call a certain library it's quite common that we need to pass as argument certain constants (for example, SIGKILL to kill()). These constants are defined in C header files and can even change it's value in different paltforms. * *These constants also are sometimes defined by the C preprocessor and so there is not way to get those values from FFI. If you don't have the value of those constants, you cannot make the FFI call. * *----------* I have tested the tool in OSX and CentOS using latest Pharo 5.0. It won't work in Windows right now. As usual, all classes and methods have comments and there are enough tests. At the end, I decided the C program will output a very naive Smalltalk literal array kind of thingy. The tool then parses that output and directly creates a init method (which is compiled into the SharedPool class) for that platform which is then called automatically at startup (only if initialization is needed). As for real examples, I started to write constants for libc: signal.h (to use kill()) , wait.h (to use wait() famility), fcntl.h (to use ... xxx()) , and errno.h. You can take a look to the package 'FFICHeaderExtractor-LibC'. Note that for running the tests you need 'cc' findable by path in OSX and 'gcc' in Unix. To load the code in a latest Pharo 5.0, execute: Metacello new baseline: 'FFICHeaderExtractor'; repository: 'github://marianopeck/FFICHeaderExtractor:master/repository'; load. Any feedback is appreciated. I will start writing the doc now. BTW: Big thanks to Eliot Miranda which helped me answering noob questions and providing useful code and guidelines. Best, On Sat, Jan 23, 2016 at 1:12 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Hi Denis, > > On Jan 23, 2016, at 7:30 AM, Denis Kudriashov <dionisiydk(a)gmail.com> > wrote: > > > 2016-01-22 22:35 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: > >> Let's measure this. Let's say we have 8 platforms (that's an >> underestimate, because different Linux distributions may have different >> values for certain constants), but 8, which is 4 basic platforms times 32- >> & 64-bits. We have Mac x86 32-bit, Mac x64 64-bit, Windows x86 >> 32-bit, Windows x64 64-bit, Linux x86 32-bit, Linux ARM 32-bit, Linux x64 >> 64-bit, and soon enough there will be more. Further, there may be >> different versions over time. >> >> So each of those initialization methods has >> - 1 slot for the global variable to be assigned >> - 1 slot for the literal value to assign to it >> - 3 bytes of bytecode per initialization for small methods, 4 for large >> methods. Let's say 4. >> >> So the overhead in 32-bits is 12 bytes per constant, and in 64-bits is 20 >> bytes. So the overhead per constant for all platforms is 96 bytes per >> constant in 32-bits and 160 bytes per constant for 64-bits. A full system >> with sockets, files, a database connexion etc could easily exceed 100 >> constants. I think it would be nearer 1000. So the overheads are in the >> 10- to 100-k byte range (100k ~= 0.5% of the image) on 32-bits. That's low >> but it's also pure overhead. Every GC has to visit them. Every senders >> and implementors has to visit them, but they offer nothing of value. >> Whereas the small parser for whatever notation is used to store the >> constants externally (if they are needed in a given deployment) has a small >> constant overhead; its simple code. >> >> Further, you still need the machinery to export the constants to be able >> to generate these initialization methods. If you've got the machinery and >> you don't need the methods why bother to generate the methods? >> >> As the Scots say, many a mickle makes a muckle. > > > Thank's Eliot for such detailed explanation. It makes sense. > But personally I prefer Smalltalk solution although Smalltalk itself is > pure overhead comparing to C. > > > I can see the draw of the pure Smalltalk. Simplicity and brows ability. > But imagine a tiny headless image deployed on containers, say 2mb. Now > 100kb of initialization code doesn't look so good :-). Anyway I'm beating > a dead horse. Mariano is generating initialization methods. > > > My question was raised by Mariano idea to save ston files in methods. I > think it can reduce problems which you described. > But then literal array syntax can be more suitable than ston. > > > I just want to be clear, I'm neutral about the notation used to export > info from the C file. Liberal array syntax, chunk source format, ston, > xml. It doesn't matter as long as it's convenient at expressing an > attribute dictionary from names to attributes such as value, size, offset. > Don't get hung up on the specific notation. If one were to go with the > external file the only real requirements are that it be reasonably compact > and quick to parse. That might kill xml but leave plenty of other > candidates. > > > _,,,^..^,,,_ (phone) > > -- Mariano http://marianopeck.wordpress.com

Jan. 25, 2016

Re: [Pharo-dev] another spotter question

by stepharo

> > Oh wow! I had no idea you click that area to toggle the preview! This is my point :) > > Even with the shift hints. > > At the very least, it needs something more in the shift hints than > just a change in background color!

Jan. 25, 2016

WARNING: Smalltalkhub could be down until tomorrow

by Christophe Demarey

The server hosting Smalltalkhub has the root partition full. It should be fixed tomorrow but you could face problems from now. Christophe

Jan. 25, 2016

Re: [Pharo-dev] monkey error

by Esteban Lorenzano

no ideaâ¦ I think is a sthub problem, I need to take a look at it but as always, Iâm running with other stuffâ¦ but I will try to add a retry when downloading (thatâs the stupid timeout). ah, then there are a couple of random glamour and other tests that fails. Just a couple. I vote to fix them or disable them :) Esteban > On 25 Jan 2016, at 20:12, Nicolai Hess <nicolaihess(a)gmail.com> wrote: > > Fixing bugs (resolve/review/check monkey result) is > extremly frustrating at the moment. > > Anything we can do ?

Jan. 25, 2016

monkey error

by Nicolai Hess

Fixing bugs (resolve/review/check monkey result) is extremly frustrating at the moment. Anything we can do ?

Jan. 25, 2016

Re: [Pharo-dev] Rubric: spotter with preview leaks some memory

by Nicolai Hess

2016-01-20 23:23 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: > > > 2016-01-20 19:18 GMT+01:00 Nicolai Hess <nicolaihess(a)gmail.com>: > >> >> >> 2016-01-20 14:47 GMT+01:00 Andrei Chis <chisvasileandrei(a)gmail.com>: >> >>> Tried it in the last latest image and if I execute before >>> #garbageCollect I just get 4 instances. >>> >>> Smalltalk garbageCollect. >>> RubTextLine allInstances size "4" >>> >>> Now indeed if I browse the same class in nautilus I get a much smaller >>> number of RubTextLine instances >>> >> >> >> I opened a bugreport >> 17437 >> <https://pharo.fogbugz.com/f/cases/17437/Rubric-spotter-with-preview-leaks-s…> >> >> Rubric: spotter with preview leaks some memory >> > > > I tried to reproduce this behavior with GTInspector instead of Spotter, > for example, inspect: > > MethodFinder methods > select on entry, -> another inspector pane shows up. Select th > "Source"-tab. Now > scroll through the list of methods in the first inspector pane. For every > method the second pane updates the > source tab with the code of the selected method. > > This generates many RubTextLine instances too. But not as many and I can > always clean them up with Smalltalk garbageCollect. > So, the is something that spotter does differently. > And even if Smalltalk garbageCollect would clean all instances, I don't think it is right, that scrolling through the method list (with preview) should create hundred thousends of RubTextLine instances. There is just not enough code visible, we definitly don't need that much. There is something wrong. > > > > >> >> >> >> >>> >>> On Wed, Jan 20, 2016 at 2:32 PM, Nicolai Hess <nicolaihess(a)gmail.com> >>> wrote: >>> >>>> Open fresh image 50536 >>>> Open Spotter, >>>> enable preview (cmd+p) >>>> search for EllipseMorph >>>> dive in instance-methods >>>> scroll through the list one item after another. >>>> >>>> open playground >>>> evaluate >>>> RubTextLine allInstances size. "553896" >>>> >>>> >>>> case 17421 is related to this, but does not fully solves this. >>>> please review and comment on that fix, >>>> so we can integrate this soon. >>>> >>>> The problem sometimes remains even with this fix. The memory >>>> (instance count of RubTextLine) just grows in the background >>>> >>> >>> >> >

Jan. 25, 2016

Re: [Pharo-dev] Color yzw

by Nicolai Hess

2016-01-24 21:38 GMT+01:00 stepharo <stepharo(a)free.fr>: > I did not forget just running against time and busy > > Le 16/1/16 22:17, Nicolai Hess a Ã©crit : > >> >> Another possible solution, don't convert strings to symbols in RBMethod/RBMessagenodes selectorParts ^ self keywords collect: #asSymbol. selector: aSelector keywordsPositions := nil. selector := aSelector asSymbol. this of course needs some changes for the compiler to actually create methoddictionary keys and literals for message send by conferting them explicit to symobls. There are suprisingly few places to modify. But the same has to be done for accesses in RBRefactoring classes too. And this is not as easy to change as for the compiler :( > see attached changesets. >> >> load with: >> >> DangerousClassNotifier disableDuring:[ >> String subclass: #Symbol >> instanceVariableNames: '' >> classVariableNames: 'NewSymbols OneCharacterSymbols SymbolTable >> SelectorTable' >> package: 'Collections-Strings' >> ]. >> >> DangerousClassNotifier disableDuring:[ >> 'selectortable.1.cs' >> asFileReference fileIn. >> ]. >> Symbol initSelectorTable. >> 'use_selector_table.cs' asFileReference fileIn. >> >> what do you think ? >> >> (selectortable.cs defines SelectorTable and some methods in Symbol and >> String) >> (use_selector_table adds selectors as internedSelectorSymbol if this >> selector is set as a selector on CompiledMethod >> and uses findInternedSelector .... in RubStyler visitMessageNode) >> > > >

Jan. 25, 2016