October 2010 - Pharo-dev - lists.pharo.org

Re: [Pharo-project] Getting my head around low space

by Max Leske

"There is not enough memory to give squeak the amount specified by the 'memory EMBED tag.". Additionally there's a "Premature end of file" error on the image and then the VM crashes (OS X crash dialog). Issuing the same commands with 1 GB memory works fine (no problems with the image). Interestingly I tried to run the task with 3.5 GB in a 5.7b1 VM over night and the low space warning popped up when the progress bar was have as far then when I ran into the same condition using the 4.2.5b1U VM and 1 GB... Any suggestions? I couldn't attach the VM statistics because of the lacking menu integration but if you tell me how to extract the statistics (supposing they are interesting...) I'll happily rerun the task and post the results. Max On 10.10.2010, at 23:47, John M McIntosh wrote: > > On 2010-10-10, at 2:00 PM, Max Leske wrote: > >> I had a first failure with 1 GB. I used the 4.2.5b1U VM. Setting the memory to anything above 1 GB would lead to an error message. I'm now trying the 5.7b1 VM with 3.5 GB. >> >> Max > > So what is the error message? > > > -- > =========================================================================== > John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 > Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com > =========================================================================== > > > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 11, 2010

Re: [Pharo-project] Issue 3090: Context not included in ImageSegment when saving morph to file

by StÃ©phane Ducasse

Thanks! and welcome back. Stef On Oct 11, 2010, at 7:16 AM, Sheridan Mahoney wrote: > I've just re-entered the Pharo world after a long(ish) absence, and wanted to report a > new bug submission; here's the breakdown: > > Pharo image: dev > Pharo core version: Pharo-1.1.1-- Latest update: #11414 > Virtual machine used: 4.0.3.2202 Unix VM > OS: Ubuntu Lucid Lynx 10.04 > Class browser used if applicable: SystemBrowser default > > Steps to reproduce: > 1. start pharo > 2. open a Shout workspace > 3. select the just-opened workspace > 4. Ctrl-right button to get the meta-click window > 5. select 'save morph in file' > 6. hit 'ok' to proffered name for file > 7. a message box comes up that reads 'please wait while objects are > counted'. > 8. shortly after that an error is generated: > 'Error: this context was not included in the ImageSegment' > > > I'm planning on doing more research, but wanted to get the issue reported. > > Cheers, > Sheridan > > PS - so far I really like 1.1.1 !!! Good work, gang! > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project

Oct. 11, 2010

Re: [Pharo-project] Xtreams port to Squeak - second wave

by mkobetic＠gmail.com

"Nicolas Cellier"<nicolas.cellier.aka.nice(a)gmail.com> wrote: > Then, the harder work begins: > - File/Socket/Pipe > - Pointers (in External Heap) > - Character encoding/decoding I just published XtreamsDevelopment(387) with following changes: * abstracted Encoder out of encoded streams to allow implementing our own encodings. * Encoder.Encoders registers known encoders * Encoder.DialectEncoder provides a hook to access dialect specific encoders * added ASCIIEncoder implementing ASCII encoding portably * added VWEncoder wrapping VisualWorks StreamEncoders Obviously VWEncoder won't work in Squeak and should eventually move to a VW specific package. But at the moment it might be useful as an example of how to take advantage of the existing encoders in Squeak. I briefly looked at the Multilingual categories in Pharo and there seem to be quite a few encodings available through the TextConverter hierarchy. You've expressed some reservations about that, but it might be worth taking advantage of those at least initially. Anyway with the above there should be at least an #ascii encoder available for now.

Oct. 11, 2010

[Pharo-project] Issue 3090: Context not included in ImageSegment when saving morph to file

by Sheridan Mahoney

Oct. 11, 2010

[Pharo-project] Experimental Cocoa OS-X based Squeak Cog JIT VM 5.8b12

by John M McIntosh

I've stuck a version (5.8b12) of the cocoa based os-x squeak cog JIT based VM in my experimental folder. http://homepage.mac.com/johnmci/.Public/experimental/Squeak%205.8b12.app.zip This is based on the latest Cog VM source (oct 10/10/10). Changes to the support logic: (a) Supply file create/mod times & size based on target of symbolic link versus the link file node. (b) Untrusted directory was set wrong, use same default value as 4.2.x VM ~/Library/Preferences/Squeak/Internet/My Squeak/ (c) There is no -pathenc option (d) Attempt to integrate changes into Cog branch to allow you to compile VM from the Cog branch. *** This does not work yet, 90% there, Eliot and I still have some pencil sharping to do... *** Now the big news is that I pushed a version of the Squeak Cog stack VM to the Apple App Store! It's rough, but it's a candidate for review by Apple's special App review committee. It does not have socket support and we are seeing if they allow us to move image/change files in and out of the Documents folder via a desktop machine. I should know more by next weekend. If accepted then I am looking to the Smalltalk community to help in building a viable image to allow people to edit source code on an iPad. Lastly, yes something broken in open/gl on os-x 10.4 I hope to look at the issue this week. -- =========================================================================== John M. McIntosh <johnmci(a)smalltalkconsulting.com> Twitter: squeaker68882 Corporate Smalltalk Consulting Ltd. http://www.smalltalkconsulting.com ===========================================================================

Oct. 11, 2010

Re: [Pharo-project] [vwnc] Xtreams port to Squeak - second wave

by Michael Lucas-Smith

On Oct 10, 2010, at 5:18 PM, mkobetic(a)gmail.com wrote: > Hi Nicolas, > > "Nicolas Cellier"<nicolas.cellier.aka.nice(a)gmail.com> wrote: >> Hi again, >> I now have ported two more packages >> - Xtreams-Transforms >> - Xtreams-Substreams >> and their tests If you get Interpreting and Marshaling going we'll have ST-ST communication between Pharo<->Squeak<->VisualWorks with a fast binary protocol. I suspect some trickery will be required to deal with namespaces, as the protocol assumes 'full names' for classes when matching between the ends. Michael

Oct. 11, 2010

Re: [Pharo-project] Xtreams port to Squeak - second wave

by Levente Uzonyi

On Sun, 10 Oct 2010, mkobetic(a)gmail.com wrote: > Hi Nicolas, > > "Nicolas Cellier"<nicolas.cellier.aka.nice(a)gmail.com> wrote: >> Hi again, >> I now have ported two more packages >> - Xtreams-Transforms >> - Xtreams-Substreams >> and their tests > > This is great! I loaded your code into Pharo 1.1 and things seem to be working quite well. There was a complaint about missing SharedQueue2, I just created a dummy subclass of SharedQueue with that name and things seemed to load fine. > XTRecyclingCenter seems to be subclass of XTWriteStream, it should be Object subclass, maybe a typo ? > >> I did not have any portability problem with those... >> But that's because I did not handle the Character encoder/decoder. >> Consequently, I have 8 tests failing (the Base64 related tests) > > I was thinking, we could implement just 'encoding: #ascii' quite easily, to make this reasonably usable at least for applications that are fine with that. We're actually contemplating about implementing our own encoders for Xtreams too. The VW ones are tied to the classic streams more than we like. You might have noticed some rather hairy parts in the encoding streams yourself, where we're trying to work around some of the issues it creates. The advantage of reusing the existing encoders was that there are quite a few of those available, so reimplementing all that would be a drag. But we can come up with a scheme where we can reimplement at least the common ones and in VW we can still preserve hooking into the old ones for the rest. I can give that a try on VW side in the meantime so you could get those for free. > >> Plus 4 other tests failing because of my poor implementation of >> #after:do: (forking processes in a SUnit TestCase can't be that >> obvious). > > I looked at this, and I think this is how #after:do: should look like: > > after: aDelay do: aBlock > "Evaluate the argument block delayed after the specified duration." > > | watchdog | > watchdog := [ > aDelay wait. > aBlock value. > ] newProcess. > watchdog priority: Processor userInterruptPriority. > watchdog resume. > > This would assume that the 2 tests calling #timeout:server:client: would use a Delay instance instead of a Duration, which I'd be fine with. However making that change doesn't quite get the tests running. It's blowing up with a DNU on the 'output close' bit in #terminate:server:client: with output being nil, which I'm having trouble figuring out. I can't find who could possibly be nilling it out. I'm somewhat struggling finding my way around the Pharo tools. The test seems to otherwise pass but the DNUs from the background process isn't nice. Also when I just click on the test in the TestRunner, I actually get four DNUs, not just one as I would expect. So I'm kinda stuck, not sure how to move forward without help from someone who knows his way around Pharo. > I also get odd failure from #testWriteCollectingMultipleBufferSize, which seems to run fine (against collection) when I run the equivalent in a workspace, but strangely fails when running via the #timeout:server:client: construct, i.e. when client and server run in separate processes. Hm, now that I think of it, they sure could fail if something preempts the client, server processes at the right moment. I'll have to rethink that again. These problems should be solved with the latest version of CoreTests. > >> Now, the easy part of the port (copy/paste) is almost ended. >> Once we manage a compatible way to handle pragmas, PEG Parser should >> port quite easily too. > > I wouldn't worry about the Parser stuff at this point. > >> Then, the harder work begins: >> - File/Socket/Pipe >> - Pointers (in External Heap) > > I wouldn't worry, about the external heap stuff either. It's neat, but probably not something many people will miss. > >> - Character encoding/decoding > > I'll see if I can help with this from the VW side. > >> - Compression/Decompression > > Is Zlib linked into the VM in Squeak too ? The compression streams are written directly against the ZLib API, so there aren't any VW specific dependencies other than how those calls are made. Similarly the crypto streams go directly against the EVP API in LibCrypto in OpenSSL, so as long as we can abstract over how those are called, the stream implementation should work as is. > >> If you think you can help in any of these, please tell. > > If you could compile a list of changes that you'd like us to adopt on the VW side, I'd certainly look at that. I did read your posts but it's not entirely clear what you'd like to handle on Squeak side and what on VW side. A fileout would be best to avoid any confusion, but a description is fine too. > > I'm also unclear about the become: discussion. Don't write streams in Squeak become: the underlying collection when they grow it ? No. Squeak uses direct pointers, so it doesn't have an object table, therefore #become: is very expensive. See the Storage Management section of http://ftp.squeak.org/docs/OOPSLA.Squeak.html for details. Levente > > Cheers, > > Martin > > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 11, 2010

Re: [Pharo-project] Xtreams port to Squeak - second wave

by Levente Uzonyi

On Sun, 10 Oct 2010, Nicolas Cellier wrote: > Hi again, > I now have ported two more packages > - Xtreams-Transforms > - Xtreams-Substreams > and their tests > > I did not have any portability problem with those... > But that's because I did not handle the Character encoder/decoder. > Consequently, I have 8 tests failing (the Base64 related tests) > Plus 4 other tests failing because of my poor implementation of > #after:do: (forking processes in a SUnit TestCase can't be that > obvious). I added Number >> #milliseconds to Xtreams-VWCompatible, because Squeak has #milliSeconds for some reason and the tests use #milliseconds I also uploaded a new version of Xtreams-CoreTests which doesn't use #after:do:. This change may use Squeak specific stuff, revert it if you don't like it. All tests (except for the base64 related) are passing reliably, no more random failures. Levente > > Now, the easy part of the port (copy/paste) is almost ended. > Once we manage a compatible way to handle pragmas, PEG Parser should > port quite easily too. > > Then, the harder work begins: > - File/Socket/Pipe > - Pointers (in External Heap) > - Character encoding/decoding > - Compression/Decompression > > If you think you can help in any of these, please tell. > > Nicolas > > _______________________________________________ > Pharo-project mailing list > Pharo-project(a)lists.gforge.inria.fr > http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project >

Oct. 11, 2010

Re: [Pharo-project] [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by David T. Lewis

On Sun, Oct 10, 2010 at 08:16:57PM -0400, Schwab,Wilhelm K wrote: > <snip> > The creation of the redundant executors is a recipe for for a double free and resulting crash. <snip> > Am I missing something? There is no such thing as a "double free and resulting crash". Dave

Oct. 11, 2010

[Pharo-project] Fwd: [squeak-dev] Re: WeakRegistry>>remove: - when you'll be in trouble

by Igor Stasenko

2010/10/11 Levente Uzonyi <leves(a)elte.hu>: > On Mon, 11 Oct 2010, Igor Stasenko wrote: > >> 2010/10/11 Levente Uzonyi <leves(a)elte.hu>: >> On Mon, 11 Oct 2010, Igor Stasenko wrote: >> >>> On 10 October 2010 23:43, Levente Uzonyi <leves(a)elte.hu> wrote: >>> On Sun, 10 Oct 2010, Igor Stasenko wrote: >>> >>>> A current implementation of this method >>>> >>>> remove: oldObject ifAbsent: exceptionBlock >>>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>>> >>>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>>> Â Â Â Â ^(self protected: [ valueDictionary removeKey: oldObject >>>> ifAbsent: >>>> nil ]) >>>> Â Â Â Â Â Â Â Â ifNil: [ exceptionBlock value ] >>>> >>>> simply removes a previously registered object from registry and voila. >>>> >>>> Now lets get back to our discussion about multiple finalizers per >>>> object and using them in weak subscriptions etc. >>>> >>>> Suppose i am added a socket to weak registry, >>>> and suppose i am added a weak subscription to it. >>>> >>>> Now, if i do 'socket close' , it tells weak registry to remove it from >>>> list. >>>> And what we'll have: >>>> - socket handle is closed >>>> - socket is wiped from weak registry >>>> - but weak subscription still listed somewhere in a list of >>>> subscriptions >>>> >>>> >>>> My suggestion is, that upon #remove:, >>>> a weak registry should notify all executors that object of interest >>>> are no longer takes part in finalization scheme, >>>> so they should not count on receiving #finalize eventually. >>>> >>>> In other words: >>>> >>>> remove: oldObject ifAbsent: exceptionBlock >>>> Â Â Â Â "Remove oldObject as one of the receiver's elements." >>>> >>>> Â Â Â Â oldObject ifNil: [ ^nil ]. >>>> Â Â Â Â ^(self protected: [ | executor | >>>> Â Â Â Â Â Â executor := valueDictionary removeKey: oldObject ifAbsent: >>>> nil. >>>> Â Â Â Â Â Â executor discardFinalization. >>>> Â Â Â Â ]) >>>> Â Â Â Â ifNil: [ exceptionBlock value ] >>> >>> It's only an issue with the new WeakRegistry implementation, previous >>> implementations don't have such problem. I think changing the method as >>> you >>> suggested, implementing WeakFinalizerItem >> #discardFinalization as >>> "executor := nil" and changing WeakFinalizerItem >> #finalizaValues to >>> ignore the executor if it's nil will fix the problem. Am I right? >>> >>> I don't get how "multiple finalizers per object" is related to this >>> problem >>> at all. >>> >> >> No, you miss the point. >> When you removing object from weak registry, it is important , time to >> time to tell all of its executors, >> that they will no longer receive #finalize, because object is no >> longer a member of weak registry. >> >> If you simply set executor := nil, it does nothing, an executor itself >> did not notified that he won't be needed >> in any possible future. >> So, if your finalization action is to remove some object(s) from the >> list , you'll get your list dirty after object will die, >> because #finalize will be never sent to your executor. >> >> Here the simple test case: >> >> | coll obj someWrapper | >> coll := OrderedCollection new. >> obj := Object new. >> someWrapper := WeakArray with: obj. >> coll add: someWrapper. >> >> obj toFinalizeSend: #remove: to: coll with: someWrapper. >> obj finalizationRegistry remove: obj. >> obj := nil. >> Smalltalk garbageCollect. >> self assert: coll isEmpty >> >> >> the point is, that once you doing a #remove: , >> your finalization scheme is broken. >> >> >> I don't get you. When you remove an object from a WeakRegistry, you tell >> the >> system that you want the object removed and all related executors deleted. >> So the executors should never receive #finalize, they should be thrown >> away. >> Even if there's action to be done with the executors (which is pointless >> IMHO since those will be thrown away, but who knows), doing them is the >> responsibility of the sender of #remove:. >> > > Let us get back to multiple finalizers per object, > when you have two separate places, which adding possibly different > executors for a single object > without knowing about each other (otherwise why would you want to add > two finalizers instead of one). > Now if one decides to do #remove: , > how to ensure that second one won't be left with dirty/invalid state? > > > It's not ensured, and it doesn't have to be. The sender of #remove: takes > all responsibility. It can decide to add some executors back to the registry > if that's necessary. > If you want to keep your executors safe from some other code, which may > #remove: your objects from a WeakRegistry, then you can create a private > WeakRegistry and store your executors there, just like Sockets and > FileStreams do. > Conclusion: weak registry don't have to support multiple finalizers per single object. Instead, if one sees the need to control a finalization of certain object(s) in own way, indepentently fron any other, then he should use a private weak registry (like Sockets and FileStreams do), and register object and its executor there. -- Best regards, Igor Stasenko AKA sig.

Oct. 11, 2010