March 2011 - Pharo-dev - lists.pharo.org

[Pharo-project] Filesystem and FS extensions in Pharo book

by Max Leske

Hey I miss Filesystem and its extensions (cURL, zip, git etc.) in the Pharo book. I'd write the entries but I want your oppinion on whether we should wait till the Filesystem rewrite in 1.2 is ready. My suggestion: ---- Pharo Tools ---- ---- Filesystem ---- ---- ---- Filesystem for Pharo 1.1 ---- ---- ---- ---- extensions 1 ...n ---- ---- ---- Filesystem for Pharo 1.2 ---- ---- ---- ---- extensions 1 ...n Should we even list the "old" versions? The downside to that is that we would have to wait for all the extensions to be updated. What do you think? Max

March 22, 2011

Re: [Pharo-project] Issue 330 in pharo: recordLoop has bug that affects recording on macintosh

by pharo＠googlecode.com

Updates: Status: FixProposed Labels: Milestone-1.3 Comment #7 on issue 330 by stephane...(a)gmail.com: recordLoop has bug that affects recording on macintosh http://code.google.com/p/pharo/issues/detail?id=330 (No comment was entered for this change.) Attachments: recordLoop.1.cs 1.0 KB

March 22, 2011

Re: [Pharo-project] test crashing the cog vm

by Igor Stasenko

On 22 March 2011 08:55, Marcus Denker <marcus.denker(a)inria.fr> wrote: > > On Mar 22, 2011, at 12:17 AM, Toon Verwaest wrote: > >> On Mon, Mar 21, 2011 at 7:23 PM, StÃ©phane Ducasse >> <stephane.ducasse(a)inria.fr> wrote: >>> >>> This would be a nice little topic... this bytecode verifier. >> >> Probably on top of Opal > > The bytecode verifier must be in the VM. Â If it is up in the image it can be > side-stepped. Â The VM is the ultimate executor of code and so it must apply > verification. Â It could be written in Smalltalk and verified in Smalltalk > and then translated. Â But it must be part of the VM and used by the VM > before running any previously unverified method. > best, > Eliot > > Indeed, Mariano's response made me want to scream: NOOOOO! :) > > HOWEVER!, Thinking about it now, it's actually a very interesting proposal! > What if we write this piece of software in Smalltak and give it to the VM as > part of the VM definition. The VM decides what the application can see, so > it might as well not give access to this smalltalk-written bytecode > validator that it knows to be safe. If it's built by a separate classbuilder > in a separate environment, it can be clearly separated, invulnerable to > become: and other evil-doing. So, in short. It's written in Smalltalk > (YAY!), but becomes part of the VM runtime, not of the image running on the > VM. If we just make sure that we can write out image segments that the VM > can use as part of its core (exactly what I do in Pinocchio incidentally), > we can just develop and test it nicely in Pharo. Ain't that super-duper > cool? > > > If we would have solved the security problem already, then we could use it > for the bytecode verifyer, too... (guarantee that nobody tampers with the > verifyer). > We need to reify that was makes "the VM" special and provide it for all > smalltalk code. But that is of course research and completely Â not clear > yet what it means. By stretching, you can treat VM as a hardware which you can't change and must follow its rules. But as someone said, hardware is an early crystallized software, which means that we can change it, but at much higher cost(s) in terms of engineering effort and time. As we hopefully approach closer to self-sustaining systems (like SqueakNOS shows) the need in VM, as a separate software layer between hardware and a language is smearing. And i would really like to see it completely disappeared at some day. There are a family of languages which is already very close to self-sustaining form - a concatenative languages, like Forth or Factor. Most of them are implemented in themselves, and have very thin layer between language and hadrware implemented in C, the rest is already in language. See http://concatenative.org/wiki/view/Factor/Optimizing%20compiler > Marcus > > -- > Marcus Denker Â --Â http://www.marcusdenker.de > INRIA Lille -- Nord Europe. Team RMoD. > -- Best regards, Igor Stasenko AKA sig.

March 22, 2011

Re: [Pharo-project] [ANN] SqueakNOS update released!

by StÃ©phane Ducasse

impressive! Stef On Mar 21, 2011, at 5:30 PM, Javier PimÃ¡s wrote: > For all those who may be intrested, we are glad to announce that we have just uploaded an updated version of SqueakNOS to our site. You can find the updated download links here. > > This release adds the following items to the previous version: (99.9% modeled with objects) > > - Improved support for ATA devices access > - Implementation of a model of files and directories > - Implementation of FAT32 filesystem > - Support for reading sources from the hard disk > - Support for X86 paging setup and handling of page faults > - Implementation of a model of free and used memory > - Implementation of image snapshotting (based on the paging mechanism) > - Based on Pharo, but also supporting Squeak and Cuis > - Added some primitives for communicating between VM and image > - Support for Alien and FFI > - Added basic support for writing to serial via VM so that it's possiblo to out debug info to the outside world. > - Added pretty basic ATA benchmarks > - Added some scripts to automatize work done from the outside world > - Many code cleanups > > Future directions are: > > - Support for Cog VM > - Investigate replacing SqueakNOS plugin with NativeBoost so that we do more on the image > - Offloading memory pages to disk > - More tests > - Benchmarks > > For the brave ones, remember that you can always generate a SqueakNOS image by using the ConfigurationOfSqueakNOS (which is a bit outdated now), or by manually loading the packages from monticello. Also to generate a SqueakNOS VM download its code from GIT (hosted on sourceforge.net) > > Don't forget to visit squeaknos.blogspot.com for news and instructions. We hope to be able to write more soon. > > Many thank's t all the people who made it possible to understand the code and to develop all this. > > SqueakNOS' Team > > -- > Javier PimÃ¡s > Ciudad de Buenos Aires

March 22, 2011

Re: [Pharo-project] Getting a functionning unstable image

by StÃ©phane Ducasse

I tried to condense the changes and I got an wonderful syntax error that I cannot access. So probably something was broken in the changes. But this is not a surprise. The more I read the internal of this part of the system, the more I wonder how it could work. Stef On Mar 21, 2011, at 10:29 PM, Nicolas Cellier wrote: > I'm inquiring and it seems related to bad handling of author/timestamps. > Here: > > (CompiledMethod allInstances select: [:e | e fileIndex = 1]) > detectMax: [:e | e filePosition] > (ZipWriteStream class>>#regressionTestFrom: "a > CompiledMethod(353370112)") filePosition 16234210 > > (SourceFiles at: 1) size 16249263 > > So, the highest position still in the source file. > Let's see the lowest: > > (CompiledMethod allInstances select: [:e | e fileIndex = 1]) > detectMin: [:e | e filePosition] > (AColorSelectorMorph>>#color: "a CompiledMethod(894697472)") filePosition 468 > > Now, let us see what is in the source file: > > (SourceFiles at: 1) position: 0; next: 468 > '''From Pharo1.0beta of ''''18 October 2009'''' [Latest update: > #10487] on 19 October 2009 at 6:18:53 pm''! > BracketSliderMorph subclass: #AColorSelectorMorph > instanceVariableNames: '''' > classVariableNames: '''' > poolDictionaries: '''' > category: ''Polymorph-Widgets''! > !AColorSelectorMorph commentStamp: ''gvc 5/18/2007 13:52'' prior: 0! > ColorComponentSelector showing an alpha gradient over a hatched background.! > > > !AColorSelectorMorph methodsFor: ''accessing'' stamp: ''gvc 9/3/2009 ' > > (SourceFiles at: 1) next: 100. > '13:43''! > color: aColor > "Set the gradient colors." > > super color: aColor asNontranslucentColor. > sel' > > Yeah, that's what I get in the browser 13:43, it's not the source, > it's the time...` > > In 1.2: > (AColorSelectorMorph>>#color: ) filePosition 476 > > (ZipWriteStream class>>#regressionTestFrom: ) filePosition 16248101 > > So, someone broke my pointers, or someone have a different sources. > > Nicolas > > 2011/3/21 StÃ©phane Ducasse <stephane.ducasse(a)inria.fr>: >> Hi nicolas >> >> this is strange because I always update all the updates from the latest unstable published by marcus >> (so yes it can be long) and I never get this error. May be you have a broken sources files or something like that. >> I update often multiple times especially when I integrate changes to make sure that I do not a strange changes around. >> and I never get such error. >> What is your OS? >> >> Stef >> >> On Mar 21, 2011, at 9:53 PM, Nicolas Cellier wrote: >> >>> Hi guys, >>> Updating Pharo from the update stream failed many times and keep on >>> failing for me... >>> But maybe it's just me... >>> >>> So I picked the unstable link from the web site >>> (http://www.pharo-project.org/pharo-download/unstable-core) >>> This drives me to a 13091 image. >>> The first thing I do is select the 'update system' menu... >>> Err, it fails... It seems related to broken sources... >>> Well the image points to which source file ? can't remember the >>> incantation, let just browse it... >>> Err, no source, I can't browse. >>> No matter, I set the browser to decompile mode... >>> Err, this does not work, I still can't browse code. >>> Hmm very bad starting experience, I know what unstable means, but I >>> was dreaming of other words like continuous development. >>> >>> So ket's open an older image (my own updated 1.2) to find the incantations. >>> (Scanner>>#scanTokens:) fileIndex -> 1, so it seems that the source >>> file is corrupted. >>> (SourceFiles at: 1) -> MultiByteFileStream: >>> '/Users/nicolas/Smalltalk/Squeak/pharo/PharoV10.sources' >>> >>> Well, for my previous Pharo images, this very file works ! >>> How damned can be the file be changed and the version not incremented ? >>> >>> Is it me who did something wrong ? >>> Or the update process somewhere did condense the sources ? >>> Or did someone elese messed with the sources ? >>> Is it a known problem ? >>> >>> Any help appreciated. >>> >>> Nicolas >>> >> >> >> >

March 22, 2011

Re: [Pharo-project] Getting a functionning unstable image

by Marcus Denker

On Mar 21, 2011, at 11:19 PM, Nicolas Cellier wrote: > > > So it was my fault finally: it's because I was too early, and started > from Pharo0.x, then applied this automated condenseSources: > Oh. > > That explains why I can never upgrade from a newer image... > I brought my 0.x image up to 1.2 by some tricky manual MC loading > and/or selective loading. > But now, things shall be easier :) > Yes... the thing is that everythign that we do not test will not work. We do not test updating 0.x --> 1.2. It *will* be broken. (And there is the question if it makes sense to keep these things working. I vote for investing into a declarative bootstrap instead, far more important) Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

March 22, 2011

Re: [Pharo-project] test crashing the cog vm

by Marcus Denker

On Mar 22, 2011, at 12:17 AM, Toon Verwaest wrote: > >> On Mon, Mar 21, 2011 at 7:23 PM, StÃ©phane Ducasse <stephane.ducasse(a)inria.fr> wrote: >> This would be a nice little topic... this bytecode verifier. >> >> Probably on top of Opal >> >> The bytecode verifier must be in the VM. If it is up in the image it can be side-stepped. The VM is the ultimate executor of code and so it must apply verification. It could be written in Smalltalk and verified in Smalltalk and then translated. But it must be part of the VM and used by the VM before running any previously unverified method. >> >> best, >> Eliot > Indeed, Mariano's response made me want to scream: NOOOOO! :) > > HOWEVER!, Thinking about it now, it's actually a very interesting proposal! What if we write this piece of software in Smalltak and give it to the VM as part of the VM definition. The VM decides what the application can see, so it might as well not give access to this smalltalk-written bytecode validator that it knows to be safe. If it's built by a separate classbuilder in a separate environment, it can be clearly separated, invulnerable to become: and other evil-doing. So, in short. It's written in Smalltalk (YAY!), but becomes part of the VM runtime, not of the image running on the VM. If we just make sure that we can write out image segments that the VM can use as part of its core (exactly what I do in Pinocchio incidentally), we can just develop and test it nicely in Pharo. Ain't that super-duper cool? > If we would have solved the security problem already, then we could use it for the bytecode verifyer, too... (guarantee that nobody tampers with the verifyer). We need to reify that was makes "the VM" special and provide it for all smalltalk code. But that is of course research and completely not clear yet what it means. Marcus -- Marcus Denker -- http://www.marcusdenker.de INRIA Lille -- Nord Europe. Team RMoD.

March 22, 2011

Re: [Pharo-project] [ANN] SqueakNOS update released!

by Igor Stasenko

On 22 March 2011 00:09, Javier PimÃ¡s <elpochodelagente(a)gmail.com> wrote: > we use "squeaknos" platform, which is mainly a unix one minus libc with a > custom makefile. All platform support files are in the git repo, and the > slang is generated with VMMakerTool selecting squeaknos platform. > My question was, should i build it under unix , or can also build under Mac and Win32? And is there any instruction about steps how to build it? I wanna to play with it a little. > On Mon, Mar 21, 2011 at 6:21 PM, Igor Stasenko <siguctua(a)gmail.com> wrote: >> >> On 21 March 2011 17:30, Javier PimÃ¡s <elpochodelagente(a)gmail.com> wrote: >> > For all those who may be intrested, we are glad to announce that we have >> > just uploaded an updated version of SqueakNOS to our site.Â You can find >> > the >> > updated download links here. >> > This release adds the following items to the previous version:Â (99.9% >> > modeled with objects) >> > - Improved support for ATA devices access >> > - Implementation of a model of files and directories >> > - Implementation of FAT32 filesystem >> > - Support for reading sources from the hard disk >> > - Support for X86 paging setup and handling of page faults >> > - Implementation of a model of free and used memory >> > - Implementation of image snapshotting (based on the paging mechanism) >> > - Based on Pharo, but also supporting Squeak and Cuis >> > - Added some primitives for communicating between VM and image >> > - Support for Alien and FFI >> > - Added basic support for writing to serial via VM so that it's possiblo >> > to >> > out debug info to the outside world. >> > - Added pretty basic ATA benchmarks >> > - Added some scripts to automatize work done from the outside world >> > - Many code cleanups >> > Future directions are: >> > - Support for Cog VM >> > - Investigate replacing SqueakNOS plugin with NativeBoost so that we do >> > more >> > on the image >> > - Offloading memory pages to disk >> > - More tests >> > - Benchmarks >> > For the brave ones, remember that you can always generate a SqueakNOS >> > image >> > by using the ConfigurationOfSqueakNOS (which is a bit outdated now), or >> > by >> > manually loading the packages from monticello. Also to generate a >> > SqueakNOS >> > VM download its code from GIT (hosted onÂ sourceforge.net) >> > Don't forget to visitÂ squeaknos.blogspot.comÂ for news and instructions. >> > We >> > hope to be able to write more soon. >> > Many thank's t all the people who made it possible to understand the >> > code >> > and to develop all this. >> > SqueakNOS' Team >> > >> >> Which platform used to build SqueakNOS VM? >> -- Best regards, Igor Stasenko AKA sig.

March 22, 2011

Re: [Pharo-project] [COTDC] 34 - MCSystemCategoryParser -- help needed, seriously

by laurent laffont

Hi people, almost no participation in COTDC last days. I'm not a Smalltalk guru and even 30mn to look at code is sometimes not enough for me to understand some stuff alone. I like COTDC because of http://code.google.com/p/pharo/wiki/CommentOfTheDayContest ==> documentation. But quality / good comment is needed and there's ~600 classes without comment remaining. So if we want a 100% good commented Pharo in 2 years from now, we should do it every day. Laurent Laffont On Mon, Mar 21, 2011 at 8:36 PM, laurent laffont <laurent.laffont(a)gmail.com>wrote: > Could it be: > > I parse PackageInfo statement from Monticello packages file. > > Laurent. > > > > On Mon, Mar 21, 2011 at 8:02 AM, laurent laffont < > laurent.laffont(a)gmail.com> wrote: > >> Today: MCSystemCategoryParser >> >> >> Comment Of The Day Contest - One Day One Comment >> Rules: >> #1: Each day a not commented class is elected. Each day the best comment >> will be integrated with name of the author(s). >> #2: If you cannot comment it, deprecate it. >> Results: http://code.google.com/p/pharo/wiki/CommentOfTheDayContest >> >> Laurent >> > >

March 22, 2011

Re: [Pharo-project] String>>#base64Encoded

by Sven Van Caekenberghe

On 21 Mar 2011, at 21:52, StÃ©phane Ducasse wrote: > On Mar 21, 2011, at 9:42 PM, Sven Van Caekenberghe wrote: > >> Hi, >> >> It took me quite some time today to figure this out and it might be relevant for others too: >> >> it turns out that String>>#base64Encoded introduces newlines which you definitively do not want when doing Basic HTTP Encoding for example. The tricky part is that it works when you stay below a certain length. > > when would it makes sense? Base64 encoding ( http://en.wikipedia.org/wiki/Base64 ) is a techique to encode binary data in ASCII strings using a radix 64 representation. It came from the early days of the internet where is was used to encode binary stuff in mail messages (part of MIME encoding). For these usages, line breaks make sense, since some email client break lines themselves, and by preformatting you prevent that, for example: (Base64MimeConverter mimeEncode: (ZnClient get: 'http://www.pharo-project.org/favicon.ico') entity readStream multiLine: true) contents 'AAABAAEAEBAAAAEAIABoBAAAFgAAACgAAAAQAAAAIAAAAAEAIAAAAAAAAAAAABMLAAATCwAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0K1+/+2tF//lpRj/0J1F/86x kP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQso7/ 1Zw1/+ekF//koBf/zZdF/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAADLrIr/yI05/9SLG//UvKP/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAA3s2//8qrjf8AAAAAAAAAAAAAAADClmr/28m4/wAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACpayz/rH1S/9K9q/8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAxKeP/4lWLv9+Tiz/oXtc/97P xf8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADYxrj/ oX1j/2RBNP+QbFT/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAC6m4H/1MGz/+7m4v/Ltqb/tJR3/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAsJWF/1tHVf+qkIH/4tTL/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAANG8rf9GPGj/IiJn/19MYP/Mt6n/AAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANXEuP9NQGj/IiiB/youf/8NFnb/Qztx/9PC t/8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADczcL/mIGA/y8ziP8nMZj/ GSGF/4l2hP/Tw7n/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMy4 q/8pMZj/JzKi/yUxov+znJb/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAADezsT/d3Om/1dt0f9cZrz/v6mf/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAJqapP+i3f//iazZ/9/Pwv8AAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADBqIv/lYps/7Caev8AAAAAAAAAAAAA AAAAAAAA/g8AAP8H8L//wwAA/nMAAP8fNcL/BwAA/4cAAP8HAAD/DwAA/wc4FP4DAAD+AwAA /wf///8H////hzp0/48AAA==' Base64 encoding is also used a lot in cryptography to encode binary data like hashes or even for silly obfuscation. For example, in HTTP, Basic Authentication is a defined as a HTTP header called 'Authorization' with as value 'Basic' <space> <credentials> where credentials is obtained by Base64 encoding <username>:<password>. ZnRequest>>#setBasicAuthenticationUsername: username password: password self headers at: 'Authorization' put: 'Basic ', (ZnUtils encodeBase64: (username, ':', password)) ZnUtils class>>#encodeBase64: string ^ (Base64MimeConverter mimeEncode: string readStream multiLine: false) contents Here you don't want line breaks since it would break the common assumption that a header is a single line (technically, multiline headers do exist, but that is another story). Now, as long as the input string is below 55 characters in length, no line breaks will be introduced. In code (assuming no line line breaks will be introduced by our email clients or the mailing list software inbetween; re-execute in a workspace if necessary): (Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: true) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB' (Base64MimeConverter mimeEncode: ((String new: 54) atAllPut: $A) readStream multiLine: false) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB' (Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: true) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFB QQ==' (Base64MimeConverter mimeEncode: ((String new: 55) atAllPut: $A) readStream multiLine: false) contents 'QUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQQ==' Please note, this is not a bug, just a gotcha that can go unnoticed for quite some time. Sven PS: Base64 decoding is clever enough to transparently deal with line breaks. >> This can easily be solved by invoking Base64MimeConvertor with the #mimeEncode: <string> multiLine: false >> >> Zinc HTTP Components now does this consistently through ZnUtils class>>#encodeBase64: >> >> Sven

March 22, 2011