- Pharo-dev - lists.pharo.org

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/60442 Home: https://github.com/pharo-project/pharo-core

March 15, 2017

Re: [Pharo-dev] Missing Zip64 support

by Esteban Lorenzano

Hi, yes, Iceberg will solve this problem by not having it, but it would be nice to be able to unzip bigger files from Pharo too (ok, maybe not a priority, but it would be cool anyway :P) Esteban > On 15 Mar 2017, at 13:07, Pavel Krivanek <pavel.krivanek(a)gmail.com> wrote: > > To clone the repository from Pharo we can use Iceberg: > > IceRepositoryCreator new > remote: (IceRemote url: 'https://github.com/guillep/pharo-core.git' <https://github.com/guillep/pharo-core.git'>); > location: ('pharo-core' asFileReference ensureCreateDirectory); > createRepository. > > 2017-03-15 11:31 GMT+01:00 Thierry Goubier <thierry.goubier(a)gmail.com <mailto:thierry.goubier@gmail.com>>: > Hi Pavel, > > Which means command like these: > > cd pharo; mkdir filetree; cd filetree; wget -O - https://api.github.com/repos/dalehenrich/filetree/tarball/pharo6.0_dev <https://api.github.com/repos/dalehenrich/filetree/tarball/pharo6.0_dev> | tar xz --strip-components=1 > > ./pharo-cli pharo/Pharo.image --no-default-preferences eval --save Metacello new baseline: \'FileTree\'\; repository: \'filetree:///`pwd`/pharo/filetree/repository\'\; load: \'Git\'. > > recovering the tarball can be a lot faster than doing a local clone, and FileTree is not interested in the git history anyway... > > Thierry > > > 2017-03-15 11:18 GMT+01:00 Pavel Krivanek <pavel.krivanek(a)gmail.com <mailto:pavel.krivanek@gmail.com>>: > Hi, > > our current Compression package does not support Zip64 which is limiting us in usage of Metacello together with Git. When we use the Metacello for installing of a baseline like: > > Metacello new > baseline: 'IDE'; > repository: 'github://guillep/pharo-core'; > load. > > then the Metacello downloads the corresponding archive from Github. However the standard Zip archives have count limit of 65535 members but the pharo-core repository and some others contain much more than this (pharo-core has 125021 files). The Zip64 extension is used for them during compression which we cannot read now. > > So if you will see Error: bad signature 16r... at position ..., then it has this reason. The workaround for now is to load projects from a local clone of such big repositories. > > Cheers, > -- Pavel > >

March 15, 2017

ZnInvalidUTF8 on response from squeaksource

by Patrick R.

Hi everyone, I have been working on bringing http://squeaksource.com/ical/ up to speed for Squeak and wanted to make sure that it also works for Pharo. Therefore, I have created a travis build job for Squeak and Pharo (https://travis-ci.org/codeZeilen/ical-smalltalk/jobs/211298950) which pulls the source from squeaksource.com. Now the issue is that loading the package in Pharo fails with a GoferException wrapping a ZnInvalidUTF8 Exception. We figured that this might be the result of the squeaksource page delivering the page as iso-8859-1 as it contains special characters. Any ideas on how to get this to work? I do not have access to the ical repository description and I would liek to avoid mirroring the whole repository on GitHub. Thanks and bests Patrick -- View this message in context: http://forum.world.st/ZnInvalidUTF8-on-response-from-squeaksource-tp4938787… Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

March 15, 2017

Re: [Pharo-dev] Capitalized

by Henrik Nergaard

>to "capitalize" is writing a word with its first letter. The Smalltalk String is not neccessarily just a word, it could be something else so the current implementation makes no assumptions of what the string is. --------------------------------------------------------------------------------------------- | newClass | newClass := Object subclass: #AbcSomeClass instanceVariableNames: 'errorCodeName referencePosition instanceVariable'. newClass instanceVariableNamesDo: [ :aString | | code | code := String streamContents: [ :aStream | | prefix cap | prefix := aString first isVowel ifTrue: ['an'] ifFalse: ['a']. cap := aString capitalized. aStream << aString; << ':'; space; << prefix; << cap; cr; cr; tab; << aString; << ' := '; << prefix; << cap. ]. newClass compile: code classified: 'accessing' ]. newClass browse -------------------------------------------------------------------------------------------- >Should we fix this? Do not know about any side effects... I think it would be better and more clear to then implement a new method called something like #onlyCapitalized that removes all other uppercase letters. Best regards, Henrik ________________________________ Fra: Pharo-dev <pharo-dev-bounces(a)lists.pharo.org> pÃ¥ vegne av Torsten Bergmann <astares(a)gmx.de> Sendt: 15. mars 2017 11:33:35 Til: Pharo Development List Emne: [Pharo-dev] Capitalized According to https://en.wikipedia.org/wiki/Capitalization to "capitalize" is writing a word with its first letter as a capital letter (upper-case letter) AND THE REMAINING LETTERS IN LOWER CASE writing systems. So 'SOMETHING' capitalized currently returns "SOMETHING" in Pharo but should return "Something" according to the definition to end up with lowercase. If we fix this I guess we would also align Pharo with other languages, like C# for example [1]. Could be easily changed by replacing "copy" with "asLowercase" in #capitalized: capitalized "Return a copy with the first letter capitalized" | cap | self isEmpty ifTrue: [ ^self copy ]. cap := self asLowercase. cap at: 1 put: (cap at: 1) asUppercase. ^ cap Should we fix this? Do not know about any side effects... Note: a) Squeak has the same issue b) it fits for #uncapitalized, see [2] c) Dont know about ANSI standard or any other ST dialect Thx T. [1] https://github.com/srkirkland/Inflector/blob/master/Inflector.Tests/Capital… [2] https://github.com/srkirkland/Inflector/blob/master/Inflector.Tests/Uncapit…

March 15, 2017

Re: [Pharo-dev] Missing Zip64 support

by Pavel Krivanek

To clone the repository from Pharo we can use Iceberg: IceRepositoryCreator new remote: (IceRemote url: 'https://github.com/guillep/pharo-core.git'); location: ('pharo-core' asFileReference ensureCreateDirectory); createRepository. 2017-03-15 11:31 GMT+01:00 Thierry Goubier <thierry.goubier(a)gmail.com>: > Hi Pavel, > > Which means command like these: > > cd pharo; mkdir filetree; cd filetree; wget -O - > https://api.github.com/repos/dalehenrich/filetree/tarball/pharo6.0_dev | > tar xz --strip-components=1 > > ./pharo-cli pharo/Pharo.image --no-default-preferences eval --save > Metacello new baseline: \'FileTree\'\; repository: > \'filetree:///`pwd`/pharo/filetree/repository\'\; load: \'Git\'. > > recovering the tarball can be a lot faster than doing a local clone, and > FileTree is not interested in the git history anyway... > > Thierry > > > 2017-03-15 11:18 GMT+01:00 Pavel Krivanek <pavel.krivanek(a)gmail.com>: > >> Hi, >> >> our current Compression package does not support Zip64 which is limiting >> us in usage of Metacello together with Git. When we use the Metacello for >> installing of a baseline like: >> >> Metacello new >> baseline: 'IDE'; >> repository: 'github://guillep/pharo-core'; >> load. >> >> then the Metacello downloads the corresponding archive from Github. >> However the standard Zip archives have count limit of 65535 members but the >> pharo-core repository and some others contain much more than this >> (pharo-core has 125021 files). The Zip64 extension is used for them during >> compression which we cannot read now. >> >> So if you will see Error: bad signature 16r... at position ..., then it >> has this reason. The workaround for now is to load projects from a local >> clone of such big repositories. >> >> Cheers, >> -- Pavel >> > >

March 15, 2017

Capitalized

by Torsten Bergmann

According to https://en.wikipedia.org/wiki/Capitalization to "capitalize" is writing a word with its first letter as a capital letter (upper-case letter) AND THE REMAINING LETTERS IN LOWER CASE writing systems. So 'SOMETHING' capitalized currently returns "SOMETHING" in Pharo but should return "Something" according to the definition to end up with lowercase. If we fix this I guess we would also align Pharo with other languages, like C# for example [1]. Could be easily changed by replacing "copy" with "asLowercase" in #capitalized: capitalized "Return a copy with the first letter capitalized" | cap | self isEmpty ifTrue: [ ^self copy ]. cap := self asLowercase. cap at: 1 put: (cap at: 1) asUppercase. ^ cap Should we fix this? Do not know about any side effects... Note: a) Squeak has the same issue b) it fits for #uncapitalized, see [2] c) Dont know about ANSI standard or any other ST dialect Thx T. [1] https://github.com/srkirkland/Inflector/blob/master/Inflector.Tests/Capital… [2] https://github.com/srkirkland/Inflector/blob/master/Inflector.Tests/Uncapit…

March 15, 2017

Re: [Pharo-dev] Missing Zip64 support

by Thierry Goubier

Hi Pavel, Which means command like these: cd pharo; mkdir filetree; cd filetree; wget -O - https://api.github.com/repos/dalehenrich/filetree/tarball/pharo6.0_dev | tar xz --strip-components=1 ./pharo-cli pharo/Pharo.image --no-default-preferences eval --save Metacello new baseline: \'FileTree\'\; repository: \'filetree:///`pwd`/pharo/filetree/repository\'\; load: \'Git\'. recovering the tarball can be a lot faster than doing a local clone, and FileTree is not interested in the git history anyway... Thierry 2017-03-15 11:18 GMT+01:00 Pavel Krivanek <pavel.krivanek(a)gmail.com>: > Hi, > > our current Compression package does not support Zip64 which is limiting > us in usage of Metacello together with Git. When we use the Metacello for > installing of a baseline like: > > Metacello new > baseline: 'IDE'; > repository: 'github://guillep/pharo-core'; > load. > > then the Metacello downloads the corresponding archive from Github. > However the standard Zip archives have count limit of 65535 members but the > pharo-core repository and some others contain much more than this > (pharo-core has 125021 files). The Zip64 extension is used for them during > compression which we cannot read now. > > So if you will see Error: bad signature 16r... at position ..., then it > has this reason. The workaround for now is to load projects from a local > clone of such big repositories. > > Cheers, > -- Pavel >

March 15, 2017

Missing Zip64 support

by Pavel Krivanek

Hi, our current Compression package does not support Zip64 which is limiting us in usage of Metacello together with Git. When we use the Metacello for installing of a baseline like: Metacello new baseline: 'IDE'; repository: 'github://guillep/pharo-core'; load. then the Metacello downloads the corresponding archive from Github. However the standard Zip archives have count limit of 65535 members but the pharo-core repository and some others contain much more than this (pharo-core has 125021 files). The Zip64 extension is used for them during compression which we cannot read now. So if you will see Error: bad signature 16r... at position ..., then it has this reason. The workaround for now is to load projects from a local clone of such big repositories. Cheers, -- Pavel

March 15, 2017

[ANN] Collecting all Pharo related publications

by Marcus Denker

Hi, We want to create a list of all Pharo related papers. I have added this page: http://pharo.org/web/PharoPublications <http://pharo.org/web/PharoPublications> it is linked from http://pharo.org/documentation <http://pharo.org/documentation> Marcus

March 15, 2017

Re: [Pharo-dev] [Vm-dev] BUG? A problem with callbacks that shows up in 64bits (but is on 32bits too)

by Igor Stasenko

On 15 March 2017 at 11:53, Igor Stasenko <siguctua(a)gmail.com> wrote: > Here's my d) > implement callback functions in C, or in native form => no need for > entering the smalltalk execution => no risk of GC => nothing to worry about. > > I guess nobody will like it (and will be right, of course ;) , but it is > how it was originally done. I used NativeBoost to implement those callback > functions and they're won't cause any GC problems. > > That, of course, gave me solution in this concrete case, but not in > general.. i.e. : if you have another callback that cannot be implemented > na(t)ively, then > you facing similar problems, mainly: how to work around the problem, that > primitive(s) that using callbacks may capture state, that are subject of GC > activity. > > In general , then, i think such primitive should be (re)written in such > way , that it won't get puzzled by GC.. and addGCRoot(s), IMO then best > way, from general interfacing/implementation standpoint. > I would just add extra interface for using it especially in primitives, so > that it > 1) won't punish primitive writer with too much coding > 2) automatically handle primitive/callback nesting e.g. > primitive1 -> adds roots1 -> calls fn -> callback -> st code -> primitive2 > -> adds roots2 -> calls fn2 -> callback2 ... > > > something like this: > > static initialized once myprimooptable = [ a,b,c]. > vm pushPrimRoots: myooptable. > self do things primitive does. > vm popPrimRoots > > or, since we have green threading, then maybe better will be in this form: > > rootsId := static initialized once myprimooptable = [ a,b,c]. > vm pushPrimRoots: myooptable. > self do things primitive does. > vm popPrimRoots: rootsId. > > oops, i meant: static initialized once myprimooptable = [ a,b,c]. rootsId := vm pushPrimRoots: myooptable. self do things primitive does. vm popPrimRoots: rootsId. the idea is mainly, that VM answers you some key, using which you can later identify, what to release. > > > On 14 March 2017 at 18:33, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > >> >> >> >> On Tue, Mar 14, 2017 at 8:56 AM, Nicolai Hess <nicolaihess(a)gmail.com> >> wrote: >> >>> >>> >>> >>> 2017-03-14 16:46 GMT+01:00 Eliot Miranda <eliot.miranda(a)gmail.com>: >>> >>>> >>>> Hi Esteban, Hi Igor, Hi All, >>>> >>>> On Fri, Mar 10, 2017 at 7:35 AM, Esteban Lorenzano <estebanlm(a)gmail.com >>>> > wrote: >>>> >>>>> >>>>> Hi, >>>>> >>>>> Iâm tumbling into an error in Pharo, because we use callbacks >>>>> intensively, in Athens(cairo)-to-World conversion in particular, and people >>>>> is sending always their crash reportsâ¦ we made the whole conversion a lot >>>>> more robust since problems started to arise, but now I hit a wall I cannot >>>>> solve: I think problem is in something in callbacks. >>>>> >>>>> And problem is showing very easy on 64bits (while in 32bits it takes >>>>> time and is more random). >>>>> >>>> >>>> I responded in the "image not opening" thread, but it's the same >>>> problem. I really want to hear what y'all think because I'll happily >>>> implement a fix, but I want to know which one y'all think is a good idea. >>>> Here's my reply (edits between [ & ] to add information): >>>> >>>> >>>> On Mon, Mar 13, 2017 at 9:11 PM, Eliot Miranda <eliot.miranda(a)gmail.com >>>> > wrote: >>>> >>>> I'm pretty confident [I know] this is to do with bugs in the Athens >>>> surface code which assumes that callbacks can be made in the existing >>>> copyBits and warpBits primitive. They can't do this safely because a GC >>>> (scavenge) can happen during a callback, which then causes chaos when the >>>> copyBits primitive tries to access objects that have been moved under its >>>> feet. >>>> >>>> I've done work to fix callbacks so that when there is a failure it is >>>> the copyBits primitive that fails, instead of apparently the callback >>>> return primitive. One of the apparent effects of this fix is to stop the >>>> screen opening up too small; another is getting the background colour >>>> right, and yet another is eliminating bogus pixels in the VGTigerDemo >>>> demo. But more work is required to fix the copyBits and warpBits >>>> primitives. There are a few approaches one might take: >>>> >>>> a) fixing the primitive so that it saves and restores oops around the >>>> callbacks using the external oop table [InterpreterProxy>>addGCRoot: & >>>> removeGCRoot:]. That's a pain but possible. [It's a pain because all the >>>> derived pointers (the start of the destForm, sourceForm, halftoneForm and >>>> colorMapTable) must be recomputed also, and of course most of the time the >>>> objects don't move; we only scavenge about once every 2 seconds in normal >>>> running] >>>> >>>> b) fixing the primitive so that it pins the objects it needs before >>>> ever invoking a callback [this is a pain because pinning an object causes >>>> it to be tenured to old space if it is in new space; objects can't be >>>> pinned in new space, so instead the pin operation forwards the new space >>>> object to an old space copy if required and answers its location in old >>>> space, so a putative withPinnedObjectsDo: operation for the copyBits >>>> primitive looks like >>>> withPinnedFormsDo: aBlock >>>> <inline: #always> >>>> self cppIf: SPURVM & false >>>> ifTrue: >>>> [| bitBltOopWasPinned destWasPinned sourceWasPinned halftoneWasPinned | >>>> (bitBltOopWasPinned := interpreterProxy isPinned: bitBltOop) ifFalse: >>>> [bitBltOop := interpreterProxy pinObject: bitBltOop]. >>>> (destWasPinned := interpreterProxy isPinned: destForm) ifFalse: >>>> [destForm := interpreterProxy pinObject: destForm]. >>>> (sourceWasPinned := interpreterProxy isPinned: sourceForm) ifFalse: >>>> [sourceForm := interpreterProxy pinObject: sourceForm]. >>>> (halftoneWasPinned := interpreterProxy isPinned: halftoneForm) ifFalse: >>>> [halftoneForm := interpreterProxy pinObject: halftoneForm]. >>>> aBlock value. >>>> bitBltOopWasPinned ifFalse: [interpreterProxy unpinObject: bitBltOop]. >>>> destWasPinned ifFalse: [interpreterProxy unpinObject: destForm]. >>>> sourceWasPinned ifFalse: [interpreterProxy unpinObject: sourceForm]. >>>> halftoneWasPinned ifFalse: [interpreterProxy unpinObject: halftoneForm]] >>>> ifFalse: [aBlock value] >>>> and tenuring objects to old space is not ideal because they are only >>>> collected by a full GC, so doing this would at least tenure the bitBltOop >>>> which is very likely to be in new space] >>>> >>>> c) fixing the primitive so that it uses the scavenge and fullGC >>>> counters in the VM to detect if a GC occurred during one of the callbacks >>>> and would fail the primitive [if it detected that a GC had occurred in any >>>> of the surface functions]. The primitive would then simply be retried. >>>> >>>> d) ? >>>> >>> >>> Wouldn't it be possible to just pause the GC (scavange) when entering a >>> primitive ? >>> >> >> I don't think so. There is a callback occurring. If the computation >> executed by the callback requires a GC the application will abort if a GC >> cannot be done. Right? This is the case here. >> >> >> I like c) as it's very lightweight, but it has issues. It is fine to use >>>> for callbacks *before* cop[yBits and warpBits move any bits (the >>>> lockSurface and querySurface functions). But it's potentially erroneous >>>> after the unlockSurface primitive. For example, a primitive which does an >>>> xor with the screen can't simply be retried as the first, falling pass, >>>> would have updated the destination bits but not displayed them via >>>> unlockSurface. But I think it could be arranged that no objects are >>>> accessed after unlockSurface, which should naturally be the last call in >>>> the primitive (or do I mean showSurface?). So the approach would be to >>>> check for GCs occurring during querySurface and lockSurface, failing if so, >>>> and then caching any and all state needed by unlockSurface and showSurface >>>> in local variables. This way no object state is accessed to make the >>>> unlockSurface and showSurface calls, and no bits are moved before the >>>> queryDurface and lockSurface calls. >>>> >>>> If we used a failure code such as #'object may move' then the >>>> primitives could answer this when a GC during callbacks is detected and >>>> then the primitive could be retried only when required. >>>> >>>> >>>> [Come on folks, please comment. I want to know which idea you like >>>> best. We could fix this quickly. But right now it feels like I'm talking >>>> to myself.] >>>> >>>> >>>>> Here is the easiest way to reproduce it (in mac): >>>>> >>>>> wget files.pharo.org/get-files/60/pharo64-mac-latest.zip >>>>> wget files.pharo.org/get-files/60/pharo64.zip >>>>> wget files.pharo.org/get-files/60/sources.zip >>>>> unzip pharo64-mac-latest.zip >>>>> unzip pharo64.zip >>>>> unzip sources.zip >>>>> ./Pharo.app/Contents/MacOS/Pharo ./Pharo64-60438.image eval >>>>> "VGTigerDemo runDemo" >>>>> >>>>> eventually (like 5-6 seconds after, if not immediately), you will have >>>>> a stack like this: >>>>> >>>>> SmallInteger(Object)>>primitiveFailed: >>>>> SmallInteger(Object)>>primitiveFailed >>>>> SmallInteger(VMCallbackContext64)>>primSignal:andReturnAs:fromContext: >>>>> GrafPort>>copyBits >>>>> GrafPort>>image:at:sourceRect:rule: >>>>> FormCanvas>>image:at:sourceRect:rule: >>>>> FormCanvas(Canvas)>>drawImage:at:sourceRect: >>>>> FormCanvas(Canvas)>>drawImage:at: >>>>> VGTigerDemo>>runDemo >>>>> VGTigerDemo class>>runDemo >>>>> UndefinedObject>>DoIt >>>>> OpalCompiler>>evaluate >>>>> OpalCompiler(AbstractCompiler)>>evaluate: >>>>> [ result := Smalltalk compiler evaluate: aStream. >>>>> self hasSessionChanged >>>>> ifFalse: [ self stdout >>>>> print: result; >>>>> lf ] ] in EvaluateCommandLineHandler>>evaluate: >>>>> in Block: [ result := Smalltalk compiler evaluate: aStream.... >>>>> BlockClosure>>on:do: >>>>> EvaluateCommandLineHandler>>evaluate: >>>>> EvaluateCommandLineHandler>>evaluateArguments >>>>> EvaluateCommandLineHandler>>activate >>>>> EvaluateCommandLineHandler class(CommandLineHandler >>>>> class)>>activateWith: >>>>> [ aCommandLinehandler activateWith: commandLine ] in >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>>>> in Block: [ aCommandLinehandler activateWith: commandLine ] >>>>> BlockClosure>>on:do: >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activateSubCommand: >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleSubcommand >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>handleArgument: >>>>> [ self >>>>> handleArgument: >>>>> (self arguments >>>>> ifEmpty: [ '' ] >>>>> ifNotEmpty: [ :arguments | arguments first ]) >>>>> ] in PharoCommandLineHandler(BasicCommandLineHandler)>>activate in >>>>> Block: [ self... >>>>> BlockClosure>>on:do: >>>>> PharoCommandLineHandler(BasicCommandLineHandler)>>activate >>>>> PharoCommandLineHandler>>activate >>>>> PharoCommandLineHandler class(CommandLineHandler class)>>activateWith: >>>>> [ super activateWith: aCommandLine ] in PharoCommandLineHandler >>>>> class>>activateWith: in Block: [ super activateWith: aCommandLine ] >>>>> >>>>> Any idea? >>>>> >>>>> thanks! >>>>> Esteban >>>>> >>>> >>>> >>>> >>>> -- >>>> _,,,^..^,,,_ >>>> best, Eliot >>>> >>>> >>> >>> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> >> > > > -- > Best regards, > Igor Stasenko. > -- Best regards, Igor Stasenko.

March 15, 2017