February 2017 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] FileStreams and encoding ( opening a readStream on a FileRerefence)

by Nicolai Hess

2017-02-04 13:40 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > > > On 4 Feb 2017, at 13:01, Nicolai Hess <nicolaihess(a)gmail.com> wrote: > > > > > > > > 2017-02-04 12:49 GMT+01:00 Sven Van Caekenberghe <sven(a)stfx.eu>: > > Hi Nicolai, > > > > The FileSystem API is a bit inconsistent, yes. > > > > This is how you can use it: > > > > (FileLocator temp / 'foo.txt') writeStreamDo: [ :out | > > out binary. > > (ZnCharacterWriteStream on: out encoding: #utf8) << 'Ã©lÃ¨ve' ]. > > > > (FileLocator temp / 'foo.txt') readStreamDo: [ :in | > > in binary. > > ZnCharacterReadStream on: in encoding: #utf8) upToEnd ]. > > > > (FileLocator temp / 'foo.txt') binaryReadStreamDo: [ :in | > > (ZnCharacterReadStream on: in encoding: #utf8) upToEnd ]. > > > > There is no #binaryWriteStreamDo: > > > > The API around File is more correct, IMHO. > > > > Does this help ? > > > > Yes, thanks for the fast response. > > > > What exactly is your question ? > > > > I am looking at the issues with FileList, there are some parts that > don't work anymore (see FileList>>#contents, it calls some unimplemted > methods), and it uses TextConverter and somoe parts of the older File api. > > It looks like most other (newer) parts are using ZnCharacterReadstream > for encoding, but I couldn't find a way to use it together with > FileReferences (most parts of the FileList already operate with the newer > FileSystem API. > > OK. > > From my standpoint and understanding, I would always use binary streams > with explicit Zn converters, this combination is much easier to understand > and better implemented with more features. > > If you encounter any problem or have any questions, I will gladly try to > help you. > :-) Is there a way to list all supported encodings, similar to TextConverter class >> #allEncodingNames ? I only found ZnSimplifiedByteEncoder class >> #knownEncodingIdentifiers and ZnByteEncoder class >> #knownEncodingIdentifiers and nothing for all supported utf-8/16 ... encodings. nicolai > > > Sven > > > > > On 4 Feb 2017, at 12:09, Nicolai Hess <nicolaihess(a)gmail.com> wrote: > > > > > > Hi > > > How can I specify the character encoding when opening a readStream on > a FileRerefence. > > > > > > I found this, that works: > > > > > > | readStream fileContent | > > > readStream := (File named: aFileName) openForRead. > > > fileContent := ZnCharacterReadStream on: readStream encoding: encoding. > > > fileContent upToEnd asString. > > > > > > But if I try to do the same with a readStream from a FileReference > > > > > > | readStream fileContent | > > > readStream := aFileName asFileReference readStream. > > > fileContent := ZnCharacterReadStream on: readStream encoding: encoding. > > > fileContent upToEnd asString. > > > > > > I get an error SmallInteger DNU #asciiValue, > > > > > > this is because, in the first method, we create a binary filestream, > and if we > > > use readStream from a FileReference, the stream is a > MultibyteFileStream. > > > > > > How can I us ZnEncoder for a readstream from a FileReference? > > > > > > (and is it on purpose that both readStream method > (openForRead/readStream) > > > return different kinds of binary streams?) > > > > > > > > > nicolai > > >

Feb. 6, 2017

Re: [Pharo-dev] [QA] Debugger buttons explained

by Andrei Chis

On Mon, Feb 6, 2017 at 9:14 PM, Hilaire <hilaire(a)drgeo.eu> wrote: > Thanks for the updated message. > > Le 06/02/2017 Ã 19:39, Sven Van Caekenberghe a Ã©crit : > > That last one is not correct. It has nothing to do with specific > iteration selectors. It means: stop when execution returns in one of the > argument blocks. It also works for #on:error: #streamContents: or any other > message that accepts blocks as argument - which is pretty cool. > > Strangely step over does not step in the bloc with #do: but does with > #to:do: > > In the later case step over and through behave mostly the same, the > stack looks a bit different though. > > > Another question, I try to locate where the tooltips are set in the > debugger. I can't find it with traditional browser tools (send, class > use) and the finder: > > The finder tells me "Through" is used in SpecThroughDebugAction, but > browser tells me there is no use case of this class. I see some pragma > in class side method, did it... > actionType > <debuggingAction> > <contextMenuDebuggingAction> > > No user of actionType, but I guess user or pragma, how do you guys track > the logic from here > Seems that the logic for changing the tooltips is not that straightforward. If you ping me once you decided for a the text of the tooltip I can add them. If you want to look at it yourself here are some details: Each action in the debugger is modeled as a subclass of DebugAction. The Glamour interface of the debugger uses DebugAction>>#asGlamourAction to transform that into a Glamour action. This action is then rendered in GLMMorphicActionRenderer>>#render:. There the tooltip is set directly by concatenating the title and the shortcut of the action. Best solution would be to add a description attribute to DebugAction and GLMAction and then modify GLMMorphicActionRenderer>>#render: to use it. For the Spec interface of the debugger you need to adapt DebugAction>>#asMenuRegistrationIn: Cheers, Andrei > > > -- > Dr. Geo > http://drgeo.eu > > >

Feb. 6, 2017

Re: [Pharo-dev] [QA] Debugger buttons explained

by Hilaire

Ok, give a look to SpecDebugActionButton, this class has no user, only progma reference, not easy to track the logic. Never mind I test on a halt in its #update method. Nothing happen when opening the debugger. May be there is instance in memory, no idea under which name it is referenced, may be not. Inspecting the debugger window does not help, it is a kind of generic morph. Never mind, can try to look at the menu item "Debug it" in the playground window, I used to find the action there. Oops, get lost there, I see #debugSelection looks promising, see announcement. Get lost again. No idea, someone else should put these tooltips in the appropriate place. Le 06/02/2017 Ã 21:14, Hilaire a Ã©crit : > No user of actionType, but I guess user or pragma, how do you guys track > the logic from here -- Dr. Geo http://drgeo.eu

Feb. 6, 2017

Re: [Pharo-dev] [QA] Debugger buttons explained

by Sean P. DeNigris

How about... Into Step into the highlighted message, i.e. dive the debugger into the method which handles it Over Step over the highlighted message, i.e. execute it, but keep the debugger focused on the current method Through Like stepping "Over", except step through block contents when executed Restart Rewind to the start of the current execution context, resetting all local variables Continue Relinquish debugger control and proceed with normal execution ----- Cheers, Sean -- View this message in context: http://forum.world.st/QA-Debugger-buttons-explained-tp4933212p4933238.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Feb. 6, 2017

Re: [Pharo-dev] [QA] Debugger buttons explained

by Hilaire

Thanks for the updated message. Le 06/02/2017 Ã 19:39, Sven Van Caekenberghe a Ã©crit : > That last one is not correct. It has nothing to do with specific iteration selectors. It means: stop when execution returns in one of the argument blocks. It also works for #on:error: #streamContents: or any other message that accepts blocks as argument - which is pretty cool. Strangely step over does not step in the bloc with #do: but does with #to:do: In the later case step over and through behave mostly the same, the stack looks a bit different though. Another question, I try to locate where the tooltips are set in the debugger. I can't find it with traditional browser tools (send, class use) and the finder: The finder tells me "Through" is used in SpecThroughDebugAction, but browser tells me there is no use case of this class. I see some pragma in class side method, did it... actionType <debuggingAction> <contextMenuDebuggingAction> No user of actionType, but I guess user or pragma, how do you guys track the logic from here -- Dr. Geo http://drgeo.eu

Feb. 6, 2017

Re: [Pharo-dev] WorkingSession UUID looks "sketchy"

by Sven Van Caekenberghe

First, thanks for the discussion, I think this is good. The idea/reason behind NeoUUIDGenerator was to get an algorithm that is fully documented and clearly implemented with proper unit tests, all at the Smalltalk level, so that we can get rid of the plugin. > On 6 Feb 2017, at 18:54, Peter Uhnak <i.uhnak(a)gmail.com> wrote: > > 1. Regarding WorkingSession > > The WS' comment claims "On each image startup the current session is invalidated and a new session is created.", > but in reality WS is reset only save&quit, and not on startup... isn't that odd? > So if image crashes or I am running it headlessly without saving I am actually still on the same session. That is a bug. Either the documentation has to be changed, but more likely the implementation. The idea behind the old, simple Session object was crystal clear: create a new, unique (in the sense of #==) Session object on each run of an image (and not when saving). WorkingSession got more complicated and less clear. People seem to have different expectations from it, and we keep on having trouble with it, which are not a good signs. Your point (3) also indicates and important issue in division of responsibility. > 2. Regarding NeoUUID > > My apologies for stating that it was made worse than the /dev/urandom one, now I know why it doesn't really matter due to the other rolling factors. > However I found some things that you may or may not find interesting (especially on Windows): > > On Windows (10) (with latest 6 image and VM) `Time microsecondClockValue` returns microseconds, but (presumably the system) cannot give precision beyond 1 second - this will imho need a VM fix; I find that quite hard to believe and I did not know that. Are you really sure ? That would be terrible. A solution might be to use another clock primitive. > I can also generate about 1.2M UUIDs per second (limited by single core I guess), which means that about every 600-1200 consequential UUIDs will have the same clock value. I can't follow your reasoning here: if the clock precision would be 1 second, you would get 1.2M consecutive UUIDs with the same clock value, right ? In the unit tests this is verified (that the time value goes forward on consecutive calls). > On Linux the microseconds are fine, also I generate only 0.8M UUIDs (it is older machine, so with >1M UUID/sec you will still have time clash), > now there's about 1-2% probability that the immediately next UUID will have the same clock, but this is countered by the counter. :) Yes, the clock + the counter + the machine id + random (this is a described in detail in the class comment of NeoUUIDGenerator). > You are also taking 8 bytes of microsecondClockValue, but the value has only 7 bytes... so 8th byte is fixed to 0. True. > 9 & 10 bytes are counter, but 9th bit is rewritten with variant, so the counter is actually 0-255 and not 0-65536. Yes, but more correctly, the top 2 bits are set to 10, making the range 1 to 2^14 (16384) (instead of 2^16). > And finally 11 & 12 are random bits (assuming the seeding isn't broken). The seeding is one aspect (the quality of the seed), the algorithm is another. Note that in the current Random class, the seed is initialised using the clock as well. > So on Windows, the conditional probability that nth and n+256th UUIDs will be identical is imho 1/65536 (assuming they are in the same second, which is easy). I am pretty/quite sure this is not correct but it would take me much more time to come up with a more correct calculation. BTW, you also have to define the context of being the same: the same instance of one NeoUUIDGenerator generating the same UUID, or several different generator instances, in the same image, in different images, on different runs, the same or different machines/networks, ... > On Linux my understanding is that clash can only happen if NTP adjusts my clock during UUID generation (at which point it is same as Windows). > > Can UUID clash be achieved on Linux if you deploy copies of the same image and let them all generate UUIDs? (It should be again 1/65536). Same remark as above. > Regarding the poor seed at startup: > > 1k outside runs of 'NeoUUIDGenerator new nextRandom16' (on a fresh image) gives me only 116 unique values, compared to the expected 990-1000 > In the above it's already second run of the generator, for (random initial) counter, there was only 69 unique values out of 1000. But that is not proper use of a random generator, you create a new instance every time. You basically test seeding, which is different from random number generator proper. It seems that is a chicken-egg problem too ;-) BTW, you can currently provide your own seed too, for example: Random seed: ('/dev/random' asFileReference binaryReadStreamDo: [ :in | in next: 4 ]) asInteger. > 3. Chicken and egg question > > How would one bootstrap session's id initialized with just NeoUUID? :) (WorkingSession wants UUID new in initialize, but UUID needs WorkingSession to generate a UUID) That is correct. I don't understand why WorkingSession needs a UUID, it was not like that before. It also does not seem to be really used. One solution I can think of would be to make id lazy initialized. In any case, I am going to try integrating/activating NeoUUIDGenerator again in the latest Pharo 6. > Peter > >> >> Reading from /dev/random is not portable to Windows and tricky too (because it sometimes hangs until there is enough entropy). >> >>>> Still, UUIDs are not 100% guaranteed to be unique, they are a (very good) best effort. >>> >>> For all intents and purposes they are considered 100% to be unique. >>> If you generate two identical V4 UUIDs then either PRNG or seeding is broken (seeding in Pharo's case). >>> >>> Peter >> >> According to https://en.wikipedia.org/wiki/Universally_unique_identifier >> >> << >> When generated according to the standard methods, UUIDs are for practical purposes unique, without depending for their uniqueness on a central registration authority or coordination between the parties generating them, unlike most other numbering schemes. While the probability that a UUID will be duplicated is not zero, it is so close to zero as to be negligible. >>>> >> >> Read the last sentence. >> >> So IMO it is certainly not 'broken'. >> >> Note also that NeoUUID uses different elements, the random part is only one of them. >> >>> On Mon, Feb 06, 2017 at 02:35:37PM +0100, Sven Van Caekenberghe wrote: >>>> >>>>> On 6 Feb 2017, at 14:17, Yuriy Tymchuk <yuriy.tymchuk(a)me.com> wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> Iâm using the session id (Smalltalk session id) for my data recording, so I can distinguish if the recorded events came from the same session. The idea is that each time an image is started a new session is created and assigned a new UUID. Now when I started to look on the data I noticed that I have some cases where I have same session IDs with different session creation times (yes a new session is initialized with a current timestamp). The time difference for the sessions with the same UUID and a different timestamp is within 2 hours. Then another thing that I did is to group the data by the timestamp and there are no cases where I have a different ID for the same timestamp, which shows that the timestamp is a more reliable ID. Now I will deal with my data just fine, but maybe we need to look in the implementation why do we get sessions with the same IDs? >>>>> >>>>> Cheers. >>>>> Uko >>>> >>>> I would be very surprised it would happen with NeoUUIDGenerator (NeoUUIDGenerator next). The idea was to replace UUIDGenerator and the VM plugin by it. That got stalled when there was unforeseen interaction with WorkingSession. I believe that should be solved by now. >>>> >>>> Still, UUIDs are not 100% guaranteed to be unique, they are a (very good) best effort. >>>> >>>> But I agree that if they repeat in such a short time frame, that should be considered a bug. >>>> >>>> Sven >>>> >>>> >>>> >>>> >>>> >>> >> >> >

Feb. 6, 2017

Segmentation fault while installing Scale

by Andrey Tykhonov

Hi all! While executing the following command: $ wget -O- https://raw.githubusercontent.com/guillep/Scale/master/setupScale.sh | sudo bash > stderr.log I got the Segmentation fault error. You can see it in the contents of stderr.log which is attached to the email. Here you can see the output of the mentioned command: ``` --2017-02-03 19:35:16-- https://raw.githubusercontent.com/guillep/Scale/master/setupScale.sh Resolving raw.githubusercontent.com... 151.101.128.133, 151.101.192.133, 151.101.64.133, ... Connecting to raw.githubusercontent.com|151.101.128.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1021 [text/plain] Saving to: âSTDOUTâ - 100%[=============================================================>] 1021 --.-KB/s in 0s 2017-02-03 19:35:16 (274 MB/s) - written to stdout [1021/1021] Cloning into '/tmp/scale'... remote: Counting objects: 1457, done. remote: Total 1457 (delta 0), reused 0 (delta 0), pack-reused 1457 Receiving objects: 100% (1457/1457), 656.60 KiB | 714.00 KiB/s, done. Resolving deltas: 100% (799/799), done. Note: checking out 'f70fdb90cf66d443140b6fe84cad9848163bf32e'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name> --2017-02-03 19:35:18-- http://get.pharo.org/50+vm Resolving get.pharo.org... 128.93.162.72 Connecting to get.pharo.org|128.93.162.72|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2901 (2.8K) [text/html] Saving to: âSTDOUTâ - 100%[=============================================================>] 2.83K --.-KB/s in 0.001s 2017-02-03 19:35:18 (3.56 MB/s) - written to stdout [2901/2901] Downloading the latest 50 Image: http://files.pharo.org/get-files/50/pharo.zip Downloading the latest pharoVM: http://files.pharo.org/get-files/50/pharo-linux-stable.zip Downloading PharoV50.sources: http://files.pharo.org/get-files/50/sources.zip Creating starter scripts pharo and pharo-ui Fetched -> BaselineOfScale-SantiagoBragagnolo.4 --- filetree:///tmp/scale/src [:] --- filetree:///tmp/scale/src./pharo: line 11: 19176 Aborted "$DIR"/"pharo-vm/pharo" --nodisplay "$@" Syntax Error on line 26: 'Unmatched '' in string literal. ->' ============================================================= 01: | scapeParentDirectory pharoParentDirectory scaleX scaleuiX | 02: 03: system stdout << 'Uninstalling scale' << String cr. 04: 05: system saveImageAs: #/ asFileReference / #tmp / #'uninstall.image'. 06: " The image must be stored elsewhere for deleting all the files " 07: 08: scaleX := '/usr/local/bin/scale' asFileReference. 09: scaleX exists ifTrue: [ system call: ' rm ', scaleX fullName ]. 10: system stdout << 'Uninstalling executable scale' << String cr ; flush. 11: 12: scaleuiX:='/usr/local/bin/scale-ui' asFileReference. 13: scaleuiX exists ifTrue: [ system call: ' rm ', scaleuiX fullName ]. 14: system stdout << 'Uninstalling executable scale-ui' << String cr ; flush. 15: 16: scapeParentDirectory := '/usr/local/scale' asFileReference. 17: pharoParentDirectory := '/usr/local/pharo' asFileReference. 18: 19: scapeParentDirectory exists ifTrue: [ system call: ' rm -rf ', scapeParentDirectory fullName ]. 20: system stdout << 'Removing scale folder' << String cr ; flush. 21: 22: pharoParentDirectory exists ifTrue: [ system call: ' rm -rf ', pharoParentDirectory fullName ]. 23: system stdout << 'Removing pharo folder' << String cr ; flush. 24: 25: 26: system stdout << 'Done _^_ Syntax Error on line 39: 'Unmatched '' in string literal. ->' ============================================================= 01: | scaleParentDirectory scaleFile scaleUIFile pharoParentDirectory| 02: 03: " DEPLOY " 04: system stdout << 'Deploying scale command';cr. 05: 06: scaleParentDirectory := '/usr/local/scale' asFileReference. 07: 08: scaleParentDirectory exists 09: ifTrue: [ 10: system stderr << 'There is already an installation. Please uninstall before' 11: ] ifFalse: [ 12: scaleParentDirectory ensureCreateDirectory. 13: scaleFile := scaleParentDirectory / 'scale'. 14: scaleUIFile := scaleParentDirectory / 'scale-ui'. 15: system workingDirectory / '..' / 'scale' copyTo: scaleFile. 16: system workingDirectory / '..' / 'scale-ui' copyTo: scaleUIFile. 17: system workingDirectory / '..' / #build / 'uninstall.st' copyTo: scaleParentDirectory / 'uninstall.st'. 18: system stdout << 'Deploying Pharo';cr. 19: 20: pharoParentDirectory := '/usr/local/pharo' asFileReference. 21: pharoParentDirectory ensureCreateDirectory. 22: #( 'pharo' 'pharo-vm' 'pharo-ui') 23: do: [ :file | system workingDirectory / file copyAllTo: pharoParentDirectory / file ]. 24: 25: #( 'Pharo.image' 'Pharo.changes') 26: do: [ :file | system workingDirectory / file copyAllTo: scaleParentDirectory / file ]. 27: 28: 29: system stdout << 'Giving permissions to run';cr. 30: 31: system call: 'chmod +x ', (pharoParentDirectory / 'pharo') fullName. 32: system call: 'chmod +x ', (pharoParentDirectory / 'pharo-ui') fullName. 33: system call: 'chmod +x ', (pharoParentDirectory / 'pharo-vm/pharo') fullName. 34: system call: 'chmod +x ', (scaleParentDirectory / 'uninstall.st') fullName. 35: system call: 'chmod +x ', scaleFile fullName. 36: system call: 'chmod +x ', scaleUIFile fullName. 37: system ln: scaleFile to: '/usr/local/bin/scale' asFileReference. 38: system ln: scaleUIFile to: '/usr/local/bin/scale-ui' asFileReference. 39: system stdout << 'Done _^_ ``` Could you please help me find a workaround in order to bypass this issue? Thank you in advance! -- Andriy

Feb. 6, 2017

Re: [Pharo-dev] [QA] Debugger buttons explained

by Sven Van Caekenberghe

> On 6 Feb 2017, at 18:55, Hilaire <hilaire(a)drgeo.eu> wrote: > > How will you describe these buttons in the debugger? > Here are proposals for tooltips, please fix it, I am sure there are > better way to write it. > > Proceed > Quit the debugger and resume the execution of the method. > > Restart > Reset the local variable and restart stepping at the beginning of the > method. > > Into > Step in the highlighted message, i.e. follow the sent message in its method. > > Over > Execute the highlighted message, i.e. do not follow the sent message in > its method. > > Through > Execute the highlighted message, when argument is a bloc to #do:, > #collect:, #select: #detect: step in the bloc. That last one is not correct. It has nothing to do with specific iteration selectors. It means: stop when execution returns in one of the argument blocks. It also works for #on:error: #streamContents: or any other message that accepts blocks as argument - which is pretty cool. Consult DebugSession for good descriptions of the actions. This is my version/take, probably not perfect either. Into Step in the highlighted message, i.e. follow the sent message, taking debugger control in the method invoked. Over Step over the highlighted message, i.e. do not follow the sent message, taking debugger control after the method invoked returns. Through Step over the highlighted message, i.e. do not follow the sent message, taking debugger control after the method invoked returns or whenever execution should return inside a block used as an argument before that. Restart Go back the start of the current execution context, resetting all local variables and take debugger control. Continue Relinquish debugger control and proceed execution from the current point of debugger control. > https://pharo.fogbugz.com/f/cases/19654/Tooltips-with-description-of-the-ov… > > -- > Dr. Geo > http://drgeo.eu > >

Feb. 6, 2017

Debugger hang when step throught

by Hilaire

Hello, The faulty code bellow when debugged and 'step through' cause the debugger to hang: self inform: '1'. #(2 3 4) select: [:i| i] thenDo: [:n| self inform: n printString ]. self inform: '5'. It seems trapped in an infinite loop, you can more or less get control back with Alt+[.] When step in, the debugger properly report the error. Pharo #60362. I did not see similar bug report, shall I? -- Dr. Geo http://drgeo.eu

Feb. 6, 2017

[QA] Debugger buttons explained

by Hilaire

How will you describe these buttons in the debugger? Here are proposals for tooltips, please fix it, I am sure there are better way to write it. Proceed Quit the debugger and resume the execution of the method. Restart Reset the local variable and restart stepping at the beginning of the method. Into Step in the highlighted message, i.e. follow the sent message in its method. Over Execute the highlighted message, i.e. do not follow the sent message in its method. Through Execute the highlighted message, when argument is a bloc to #do:, #collect:, #select: #detect: step in the bloc. https://pharo.fogbugz.com/f/cases/19654/Tooltips-with-description-of-the-ov… -- Dr. Geo http://drgeo.eu

Feb. 6, 2017