December 2015 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Cog Rump Xen unikernel

by Ben Coman

Whoops I notice an erroneous line in that recipe... On Sat, Dec 12, 2015 at 2:27 AM, Ben Coman <btc(a)openinworld.com> wrote: > I got curious about following up these two threads making Cog a unikernel... > So I thought I would try the "Building Rumprun Unikernels" tutorial here... > https://github.com/rumpkernel/wiki/wiki/Tutorial:-Building-Rumprun-Unikerne… > which I report here in case it was interest. The tutorial is based on > QEMU (and I think I saw a comment somewhere that Cog doesn't run so > well on QEMU and needs Bochs), so after that I adapted it from QEMU to > Xen. ... > $ cmake -G "Unix Makefiles" > -DCMAKE_TOOLCHAIN_FILE=../../rumprun/rumprun-x86_64/share/x86_64-rumprun-netbsd-toolchain.cmake > .. > $ make > $ rumprun-bake xen_pv helloer.bin helloer ERRONEOUS> $ rumprun-bake hw_generic helloer.bin helloer > $ rumprun xen -i helloer.bin cheers -ben

Dec. 12, 2015

Re: [Pharo-dev] [squeak-dev] Re: Unicode Support

by Eliot Miranda

Hi Euan, On Fri, Dec 11, 2015 at 5:45 PM, EuanM <euanmee(a)gmail.com> wrote: > Elliot, what's your take on having heterogenous collections for the > composed Unicode? > I'm not sure I'm understanding the question, but... I'm told by someone in the know that string concatenation is a big deal in certain applications, so providing tree-like representations for strings can be a win since concatenation is O(1) (allocate a new root and assign the two subtrees). It seems reasonable to have a rich library with several representations available with different trade-offs. But I'd let requirements drive design, not feature dreams. > i.e. collections with one element for each character, with some > characters being themselves a collection of characters > > (Simple character like "a" is one char, and a character which is a > collection of characters is the fully composed version of Ç (01d5), a > U (0055) with a diaeresis - Â¨ - ( 00a8 aka 0308 in combining form ) > on top to form the compatibility character Ã (ooDC) which then gets a > macron- Ì -( 0304) on top of that > > so > a #(0061) > > Ç #(01d5) = #( 00dc 0304) = #( 0055 0308 0304) > > i.e a string which alternated those two characters > > 'aÇaÇaÇaÇ' > > would be represented by something equivalent to: > > #( 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) 0061 #( 0055 0308 > 0304) 0061 #( 0055 0308 0304) ) > > as opposed to a string of compatibility characters: > #( 0061 01d5 0061 01d5 0061 01d5 0061 01d5) > > Does alternating the type used for characters in a string have a > significant effect on speed? > I honestly don't know. You've just gone well beyond my familiarity with the issues :-). I'm just a VM guy :-). But I will say that in cases like this, real applications and the profiler are your friends. Be guided by what you need now, not by what you think you'll need further down the road. > On 11 December 2015 at 23:08, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > > Hi Todd, > > > > On Dec 11, 2015, at 12:57 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: > > > > > > On Dec 11, 2015, at 12:19, EuanM <euanmee(a)gmail.com> wrote: > > > > "If it hasn't already been said, please do not conflate Unicode and > > UTF-8. I think that would be a recipe for > > a high P.I.T.A. factor." --Richard Sargent > > > > > > Well, yes. But I think you guys are making this way too hard. > > > > A unicode character is an abstract idea - for instance the letter 'a'. > > The letter 'a' has a code point - its the number 97. How the number 97 > is > > represented in the computer is irrelevant. > > > > Now we get to transfer encodings. These are UTF8, UTF16, etc.... A > > transfer encoding specifies the binary representation of the sequence of > > code points. > > > > UTF8 is a variable length byte encoding. You read it one byte at a time, > > aggregating byte sequences to 'code points'. ByteArray would be an > > excellent choice as a superclass but it must be understood that #at: or > > #at:put refers to a byte, not a character. If you want characters, you > have > > to start at the beginning and process it sequentially, like a stream (if > > working in the ASCII domain - you can generally 'cheat' this a bit). A C > > representation would be char utf8[] > > > > UTF16 is also a variable length encoding of two byte quantities - what C > > used to call a 'short int'. You process it in two byte chunks instead of > > one byte chunks. Like UTF8, you must read it sequentially to interpret > the > > characters. #at and #at:put: would necessarily refer to byte pairs and > not > > characters. A C representation would be short utf16[]; It would also to > > 50% space inefficient for ASCII - which is normally the bulk of your > text. > > > > Realistically, you need exactly one in-memory format and stream > > readers/writers that can convert (these are typically table driven state > > machines). My choice would be UTF8 for the internal memory format and > the > > ability to read and write from UTF8 to UTF16. > > > > But I stress again...strings don't really need indexability as much as > you > > think and neither UTF8 nor UTF16 provide this property anyhow as they are > > variable length encodings. I don't see any sensible reason to have more > > than one in-memory binary format in the image. > > > > > > The only reasons are space and time. If a string only contains code > points > > in the range 0-255 there's no point in squandering 4 bytes per code point > > (same goes for 0-65535). Further, if in some application interchange is > > more important than random access it may make sense in performance > grounds > > to use utf-8 directly. > > > > Again, Smalltalk's dynamic typing makes it easy to have one's cake and > eat > > it too. > > > > My $0.02c > > > > > > _,,,^..^,,,_ (phone) > > > > > > I agree. :-) > > > > Regarding UTF-16, I just want to be able to export to, and receive > > from, Windows (and any other platforms using UTF-16 as their native > > character representation). > > > > Windows will always be able to accept UTF-16. All Windows apps *might > > well* export UTF-16. There may be other platforms which use UTF-16 as > > their native format. I'd just like to be able to cope with those > > situations. Nothing more. > > > > All this is requires is a Utf16String class that has an asUtf8String > > method (and any other required conversion methods). > > > > > > -- _,,,^..^,,,_ best, Eliot

Dec. 12, 2015

Re: [Pharo-dev] Unicode Support

by EuanM

Elliot, what's your take on having heterogenous collections for the composed Unicode? i.e. collections with one element for each character, with some characters being themselves a collection of characters (Simple character like "a" is one char, and a character which is a collection of characters is the fully composed version of Ç (01d5), a U (0055) with a diaeresis - Â¨ - ( 00a8 aka 0308 in combining form ) on top to form the compatibility character Ã (ooDC) which then gets a macron- Ì -( 0304) on top of that so a #(0061) Ç #(01d5) = #( 00dc 0304) = #( 0055 0308 0304) i.e a string which alternated those two characters 'aÇaÇaÇaÇ' would be represented by something equivalent to: #( 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) 0061 #( 0055 0308 0304) ) as opposed to a string of compatibility characters: #( 0061 01d5 0061 01d5 0061 01d5 0061 01d5) Does alternating the type used for characters in a string have a significant effect on speed? On 11 December 2015 at 23:08, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Todd, > > On Dec 11, 2015, at 12:57 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: > > > On Dec 11, 2015, at 12:19, EuanM <euanmee(a)gmail.com> wrote: > > "If it hasn't already been said, please do not conflate Unicode and > UTF-8. I think that would be a recipe for > a high P.I.T.A. factor." --Richard Sargent > > > Well, yes. But I think you guys are making this way too hard. > > A unicode character is an abstract idea - for instance the letter 'a'. > The letter 'a' has a code point - its the number 97. How the number 97 is > represented in the computer is irrelevant. > > Now we get to transfer encodings. These are UTF8, UTF16, etc.... A > transfer encoding specifies the binary representation of the sequence of > code points. > > UTF8 is a variable length byte encoding. You read it one byte at a time, > aggregating byte sequences to 'code points'. ByteArray would be an > excellent choice as a superclass but it must be understood that #at: or > #at:put refers to a byte, not a character. If you want characters, you have > to start at the beginning and process it sequentially, like a stream (if > working in the ASCII domain - you can generally 'cheat' this a bit). A C > representation would be char utf8[] > > UTF16 is also a variable length encoding of two byte quantities - what C > used to call a 'short int'. You process it in two byte chunks instead of > one byte chunks. Like UTF8, you must read it sequentially to interpret the > characters. #at and #at:put: would necessarily refer to byte pairs and not > characters. A C representation would be short utf16[]; It would also to > 50% space inefficient for ASCII - which is normally the bulk of your text. > > Realistically, you need exactly one in-memory format and stream > readers/writers that can convert (these are typically table driven state > machines). My choice would be UTF8 for the internal memory format and the > ability to read and write from UTF8 to UTF16. > > But I stress again...strings don't really need indexability as much as you > think and neither UTF8 nor UTF16 provide this property anyhow as they are > variable length encodings. I don't see any sensible reason to have more > than one in-memory binary format in the image. > > > The only reasons are space and time. If a string only contains code points > in the range 0-255 there's no point in squandering 4 bytes per code point > (same goes for 0-65535). Further, if in some application interchange is > more important than random access it may make sense in performance grounds > to use utf-8 directly. > > Again, Smalltalk's dynamic typing makes it easy to have one's cake and eat > it too. > > My $0.02c > > > _,,,^..^,,,_ (phone) > > > I agree. :-) > > Regarding UTF-16, I just want to be able to export to, and receive > from, Windows (and any other platforms using UTF-16 as their native > character representation). > > Windows will always be able to accept UTF-16. All Windows apps *might > well* export UTF-16. There may be other platforms which use UTF-16 as > their native format. I'd just like to be able to cope with those > situations. Nothing more. > > All this is requires is a Utf16String class that has an asUtf8String > method (and any other required conversion methods). > >

Dec. 12, 2015

Re: [Pharo-dev] Unicode Support

by Eliot Miranda

Hi Todd, > On Dec 11, 2015, at 12:57 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: > > >> On Dec 11, 2015, at 12:19, EuanM <euanmee(a)gmail.com> wrote: >> >> "If it hasn't already been said, please do not conflate Unicode and >> UTF-8. I think that would be a recipe for >> a high P.I.T.A. factor." --Richard Sargent > > Well, yes. But I think you guys are making this way too hard. > > A unicode character is an abstract idea - for instance the letter 'a'. > The letter 'a' has a code point - its the number 97. How the number 97 is represented in the computer is irrelevant. > > Now we get to transfer encodings. These are UTF8, UTF16, etc.... A transfer encoding specifies the binary representation of the sequence of code points. > > UTF8 is a variable length byte encoding. You read it one byte at a time, aggregating byte sequences to 'code points'. ByteArray would be an excellent choice as a superclass but it must be understood that #at: or #at:put refers to a byte, not a character. If you want characters, you have to start at the beginning and process it sequentially, like a stream (if working in the ASCII domain - you can generally 'cheat' this a bit). A C representation would be char utf8[] > > UTF16 is also a variable length encoding of two byte quantities - what C used to call a 'short int'. You process it in two byte chunks instead of one byte chunks. Like UTF8, you must read it sequentially to interpret the characters. #at and #at:put: would necessarily refer to byte pairs and not characters. A C representation would be short utf16[]; It would also to 50% space inefficient for ASCII - which is normally the bulk of your text. > > Realistically, you need exactly one in-memory format and stream readers/writers that can convert (these are typically table driven state machines). My choice would be UTF8 for the internal memory format and the ability to read and write from UTF8 to UTF16. > > But I stress again...strings don't really need indexability as much as you think and neither UTF8 nor UTF16 provide this property anyhow as they are variable length encodings. I don't see any sensible reason to have more than one in-memory binary format in the image. The only reasons are space and time. If a string only contains code points in the range 0-255 there's no point in squandering 4 bytes per code point (same goes for 0-65535). Further, if in some application interchange is more important than random access it may make sense in performance grounds to use utf-8 directly. Again, Smalltalk's dynamic typing makes it easy to have one's cake and eat it too. > My $0.02c _,,,^..^,,,_ (phone) > >> I agree. :-) >> >> Regarding UTF-16, I just want to be able to export to, and receive >> from, Windows (and any other platforms using UTF-16 as their native >> character representation). >> >> Windows will always be able to accept UTF-16. All Windows apps *might >> well* export UTF-16. There may be other platforms which use UTF-16 as >> their native format. I'd just like to be able to cope with those >> situations. Nothing more. >> >> All this is requires is a Utf16String class that has an asUtf8String >> method (and any other required conversion methods). >

Dec. 11, 2015

Re: [Pharo-dev] Some GT tool for this?

by Andrei Chis

On Fri, Dec 11, 2015 at 8:10 PM, Mariano Martinez Peck < marianopeck(a)gmail.com> wrote: > > > On Fri, Dec 11, 2015 at 4:01 PM, Andrei Chis <chisvasileandrei(a)gmail.com> > wrote: > >> You can add the two methods below. Then when you open Spotter on a >> package you can also search through it's history. >> This will trigger an update if the history is not present, but Spotter >> will still respond quite fast. >> >> You can also dive in a MCVersionInfo to see its ancestors. >> >> RPackage>>spotterHistoryFor: aStep >> <spotterOrder: 100> >> aStep listProcessor >> title: 'History'; >> allCandidates: [ |ancestry| >> ancestry := (MCWorkingCopy allManagers detect: [:each | each packageName >> = self name]) ancestry. >> ancestry withBreadthFirstAncestors select: [:each | each isKindOf: >> MCVersionInfo] ]; >> candidatesLimit: 50; >> itemName: [ :each | each name, ' - ', each message ]; >> filter: GTFilterSubstrings >> >> MCVersionInfo>>spotterPreviewCodeIn: aComposite >> <spotterPreview: 10> >> aComposite text >> title: 'Summary'; >> display: [ >> self summaryHeader, String cr, >> 'Message:', String cr, self message ]; >> entity: self. >> Cheers, >> Andrei >> >> > Wow.... I send the other email before reading this one. This is much > better hahahaha. > Could these 2 extensions be incorporated as part of GT for further Pharo > releases? > I don't like the #isKindOf: but it happened to me that if I sent > #allAncestors I would not get the latest commits. > And the #isKindOf: is because otherwise the first elements > are MCLazyVersionInfo. > I am not sure. Maybe someone with better understanding of MC could tell us > the correct code. > Let's see if we can find a cleaner way. If now I'll push them like this :) Cheers, Andrei > > Thanks! > > > >> >> On Fri, Dec 11, 2015 at 4:52 PM, Eliot Miranda <eliot.miranda(a)gmail.com> >> wrote: >> >>> Hi Mariano, >>> >>> for this in Squeak I added a simple menu pick to the Monticello >>> Browser's package list menu called "search history" which opens up a >>> workspace containing the entire history as a flat string. I then just >>> search using ctrl-f. You can find it in a current squeak trunk image. >>> >>> _,,,^..^,,,_ (phone) >>> >>> On Dec 10, 2015, at 9:12 AM, Mariano Martinez Peck < >>> marianopeck(a)gmail.com> wrote: >>> >>> How many times have you tried to search a string as a comment of a >>> commit in all the history? Myself: many many times. I know we need much >>> better tools for that, store the history in a another way, provide a nicer >>> API, etc etc. And I know there were topics about that. >>> >>> However, I would deeply appreciate a very short term solution for ease >>> that. The model side is as simple as this: >>> >>> | packageName ancestry anscestors substring | >>> packageName := 'MyPackageXX'. >>> substring := 'whatever I want to search'. >>> ancestry := (MCWorkingCopy allManagers detect: [:each | each packageName >>> = packageName] ) ancestry. >>> anscestors := ancestry withBreadthFirstAncestors select: [:each | each >>> isKindOf: MCVersionInfo]. >>> anscestors select: [ :each | each message includesSubstring: substring ] >>> >>> Do you think it's worth a simple UI tool? Custom inspector of MCAncestry? >>> >>> Cheers, >>> >>> >>> -- >>> Mariano >>> http://marianopeck.wordpress.com >>> >>> >> > > > -- > Mariano > http://marianopeck.wordpress.com >

Dec. 11, 2015

Re: [Pharo-dev] Unicode Support

by Todd Blanchard

> On Dec 11, 2015, at 12:19, EuanM <euanmee(a)gmail.com> wrote: > > "If it hasn't already been said, please do not conflate Unicode and > UTF-8. I think that would be a recipe for > a high P.I.T.A. factor." --Richard Sargent > Well, yes. But I think you guys are making this way too hard. A unicode character is an abstract idea - for instance the letter 'a'. The letter 'a' has a code point - its the number 97. How the number 97 is represented in the computer is irrelevant. Now we get to transfer encodings. These are UTF8, UTF16, etc.... A transfer encoding specifies the binary representation of the sequence of code points. UTF8 is a variable length byte encoding. You read it one byte at a time, aggregating byte sequences to 'code points'. ByteArray would be an excellent choice as a superclass but it must be understood that #at: or #at:put refers to a byte, not a character. If you want characters, you have to start at the beginning and process it sequentially, like a stream (if working in the ASCII domain - you can generally 'cheat' this a bit). A C representation would be char utf8[] UTF16 is also a variable length encoding of two byte quantities - what C used to call a 'short int'. You process it in two byte chunks instead of one byte chunks. Like UTF8, you must read it sequentially to interpret the characters. #at and #at:put: would necessarily refer to byte pairs and not characters. A C representation would be short utf16[]; It would also to 50% space inefficient for ASCII - which is normally the bulk of your text. Realistically, you need exactly one in-memory format and stream readers/writers that can convert (these are typically table driven state machines). My choice would be UTF8 for the internal memory format and the ability to read and write from UTF8 to UTF16. But I stress again...strings don't really need indexability as much as you think and neither UTF8 nor UTF16 provide this property anyhow as they are variable length encodings. I don't see any sensible reason to have more than one in-memory binary format in the image. My $0.02c > I agree. :-) > > Regarding UTF-16, I just want to be able to export to, and receive > from, Windows (and any other platforms using UTF-16 as their native > character representation). > > Windows will always be able to accept UTF-16. All Windows apps *might > well* export UTF-16. There may be other platforms which use UTF-16 as > their native format. I'd just like to be able to cope with those > situations. Nothing more. > > All this is requires is a Utf16String class that has an asUtf8String > method (and any other required conversion methods).

Dec. 11, 2015

Re: [Pharo-dev] [Vm-dev] Cog Rump Xen unikernel

by Holger Freyther

> On 11 Dec 2015, at 20:09, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > Ben, > > great to hear you're working on this! I'm watching with bated breath! BTW, you might find that Docker is easier and more general; I believe there's no Xen for ARM for example, and ARM is increasingly interesting with up coming extremely cheap 64-bit versions of Pi and PINE. Good luck!! @Eliot: There is Xen for ARMv8 (because ARM is interesting for multi-core servers) @Ben: Awesome. Will be interesting how small one can get it (with still functioning TCP/IP)

Dec. 11, 2015

Re: [Pharo-dev] [squeak-dev] Re: Unicode Support

by EuanM

Eliot - thank you for explaining to me why my original idea was bad. :-) I always assumed it would be. Otherwise I'd've just built it the way I proposed. I'm thoroughly delighted to have more knowledgeable people contributing. On 11 December 2015 at 09:29, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > Hi Euan, > >> On Dec 10, 2015, at 6:43 PM, EuanM <euanmee(a)gmail.com> wrote: >> >> I agree with all of that, Ben. >> >> I'm currently fairly certain that fully-composed abstract characters >> is a term that is 1:1 mapped with the term "grapheme cluster" (i.e. >> one is an older Unicode description of a newer Unicode term). >> >> And once we create these, I think this sort of implementation is >> straightforward. For particular values of "straightforward", of >> course :-) >> >> i.e. the Swift approach is equivalent to the approach I originally >> proposed and asked for critiques of. >> >> One thing I don't understand.... why does the fact the composed >> abstract character (aka grapheme cluster) is a sequence mean that an >> array cannot be used to hold the sequence? > > Of course an Array can be used, but one good reason to use bits organized as four-byte units is that the garbage collector spends no time scanning them, whereas as far as its concerned the Array representation is all objects and must be scanned. Another reason is that foreign code may find the bits representation compatible and so they can be passed through the FFI to other languages whereas the Array of tagged characters will always require conversion. Yet another reason is that in 64-bits the Array takes twice the space of the bits object. > >> If people then also want a compatibility-codepoints-only UTF-8 >> representation, it is simple to provide comparable (i.e >> equivalence-testable) versions of any UTF-8 string - because we are >> creating them from composed forms by a *single* defined method. >> >> For my part, the reason I think we ought to implement it *in* >> Smalltalk is ... this is the String class of the new age. I want >> Smalltalk to be handle Strings as native objects. > > There's little if any difference in convenience of use between an Array of characters and a bits array with the string at:/at:put: primitives since both require at:/at:put: to access, but the latter is (efficiently) type checked (by the VM), whereas there's nothing to prevent storing other than characters in the Areay unless one introduces the overhead of skier explicit type checks in Smalltalk, and the Areay starts life as a sequence of nils (invalid until every element is set to a character) whereas the bits representation begins fully initialized with 0 asCharacter. So there's nothing more "natively objecty" about the Array. Smalltalk objects hide their representation from clients and externally they behave the same, except for space and time. > > Given that this is a dynamically-typed language there's nothing to prevent one providing both implementations beyond maintenance cost and complexity/confusion. So at least it's easy to do performance comparisons between the two. But I still think the bits representation is superior if what you want is a sequence of Characters. > >>> On 10 December 2015 at 23:41, Ben Coman <btc(a)openinworld.com> wrote: >>> On Wed, Dec 9, 2015 at 5:35 PM, Guillermo Polito >>> <guillermopolito(a)gmail.com> wrote: >>>> >>>>> On 8 dic 2015, at 10:07 p.m., EuanM <euanmee(a)gmail.com> wrote: >>>>> >>>>> "No. a codepoint is the numerical value assigned to a character. An >>>>> "encoded character" is the way a codepoint is represented in bytes >>>>> using a given encoding." >>>>> >>>>> No. >>>>> >>>>> A codepoint may represent a component part of an abstract character, >>>>> or may represent an abstract character, or it may do both (but not >>>>> always at the same time). >>>>> >>>>> Codepoints represent a single encoding of a single concept. >>>>> >>>>> Sometimes that concept represents a whole abstract character. >>>>> Sometimes it represent part of an abstract character. >>>> >>>> Well. I do not agree with this. I agree with the quote. >>>> >>>> Can you explain a bit more about what you mean by abstract character and concept? >>> >>> This seems to be what Swift is doing, where Strings are not composed >>> not of codepoints but of graphemes. >>> >>>>>> "Every instance of Swiftâs Character type represents a single extended grapheme cluster. An extended grapheme cluster is a sequence** of one or more Unicode scalars that (when combined) produce a single human-readable character. [1] >>> >>> ** i.e. not an array >>> >>>>>> Hereâs an example. The letter Ã© can be represented as the single Unicode scalar Ã© (LATIN SMALL LETTER E WITH ACUTE, or U+00E9). However, the same letter can also be represented as a pair of scalarsâa standard letter e (LATIN SMALL LETTER E, or U+0065), followed by the COMBINING ACUTE ACCENT scalar (U+0301). TheCOMBINING ACUTE ACCENT scalar is graphically applied to the scalar that precedes it, turning an e into an Ã©when it is rendered by a Unicode-aware text-rendering system. [1] >>> >>>>>> In both cases, the letter Ã© is represented as a single Swift Character value that represents an extended grapheme cluster. In the first case, the cluster contains a single scalar; in the second case, it is a cluster of two scalars:" [1] >>> >>>>>> SwiftÊ¼s string implemenation makes working with Unicode easier and significantly less error-prone. As a programmer, you still have to be aware of possible edge cases, but this probably cannot be avoided completely considering the characteristics of Unicode. [2] >>> >>> Indeed I've tried searched for what problems it causes and get a null >>> result. So I read *all*good* things about Swift's unicode >>> implementation reducing common errors dealing with Unicode. Can >>> anyone point to complaints about Swift's unicode implementation? >>> Maybe this... >>> >>>>>> An argument could be made that the implementation of String as a sequence that requires iterating over characters from the beginning of the string for many operations poses a significant performance problem but I do not think so. My guess is that AppleÊ¼s engineers have considered the implications of their implementation and apps that do not deal with enormous amounts of text will be fine. Moreover, the idea that you could get away with an implementation that supports random access of characters is an illusion given the complexity of Unicode. [2] >>> >>> Considering our common pattern: Make it work, Make it right, Make it >>> fast -- maybe Strings as arrays are a premature optimisation, that >>> was the right choice in the past prior to Unicode, but considering >>> Moore's Law versus programmer time, is not the best choice now. >>> Should we at least start with a UnicodeString and UnicodeCharacter >>> that operates like Swift, and over time *maybe* move the tools to use >>> them. >>> >>> [1] https://developer.apple.com/library/ios/documentation/Swift/Conceptual/Swif… >>> [2] http://oleb.net/blog/2014/07/swift-strings/ >>> >>> cheers -ben >>> >>>> >>>>> >>>>> This is the key difference between Unicode and most character encodings. >>>>> >>>>> A codepoint does not always represent a whole character. >>>>> >>>>> On 7 December 2015 at 13:06, Henrik Johansen > > _,,,^..^,,,_ (phone)

Dec. 11, 2015

Re: [Pharo-dev] Unicode Support

by EuanM

"If it hasn't already been said, please do not conflate Unicode and UTF-8. I think that would be a recipe for a high P.I.T.A. factor." --Richard Sargent I agree. :-) Regarding UTF-16, I just want to be able to export to, and receive from, Windows (and any other platforms using UTF-16 as their native character representation). Windows will always be able to accept UTF-16. All Windows apps *might well* export UTF-16. There may be other platforms which use UTF-16 as their native format. I'd just like to be able to cope with those situations. Nothing more. All this is requires is a Utf16String class that has an asUtf8String method (and any other required conversion methods). And other string classes to have asUtf16String classes. Once we have the other classes and methods, this should be a trivial extensions. Export will just be transformations of existing formats of valid strings. Import just needs to transform to (one of) our preferred format(s), and have a validity check performed after the transform is complete. On 11 December 2015 at 15:37, Richard Sargent <richard.sargent(a)gemtalksystems.com> wrote: > EuanM wrote >> ... >> all ISO-8859-1 maps 1:1 to Unicode UTF-8 >> ... > > I am late coming in to this conversation. If it hasn't already been said, > please do not conflate Unicode and UTF-8. I think that would be a recipe for > a high P.I.T.A. factor. > > Unicode defines the meaning of the code points. > UTF-8 (and -16) define an interchange mechanism. > > In other words, when you write the code points to an external medium > (socket, file, whatever), encode them via UTF-whatever. Read UTF-whatever > from an external medium and re-instantiate the code points. > (Personally, I see no use for UTF-16 as an interchange mechanism. Others may > have justification for it. I don't.) > > Having characters be a consistent size in their object representation makes > everything easier. #at:, #indexOf:, #includes: ... no one wants to be > scanning through bytes representing variable sized characters. > > Model Unicode strings using classes such as e.g. Unicode7, Unicode16, and > Unicode32, with automatic coercion to the larger character width. > > > > > -- > View this message in context: http://forum.world.st/Unicode-Support-tp4865139p4866610.html > Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com. >

Dec. 11, 2015

Re: [Pharo-dev] Some GT tool for this?

by Mariano Martinez Peck

On Fri, Dec 11, 2015 at 4:01 PM, Andrei Chis <chisvasileandrei(a)gmail.com> wrote: > You can add the two methods below. Then when you open Spotter on a package > you can also search through it's history. > This will trigger an update if the history is not present, but Spotter > will still respond quite fast. > > You can also dive in a MCVersionInfo to see its ancestors. > > RPackage>>spotterHistoryFor: aStep > <spotterOrder: 100> > aStep listProcessor > title: 'History'; > allCandidates: [ |ancestry| > ancestry := (MCWorkingCopy allManagers detect: [:each | each packageName = > self name]) ancestry. > ancestry withBreadthFirstAncestors select: [:each | each isKindOf: > MCVersionInfo] ]; > candidatesLimit: 50; > itemName: [ :each | each name, ' - ', each message ]; > filter: GTFilterSubstrings > > MCVersionInfo>>spotterPreviewCodeIn: aComposite > <spotterPreview: 10> > aComposite text > title: 'Summary'; > display: [ > self summaryHeader, String cr, > 'Message:', String cr, self message ]; > entity: self. > Cheers, > Andrei > > Wow.... I send the other email before reading this one. This is much better hahahaha. Could these 2 extensions be incorporated as part of GT for further Pharo releases? I don't like the #isKindOf: but it happened to me that if I sent #allAncestors I would not get the latest commits. And the #isKindOf: is because otherwise the first elements are MCLazyVersionInfo. I am not sure. Maybe someone with better understanding of MC could tell us the correct code. Thanks! > > On Fri, Dec 11, 2015 at 4:52 PM, Eliot Miranda <eliot.miranda(a)gmail.com> > wrote: > >> Hi Mariano, >> >> for this in Squeak I added a simple menu pick to the Monticello >> Browser's package list menu called "search history" which opens up a >> workspace containing the entire history as a flat string. I then just >> search using ctrl-f. You can find it in a current squeak trunk image. >> >> _,,,^..^,,,_ (phone) >> >> On Dec 10, 2015, at 9:12 AM, Mariano Martinez Peck <marianopeck(a)gmail.com> >> wrote: >> >> How many times have you tried to search a string as a comment of a commit >> in all the history? Myself: many many times. I know we need much better >> tools for that, store the history in a another way, provide a nicer API, >> etc etc. And I know there were topics about that. >> >> However, I would deeply appreciate a very short term solution for ease >> that. The model side is as simple as this: >> >> | packageName ancestry anscestors substring | >> packageName := 'MyPackageXX'. >> substring := 'whatever I want to search'. >> ancestry := (MCWorkingCopy allManagers detect: [:each | each packageName >> = packageName] ) ancestry. >> anscestors := ancestry withBreadthFirstAncestors select: [:each | each >> isKindOf: MCVersionInfo]. >> anscestors select: [ :each | each message includesSubstring: substring ] >> >> Do you think it's worth a simple UI tool? Custom inspector of MCAncestry? >> >> Cheers, >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> > -- Mariano http://marianopeck.wordpress.com

Dec. 11, 2015