January 2011 - Pharo-dev - lists.pharo.org

[Pharo-project] An intuitive (or screwed) #copyFrom:

by Sebastian Sastre

('if this isn''t broken' copyFrom: 15) ~= 'broken' ifTrue:['this is screwed'] ifFalse:['this is intuitive'] Evaluate that in any Pharo workspace and it will show you. Reported here. Status? invalid Came on guys. Really? I ask because, if so, situation goes like this: A. we think again and decide to do the right thing or we go with the alternative which is B. we leave it as invalid, as it is right now, and 1. we mislead even to smalltalkers not familiarized to squeak/pharo 2. we rationalize some clever way to see it as a feature even if it will mislead everybody (even ourselves in a hurry) 3. we lay a foundation to lightly use protocol that is typically used in collections (to do dangerous things like instVar manipulation) 4. we break encapsulation and manipulate extremely primitive things in a common sounding selector. 5. we work harder on trying to give the impression that we're leaving it like that because we're smarter than the confused people that tried to use it (proving to them that we're dumb) 6. we get involved in an unnecessarily complicated way of thinking that will complicate unnecessarily our future (guaranteed) 7. we learn how to maintain a screwed attitude in front of people trying to use intuition when using pharo 8. we stay comfortable (on the wrong foundation and for the wrong reasons) That would leave us with this question in the table: what is compatible with the Pharo's mission? is it A or B?

Jan. 1, 2011

Re: [Pharo-project] Issue 3436 in pharo: #copyFrom: is broken

by pharo＠googlecode.com

Comment #3 on issue 3436 by sebastianconcept: #copyFrom: is broken http://code.google.com/p/pharo/issues/detail?id=3436 I don't get it. When you take any non squeak based smalltalk you'll see that #copyFrom: is consistent and intuitive. When you take Pharo you get that: 1. it's broken even for smalltalkers (imagine random citizens) 2. is manipulating (admitedly) dangerously (check implementor comment) 3. is doing something extremely primitive without using the prefix basic (or whatever that suggest that this is a thing to use unless you know what you're doing) 4. our community response is that reporting this is invalid? Can we please fix this instead of rationalize? Suggestion: a rename of the current Object>>copyFrom: will be enough. How to rename it? with something that suggest is going to manipulate the freaking instance variables) there is code depending on that? screw it. If they started with the wrong foot, they should pay the price of their upgrade (they are guilty of creating a dependence on something broken).

Jan. 1, 2011

Re: [Pharo-project] [squeak-dev] Proper Smalltalk lots of classes

by Jan van de Sandt

Hello Jimmie, I had to do something similar a while ago. I had to support a complex XML schema for a standard message format of the Dutch government (StUF). The Java implementation used JAXB to generate a few thousand data objects. Very clumsy. In Smalltalk I made some code to generate Magritte descriptions for all the structure and field definitions in the XML schema. Using the Magritte descriptions it was very easy to read and write messages in the correct format. I created real classes only for the few message types that required some logic. For all the other message types I used some generic class that stores the values in a dictionary. So my advice is to have a look at Magritte. It can help you handle the type information without creating/generating lots of classes. Jan. On Fri, Dec 31, 2010 at 11:24 PM, Jimmie Houchin <jlhouchin(a)gmail.com>wrote: > Hello, > > I am porting the FIX protocol to Squeak/Pharo. > http://www.fixprotocol.org/ > > It is the Financial Information eXchange protocol. It is used by many firms > in the financial industries, stocks, forex, etc. > > There is good documentation and also a semi-reference implementation in > Java. > http://www.quickfixj.org > > I have a partial port started but I wanted a check with those who know more > than I about proper Smalltalk coding. > > In the Java implementation there are over 2000 classes. In the protocol > there are approximately 90 message types and 956 field types. > > I am initially creating the message classes as they form the basis of the > protocol and the fields are simply instance variables and methods in the > message classes. > > I am not porting the Java library. I am implementing the protocol in > Smalltalk from the documentation. > > Initially I have created a Fix44Field class with its necessary variables. > There are several versions of the Protocol and FIX 4.4 is the one required > for my application. > > With this Fix44Field class I am creating a Dictionary with the 956 > different field types. I am planning on creating a constructor method which > will create the desired field from the information in the dictionary about > the field and its properties. > > Am I way off base here? Should I really create the 956 classes necessary to > for each field in addition to the 90 or so message classes, and then I don't > know what else I haven't discovered yet. Having 1000+ classes just seems > unwieldy and naively it just doesn't feel like the Smalltalk way. I could be > wrong, but I haven't seen such an example to my memory. I don't know enough > about Java to know if it is good Java form either. > > When I have a reasonably fleshed out, tested and working solution, I will > make the source available under a MIT license. I would love to see > Squeak/Pharo become an out of the box viable option for trading > applications. > > Advise and wisdom greatly appreciated. > > Thanks. > > Jimmie > >

Jan. 1, 2011

Re: [Pharo-project] about growing streams and allocating buffers

by Philippe Marschall

On 01.01.2011 15:07, StÃ©phane Ducasse wrote: >> Hi >> >> I've been doing some performance work lately in Seaside. Long story >> short Seaside (and I guess AIDA too) spends of if it's rendering time in >> WriteStream (#nextPutAll:, #nextPut:). >> >> The way WriteStream >> #pastEndPut: behaves is not really ideal for >> Seaside. It grows the underlying collection by just enough to >> accommodate the argument collection (or 20 which ever is bigger). Now >> image the following not very unlikely scenario. You start with a 4k >> buffer and put on average a 10 element collection (remember all those >> tags are put individually) until you have a 16k response. You allocate >> more than a thousand intermediary collections to get there. >> What would be better suited for Seaside is doubling the required size. >> In the worst case that would mean wasting 50% of memory but it would >> make the overhead of creating intermediary collections logarithmic. In >> the given example that would take us only three allocations to get there. >> Now I do realize there are other applications for Pharo where this >> strategy is not ideal and this is not a killer for us. I just wanted to >> shed some light and this and ask whether other projects are in a similar >> situation. > > Thanks for the info. > Do you have an idea in which case allocating more would be a real problem? Whenever you're memory constrained, I guess. When doubling would put you past the size limit of the image. > Because in VW there were some patterns: > turning , into nextPut: to avoid exact the underlying string allocation. I don't get that. > preallocation OrderedCollection new: instead of relying on its growing behavior. Right, but you never know how big the response is going to before actually rendering. Otherwise you could just do Array/String new: beforehand. > Final question > May be we should be able to plug the stream behavior we want. Like that seaside people can > get the speed out of it. I think that having fast web app is important. Oh, it's already way faster than it was previously ;-) And lets not forget, even when you run Seaside, you will other other code like a database driver as well, that will need streams as well. Whatever you plug in, has to work with whatever else you have in your image. Cheers Philippe

Jan. 1, 2011

Re: [Pharo-project] about growing streams and allocating buffers

by Philippe Marschall

On 01.01.2011 14:39, Levente Uzonyi wrote: > On Sat, 1 Jan 2011, Philippe Marschall wrote: > >> Hi >> >> I've been doing some performance work lately in Seaside. Long story >> short Seaside (and I guess AIDA too) spends of if it's rendering time in >> WriteStream (#nextPutAll:, #nextPut:). >> >> The way WriteStream >> #pastEndPut: behaves is not really ideal for >> Seaside. It grows the underlying collection by just enough to >> accommodate the argument collection (or 20 which ever is bigger). Now > > No it doesn't. The code in #pastEndPut: is > > collection := collection grownBy: ((collection size max: 20) min: 1000000). > > so the argument of #grownBy: is at least 20, at most 1000000 and it's > size of the internal buffer (collection) if it's between 20 and 1000000. > So in most practical cases it's the value of [collection size]. > #grownBy: allocates a new collection that has the size of the > collection's size + the argument, which is 2 * self size in the most > common case. So the size grows exponentially till it reaches 1000000 and > then linearly (with a relatively high constant 1000000). This guarantees > that the cost of #nextPut: is constant in most cases. You're right, I misread the code, that's the behaviour for #nextPut:. However #nextPutAll: does: newEnd := position + aCollection size. newEnd > writeLimit ifTrue: [self growTo: newEnd + 10]. and #growTo: does then: newSize := anInteger + (oldSize // 4 max: 20). grownCollection := collection class new: newSize. aCollection is the argument collection, not the underlying collection. So that's the required size puls ten plus the max of a fourth of the current size and 20. >> image the following not very unlikely scenario. You start with a 4k >> buffer and put on average a 10 element collection (remember all those >> tags are put individually) until you have a 16k response. You allocate >> more than a thousand intermediary collections to get there. > > So just 3: one for 4k (the initial), one for 8k and one for 16k. Yes. >> What would be better suited for Seaside is doubling the required size. > > That's exactly what's happening in most cases. In the #nextPut: case, not in the #nextPutAll: case. >> In the worst case that would mean wasting 50% of memory but it would >> make the overhead of creating intermediary collections logarithmic. In > > The overhead is constant (amortized) in most cases. > >> the given example that would take us only three allocations to get there. >> Now I do realize there are other applications for Pharo where this >> strategy is not ideal and this is not a killer for us. I just wanted to >> shed some light and this and ask whether other projects are in a similar >> situation. >> >> To get a feel how allocation limited Seaside is: avoiding one allocation >> of a 16k ByteArray per request can make a difference in throughput >> between 10 Mbyte/s and 30 Mbyte/s (see "[Progress Report] Zinc HTTP >> Components"). If anybody knows a way to make allocation of large young >> space objects faster (Set GC Bias to Grow?, #vmParameterAt:put:?) I'd >> like to hear it. > > IIRC CogVM's GC is different from SqueakVM's GC in the sense that it is > activated when a predefined amount of data is allocated, not when the > number of allocations since the last GC reaches a predefined limit. So > you have to tune the GC differently on Cog and non-Cog VMs. Do you know where I can find more information about this? Cheers Philippe

Jan. 1, 2011

Re: [Pharo-project] about growing streams and allocating buffers

by StÃ©phane Ducasse

> Hi > > I've been doing some performance work lately in Seaside. Long story > short Seaside (and I guess AIDA too) spends of if it's rendering time in > WriteStream (#nextPutAll:, #nextPut:). > > The way WriteStream >> #pastEndPut: behaves is not really ideal for > Seaside. It grows the underlying collection by just enough to > accommodate the argument collection (or 20 which ever is bigger). Now > image the following not very unlikely scenario. You start with a 4k > buffer and put on average a 10 element collection (remember all those > tags are put individually) until you have a 16k response. You allocate > more than a thousand intermediary collections to get there. > What would be better suited for Seaside is doubling the required size. > In the worst case that would mean wasting 50% of memory but it would > make the overhead of creating intermediary collections logarithmic. In > the given example that would take us only three allocations to get there. > Now I do realize there are other applications for Pharo where this > strategy is not ideal and this is not a killer for us. I just wanted to > shed some light and this and ask whether other projects are in a similar > situation. Thanks for the info. Do you have an idea in which case allocating more would be a real problem? Because in VW there were some patterns: turning , into nextPut: to avoid exact the underlying string allocation. preallocation OrderedCollection new: instead of relying on its growing behavior. Final question May be we should be able to plug the stream behavior we want. Like that seaside people can get the speed out of it. I think that having fast web app is important. > > To get a feel how allocation limited Seaside is: avoiding one allocation > of a 16k ByteArray per request can make a difference in throughput > between 10 Mbyte/s and 30 Mbyte/s (see "[Progress Report] Zinc HTTP > Components"). If anybody knows a way to make allocation of large young > space objects faster (Set GC Bias to Grow?, #vmParameterAt:put:?) I'd > like to hear it. > > Cheers > Philippe > >

Jan. 1, 2011

[Pharo-project] about growing streams and allocating buffers

by Philippe Marschall

Hi I've been doing some performance work lately in Seaside. Long story short Seaside (and I guess AIDA too) spends of if it's rendering time in WriteStream (#nextPutAll:, #nextPut:). The way WriteStream >> #pastEndPut: behaves is not really ideal for Seaside. It grows the underlying collection by just enough to accommodate the argument collection (or 20 which ever is bigger). Now image the following not very unlikely scenario. You start with a 4k buffer and put on average a 10 element collection (remember all those tags are put individually) until you have a 16k response. You allocate more than a thousand intermediary collections to get there. What would be better suited for Seaside is doubling the required size. In the worst case that would mean wasting 50% of memory but it would make the overhead of creating intermediary collections logarithmic. In the given example that would take us only three allocations to get there. Now I do realize there are other applications for Pharo where this strategy is not ideal and this is not a killer for us. I just wanted to shed some light and this and ask whether other projects are in a similar situation. To get a feel how allocation limited Seaside is: avoiding one allocation of a 16k ByteArray per request can make a difference in throughput between 10 Mbyte/s and 30 Mbyte/s (see "[Progress Report] Zinc HTTP Components"). If anybody knows a way to make allocation of large young space objects faster (Set GC Bias to Grow?, #vmParameterAt:put:?) I'd like to hear it. Cheers Philippe

Jan. 1, 2011

[Pharo-project] [OT] questions answered on stackexchange: Why do good programmers have ugly websites?

by Igor Stasenko

Funny. Really funny. Especially this one: " Just be thankful there's more than just a command-prompt. " I hope that with Pharo we have enough 'bad' programmers, which can make good designs.. otherwise we're in trouble :) -- Best regards, Igor Stasenko AKA sig.

Jan. 1, 2011

Re: [Pharo-project] Issue 3500 in pharo: Create deprecated13

by Igor Stasenko

looks fine as to me. On 1 January 2011 12:35, <pharo(a)googlecode.com> wrote: > Status: Fixed > Owner: stephane.ducasse > > New issue 3500 by stephane.ducasse: Create deprecated13 > http://code.google.com/p/pharo/issues/detail?id=3500 > > We should create a package deprecated13 as one of the first move in 1.3. > > > > > -- Best regards, Igor Stasenko AKA sig.

Jan. 1, 2011

Re: [Pharo-project] new Cog VMs

by Philippe Marschall

On 31.12.2010 03:02, Eliot Miranda wrote: > > > > > Hi All, > > I've released a new version of Cog that has a substantially > improved code generator along the lines of Peter Deutsch's HPS > (VisualWorks) and various of Ian Piumarta's VMs. These all use a simple > tecnique to identify constant references in bytecode and to support a > register-based calling convention. While this does produce faster code > it tends to accelerate low-level code much more than high-level code as > you can see by the following benchmarks: > > SimpleStackBasedCogit: [1 to: 100000000 do: [:i|]] timeToRun 691 > StackToRegisterMappingCogit: [1 to: 100000000 do: [:i|]] timeToRun 192 > 192 - 691 / 6.91 -72% > > SimpleStackBasedCogit: 0 tinyBenchmarks '753495217 bytecodes/sec; > 64769127 sends/sec' > StackToRegisterMappingCogit: 0 tinyBenchmarks '931756141 bytecodes/sec; > 128157989 sends/sec' > 931756141 - 753495217 / 7534952.17 -24% > 128157989 - 64769127 / 647691.27 -98% > > SimpleStackBasedCogit: [Compiler recompileAll] timeToRun 47013 (no > transcript > StackToRegisterMappingCogit: [Compiler recompileAll] timeToRun 43406 (no > transcript) > 43406 - 47013 / 470.13 -7.67234594686576 > > The status of this code is essentially beta. The test suite runs the > same on the new code generator as on the old, but I think there are > still bugs because I get the occasional transient error. I am therefore > very interested in any reproducible errors you can find. > > The VMs (http://www.mirandabanda.org/files/Cog/VM/VM.r2334/) contain a > few other important changes: > > - a bug fix to bytecode<->native pc mappng that produced incorrect > results for methods containing blocks with ^-returns in them. One > symptom is incorrect highlighting of the pc in the debugger, althoguh > symptoms could be much serious. > > - jitting interpreted methods on backward branches. Currently any > interpreted method that performs more than 20 backward branches will be > considered for JIT compilation and if it is suitable (default, <= 60 > literals) will be compiled to native code. > > - new callback support. I need to commit some changes to the Alien > package to provide access to this but essentially the VM's callback > support is now able to be ported to architectures with register-based > calling conventions (ARM, PowerPC, SPARC etc). I'll try and get the > Alien code released soon, and to back-port the changes to the standard > VM before the end of the holiday. > > One thing that is still /not/ fixed is the lack of a SoundPlugin on > win32. Apologies. I'll try and get a fix for this before the end of > the holidays too, but time might be too tight. There are other > priorities such as harmonising the standard and Cog VMs for the 4.2 release. I gave it a short test with the latest Seaside. First the good news, I seem to get an average performance improvement of about 10%. However in some very rare cases the temps seem to be mixed up the very first time a method is executed and fine afterwards. For example in the method below when the div 'bench' is created. The very first time it is executed #div is sent to a BlockClosure, the next time it is fine. This is reproducible, if I recompile the package or restart the image it happens again the first time and is fine afterwards. I don't know how I can provide more information but I could upload the image somewhere. renderInline: aBlock factor: factor key: key on: html | startTime endTime count backColor anAssociation title referenceValue spi context document renderer stream runTime | count := 0. runTime := 200. anAssociation := referenceDict at: key ifAbsent: [ 'Undefined' -> 100000 ]. title := anAssociation key. referenceValue := anAssociation value. stream := WriteStream on: String new. document := builder documentClass on: stream codec: builder codec. context := WARenderContext new. context document: document. context actionUrl: builder actionUrl; resourceUrl: builder resourceUrl. renderer := builder rendererClass context: context. builder openDocument: document context: context. html div class: 'bench'; with: [ html heading: title. startTime := Time millisecondClockValue. endTime := startTime + runTime. [ Time millisecondClockValue < endTime ] whileTrue: [ count := count + 1. renderer render: aBlock; flush ] ]. builder closeDocument: document. spi := (count / referenceValue * (10000 / runTime)) rounded. backColor := spi > 100 ifTrue: [ '#5f5' ] ifFalse: [ spi < 50 ifTrue: [ '#f55' ] ifFalse: [ '#55f' ] ]. duration := duration + spi. html div style: 'float:left;padding:4px'; with: [ html big: spi greaseString , ' SPI'. html span class: 'iteration'; with: '(' , count greaseString , ' iterations)'. html break. html div class: 'rect'; style: 'background:' , backColor , ';width:' , (spi * 2) greaseString , 'px' ]. html div style: 'clear:both' Cheers Philippe

Jan. 1, 2011