- Pharo-dev - lists.pharo.org

[Pharo-project] Towards better HTTP client usage
by Sven Van Caekenberghe Sept. 6, 2011

Sept. 6, 2011

Towards better HTTP client usage At the last Pharo Sprint in Lille (July 8th), StÃ©phane and I were at one point trying to remove some old HTTPSocket usage (which indirectly uses Zn) and replace it with direct and clean Zn usage. We hadn't much time left and didn't get very far. But I realized afterwards that although technically everything was there to write good HTTP client code, it was way too difficult (it required to much code on the user's behalf). Hence I decided to try write a new client that would support this usage much better and encourage better HTTP client usage in general. Another goal of this new client is to ultimately replace all other clients currently in Zn (these different client are creating some confusion among users). ZnNeoClient (a temporary name, to be renamed to ZnClient in the future) is a single object with a lot of (convenience) API to build, execute and process HTTP client requests. It is somewhat similar to Gofer like classes. It is also somewhat comparable to ZnHttpClient but it contains even more functionality. The new tests in ZnNeoClientTests show some of the ways the client can be used. In the current Zn version all ZnClient class functionality as well as all ZnHTTPSocketFacade (and thus HTTPSocket and thus the rest of the image) has already been reimplemented using ZnNeoClient. These are good examples to look at. At one point I plan to write some proper documentation, when the design settles down a bit, but here are some examples: The simplest possible usage: ZnNeoClient new get: 'http://zn.stxfx.eu/zn/small.html'. This is shorthand for: ZnNeoClient new url: 'http://zn.stxfx.eu/zn/small.html'; get. Which is actually shorthand for: ZnNeoClient new url: 'http://zn.stxfx.eu/zn/small.html'; get; contents. If you know upfront that you will do only one request, you can help conserve resources by doing: ZnNeoClient new beOneShot; get: 'http://zn.stxfx.eu/zn/small.html'. Connections are kept open whenever possible unless #beOneShot is chosen. Multiple requests (possibly to the same host) can be issued using the same client instance. At the end, #close should be send to the client, but garbage collection cleans up as well. Specifying URLs as strings can sometimes be tricky when special characters such as spaces are involved, for this ZnNeoClient has a whole URL and request construction API. Here is an example: ZnNeoClient new http; host: 'www.google.com'; addPath: 'search'; queryAt: 'q' put: 'Pharo Smalltalk'; get. There is also API for specifying headers, forms, multipart uploads and so on. See the unit tests, the ZnClient class side as well as the ZnHTTPSocketFacade for examples. The result of an HTTP request is an HTTP response, modelled by ZnResponse. Hence, you can ask a client for the last #response, #entity, #contents or test for #isSuccess. Which brings us to the problem that it is not easy to properly set up and handle the various things that can go wrong. It is here that ZnNeoClient aims to make a big contribution. Assume the following example: at some URL there is a list of numbers as lines in a text file that we want to download and use. Let's start simple: ^ ZnNeoClient new get: 'http://www.example.com/numbers.txt'. When all is well, this will return the text file as string. Let's add parsing: ^ ZnNeoClient new contentReader: [ :entity | (entity contents lines do: [ :each | Integer readFrom: each ifFail: [ nil ] ]) select: [ :each | each notNil ] ]; get: 'http://www.example.com/numbers.txt'. This will make sure we get a (possibly empty) list of numbers. The problem is, when number.txt is not found by the server, we don't deal properly with that situation. We can fix that: ^ ZnNeoClient new enforceHttpSuccess: true; contentReader: [ :entity | (entity contents lines do: [ :each | Integer readFrom: each ifFail: [ nil ] ]) select: [ :each | each notNil ] ]; get: 'http://www.example.com/numbers.txt'. Now, a non-success code will throw an error. We should also make sure that we do get a text/plain document back, and we want a uniform error handler reaction: ^ ZnNeoClient new enforceHttpSuccess: true; enforceAcceptContentType: true; accept: ZnMimeType textPlain; contentReader: [ :entity | (entity contents lines do: [ :each | Integer readFrom: each ifFail: [ nil ] ]) select: [ :each | each notNil ] ]; ifFail: [ :exception | self log: exception printString, ' while fetching numbers'. ^ #() ]; get: 'http://www.example.com/numbers.txt'. The failBlock will be execute on any exception, including ZnHttpUnsuccessful and ZnUnexpectedContentType. What about unreliable networking in general: we need a timeout, but better still, we could retry the request once or twice to cover for (possibly transient) networking/server problems. ^ ZnNeoClient new timeout: 15; numberOfRetries: 1; retryDelay: 2; enforceHttpSuccess: true; enforceAcceptContentType: true; accept: ZnMimeType textPlain; contentReader: [ :entity | (entity contents lines do: [ :each | Integer readFrom: each ifFail: [ nil ] ]) select: [ :each | each notNil ] ]; ifFail: [ :exception | self log: exception printString, ' while fetching numbers'. ^ #() ]; get: 'http://www.example.com/numbers.txt'. If anything goes wrong, there will be one retry after a delay of 2 seconds. Although there are sensible defaults for all options, it will probably make sense to group the options, like this: ^ ZnNeoClient new systemPolicy; accept: ZnMimeType textPlain; contentReader: [ :entity | (entity contents lines do: [ :each | Integer readFrom: each ifFail: [ nil ] ]) select: [ :each | each notNil ] ]; ifFail: [ :exception | self log: exception printString, ' while fetching numbers'. ^ #() ]; get: 'http://www.example.com/numbers.txt'. Finding the proper defaults and policies will take some trial and error. Another strategy is factory method like ZnClient class>>#client or ZnHTTPSocketFacade class>>#client. Here is another real-world example, invoking a REST service that returns JSON (you need the JSJsonParser class that comes with Seaside): ^ ZnNeoClient new systemPolicy; beOneShot; url: 'http://easy.t3-platform.net/rest/geo-ip'; queryAt: 'address' put: '81.83.7.35'; accept: ZnMimeType applicationJson; contentReader: [ :entity | (JSJsonParser parse: entity contents) at: #country ]; ifFail: [ nil ]; get. Or as a utility method: XYZUtils class>>#countryForIpAddress: ipAddressString ifFail: failBlock ^ ZnNeoClient new systemPolicy; beOneShot; url: 'http://easy.t3-platform.net/rest/geo-ip'; queryAt: 'address' put: ipAddressString; accept: ZnMimeType applicationJson; contentReader: [ :entity | (JSJsonParser parse: entity contents) at: #country ]; ifFail: failBlock; get That's it for now. There is still some implementation and testing work to be done. As always, all feedback is welcome. Sven

5 7

[Pharo-project] Pharo is getting fast
by Sean P. DeNigris Sept. 6, 2011

Sept. 6, 2011

On my new MBP with Cog Jit VM and Pharo 1.3, operations are getting /very/ fast. Sean -- View this message in context: http://forum.world.st/Pharo-is-getting-fast-tp3791983p3791983.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

3 3

[Pharo-project] [COTDC] 80 - CategoryEnvironment
by laurent laffont Sept. 5, 2011

Sept. 5, 2011

Today: CategoryEnvironment Comment Of The Day Contest - One Day One Comment Rules: #1: Each day a not commented class is elected. Each day the best comment will be integrated with name of the author(s). #2: If you cannot comment it, deprecate it. Results: http://code.google.com/p/pharo/wiki/CommentOfTheDayContest Laurent

1 0

[Pharo-project] [COTDC] 79 - RefactoryChange
by laurent laffont Sept. 5, 2011

Sept. 5, 2011

A comment per day is known to expend your life expectancy. Today: RefactoryChange Comment Of The Day Contest - One Day One Comment Rules: #1: Each day a not commented class is elected. Each day the best comment will be integrated with name of the author(s). #2: If you cannot comment it, deprecate it. Results: http://code.google.com/p/pharo/wiki/CommentOfTheDayContest Laurent

2 2

[Pharo-project] AutoStart, where are you?
by Sean P. DeNigris Sept. 5, 2011

Sept. 5, 2011

This class was in 1.2, but is missing in 1.3. Why was it removed and what has replaced it? Also, is there a flaw in the deprecation system that has me ask these questions in the first place? I can't load XMLRPC because of: Smalltalk addToStartUpList: self after: AutoStart. What do I replace this with? Sean p.s. I found mentions in several issues and ml posts, but nothing definitive. -- View this message in context: http://forum.world.st/AutoStart-where-are-you-tp3791613p3791613.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

5 7

[Pharo-project] Pharo Google Code Labels
by Sean P. DeNigris Sept. 5, 2011

Sept. 5, 2011

On http://code.google.com/p/pharo/ : Labels: smalltalk, dynamiclanguages, object-oriented, *****fun***** (my emphasis) Classic. And true. Sean -- View this message in context: http://forum.world.st/Pharo-Google-Code-Labels-tp3791747p3791747.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

1 0

[Pharo-project] [VM-team] [ANN] VMMaker repository home is now http://source.squeak.org/VMMaker
by David T. Lewis Sept. 5, 2011

Sept. 5, 2011

The VMMaker source repository is now located at: MCHttpRepository location: 'http://source.squeak.org/VMMaker' user: '' password: '' This is a mirror of the http://www.squeaksource.com/VMMaker repository, with all historical files, and with no change to policy. The intent of the move is to improve reliability and availability of the VMMaker repository for all project participants. The automated commit notices to the vm-dev list will now come from source.squeak.org, and the commit notices from squeaksource.com have been disabled. This is to permit files to be copied between source.squeak.org/VMMaker and squeaksource.com/VMMaker without duplicate commit notices. Commits to VMMaker should now be made to the new repository, and direct references to the VMMaker repository (e.g. ConfigurationOfVMMaker) should be updated to reflect the change. For Squeak and Pharo users who use VMMaker through Monticello or Metacello, there should be no immediate change. However, adding a repository for http://source.squeak.org/VMMaker is recommended at your earliest convenience. I wish to add my appreciation for SqueakSource, which serves as a valuable community resource and has for many years been the home of the VMMaker project. I further note that many plugins used by VMMaker are hosted on SqueakSource, and this will continue to be the case. Thanks also to Bert Freudenberg for setting up the new repository with a complete mirror copy of the VMMaker project. Dave

2 3

Sept. 5, 2011

14120 ----- - Issue 4646: Provide better Class Selection Widget. Thanks Camillo Bruni. http://code.google.com/p/pharo/issues/detail?id=4646 - Issue 4652: TestRunner search bars. Thanks Camillo Bruni. http://code.google.com/p/pharo/issues/detail?id=4652 - Fix settings opening. - Fix size of search bars Stef, Ben and Camillo

1 0

[Pharo-project] #copyReplaceAll:with: vs. the schlemiel
by Camillo Bruni Sept. 5, 2011

Sept. 5, 2011

I stumbled upon the horrible implementation of SequenceableCollection >> #copyReplaceAll:with: Not only was it horribly documented (only for strings) but also horribly inefficient nicely following the painters algorithm :). On to replacing this craftwork with a stream based solution I figured out that it breaks quite a few tests: Namely the #testCopyReplaceAllWith1Occurence test doesn't work with the following classes anymore: - LinkedList - Interval - SortedCollection - Symbol Now I think this test does't make much sense for SortedCollection/Intervals. SortedCollection enforces a certain order on the objects, meaning that replacing a slice of elements with something else yields a completely new intermediate collection. So unless the collection is first converted to an OrderedCollection I don't see much sense in supporting this method on SortedCollection. For Intervals there's a similar issue. Replacing elements in an Interval doesn't work (yetâ¦) so again, only an indirect OrderedCollection helps thereâ¦ On Symbol there's the problem of #streamContents which should use #species and an intermediate String to build up the Symbol. cami

1 0

Sept. 5, 2011

Alain either we should change some textEditor into CodeEditor (and the inverse) or we should make sure that by default texts are not shouted. Because we have a lot of place (like workspace, some inspector panes So what do you think? Stef

4 22