April 2015 - Pharo-users - lists.pharo.org

ZnClient: getting more that 19 tweet for data scrapping
by Offray Vladimir Luna CÃ¡rdenas April 8, 2015

April 8, 2015

Hi, Recently Paul DeBruicker taught me how to refine my code for getting tweets properly. Consider this: =[1]==================================== | source anUrl tweet | anUrl := 'https://twitter.com/offrayLC'. source := Soup fromString: (ZnEasy get: anUrl ) contents asString. tweets := (source findAllTagsByClass: 'ProfileTweet-text') collect:[:ea | ea text]. ======================================== Is working fine, but I would like to get more that 19 tweets, that is what you get by default. There is any way to tell ZnEasy and friends to get more tweets, something similar to what you do when you scroll down into a twitter page? And by the way, I would like to make more sense of the Soup I got in the last line. ea text gives me the tweet contents, but how can I interpret the metadata in the soup? (is a retweet, date of publishing and so on). I could make this for most part of the twitter profile page, but the tweet is kind of elusive, for example how to know that "text" is the proper message for getting the tweet content? Any pointer to how to make sense of it by myself is greatly appreciated. Cheers, Offray

3 4

Messaging solutions for Pharo
by Esteban A. Maringolo April 8, 2015

April 8, 2015

Hi, I'll be needing to add bidirectional messaging to our current Android (Java) application, and as usual I'm expecting to manage the server side of the communication with Pharo. I'm evaluating two alternatives, but I'm in open to other, proven, options. Option 1. WebSockets Open a WebSocket on each device, and push/retrieve messages from each of these, tracking the device id if I need to recreate a new socket (avoiding recreating new sockets in case of connectivity issues, very common). Option 2. MQTT [1] This is basically a mobile oriented MQ, super low footprint on mobile. For what I saw, I could implement the server using an intermediate MQ (RabbitMQ) and use STOMP to connect to it. Option 3. XMPP [3] Provides several features I'll need in the future, like file transfer in addition to regular text messaging. It is very well supported in Android with Smack [4], but I don't know if we have a Pharo server for it, or if somebody ever played with it. By means of simplicity and use load I favor option 1, because I can understand it better, and as everything moves towards web based technologies, I could implement WAMP [5] on top of that in the future. Right now I'd need to have a hundred websockets opened at the same time, which doesn't sound like a heavy load to me, but I certainly don't know. In the future it could be an order of magnitude bigger, and that's why I don't discard more complex solutions like XMPP or MQTT. Regards! [1] http://mqtt.org/ [2] https://github.com/svenvc/docs/blob/master/neo/stamp.md [3] http://xmpp.org/ [4] http://www.igniterealtime.org/projects/smack/ [5] http://wamp.ws/

6 10

happy easter :-)
by Alexandre Bergel April 8, 2015

April 8, 2015

A few days late :-) http://ws.stfx.eu/B8D2V8TO8BIE <http://ws.stfx.eu/B8D2V8TO8BIE> -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

2 2

Spec with dynamic tabs
by Hartmut April 8, 2015

April 8, 2015

Hi to all, I'm working with Spec and appreciate the modularity a lot. However, I had some difficulties to discover, how I can have a dynamic widget in a tab. Plugging together the TabsExample and the DynamicWidgetChange didn't work either. In the end I discovered the flaw: - In DynamicWidgetChange>>change the last line from "self openWithSpec" to "self buildWithSpec" . The DynamicWidgetChange example works as before. But now also the following works (I propose to enhance the example): - Add a method "dynamicTab | tab | tab := self newTab. tab label: 'Dynamic'; icon: Smalltalk ui icons nautilusIcon; model: (DynamicWidgetChange new). ^ tab " to DynamicWidgetChange. - add a line "addTab: self dynamicTab " to DynamicWidgetChange>>populateManager Now you can dynamically change the widgets in the fourth tab. Cheers, Hartmut -- Hartmut Krasemannâ¨ KÃ¶nigsberger Str. 41 c â¨D 22869 Schenefeldâ¨ Tel. 040.8307097â¨ Mobil 0171.6451283â¨ krasemann(a)acm.org

2 2

Mapping graphs on a open street map
by Alexandre Bergel April 7, 2015

April 7, 2015

Hi! Several times we were asked how to map chart and graph on top of a geographical map. Here is a short example. Hope it is useful to someone. http://ws.stfx.eu/11NOBLH3SIYO <http://ws.stfx.eu/11NOBLH3SIYO> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= v := RTView new. map := RTOSM new. e := map element. v add: e. paris := 48.8567 @ 2.3508. newyork := 40.7127 @ -74.0059. london := 51.507222@ -0.1275. v @ RTDraggableView. data := { { paris . { 10 . 5 . 10 . 3 . 10 .6 . 8 } }. {london . { 5 . 3 .3 . -5 } } . {newyork . { 5 . -2 . 10 . 15 . -10 }} }. data do: [ :tupple | | grapher dataSet | grapher := RTGrapher new. grapher extent: 150 @ 100. dataSet := RTStackedDataSet new points: tupple second. dataSet barShape width: 10; color: Color red. grapher add: dataSet. grapher axisY. grapher axisX noLabel; noTick. grapher build. allGraphElements := grapher view elements. v addAll: allGraphElements. allGraphElements translateTo: (map latLonToRoassal: tupple first) ]. v canvas camera translateTo: (map latLonToRoassal: paris). v canvas camera noInitializationWhenOpen. v canvas camera scale: 0.1. v -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;.

1 0

Updating ZeroConf! Please do not use it until new advice!
by Esteban Lorenzano April 7, 2015

April 7, 2015

thatâ¦ we need to change it :P

3 5

Showcase of initial support for word clouds and data scrapping (and the usual newbie questions)
by Offray Vladimir Luna CÃ¡rdenas April 7, 2015

April 7, 2015

Hi all, On [1] you can see a small video showing the support for word clouds and data scrapping from Twitter for some public profiles (including mine). As you can see the visualization is embedded on grafoscopio trees/documents that provide context for the scripts and combine them with other documentation. You can create your own visualization by running the code at [2] and changing the profile name. [1] https://archive.org/details/gfcp-wordclouds-initial.mp4 [2] http://ws.stfx.eu/DVG4C8VFON1L So here are my usual questions: a) The code that puts an image in a name cloud accepts only png images and when the avatar has jpeg images I got an error. There is any way to make the RTNameCloud to work with jpg or to convert it to png. I was looking at Sven's documentation[3], but I can't make it work in my context (jpeg is taken from the web and converted to png without using the file system). [3] https://medium.com/concerning-pharo/elegant-pharo-code-bb590f0856d0 b) There is any way to go beyond 19 tweets. In recent conversations with Paul[4] he taught me how to extract tweets properly from a page using ZnClient, but it gets the last 19. There is any way to make ZnClient to go beyond that initial request and get, lets say 100 or more tweets? [4] http://forum.world.st/Data-scrapping-in-pharo-Extracting-tweets-contents-td… Thanks as always for keeping Pharo/Moose and their communities awesome! Cheers, Offray

2 2

How do I get the list of projects on SmalltalkHub?
by stepharo April 7, 2015

April 7, 2015

Where can we find this information as a newbie? Stef

6 10

Data scrapping in pharo: converting text with dates in Spanish
by Offray Vladimir Luna CÃ¡rdenas April 6, 2015

April 6, 2015

Hi all, I'm making a small data scrapper[1] in pharo to feed some visualizations. The data I'm scrapping contains strings with dates in Spanish like '16:21 - 15 de jun. de 2011' and I would like to convert them to proper dates in Smalltalk. So I started prototyping an Small script at [2], but the problem is that monthIndex at: month doesn't work because 'jun' is not in the dictionary (please refer to [2] for details). This is motivated by month = 'jun' giving false, but when I inspect month its content is 'jun', so I think that I'm missing something important. [1] http://smalltalkhub.com/#!/~Offray/Dataviz [2] http://ws.stfx.eu/IOMTYZ0N9W29 So here come my questions: a). How to get "monthIndex at: month" working properly so I can get '06' as the proper month index for the month 'jun' (junio).? b) Why month = 'jun' gives false? c) There is any way to convert strings which has months in different languages (for example Spanish) more directly? Thanks, Offray

4 8

Data scrapping in pharo: Extracting tweets contents
by Offray Vladimir Luna CÃ¡rdenas April 6, 2015

April 6, 2015

Hi, I'm making a data scrapper from twitter. I know that twitter API is there, but I would like to make the scrapped data available to anyone, even if the person has not signed an API agreement. Also I think that this kind of external data is important for making agile visualization less self-referential and could bring some interesting examples with the data is common to the usual "netizen". I have some advances that you can test easily executing the code at [1] and I have already scrapped and filled out the data from a twitter profile page. [1] http://ws.stfx.eu/E3LD464QI0GR Now I'm having problems extracting tweets data. If I execute the code at [2] I can get a list of tweets (first 19) and I can explore inside any member of the collection, but I can't make sense of the SoupTag data inside. How can I extract particularly the tweet contents? [2] http://ws.stfx.eu/JXYM7W7WL1H9 Any help with this will be greatly appreciated. Cheers, Offray

2 2