November 2017 - Pharo-users

Re: [Pharo-users] LiteratureResearcher - where graphs, PDFs, and BibTex happily live together

by Christian Haider

Yes, reading PDFs is fine with PDFtalk, but what you want is more: Text extraction (there is a chapter in the spec about this). This feature is not yet readily available. Some ground work has been done (content analysis), but for full text extraction more work is needed. Cheers, Christian > -----UrsprÃ¼ngliche Nachricht----- > Von: Pharo-users [mailto:pharo-users-bounces@lists.pharo.org] Im Auftrag > von Stephane Ducasse > Gesendet: Freitag, 3. November 2017 09:46 > An: Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > Betreff: Re: [Pharo-users] LiteratureResearcher - where graphs, PDFs, and > BibTex happily live together > > Hi manuel > > thanks for the details. I think that the framework of christian haidler should > be able to read pdf. > > Stef > > On Thu, Nov 2, 2017 at 8:33 PM, Manuel Leuenberger > <leuenberger(a)inf.unibe.ch> wrote: > > Hi Stef, > > > > The PDF integration consists of three parts: > > > > 1. CERMINE (https://github.com/CeON/CERMINE) is fed with the PDF and > > outputs metadata as BibTex and a structured XML (title, authors, > > affiliations, abstract, keyword, references, â¦). This is not perfect, > > but way better than any other metadata extractor I could find. > > 2. From the metadata I generate hyperlinks that are anchored in the > > PDF by a text key. pdf-linker (https://github.com/maenu/pdf-linker) > > then searches for the anchors in the PDF text, using heuristics, as > > PDF has a document model that is primarily intended for rendering and > > printing, but not for processing. The hyperlinks are then inserted > > using the awesome Apache PDFBox (https://pdfbox.apache.org/) > > 3. Those hyperlinks point to an URI like > > âpharo://handle/clickReference.in.?args=1&args=2â to represent a > > reference 1 in the paper 2. Now comes the magic part: The OS allows > > you to register custom handlers for custom URI schemes like pharo://. > > For that I created a simple Objective-C app that handles the event and > > passes it over as a HTTP message to a server running in Pharo > > (https://github.com/maenu/PharoUriScheme) The OS will even start the > > application if it is not yet running. > > > > While the custom URI scheme approach is super powerful, it has > > critical drawbacks. Any application can request to be the receiver of > > a URI scheme, just as browser are for http://. Especially on mobile > > devices with limited access to the OS, this opens up an attack point > > for malware apps that replicate original apps that make use of schemes > > like facebook:// and eavesdrop all interactions. If an original app > > transmits any unencrypted secrets or user data encoded in those URIs, > > malware can easily intercept it without the user noticing the leak. I > > guess this is the reason why many PDF viewer just support the standard > > http:// and mailto:// schemes. E.g., macOS Preview gives just an > > audible beep when I click on a pharo:// link, Chromes viewer doesnât > > even bother giving any feedback. Only Adobe Acrobat allows you to > > relax security settings to make them work (How could it be someone else > than Adobe, when itâs a security issue? ;). > > > > I finished basic packaging today and will continue with some READMEs > > and a nearly-all-in-one distribution tomorrow, Iâll keep you posted in > > this thread. > > > > Cheers, > > Manuel > > > > On 2 Nov 2017, at 18:08, Stephane Ducasse <stepharo.self(a)gmail.com> > wrote: > > > > Hi manuel > > > > this is super cool :) > > Could you describe how you did the pdf integration? > > And yes please package it :) > > I want to try it. > > > > Stef > > > > On Wed, Nov 1, 2017 at 10:16 PM, Manuel Leuenberger > > <leuenberger(a)inf.unibe.ch> wrote: > > > > Hi everyone, > > > > I was experimenting in the last few weeks with my take on literature > > research. For me, the corpus of scientific papers form an > > interconnected graph, not those plain lists and tables we keep in our > > bibliographies. So, here is the first prototype that has Google > > Scholar integration for search, can fetch PDFs from IEEE and ACM, > > extracts metadata from PDFs - all this results in hyperlinked PDFs! > > > > See a demo here: https://youtu.be/EcK3Pt_WnEw Also slides from the > SCG > > seminar here: > > http://scg.unibe.ch/download/softwarecomposition/2017-10-31- > Leuenberge > > r-ILE.pdf > > > > I plan on packaging it, so that those who are interested can check it > > out themselves (help wanted!). Currently, it only works on macOS. > > > > What do you think of my approach? Which use cases should be added? > > > > Cheers, > > Manuel > > > > > >

Nov. 3, 2017

Re: [Pharo-users] NeoCSV #skipToNextLine or #skipRemainingColumns ?

by Sven Van Caekenberghe

Hi Paul, > On 3 Nov 2017, at 04:14, PAUL DEBRUICKER <pdebruic(a)gmail.com> wrote: > > Hi - > > I want to import the first and seventh column from a set of 50 csv files each of which could have 28 - 34 columns. Is there a way to have NeoCSV skip the remaining columns automatically (e.g. 8 and up) ? > > Thanks > > Paul There are NeoCSVReader>>#addIgnoredField and #addIgnoredFields: You could maybe use the header to determine how many fields to skip. There is no, 'skip all remaining fields' yet. HTH, Sven

Nov. 3, 2017

Looking for game state

by Stephane Ducasse

Hi I'm reading a nice book explaining Game AI logic and it uses state design patterns and I would like to take it as an example for a future book on AI in the future They have two little actors talking via a a queued message list. There is one miner and its wife. Now I'm looking for a better example :) I was thinking about Pokemon hunting but if you have other ideas I'm interested. Stef

Nov. 3, 2017

Re: [Pharo-users] LiteratureResearcher - where graphs, PDFs, and BibTex happily live together

by Stephane Ducasse

Hi manuel thanks for the details. I think that the framework of christian haidler should be able to read pdf. Stef On Thu, Nov 2, 2017 at 8:33 PM, Manuel Leuenberger <leuenberger(a)inf.unibe.ch> wrote: > Hi Stef, > > The PDF integration consists of three parts: > > 1. CERMINE (https://github.com/CeON/CERMINE) is fed with the PDF and outputs > metadata as BibTex and a structured XML (title, authors, affiliations, > abstract, keyword, references, â¦). This is not perfect, but way better than > any other metadata extractor I could find. > 2. From the metadata I generate hyperlinks that are anchored in the PDF by a > text key. pdf-linker (https://github.com/maenu/pdf-linker) then searches for > the anchors in the PDF text, using heuristics, as PDF has a document model > that is primarily intended for rendering and printing, but not for > processing. The hyperlinks are then inserted using the awesome Apache PDFBox > (https://pdfbox.apache.org/) > 3. Those hyperlinks point to an URI like > âpharo://handle/clickReference.in.?args=1&args=2â to represent a reference 1 > in the paper 2. Now comes the magic part: The OS allows you to register > custom handlers for custom URI schemes like pharo://. For that I created a > simple Objective-C app that handles the event and passes it over as a HTTP > message to a server running in Pharo > (https://github.com/maenu/PharoUriScheme) The OS will even start the > application if it is not yet running. > > While the custom URI scheme approach is super powerful, it has critical > drawbacks. Any application can request to be the receiver of a URI scheme, > just as browser are for http://. Especially on mobile devices with limited > access to the OS, this opens up an attack point for malware apps that > replicate original apps that make use of schemes like facebook:// and > eavesdrop all interactions. If an original app transmits any unencrypted > secrets or user data encoded in those URIs, malware can easily intercept it > without the user noticing the leak. I guess this is the reason why many PDF > viewer just support the standard http:// and mailto:// schemes. E.g., macOS > Preview gives just an audible beep when I click on a pharo:// link, Chromes > viewer doesnât even bother giving any feedback. Only Adobe Acrobat allows > you to relax security settings to make them work (How could it be someone > else than Adobe, when itâs a security issue? ;). > > I finished basic packaging today and will continue with some READMEs and a > nearly-all-in-one distribution tomorrow, Iâll keep you posted in this > thread. > > Cheers, > Manuel > > On 2 Nov 2017, at 18:08, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > > Hi manuel > > this is super cool :) > Could you describe how you did the pdf integration? > And yes please package it :) > I want to try it. > > Stef > > On Wed, Nov 1, 2017 at 10:16 PM, Manuel Leuenberger > <leuenberger(a)inf.unibe.ch> wrote: > > Hi everyone, > > I was experimenting in the last few weeks with my take on literature > research. For me, the corpus of scientific papers form an interconnected > graph, not those plain lists and tables we keep in our bibliographies. So, > here is the first prototype that has Google Scholar integration for search, > can fetch PDFs from IEEE and ACM, extracts metadata from PDFs - all this > results in hyperlinked PDFs! > > See a demo here: https://youtu.be/EcK3Pt_WnEw > Also slides from the SCG seminar here: > http://scg.unibe.ch/download/softwarecomposition/2017-10-31-Leuenberger-ILE… > > I plan on packaging it, so that those who are interested can check it out > themselves (help wanted!). Currently, it only works on macOS. > > What do you think of my approach? Which use cases should be added? > > Cheers, > Manuel > > >

Nov. 3, 2017

Re: [Pharo-users] New Pharo Booklet

by Stephane Ducasse

Hernan I was thinking about it this night. For the coloring we can also change the template. What we can do is to copy the book template and experiment. Stef On Thu, Nov 2, 2017 at 7:57 PM, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: > Hi renaud > > thanks for the feedback. Settings is way to describe settings. > Now once you have a settings and save it you can take the setting code > and wrap it into a startUpLoader actions. > So there are really complementary. > > Ok I did not know that we change the class name. I will check because > my preferences seems to work in pharo 70. > Can you do pull requests for your suggestions so that the chapter gets > improved. > > > > > On Wed, Nov 1, 2017 at 6:34 PM, Renaud de Villemeur > <renaud.devillemeur(a)free.fr> wrote: >> Hi Stephane >> >> Your booklet is very usefull, and show stuff that can only be done by code. >> >> However, in your example, you show how to change syntax style, or enable >> freetype, which can also be done using the setting framework, so I was >> wondering, how would you position Startup Loader with the settings framework >> ? >> >> Next, you always use StartupLoader. In my recent Pharo 6.1 image, this seems >> to be replaced by class StartupPreferencesLoader. Not a big deal, I was >> able to find the correct class through the message, but quite confusing if >> you only read the booklet. >> >> I tend to use raw image, and load everything from scratch, instead of >> keeping multiple image around. So I try to clean things, or put things >> automatically at startup >> >> How to remove automatically at startup the welcome windows (probably >> specific to Pharo 6) >> World submorphs >> select: [ :sm | sm isSystemWindow and: [ sm label = ((Character value: 0) >> asString, 'Welcome')]] >> thenDo: [ :window | window delete ]. >> ] >> >> Open browser automatically on package specific to my (little) projects >> |browser| >> browser := Smalltalk tools openClassBrowser. >> browser open. >> browser ui packageSearchUpdate: 'MyPack' >> >> Cheers >> Renaud >> >> >> 2017-10-29 16:53 GMT-04:00 Stephane Ducasse <stepharo.self(a)gmail.com>: >>> >>> Hi guys >>> >>> I just published a new little booklet on Pharo tips and tricks! >>> Available at: http://books.pharo.org >>> >>> Feedback and contributions are welcome as usual. >>> >>> Stef >>> >>

Nov. 3, 2017

Re: [Pharo-users] FYI about Pharo MOOC

by Stephane Ducasse

Sanjay where did you get this file? Stef On Wed, Nov 1, 2017 at 5:26 PM, Sanjay Minni <sm(a)planage.com> wrote: > There seems to be an issue with the english dubbed version of Week 5 lesson > 4 - it has no content > C019SD-W5-S4-EN-V1.mp4 > > --- > Sanjay Minni > > On Fri, Oct 13, 2017 at 1:29 AM, Stephane Ducasse <stepharo.self(a)gmail.com> > wrote: >> >> I'm about to release the en versions. >> you can find them unofficially on http://www.stephaneducasse.eu/MOOC/ >> >> Stef >> >> On Tue, Oct 10, 2017 at 10:10 PM, Gour <gour(a)atmarama.com> wrote: >> > On Tue, 10 Oct 2017 21:31:55 +0200 >> > Stephane Ducasse >> > <stepharo.self(a)gmail.com> wrote: >> > >> > Hello Stef, >> > >> >> I will ask one guy thursday and let you know. >> > >> > Thanks a lot! >> > >> >> We will release Mooc with english voices (not mine else english >> >> natives would get an heart attack - I have what they call a sexy >> >> french accents ;) >> > >> > I did watch few of your Pharo-related presentations and, although not >> > native, >> > happily survived. :-) >> > >> > Moreover, I'd say that your English is charming! At least, one is sure >> > that the >> > real human is speaking and not some "robot" put on auto-pilot, so if the >> > new >> > Mooc is going to be the same as the current/old one, I'd prefer to >> > download >> > the current files and watched them along with *.srt subtitles? >> > >> > Iow. my point is that the accent is just one part of the talk/teaching, >> > but the >> > energy behind it is much more imporant - this is, my conviction, based >> > on my >> > own teaching experiences. >> > >> > >> > Sincerely, >> > Gour >> > >> > -- >> > From anger, complete delusion arises, and from delusion >> > bewilderment of memory. When memory is bewildered, >> > intelligence is lost, and when intelligence is lost >> > one falls down again into the material pool. >> > >> > >> > >> >

Nov. 3, 2017

Re: [Pharo-users] Territorial loading problems & alternative ways of loading data for making a choropleth map

by HernÃ¡n Morales Durand

Thank you Tudor. For Offray, I found some time and enabled the country visualizations through GT over a RTView. To access all the country names as of this map provider: TerritorialAMChartsMapsProvider readCountryNames. To access the GT-Inspector, inspect the following expression: TerritorialMapObject new mapProvider: TerritorialAMChartsMapsProvider; viewCountry: 'USA2'. Now you can access to the GT buttons: zoom, export, etc. Also if you do not want random colors for administrative 1 levels, you could setup an uniform color: TerritorialMapObject new mapProvider: (TerritorialAMChartsMapsProvider adm1ColorLevel: Color white); viewCountry: 'australia'. Cheers HernÃ¡n 2017-10-25 7:03 GMT-03:00 Tudor Girba <tudor(a)tudorgirba.com>: > You can directly use the Roassal presentation > > > gtInspectorExtension: composite > <gtInspectorPresentationOrder: 20> > composite roassal2 > title: â...' > initializeView: [ YourViewOrBuilder ]; > painting: [ :yourViewOrBuilder | â¦ ] > > Doru > >> On Oct 25, 2017, at 5:57 AM, HernÃ¡n Morales Durand <hernan.morales(a)gmail.com> wrote: >> >> 2017-10-24 15:24 GMT-03:00 Offray Vladimir Luna CÃ¡rdenas >> <offray.luna(a)mutabit.com>: >>> Gracias HernÃ¡n! >>> >> >> No problema :) >> >>> This is working fine. I would like to put the displayed map inside the >>> GT playground right panel, to get something similar to what you get when >>> you execute "RTMapLocationExample new exampleSeismOnEarth" from a >>> playground. Do you have any planned support for that? >>> >> >> I know this should be as easy as adding a >> <gtInspectorPresentationOrder: ... > pragma >> >> gtInspectorViewIn: composite >> <gtInspectorPresentationOrder: 60> >> >> composite morph >> title: 'Territorial Map'; >> display: [ view build ] >> >> but still I have to investigate how to connect a RTView or TRMorph >> with GT inspector >> >>> I could try to use some similar approach to the one I take on the Panama >>> Papers choropleth map [1][2] to export some Territorial data for >>> visualizations in a light format (120kb file example at: [3]), if you're >>> interested. >>> >>> [1] http://mutabit.com/offray/blog/en/entry/panama-papers-1 >>> [2] http://mutabit.com/repos.fossil/panama-papers/doc/tip/index.html >>> [3] http://mutabit.com/repos.fossil/panama-papers/doc/tip/territories.ston >>> >> >> Super! It would be nice to see usage in other scenarios! >> Let me know how it goes. >> >> Cheers, >> >> HernÃ¡n >> >>> Cheers, >>> >>> Offray >>> >>> >>> On 23/10/17 23:21, HernÃ¡n Morales Durand wrote: >>>> Hola Offray, >>>> >>>> I tried loading in Windows 8.1 and it loads fine using: >>>> >>>> Metacello new >>>> smalltalkhubUser: 'hernan' project: 'Territorial'; >>>> configuration: 'Territorial'; >>>> version: #bleedingEdge; >>>> load. >>>> >>>> Then load TerritorialData-AMCharts-HernanMoralesDurand.2 from the repository. >>>> >>>> I published some "fixes" related to Spec and Roassal. (I have to >>>> review them because I'm porting other packages to Pharo 6.1, but it >>>> should work for displaying maps). >>>> >>>> The following expression displays a window with Colombia departments: >>>> >>>> TerritorialAMChartsMapsProvider viewHighResCountry: 'Colombia' >>>> >>>> For a choropleth with a heatmap or centroids have a look at how I did >>>> it in PhyloclassTalk in SmalltalkHub, again I should update it to work >>>> in Pharo 6.1. >>>> >>>> Cheers, >>>> >>>> HernÃ¡n >>>> >>>> >>>> >>>> 2017-10-23 22:59 GMT-03:00 Offray Vladimir Luna CÃ¡rdenas >>>> <offray.luna(a)mutabit.com>: >>>>> Hi, >>>>> >>>>> I would like a choropleth map of Colombian departments [1]. The first >>>>> approach would be to use Territorial, but loading it is not working on >>>>> Pharo 6.1 (used on Manjaro Linux): It doesn't appears in the Catalog, >>>>> Gopher load gives me: "XMLFileException: File does does not exist: >>>>> /home/offray/Programas/Pharo/6.1a/Dev24/territorial_files/fao/fao_country_names.xml" >>>>> and loading Monticello bleeding edge gives me "FileDoesNotExist: File @ >>>>> /home/offray/Programas/Pharo/6.1a/Dev24/territorial_files/opengeocode/wc-ASCII.csv". >>>>> There is another way to make such choropleth map? >>>>> >>>>> [1] https://en.wikipedia.org/wiki/Departments_of_Colombia >>>>> >>>>> Thanks, >>>>> >>>>> Offray >>>>> >>>>> >>>>> >>> >>> >>> >> > > -- > www.tudorgirba.com > www.feenk.com > > "Problem solving efficiency grows with the abstractness level of problem understanding." > > > > >

Nov. 3, 2017

Re: [Pharo-users] UFFI and Fortran

by Todd Blanchard

Yes - the reason I've been trying to learn FFI is specifically to get TensorFlow integration. I want a better ML workbench. > On Nov 2, 2017, at 9:08 PM, Ben Coman <btc(a)openinworld.com> wrote: > > > > On Fri, Nov 3, 2017 at 11:10 AM, horrido <horrido.hobbies(a)gmail.com <mailto:horrido.hobbies@gmail.com>> wrote: > Sounds like it's possible to call into TensorFlow (Python) from Pharo. That > would give Pharo a tremendous boost for machine learning applications. > > Am I right? > > Tensorflow has a C API for FFI from other languages. > * https://www.tensorflow.org/install/install_c <https://www.tensorflow.org/install/install_c> > * https://www.tensorflow.org/extend/language_bindings <https://www.tensorflow.org/extend/language_bindings> > > cheers -ben > > > > kilon.alios wrote > > Maybe you talk about TalkFFI which aumatically wrapped C libraries for > > Pharo and i think Squeak as well but used Nativeboos, i think > > > > http://forum.world.st/TalkFFI-automatic-FFI-generation-for-Pharo-td4662239.… <http://forum.world.st/TalkFFI-automatic-FFI-generation-for-Pharo-td4662239.…> > > > > On the subject of Fortran yes you can use UFFI if Fortran code is compiled > > as a DLL (or equivelant non Windoom OSes) > > > > https://software.intel.com/en-us/node/535305 <https://software.intel.com/en-us/node/535305> > > > > Dynamic Library format are not exclusive to C for generation , many > > languages can generate them so even though UFFI is used predominatly for C > > any language that can generate a DLL without any name mangling should be > > accessible via UFFI. > > > > if DLL generation is not possible, then you can drop down to my solution > > of > > Shared Memory Mapped Files. Which means you make an executable in Frotran > > (or any other language) that contains the code you want to access and > > creates a shared memory region and you can access that region from Pharo > > via UFFI. > > > > This is how I built my CPPBridge project. Which one can use as a template > > for generating similar bridge for Fortran or any other language where the > > generation of DLLs is not practical or possible. > > > > The shared memory region can be used also as a means of communication > > additional to sharing live state, memory mapped files automatically save > > the live state so next time you open the file it behaves as you would > > expect a Pharo image to behave by restoring the live state. A means to > > extend Pharo image to include memory not managed by the VM. > > > > Because memory mapped files is mechanism of the OS Kernel and is also the > > mechanism used for the loading of dynamic libraries like DLLs there is > > also > > no loss of performance and is the fastest way of communication between > > languages, libraries and applications. > > > > So yes using Fortran code is possible via UFFI in more than one way. > > > > But in the end it should be noted here that because IPC mechanisms (common > > way of using libraries from other languages) are based on core OS > > functionality like , memory managment, sockets , pipes, etc its not hard > > to > > use libraries from any language from inside any language. > > > > So Pharo can use Fortran libraries and Fortran can use Pharo libraries, > > its > > also possible to retain live coding even when executing code written in > > another language through various means. Recently I was succesful in making > > Python into a basic live coding enviroment , meaning code that when > > changed > > in the source file it updates also existing live intances as you expect in > > Pharo. Python also supports memory mapped files so its possible to create > > a > > joined live coding enviroment and live image that containes both Pharo and > > Python bytecode. Of course without touching VM source code. > > > > Even though live state is not tricky to retain for compiled languages, > > live > > code can be a bit trickier to do but still not impossible or that hard as > > most would assume. > > > > Sky is the limit. > > > > Bottom line is that if you have a favorite library in any language you can > > use it from inside Pharo with at worst a few hundrend lines of setup code > > you can create as a library and of coure reuse next time you want to use > > again a library from another language without having to write a line of > > additional code. The technology is available is just a matter of learning > > how to use it. Especially if its a very large libraries it will be > > thousands times easier than trying to replicate the functionality in > > Pharo. > > > > > > > > On Fri, Oct 27, 2017 at 6:26 PM Sean P. DeNigris < > > > sean@ > > > > > > wrote: > > > >> Ben Coman wrote > >> > it seems to hint how to do it from Pharo UFFI. > >> > >> Slightly OT: I remember years ago, someone (Dave Mason?) demoed a library > >> which automatically created FFI wrappers for C libs. I never heard > >> anything > >> about it after that, which is sad, because I was amazed and wanted to use > >> it! > >> > >> > >> > >> ----- > >> Cheers, > >> Sean > >> -- > >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html <http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> > >> > >> > > > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html <http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html> > >

Nov. 3, 2017

Re: [Pharo-users] UFFI and Fortran

by Ben Coman

On Fri, Nov 3, 2017 at 11:10 AM, horrido <horrido.hobbies(a)gmail.com> wrote: > Sounds like it's possible to call into TensorFlow (Python) from Pharo. That > would give Pharo a tremendous boost for machine learning applications. > > Am I right? > Tensorflow has a C API for FFI from other languages. * https://www.tensorflow.org/install/install_c * https://www.tensorflow.org/extend/language_bindings cheers -ben > > kilon.alios wrote > > Maybe you talk about TalkFFI which aumatically wrapped C libraries for > > Pharo and i think Squeak as well but used Nativeboos, i think > > > > http://forum.world.st/TalkFFI-automatic-FFI-generation-for- > Pharo-td4662239.html > > > > On the subject of Fortran yes you can use UFFI if Fortran code is > compiled > > as a DLL (or equivelant non Windoom OSes) > > > > https://software.intel.com/en-us/node/535305 > > > > Dynamic Library format are not exclusive to C for generation , many > > languages can generate them so even though UFFI is used predominatly for > C > > any language that can generate a DLL without any name mangling should be > > accessible via UFFI. > > > > if DLL generation is not possible, then you can drop down to my solution > > of > > Shared Memory Mapped Files. Which means you make an executable in Frotran > > (or any other language) that contains the code you want to access and > > creates a shared memory region and you can access that region from Pharo > > via UFFI. > > > > This is how I built my CPPBridge project. Which one can use as a template > > for generating similar bridge for Fortran or any other language where the > > generation of DLLs is not practical or possible. > > > > The shared memory region can be used also as a means of communication > > additional to sharing live state, memory mapped files automatically save > > the live state so next time you open the file it behaves as you would > > expect a Pharo image to behave by restoring the live state. A means to > > extend Pharo image to include memory not managed by the VM. > > > > Because memory mapped files is mechanism of the OS Kernel and is also the > > mechanism used for the loading of dynamic libraries like DLLs there is > > also > > no loss of performance and is the fastest way of communication between > > languages, libraries and applications. > > > > So yes using Fortran code is possible via UFFI in more than one way. > > > > But in the end it should be noted here that because IPC mechanisms > (common > > way of using libraries from other languages) are based on core OS > > functionality like , memory managment, sockets , pipes, etc its not hard > > to > > use libraries from any language from inside any language. > > > > So Pharo can use Fortran libraries and Fortran can use Pharo libraries, > > its > > also possible to retain live coding even when executing code written in > > another language through various means. Recently I was succesful in > making > > Python into a basic live coding enviroment , meaning code that when > > changed > > in the source file it updates also existing live intances as you expect > in > > Pharo. Python also supports memory mapped files so its possible to create > > a > > joined live coding enviroment and live image that containes both Pharo > and > > Python bytecode. Of course without touching VM source code. > > > > Even though live state is not tricky to retain for compiled languages, > > live > > code can be a bit trickier to do but still not impossible or that hard as > > most would assume. > > > > Sky is the limit. > > > > Bottom line is that if you have a favorite library in any language you > can > > use it from inside Pharo with at worst a few hundrend lines of setup code > > you can create as a library and of coure reuse next time you want to use > > again a library from another language without having to write a line of > > additional code. The technology is available is just a matter of learning > > how to use it. Especially if its a very large libraries it will be > > thousands times easier than trying to replicate the functionality in > > Pharo. > > > > > > > > On Fri, Oct 27, 2017 at 6:26 PM Sean P. DeNigris < > > > sean@ > > > > > > wrote: > > > >> Ben Coman wrote > >> > it seems to hint how to do it from Pharo UFFI. > >> > >> Slightly OT: I remember years ago, someone (Dave Mason?) demoed a > library > >> which automatically created FFI wrappers for C libs. I never heard > >> anything > >> about it after that, which is sad, because I was amazed and wanted to > use > >> it! > >> > >> > >> > >> ----- > >> Cheers, > >> Sean > >> -- > >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > >> > >> > > > > > > -- > Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html > >

Nov. 3, 2017

Re: [Pharo-users] [squeak-dev] [Pharo-dev] [ANNOUNCE] ParrotTalk release/design change considerations

by Ben Coman

On Fri, Nov 3, 2017 at 4:06 AM, henry <henry(a)callistohouse.club> wrote: > I have a question about my derivation of a MAC key. It is very complicated > and I am considering ways to simplify for use by potential language > implementations. Here is the current derivation in Squeak/Pharo. Should I > just use the sharedKey as the SHA1HMAC key or some simplified hash of the > sharedKey, without all the pad hashings I am doing? What do you think? I > want to hear from you. > Hi Henry, I'd guess not many people here are *really* competent enough in crypto to confidently advise you. I know I'm not. I only know enough to know crypto is *hard*. * https://www.happybearsoftware.com/you-are-dangerously-bad- at-cryptography.html * https://www.schneier.com/blog/archives/2011/04/schneiers_law.html * https://www.schneier.com/essays/archives/1997/01/why_cryptography_is.html I hope that doesn't discourage you, and I'm interested to hear a more useful reply from anyone else. cheers -ben P.S. I just bumped into what looks like an interesting intellectual exercise... http://cryptopals.com/ > > makeHMAC > > | sharedKey hashPadder macKey | > sharedKey := diffieHellman sharedKeyPadPositiveByteArray. > hashPadder := self class. > macKey := MD5 hashMessage: ( > (hashPadder hash: sharedKey pad: 16rCC), > (hashPadder hash: sharedKey pad: 16rBB), > (hashPadder hash: sharedKey pad: 16rAA), > (hashPadder hash: sharedKey pad: 16r99)). > macKey := macKey, (MD5 hashMessage: ( > (hashPadder hash: sharedKey pad: 16r88), > (hashPadder hash: sharedKey pad: 16r77), > (hashPadder hash: sharedKey pad: 16r66), > (hashPadder hash: sharedKey pad: 16r55))). > macKey := macKey, (MD5 hashMessage: ( > (hashPadder hash: sharedKey pad: 16r44), > (hashPadder hash: sharedKey pad: 16r33), > (hashPadder hash: sharedKey pad: 16r22), > (hashPadder hash: sharedKey pad: 16r11))). > ^ SHA1 new hmac key: macKey. > > Would a better way forwards is simply this? > > makeHMAC > > macKey := diffieHellman sharedKeyPadPositiveByteArray. > ^ SHA1 new hmac key: macKey. > > > Thank you. > > - HH > > > -------- Original Message -------- > Subject: Re: [Pharo-users] [Pharo-dev] [ANNOUNCE] ParrotTalk > release/design change considerations > Local Time: November 2, 2017 3:50 PM > UTC Time: November 2, 2017 7:50 PM > From: henry(a)callistohouse.club > To: henry <henry(a)callistohouse.club> > Squeak-dev <squeak-dev(a)lists.squeakfoundation.org>, Pharo-users list < > pharo-users(a)lists.pharo.org>, Pharo Development List < > pharo-dev(a)lists.pharo.org> > > I have developed the ParrotTalk Protocol, documented in part here[1], > while I have two implementations: 1 in Squeak/Pharo [2 a/b] and the other > in Java [3 a/b]. Particulars of some decisions are absent from the > specification: namely the MAC key and ivSequence derivations, as well as > constrained traffic signing; they are in the implementations. I will update > the 3.6 slideshow to specify these items. > > Thank you. > > - HH > > [1] - ParrotTalk Protocol : http://jmp.sh/OqlYpyg > > Squeak/Pharo implementation > [2 a] - http://www.squeaksource.com/Cryptography/Cryptography-Henr > yHouse.113.mcz > [2 b] - http://www.squeaksource.com/Cryptography/ParrotTalk-HenryH > ouse.9.mcz > > Java implementation > [3 a] - https://github.com/ZiroZimbarra/ASN1 > [3 b] - https://github.com/ZiroZimbarra/ParrotTalk > > - HH > > > -------- Original Message -------- > Subject: Re: [Pharo-users] [Pharo-dev] [ANNOUNCE] ParrotTalk > release/design change considerations > Local Time: November 2, 2017 3:43 PM > UTC Time: November 2, 2017 7:43 PM > From: henry(a)callistohouse.club > To: Squeak-dev <squeak-dev(a)lists.squeakfoundation.org> > Pharo-users list <pharo-users(a)lists.pharo.org>, Pharo Development List < > pharo-dev(a)lists.pharo.org> > > More celebration requested. I got encrypted data passing between > Squeak/Pharo and Java: AES with CBC and IVs and SHA1 HMAC. It works! > NSA-proof crypto rocking Squeak, Pharo & Java! > > *Celebrates again, drinks for everyone* > > > - HH > > > -------- Original Message -------- > Subject: Re: [Pharo-users] [Pharo-dev] [ANNOUNCE] ParrotTalk > release/design change considerations > Local Time: October 31, 2017 1:54 PM > UTC Time: October 31, 2017 5:54 PM > From: henry(a)callistohouse.club > To: henry <henry(a)callistohouse.club> > Ben Coman <btc(a)openinworld.com>, Squeak-dev <squeak-dev(a)lists.squeakfounda > tion.org>, Pharo-users list <pharo-users(a)lists.pharo.org>, Pharo > Development List <pharo-dev(a)lists.pharo.org> > > I wanted to share my good news, though I have not yet gotten Pharo, Squeak > and Java talking. What I was able to accomplish, after finishing the port > of Cryptography's ASN1Types, Module and Streams is to get Java's rendezvous > and key exchange working. After I got that connecting, I looked at the > ciphers, ivSequences and MAC and got that working. In Java, I am now able > to encrypt and decrypt "hello world". > > *Celebrates* > > - HH > > > -------- Original Message -------- > Subject: Re: [Pharo-users] [Pharo-dev] [ANNOUNCE] ParrotTalk > release/design change considerations > Local Time: October 25, 2017 12:00 PM > UTC Time: October 25, 2017 4:00 PM > From: henry(a)callistohouse.club > To: Ben Coman <btc(a)openinworld.com>, Squeak-dev < > squeak-dev(a)lists.squeakfoundation.org>, Pharo-users list < > pharo-users(a)lists.pharo.org> > Pharo Development List <pharo-dev(a)lists.pharo.org> > > I disabled vatId authorization in version 3, located in > SessionOperations>>processIWant | SessionOperations>>processIAm:. > > I was asked to disseminate my news to Pharo Users, hello there. I was > asked to describe ParrotTalk well, provide use cases and adopters of its > use. Alright, I will give an attempt. > > ParrotTalk is an encrypted connection framework. Currently allowing > anonymous 2048-bit key negotiation to establish user-provided encryption > cipher and user-provided encoding and decoding, both through a provided > SessionAgentMap to a starting SessionAgent server. Please look in the test > case ThunkHelloWorldTest for building these maps and running a connection > iwth data passing after encryption is established. There is a 4-way > negotiation, from ProtocolOffered/Accepted to Go/GoToo. In using RSA 2048 > signature validation and DH 2048 primes to establish the key used within > the selected Cipher. The Cipher and Encoder are selected by name through > the negotiation protocol. Currently three Ciphers are selectable, AESede, > DESede, and DES. There are two encoders tested, asn1der, and Bytes. This > protocol is described here, in this document. > > https://github.com/ZiroZimbarra/callistohouse/blob/master/ > docs/ParrotTalkFrameDesign-3.5.pdf > > For as to use cases, this encrypted connection has no third party, > man-in-the-middle situation by not using Certificates. As such, this is a > tight implementation of NSA-proof encryption without explicit authorization > beyond knowledge of a host:port. The use cases involve any communication > desired to be encrypted with such high encryption. The support will last my > lifetime, so we have a settled solution, here in the third version, > provided here. It requires version 111 of Cryptography, as a prerequisite. > Both run on Squeak and Pharo. > > http://www.squeaksource.com/Cryptography/Cryptography-zzz.111.mcz > http://www.squeaksource.com/Cryptography/ParrotTalk-HenryHouse.3.mcz > > The current use is with my hubbub system, a promise-based distributed > object implementation. I am working to bring ParrotTalk to Java and allow > hubbub to operate interdependently between Squeak, Pharo, Java and any > other languages which can support ParrotTalk and STON. My latest efforts > with hubbub are to bring STON as the Layer 6 encoding. Hubbub depends on > eLinda. > > http://www.squeaksource.com/Cryptography/elinda-HenryHouse.14.mcz > http://www.squeaksource.com/Oceanside/hubbub-HenryHouse.38.mcz > > Hubbub currently fails to load correctly in Pharo. In Squeak the two > ParrotRemoteTestCases: LookupTestCase and OperationalTestCase really screw > everything up, do not save an image after running. I never run these tests > until I get a better handle of serialization. Currently, do to a shift in > progress to STON, the serialization tests fail, just the way we like it. > Fix the tests then the system starts to work right. > > The shift away from vatId authorization should not adversely affect > hubbub, we hope! When STON starts to work, and I think I need to turn off > jsonMode, then traffic between vats can start to be analyzed and figure out > how resolving is working with redirectors invoked remotely. > > - HH > > > -------- Original Message -------- > Subject: Re: [Pharo-dev] [ANNOUNCE] ParrotTalk release/design change > considerations > Local Time: October 25, 2017 12:43 AM > UTC Time: October 25, 2017 4:43 AM > From: btc(a)openinworld.com > To: henry <henry(a)callistohouse.club>, Pharo Development List < > pharo-dev(a)lists.pharo.org> > Stephane Ducasse <stepharo.self(a)gmail.com> > > > If I google... parrottalk protocol > nothing seems related. So what is ParrotTalk? > > P.S... since this is not "part of" pharo, [pharo-users] would be a better > place for discussion than [pharo-dev]. > Could you re-announce there, with some description of use cases and > who/where the protocol is used? > > cheers -ben > > On Wed, Oct 25, 2017 at 1:42 AM, henry <henry(a)callistohouse.club> wrote: > >> Please excuse all the low-level detail, if inappropriate to the >> discussion. The protocol changes are here specified. >> >> >> - HH >> >> >> On Tue, Oct 24, 2017 at 13:40, henry <henry(a)callistohouse.club> wrote: >> >> In order to bring ParrotTalk-3.6 support, alongside historical 3.5 >> support, the frame header stays the same and the processing of the >> ProtocolOffered would select "ParrotTalk-3.6" to use the compact protocol >> of >> >> - ProtocolOffered { offered, preferred } >> - ProtocolAccepted { accepted } >> - IWant|GiveInfo|ReplyInfo { vatId, domain, publicKey, cryptoProtocols, >> dataEncoders } >> - IAm|ReplyInfo { vatId, domain, publicKey, cryptoProtocol, dataEncoder, >> dhParam } >> - Go { cryptoProtocol, dataEncoder, dhParam, signature } >> - GoToo { signature } >> >> >> Using eLinda : >> >> http://www.squeaksource.com/Cryptography/elinda-HenryHouse.14.mcz >> >> Above the FrameBuffer, below the SessionOperations for each version of >> the protocol, an ELindaSession could be inserted. This session think would >> have the stacks for each protocol version SessionOperation registered by >> protocol in a Tuple, for eventual callback. When the ProtocolOffered comes >> in we publish the tuple frame, with set selected version, to route to the >> correct SessionOperation to support each version of the protocol. >> >> With a non-specific vatId required, anonymous connections would be >> supported. >> >> - HH >> >> >> On Tue, Oct 24, 2017 at 12:52, Stephane Ducasse <stepharo.self(a)gmail.com> >> wrote: >> >> Hi henry thanks for this announce. Can you tell where such protocols are >> used? Stef On Tue, Oct 24, 2017 at 6:33 PM, henry wrote: > Hi all, > > I >> am happy to announce the release of version 3.5 of ParrotTalk, for Squeak > >> and Pharo, found here: > > http://www.squeaksource.com/Cr >> yptography/ParrotTalk-zzz.2.mcz > > It follows this specification: > >> https://github.com/ZiroZimbarra/callistohouse/blob/master/do >> cs/ParrotTalkFrameDesign-3.5.pdf > > One item of note, in version 3.5, >> the system connecting to a server, sending > the IWant msg, must know the >> vatId of the system being connected to. I am > considering changing this to >> version 3.6 by removing one round-trip in > messaging. Therefore, these >> messages would be combined: IWant/GiveInfo, > IAm/ReplyInfo. I will keep >> ProtocolOffered and ProtocolAccepted to allow > eLindaSession to support >> both versions: 3.5 and 3.6. > > Thoughts please? > > - HH >> >> > > > > > > > >

Nov. 3, 2017