- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] NativeBoost libraries hosting

by HernÃ¡n Morales Durand

2014-12-21 19:02 GMT-03:00 Tommaso Dal Sasso <tommaso.dalsasso(a)gmail.com>: > What about writing some glue classes that check for the presence of the > libraries and download them automatically if the libraries are not present? > In this way, the developer would only have to specify a download url (or > at most one for each supported platform). > Yes, actually I am using that "mechanism" - only 6 methods - for several of my projects (see ConfigurationOfBioSmalltalk in 'doits' category). I think it could be included in Metacello. The problem is getting a host which could be - Reliable = no downtime (Dropbox is ok, GitHub is ok) - Allows to link multiple accounts (Dropbox ??, GitHub ??) - Custom URLs (DropBox is ok, GitHub is ok) - Easy to link (Dropbox somewhat, GitHub ??) - Easy to update (Dropbox is ok, GitHub not) HernÃ¡n This sort of "caching" mechanism could be included in NativeBoost to reduce > the problem of hosting the libraries and would also reduce the application > size, since it would only download the binaries of the current platform. > > Tommaso > > > > On 20/12/14 10:20, Ben Coman wrote: > >> Esteban Lorenzano wrote: >> >>> I donât think so. We cannot pollute pharo file server with specific >>> libraries :( >>> >>> we could in the future set up a service for that, but then we will need >>> to take into account licenses, etc. my recommendation is that you do not >>> include the library but instructions on where they can download/build the >>> proper libraries. >>> Esteban >>> >>> On 19 Dec 2014, at 12:24, Blondeau Vincent <vincent.blondeau@worldline. >>>> com <mailto:vincent.blondeau@worldline.com>> wrote: >>>> >>>> Hi, >>>> >>>> Some of us are developing some applications by using NativeBoost, which >>>> is a great tool for Pharo. >>>> But when we want to distribute our application, we have to give the >>>> libraries with it. >>>> We can integrate the library in the package, give a link to download an >>>> application and copy the libraries near the Pharo VM, or put it on dropbox, >>>> but it seems that are not good solutions. >>>> >>>> Maybe we should have a hosting platform for these libraries? >>>> Does http://smalltalkhub.com <http://smalltalkhub.com/> or >>>> http://files.pharo.org/ can do the hosting? >>>> >>>> Cheers, >>>> >>>> Vincent >>>> >>> >> This is an interesting question I might have to deal with some time, so I >> had a look around for options. (As an aside, I came across this >> interesting article "How I moved my websites to DropBox and Github"... >> http://alexcican.com/post/guide-hosting-website-dropbox-github/ ) >> >> That makes me wonder what is wrong with using dropbox? You might use it >> with a custom domain name so that later you can move services transparently >> to users. >> http://hostlater.com/2014/04/dropbox-custom-domain-name-url-2246/ >> >> Github recommends using either "Releases" or Amazon S3 & CloudFront. >> Indeed it seems that "Releases" itself is built on top of CloudFront. >> https://help.github.com/articles/distributing-large-binaries/ >> https://github.com/blog/1547-release-your-software >> >> I followed this instruction as a trial... >> https://help.github.com/articles/creating-releases/ >> and found it quite straightforward - see... >> https://github.com/bencoman/Hello-World/releases >> (I just used a jpg as an example) >> >> cheers -ben >> >> >> > >

Dec. 21, 2014

Re: [Pharo-dev] Cog in the cloud

by Ben Coman

Eliot Miranda wrote: > Hi Ben, > > > On Dec 19, 2014, at 8:08 PM, Ben Coman <btc(a)openInWorld.com> wrote: > >> Just some thoughts that arose as I skipped along the web... >> >> With the idea of using Pharo in the cloud, I was thinking of how PharoNOS[1] is basing off top of the Linux kernel, but the examples indicate its operating in User Mode. So I was wondering about performance being greater[2] if the CogVM ran in Kernel Mode. However maybe this could get tangled in the GPL license of the kernel. >> >> Now I believe FreeBSD/NetBSD license is compatible with COG's MIT license, so that seems a better option for eliminating layers of the operating system. They both[3][4] have pre-built Amazon Machine Images to run on EC2, as well as instructions[5] to build your own AMI. So potentially we could build an AMI with the CogVM linked to the FreeBSDKernel operating in Kernel Mode, with no User Mode. The lack of direct access to memory from the Image, and probably single application focus should be sufficient security to forgo User Mode, and run faster. >> >> But we could go a step further. Runing in the cloud relies heavily on virtualisation, and for performance[6], most likely on the OS's paravirtualisation interface - to Xen for example. So from Cog, why not interface direct to the Xen FrontEndDriver[7] and eliminate the DomU operating system all together. What I understand from [6] is that Xen's paravirtualisation hooks makes it much easier to boot, than booting on the bare metal of a fully-virtualised system. So the former seems more achievable than the latter (which was the only option in years past), and also be more portable as technology evolves, to any platform Xen runs on. >> >> Squeak/Pharo on Cog-Xen could be a good choice for a Cloud Operating System[8]. Maybe a good student research project? > > This sounds really cool, but could you analyse a bit further? > > - what key economic and managerial benefits accrue from this configuration? I'll put some more thought into this, but just off the top of my head (so it turned out more technical than economic/managerial): * Eliminates performance overhead of system call context switching. Although this is replaced by hypercalls, it removes one layer on hypervisor systems, which plausibly is going to be EVERYTHING in the future. * Eliminate performance overhead of OS networking stack http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/ * Data from virtual devices is event driven. * More easily portable to different ARM platforms. Xen will present a common interface to hide platform idiosyncrasies. --> More widespread embedded use Cog. http://www.slideshare.net/xen_com_mgr/free-rtos-xensummit * Run databases in their own domain. * Isolate plugin memory space. Rather than linking plugins into Cog, potentially threatening its stability, run them in a separate domain and use high-speed inter-domain shared memory. http://www.cs.ucsb.edu/~nanli/projects/cs270.pdf * Have a cloud community of thousands** of Images running each in their own domain, communicating with high-speed inter-domain shared memory. ** presumptive figure - would need to check scaling ability * Data sharing on high density hosting providers. Not sure if its there yet, but may get Copy-On-Write http://www-archive.xenproject.org/files/xensummit_fall07/18_GregorMilos.pdf * In very large Images, which Spur will allow, perhaps a better way to snapshot an Image is via virtual machine snapshotting using Copy-On. http://www.cercs.gatech.edu/tech-reports/tr2010/git-cercs-10-05.pdf * Rather than convert the Cog/Image to be multi-threaded, could have many grid nodes running their own domain, communicating with high speed shared memory. http://www.cs.ucsb.edu/~nanli/projects/cs270.pdf http://osnet.cs.binghamton.edu/publications/hines07memx.pdf * AMD's hardware virtualisation eliminated a Ring and also segmented memory protection used by virtualisation to separate address spaces, so 64-bit AMD can be slower. Eliminating the OS kernel layer probably bypasses this limitation since there would be only two levels - Hypervisor & Cog. https://lse.epita.fr/data/2011-lse-summer-week/xen.pdf * Its where all the cool kids hang out --> marketing opportunity > > - what infrastructure is missing? (e.g. what key drivers need implementing?) I'll need to understand Cog more. As a start, here are some porting and API docs... http://events.linuxfoundation.org/sites/events/files/slides/Porting%20FreeB… http://oss.org.cn/ossdocs/server_storage/xen/interface/interface.html http://www.informit.com/articles/article.aspx?p=1160234&seqNum=4 https://lse.epita.fr/data/2011-lse-summer-week/xen.pdf https://github.com/cloudius-systems/osv/blob/master/drivers/xenfront-xenbus… cheers -ben >> >> [1] http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos >> [2] http://blog.codinghorror.com/understanding-user-and-kernel-mode/ >> [3] http://www.daemonology.net/freebsd-on-ec2/ >> [4] http://wiki.netbsd.org/amazon_ec2/amis/ >> [5] http://wiki.netbsd.org/amazon_ec2/build_your_own_ami/ >> [6] http://wiki.xen.org/wiki/Virtualization_Spectrum >> [7] http://wiki.xen.org/wiki/FrontendDriver >> [8] http://wiki.xen.org/wiki/Cloud_Operating_Systems > > > Eliot (phone) >

Dec. 21, 2014

Re: [Pharo-dev] NativeBoost libraries hosting

by Tommaso Dal Sasso

What about writing some glue classes that check for the presence of the libraries and download them automatically if the libraries are not present? In this way, the developer would only have to specify a download url (or at most one for each supported platform). This sort of "caching" mechanism could be included in NativeBoost to reduce the problem of hosting the libraries and would also reduce the application size, since it would only download the binaries of the current platform. Tommaso On 20/12/14 10:20, Ben Coman wrote: > Esteban Lorenzano wrote: >> I donât think so. We cannot pollute pharo file server with specific >> libraries :( >> >> we could in the future set up a service for that, but then we will >> need to take into account licenses, etc. my recommendation is that >> you do not include the library but instructions on where they can >> download/build the proper libraries. >> Esteban >> >>> On 19 Dec 2014, at 12:24, Blondeau Vincent >>> <vincent.blondeau(a)worldline.com >>> <mailto:vincent.blondeau@worldline.com>> wrote: >>> >>> Hi, >>> >>> Some of us are developing some applications by using NativeBoost, >>> which is a great tool for Pharo. >>> But when we want to distribute our application, we have to give the >>> libraries with it. >>> We can integrate the library in the package, give a link to download >>> an application and copy the libraries near the Pharo VM, or put it >>> on dropbox, but it seems that are not good solutions. >>> >>> Maybe we should have a hosting platform for these libraries? >>> Does http://smalltalkhub.com <http://smalltalkhub.com/> or >>> http://files.pharo.org/ can do the hosting? >>> >>> Cheers, >>> >>> Vincent > > This is an interesting question I might have to deal with some time, > so I had a look around for options. (As an aside, I came across this > interesting article "How I moved my websites to DropBox and Github"... > http://alexcican.com/post/guide-hosting-website-dropbox-github/ ) > > That makes me wonder what is wrong with using dropbox? You might use > it with a custom domain name so that later you can move services > transparently to users. > http://hostlater.com/2014/04/dropbox-custom-domain-name-url-2246/ > > Github recommends using either "Releases" or Amazon S3 & CloudFront. > Indeed it seems that "Releases" itself is built on top of CloudFront. > https://help.github.com/articles/distributing-large-binaries/ > https://github.com/blog/1547-release-your-software > > I followed this instruction as a trial... > https://help.github.com/articles/creating-releases/ > and found it quite straightforward - see... > https://github.com/bencoman/Hello-World/releases > (I just used a jpg as an example) > > cheers -ben > >

Dec. 21, 2014

Re: [Pharo-dev] Cog in the cloud

by Eliot Miranda

Hi Ben, On Dec 19, 2014, at 8:08 PM, Ben Coman <btc(a)openInWorld.com> wrote: > > Just some thoughts that arose as I skipped along the web... > > With the idea of using Pharo in the cloud, I was thinking of how PharoNOS[1] is basing off top of the Linux kernel, but the examples indicate its operating in User Mode. So I was wondering about performance being greater[2] if the CogVM ran in Kernel Mode. However maybe this could get tangled in the GPL license of the kernel. > > Now I believe FreeBSD/NetBSD license is compatible with COG's MIT license, so that seems a better option for eliminating layers of the operating system. They both[3][4] have pre-built Amazon Machine Images to run on EC2, as well as instructions[5] to build your own AMI. So potentially we could build an AMI with the CogVM linked to the FreeBSDKernel operating in Kernel Mode, with no User Mode. The lack of direct access to memory from the Image, and probably single application focus should be sufficient security to forgo User Mode, and run faster. > > But we could go a step further. Runing in the cloud relies heavily on virtualisation, and for performance[6], most likely on the OS's paravirtualisation interface - to Xen for example. So from Cog, why not interface direct to the Xen FrontEndDriver[7] and eliminate the DomU operating system all together. What I understand from [6] is that Xen's paravirtualisation hooks makes it much easier to boot, than booting on the bare metal of a fully-virtualised system. So the former seems more achievable than the latter (which was the only option in years past), and also be more portable as technology evolves, to any platform Xen runs on. > > Squeak/Pharo on Cog-Xen could be a good choice for a Cloud Operating System[8]. Maybe a good student research project? This sounds really cool, but could you analyse a bit further? - what key economic and managerial benefits accrue from this configuration? - what infrastructure is missing? (e.g. what key drivers need implementing?) > cheers -ben > > [1] http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos > [2] http://blog.codinghorror.com/understanding-user-and-kernel-mode/ > [3] http://www.daemonology.net/freebsd-on-ec2/ > [4] http://wiki.netbsd.org/amazon_ec2/amis/ > [5] http://wiki.netbsd.org/amazon_ec2/build_your_own_ami/ > [6] http://wiki.xen.org/wiki/Virtualization_Spectrum > [7] http://wiki.xen.org/wiki/FrontendDriver > [8] http://wiki.xen.org/wiki/Cloud_Operating_Systems Eliot (phone)

Dec. 21, 2014

Re: [Pharo-dev] Cog in the cloud

by Michael J. Forster

Thanks, Ben! -Mike -- View this message in context: http://forum.world.st/Cog-in-the-cloud-tp4796514p4796597.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Dec. 21, 2014

Re: [Pharo-dev] Cog in the cloud

by Ben Coman

Michael J. Forster wrote: > Having spent some time with Erlang on Xen and OpenMirage, I'm convinced that > Pharo/Cog on Xen is something I must do. However, I still have much to learn > about the Squeak/Cog VM before I could dream of porting it. > > Cheers! > -Mike Great to hear your enthusiasm. I only recently built my first VM, and discovered it was much easier than I thought. Have a go with these instructions... https://github.com/pharo-project/pharo-vm Even if your not up for modifying the code yet, its a nice buzz to get a VM build under the belt. One tip... in generator.image, review implementors of #initializePrimitiveTable to see which primitive numbers map to which primitive methods. cheers -ben

Dec. 21, 2014

Re: [Pharo-dev] Cog in the cloud

by Michael J. Forster

Having spent some time with Erlang on Xen and OpenMirage, I'm convinced that Pharo/Cog on Xen is something I must do. However, I still have much to learn about the Squeak/Cog VM before I could dream of porting it. Cheers! -Mike -- View this message in context: http://forum.world.st/Cog-in-the-cloud-tp4796514p4796591.html Sent from the Pharo Smalltalk Developers mailing list archive at Nabble.com.

Dec. 21, 2014

Re: [Pharo-dev] outputing to PharoDebug.log

by Sven Van Caekenberghe

Hi Ben, See SmalltalkImage>>#logDuring: or #openLog I'm sure you'll find your way from there on. Sven > On 21 Dec 2014, at 11:58, Ben Coman <btc(a)openInWorld.com> wrote: > > > What is the way to output to PharoDebug.log from inside the Image. > > Use case: Should the Delay timer event loop fail, I want to log this. > > cheers, Ben

Dec. 21, 2014

outputing to PharoDebug.log

by Ben Coman

What is the way to output to PharoDebug.log from inside the Image. Use case: Should the Delay timer event loop fail, I want to log this. cheers, Ben

Dec. 21, 2014

Re: [Pharo-dev] running MS Surface Pro 3 - test failures

by stepharo

I suggest using Pharo40 because you may try to fix issues that have already been fixed. Le 19/12/14 20:54, sheridan(a)sheridan-mahoney.com a Ã©crit : > Hi all, > > I ran test case coverage using my MS Surface Pro 3 with the following > specs: > Processor: Intel(R)Core(TM) i5-4300U CPU @ 1.90 GHz 2.50 GHz > Installed RAM : 8 GB > System type: 64-bit operating system, x64-based processor > Windows Edition: Windows 8.1 Pro > > I recieved 11 test failures and 2 errors. > > Here's a snip attached of the failing tests. I hope it makes it > through... > Otherwise I'll resend with the info. > > I was wondering if each failing test requires a bug report, or is this > not a supported machine? > > Cheers, > Sheridan

Dec. 21, 2014