January 2016 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] Seeing class method template when there is no protocol

by Cyril Ferlicot D.

Le 17/01/2016 12:42, stepharo a Ã©crit : > HI > > yesterday during my exercise session a student was puzzled because > he could not see the method template on the class side when there is not > protocol. > I had to create first a protocol. > Hi, Nautilus should always have a protocol on class side. If there is none, there is the "no messages" one. > I do not have the solution because I understand why it is like that. > Because Nautilus displays > the class definition first and when we click on protocol the template. > But I would be interested to see if there is an alternative. > > Stef > -- Cyril Ferlicot http://www.synectique.eu 165 Avenue Bretagne Lille 59000 France

Jan. 17, 2016

Re: [Pharo-dev] gtdebugger in pharo 5.0

by Cyril Ferlicot D.

Le 17/01/2016 15:51, Eliot Miranda a Ã©crit : > > > I have to say that the column names are entirely redundant. They're > obvious from the information presented (unless empty when no stack frame > is selected). They eat vertical screen real estate and are one more set > of strings to localize. Nuke em. > > _,,,^..^,,,_ (phone) > + 1 -- Cyril Ferlicot http://www.synectique.eu 165 Avenue Bretagne Lille 59000 France

Jan. 17, 2016

Re: [Pharo-dev] Why there is no setting to disable QA?

by stepharo

We have also instance variables not written or read class comment is empty And I find that disruptive for a student starting with Pharo. Le 17/1/16 12:33, Yuriy Tymchuk a Ã©crit : > You can simply do `QANautilusPlugin uninstall`. I have not figured out > how to put it in the prefs. > > Now about the false positivesâ¦ I donât know what you are talking > about, but I suspect that it is about something like âclass not being > usedâ. Now if it is not being used itâs not a false positive, itâs > just a useless information. What we can do, is put this rules into > âinformationâ category and disable it in settings. > > Cheers. > Uko > >> On 17 Jan 2016, at 11:47, stepharo <stepharo(a)free.fr >> <mailto:stepharo@free.fr>> wrote: >> >> Hi nicolai >> >> Le 17/1/16 11:29, Nicolai Hess a Ã©crit : >>> >>> Hi >>> >>> Why there is no setting for that? >>> >>> Stef >>> >>> >>> >>> You can remove the Nautilus plugin (Nautilus window menu -> plugin >>> manager) >> Yes I saw looking around. Now I have to dive into it to script it so >> that I avoid to be forced as today to redo >> my screenshots (to avoid all the false positive on class creation). >> Because else >>> For the spotter and inspector integration, there is a setting >>> (default "disabled") >>> >> >

Jan. 17, 2016

Re: [Pharo-dev] gtdebugger in pharo 5.0

by Tudor Girba

Hi, > On Jan 17, 2016, at 3:51 PM, Eliot Miranda <eliot.miranda(a)gmail.com> wrote: > > > > On Jan 17, 2016, at 12:05 AM, Tudor Girba <tudor(a)tudorgirba.com> wrote: > >> Hi, >> >> Indeed. See below how it looks like in the latest development version. >> >> We have not yet integrate this one because we still need little changes in FastTable to be able to resize columns and not make the column names clickable. > > I have to say that the column names are entirely redundant. They're obvious from the information presented (unless empty when no stack frame is selected). They eat vertical screen real estate and are one more set of strings to localize. Nuke em. We are looking at that option as well. However, we still need a way to resize the columns, especially when we have large class or method names. We need to play a bit here. Cheers, Doru > _,,,^..^,,,_ (phone) > >> >> >> <table-stack-details.png> >> >> Cheers, >> Doru >> >> >>> On Jan 17, 2016, at 8:07 AM, Ben Coman <btc(a)openInWorld.com> wrote: >>> >>> On Sun, Jan 17, 2016 at 9:48 AM, Cyril Ferlicot D. >>> <cyril.ferlicot(a)gmail.com> wrote: >>>> >>>> And more, it make the debugger "ventilated" (I don't know if the term >>>> work in English too). I mean that with more space between the classes >>>> and the methods it's better for the eye of the user. >>> >>> Never heard the term used that way in graphic design, but it makes >>> perfect sense. >>> cheers -ben >>> >> >> -- >> www.tudorgirba.com >> www.feenk.com >> >> "If you can't say why something is relevant, >> it probably isn't." >> -- www.tudorgirba.com www.feenk.com "Be rather willing to give than demanding to get."

Jan. 17, 2016

Re: [Pharo-dev] Seeing class method template when there is no protocol

by Gabriel Cotelli

Maybe add a "New Method" button somewhere that changes the class definition with a method template ? On Sun, Jan 17, 2016 at 8:42 AM, stepharo <stepharo(a)free.fr> wrote: > HI > > yesterday during my exercise session a student was puzzled because > he could not see the method template on the class side when there is not > protocol. > I had to create first a protocol. > > I do not have the solution because I understand why it is like that. > Because Nautilus displays > the class definition first and when we click on protocol the template. > But I would be interested to see if there is an alternative. > > Stef > >

Jan. 17, 2016

Re: [Pharo-dev] gtdebugger in pharo 5.0

by Eliot Miranda

> On Jan 17, 2016, at 12:05 AM, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > Hi, > > Indeed. See below how it looks like in the latest development version. > > We have not yet integrate this one because we still need little changes in FastTable to be able to resize columns and not make the column names clickable. I have to say that the column names are entirely redundant. They're obvious from the information presented (unless empty when no stack frame is selected). They eat vertical screen real estate and are one more set of strings to localize. Nuke em. _,,,^..^,,,_ (phone) > > > <table-stack-details.png> > > Cheers, > Doru > > >> On Jan 17, 2016, at 8:07 AM, Ben Coman <btc(a)openInWorld.com> wrote: >> >> On Sun, Jan 17, 2016 at 9:48 AM, Cyril Ferlicot D. >> <cyril.ferlicot(a)gmail.com> wrote: >>> >>> And more, it make the debugger "ventilated" (I don't know if the term >>> work in English too). I mean that with more space between the classes >>> and the methods it's better for the eye of the user. >> >> Never heard the term used that way in graphic design, but it makes >> perfect sense. >> cheers -ben > > -- > www.tudorgirba.com > www.feenk.com > > "If you can't say why something is relevant, > it probably isn't." >

Jan. 17, 2016

Re: [Pharo-dev] [Pharo-project] [Vm-dev] Re: Can OSProcess functionality be implemented using FFI instead of plugin?

by Eliot Miranda

Hi Thierry, > On Jan 17, 2016, at 6:10 AM, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > Hi Eliot, > > I still don't understand why this platform-specific data (external datatypes sizes, constants values) has to be written as a json file and not as Smalltalk code ? It can be any notation that is easy to parse. One could use pure Smalltalk, json, ston, xml. I thought ston /was/ a form of Smalltalk aimed at representing key/value pairs, in which case it seems ideal. What are the objectives for this notation? My take (feel free to suggest a better set) is: a) be quick to parse since these files are parsed on start-up when the platform is different from that the image was snapshot red on b) be human readable. The flexibility of being able to version, edit, search and diff these files seems useful c) be reasonably straight-forward to generate from C. But hey, it's only code :-) > > If I write an interface to an external lib, I'd like to benefit from nice access to Smalltalk code to, say, test versions of the external libraries, set external types based on system information, etc, etc... instead of having to deal with all those various json files outside my normal workflow. > > Okay, I can write smalltalk code which runs those C tests programs, extract their output, model them in smalltalk and generate the jsons from there... but, then, I really don't understand why I have to write those jsons then every user loads those jsons instead of just running that setup code I have in my external library smalltalk support package ? You /don't/ have to write them ever. They are written for you by the FFI shared pool. So you're suggesting the C program output a Smalltalk method that looks like:e.g. initializeMYSQLMac32 MYSQL_ENABLE_CLEARTEXT_PLUGIN := 123. MYSQL_INIT_COMMAND := 234. MYSQL_OPT_BIND := 345 So the trade off you're discussing is whether it is valuable to keep these in the image or not. For me these initialize toon methods are overhead. They get parsed as infrequently as possible. So personally I prefer to use several external files that are parsed infrequently instead of the overhead of several initialization methods that are run infrequently. For example, if I were deploying on some embedded arm device, the initialization method would be something I'd have to strip from the image. Seems easier to just parse an external file to me. I like the use of Smalltalk for eg user interface opening code where - we want a UI construction tool to edit an e excitable representation of the interface - there is only one version of the UI method But with these interface descriptions we could easily end up with a 32-bit and a 64-bit version for every UNIX variant plus Mac OS and Windows. You could easily have ten long methods with overheads in the tens of kilobytes. Seems a waste to me. > Thierry > > (Which is going through a hardware/system capabilities description language for high performance computing adaptation meeting at the moment and where one see issues about "static" external information kept in files) > > Le 17/01/2016 04:40, Eliot Miranda a Ã©crit : >> Hi Mariano, >> >> On Sat, Jan 16, 2016 at 6:25 PM, Mariano Martinez Peck >> <marianopeck(a)gmail.com <mailto:marianopeck@gmail.com>> wrote: >> >> >> >> On Sat, Jan 16, 2016 at 11:02 PM, Eliot Miranda >> <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>> wrote: >> >> >> >> On Sat, Jan 16, 2016 at 6:00 AM, Mariano Martinez Peck >> <marianopeck(a)gmail.com <mailto:marianopeck@gmail.com>> wrote: >> >> >> Hi all, >> >> Sorry for reviving an old thread but I thought it was better >> to continue the discussion here because of the context. >> As you may have read, the other day I released a first >> approeach to a subset of OSProcess based on FFI >> (posix_spwan() family of functions): >> >> https://github.com/marianopeck/OSSubprocess >> >> And with that in mind, I wanted to share a few things with >> you. The main 2 problems I found with implementing this with >> FFI was: >> >> 1) We have all already agree and discussed that fork+exec >> cannot be done in separate FFI calls. So at the very min you >> need either a plugin method that does the fork()+exec() OR >> wrapping a lib like posix_spwan() >> >> 2) The other main problem, is, as you all said (and mostly >> Nicolas), is the problems with the preprocessor >> (constants, macros, etc). >> >> With all that said, I was able to get my stuff working. >> However, I am still using some primitives of OSProcess >> plugin because of 2). >> >> I read Eliot idea and what I don't like is the need of a C >> compiler in the user machine. I think that's a high >> constrain. Then Igor suggested that WE (developers and >> maintainers of a certain tool) are the ones that compiles >> the little C program to extract constant values etc and then >> WE provide as part of our source code, some packages with >> some SharedPool depending on the platform/OS. And Igor >> approach looked a bit better to me. >> >> >> >> >> You misunderstand the proposal. >> >> >> I think I did. But let me confirm that below ;) >> >> The C compiler is needed /only when changing the set of >> constants/, i.e. when /developing/ the interface. The C >> compiler is /not/ needed when deploying. >> >> The idea is to >> a) at development time, e.g. when a new variable is added to a >> SharedPool containing platform constants, a C program is >> autogenerated that outputs in some format a description of the >> names and values of all the constants defined in the pool. One >> convenient notation is e.g. STON. For the purposes of this >> discussion let's assume we're using ston, but any format the >> image an parse (or indeed a shared object the image can load on >> teh current pkatform) will do. The output of the autogenerated >> C program would be called something like >> <SharedPoolName>.<PlatformName>.ston, e.g. >> UnixConstants.MacOSX64.ston or UnixConstants.Linux32.ston. The >> ston files can easily be parsed by facilities in the Smalltalk >> image. >> >> b) when deploying the system to a set of platforms one includes >> all the relevant platform-specific ston files. >> >> >> OK. But let me ask something. Below you said "be it a plugin or a >> dll doesn't matter". To autogenerate the C program, I must know >> which header files to include for each platform and probably a few >> others things. For example, besides exporting the value, I would >> also like to export the sizeof(). At that depends how was the VM >> compiled, right? So...my question is...if such a autogenerated C >> code could be part of the VM building (considering all the settings >> being assume when building), cannot I reuse the knowledge the VM >> already has? Like which header files to include, if it was compiled >> 32 bits or 64 bits, which C compiler to use, etc.. >> >> >> I actually said that using text is easier than a dll. So I'm saying >> autogenerate a C program that outputs name-value pairs in some >> convenient textual representation, e.g. ston. But answering your >> question... >> >> The knowledge in the VM as to what header files are included *applies >> only to the include files the VM uses*. The VM uses a subset of the >> platform. It doesn't for example include any headers that define a >> database interface. It doesn't include header files that define the >> interface to a UI tooklit such at GTK. Etc, etc. So in fact the VM >> *doesn't* include the knowledge one needs to determine the set of >> include files for an arbitrary FFI interface. And even so, the include >> files that it does use are in the VM's platform source files, and that >> information is not readily accessible. >> >> Let me summarise. No, the VM cannot be used to determine the set of >> include files needed to generate constants used in an arbitrary FFI >> interface. >> >> What I mean is if it would be easier if I take the SharedPool at VM >> building time, and from there I autogenerate (and run) the C code >> that would generate the output. Then, when we "deploy" the VM, we >> can deploy it with relevant platform specific ston files as you said. >> >> >> No. The VM is something that provides an FFI. It doesn't *define* an >> FFI. One must be able to develop an FFI interface without needing to >> rebuild the VM. So computing the values of constants should be >> *separate* from building a VM. Now let me give you more of an example. >> >> Let's say we define a subclass of SharedPool called FFISharedPool. >> FFISharedPool 's job is to manage autogenerating a C file, compiling it >> for the platform, and organizing parsing the relevant output. Let's say >> we use a convention like class-side pragmas to define include files, and >> compiler flags. The VM provides two crucial pieces of information: >> >> 1. the platform name >> 2. the word size >> >> One can't run a Mac OS VM on Linux, and one can't run a 64-bit VM on a >> 32-bit operating system. So taking this information from the VM >> accurately tells the current system what ABI (application binary >> interface) to use, and that's what's important in generating the right >> constants. >> >> So we use these two pieces of information to index the method pragmas >> that tell us what specific files to include. >> >> Let's imagine we subclass FFISharedPool to add a shared pool for >> constants for an SQL database. We might have a class declaration like >> >> FFISharedPool subclass: #MYSQLInterface >> instanceVariableNames: '' >> classVariableNames: 'MYSQL_DEFAULT_AUTH MYSQL_ENABLE_CLEARTEXT_PLUGIN >> MYSQL_INIT_COMMAND MYSQL_OPT_BIND MYSQL_OPT_CAN_HANDLE_EXPIRED_PASSWORDS >> MYSQL_OPT_COMPRESS >> MYSQL_OPT_CONNECT_ATTR_DELETE MYSQL_OPT_CONNECT_ATTR_RESET' >> poolDictionaries: '' >> category: 'MYSQLInterface-Pools' >> >> The job of FFISharedPool is to compute the right values for the class >> variables on every platform we want to deploy the MYSQL interface on. >> >> So we need to know the relevant include files and C flags for each >> platform/word-size combination. A few of them might look like >> >> >> MYSQLInterface class methods for platform information >> mac32 >> "I describe the include files and C flags to use when developing a >> 32-bit MYSQL FFI interface on Mac OS X" >> <platformName: 'Mac OS' wordSize: 4> >> <cFlags: #('-m32') includeFiles: #('/opt/mysql/include32')> >> ^self "all the info is in the pragmas" >> >> mac64 >> "I describe the include files and C flags to use when developing a >> 64-bit MYSQL FFI interface on Mac OS X" >> <platformName: 'Mac OS' wordSize: 8> >> <cFlags: #('-m64') includeFiles: #('/opt/mysql/include64')> >> >> The above might cause FFISharedPool to autogenerate files called >> MYSQLInterface.mac32.c & MYSQLInterface.mac64.c. And these, when run, >> might output ston notation to MYSQLInterface.mac32.ston & >> MYSQLInterface.mac64.ston (or maybe to stdout which has to be redirected >> to MYSQLInterface.mac32.ston; whatever). >> >> Now, you might use pragmas, or you might answer a Dictionary instance. >> What ever style pleases you and seems convenient and readable. But >> these methods define the necessary metadata (C flags, include paths, and >> ...?) for FFISharedPool to autogenerate the C program that, when >> compiled with the supplied C flags and run on the current platform, >> outputs the values for the constants the shared pool wants to define. >> >> >> You can get fancy and have FFISharedPool autogenerate the C programs >> whenever one adds or removes a constant name. Or you can require the >> programmer run something, e.g. MYSQLInterface generateInterfaces. It's >> really nice if FFISharedPool submits the file to the C compiler >> automatically, but this can only work for e.g. 32 & 64 bit versions on a >> single platform. You have to compile the autogenerated program on the >> relevant platform, with the necessary libraries and include files installed. >> >> You could imagine a set of servers for different platforms so one could >> submit the autogenerated program for compilation and execution on each >> platform. That's a facility I'd make it easy to implement. I could >> imagine that a programmer whose company develops an FFI interface and >> deploys it on a number of platforms would love to be able to automate >> compiling and running the relevant autogenerated code on a set of >> servers. I could imagine the Pharo community providing a set of servers >> upon which lots of software is installed for precisely this purpose. >> That means that people could develop FFI interfaces without even having >> to have the C compiler installed on their platform. >> >> You could also add a C parser to FFISharedPool that parses the >> post-preprocessed code and extracts function declarations. But the >> important thing is autogenerating the C program so that it generates >> easily parsable output containing the values for the constants. You can >> extend the system in interesting ways once you ave this core >> functionality implemented. >> >> So once the program is autogenerated and compiled for the current >> platform, it is run and its output collected in a file whose name can be >> recognised by FFISharedPool. >> >> >> Now the class side of FFISharedPool might be declared as >> >> FFIShardPool class >> instanceVariableNames: 'platformName wordSize' >> >> and on start-up FFIShardPool could examine its subclasses, and for each >> whose platformName & wordSize do not match the current platform, search >> for all the matching FOOInterface.plat.ston files, parse them and update >> the subclasses' variables, and update that pool's platformName & >> wordSize. It could emit a warning on the Transcript or stdout (headful >> vs headless) indicating which subclasses it couldn't find the relevant >> FOOInterface.plat.ston files for. >> >> But the end result is that >> >> a) providing the system is deployed with FOOInterface.plat.ston files >> for each interface and platform used, a cross-platform application can >> be deployed *that does not require a C compiler*. >> b) providing that a system's FOOInterface files have been initialized on >> the intended platform, a platform-specific application can be deployed >> for a single platform *without needing the ston files*. >> >> Does this make more sense now? >> >> c) at startup the image checks its current platform. If the >> platform is the same that it was saved on, no action is taken. >> But if the platform as changed then the relevant ston file is >> selected, parsed, and the values for the variables in the shared >> pool updated to reflect the values of the current platform. >> >> So the C compiler is only needed when developing the interface, >> not when deploying it. >> >> OK >> >> >> Then Nicolas made a point that if we plan to manage all that >> complexity at the image level it may become a hell too. >> >> So.... what if we take a simpler (probably not better) >> approach and we consider the "c program that exports >> constants and sizes" a VM Plugin? Let's say we have a >> UnixPreprocessorPlugin (that would work for OSX, Linux and >> other's Unix I imagine for the time being) which provides a >> function (that is exported) which answers an array of >> arrays. For each constant, we include the name of the >> constant, the value, and the sizeof(). Then from image >> side, we simply do one FFI call, we get the large array and >> we adapt it to a SharedPool or whatever kind of object >> representing that info. >> >> >> >> >> This is what I suggestred in teh first place. That what is >> autogenerated is a shared object (be it a plgin or a dll doesn't >> matter, it is machine code generated by a C compiler form an >> autogenerated C program compiled with the platform's C compiler) >> that can be loaded at run-time and interrogated to fetch the >> values of a set of variables >> >> >> OK, got it. But still, it would be easier if the "platform" in this >> case is the "machine where we build the VM we will then distribute" >> right? i mean, I would like to put this in the CI jobs that >> automatically builds the VM, and not myself building for each platform. >> >> >> NO! For example, why would a company that has some proprietary >> arithmetic package implemented in its secret labs in C or C++ and >> accessed through the FFI want to have that code on the Pharo community's >> build servers? >> >> >> *I mean, my main doubt is if this job of autogenerating C code, >> compile it, run it, export text file, and distribute text file with >> the VM, could be done as part of the VM building. * >> >> >> For fuck's sake. Developing an FFI is not something one does when >> building a VM. It is something one does wen using the system. f you >> want to do this you *use a plugin*. The FFI is a different beast. It >> is to allow programers to interface to external librarys that are >> *independent from teh VM*. >> >> I'm not going to answer this one again. OK? >> >> >> . But I think that the textual notation suggested above is >> simpler. The test files are easier to distribute and change. >> Shared objects and plugins have a habit of going stale, and >> there needs to be metadata in there to describe the set of >> constants etc, which is tricky to generate and parse because it >> is binary (pointer sizes, etc, etc). Instead a simple textual >> format should be much more robust. One could even edit by hand >> to add new constants. It would be easy to make the textual file >> a versioned file. Etc, etc. >> >> >> OK. Got it. And do you think using X Macros for the autogenerated C >> (from the SharedPool) is a good idea? >> And then I simply write a text file out of it. >> >> >> I know that different users will need different constants. >> But let's say the infrastructure (plugin etc) is already >> done. And let's say I am a user that I want to build >> something with FFI and I need some constants that I see are >> not defined. Then I can simply add the ones I need in the >> plugin, and next VM release will have those. If Cog gets >> moved to Github, then this is even easier. Everybody can do >> a PR with the constants he needs. And in fact, if we have >> the infrastructure in place, I think that we each of us >> spend half an hour, we may have almost everything we need. >> >> For example, I can add myself all those for signals (to use >> kill() from FFI), all those from fcntl (to make none >> blocking pipes), all those from wait()/waitpid() family (so >> that I can do a waitpid() with WNOHANG), etc etc etc. >> >> I know it's not the best approach but it's something that >> could be done very easily and would allow A LOT of stuff to >> be moved to FFI just because we have no access to preprocess >> constants or sizeof() (to know how to allocate). I also >> know this won't cover macros and other stuff. But still. >> >> If you think this is a good idea, I can spend the time to do >> it. >> >> Cheers, >> >> >> >> >> >> >> >> >> >> On Thu, May 10, 2012 at 10:09 AM, Nick Ager >> <nick.ager(a)gmail.com <mailto:nick.ager@gmail.com>> wrote: >> >> <snip> >> Well, like opendbx, maybe because opengl has quite >> standard interface... >> </snip> >> >> and >> >> <snip> >> It's not that it's not doable, it's that we gonna >> reinvent gaz plant >> and it gonna be so boring... >> I'd like to see a proof of concept, even if we restrict >> to libc, libm, >> kernel.dll, msvcrt.dll ... >> </snip> >> >> <snip> >> Is the unix style select() >> ubiquitous or should I use WaitForMultipleObject() on >> Windows? Are >> specification of read/write streams implementation >> machine independant >> (bsd/sysv/others...) >> </snip> >> >> Perhaps *a* way forward is to try to find existing >> projects which have already created cross-platform >> abstractions for platform specific functionality. Then >> we can use FFI to access that interface in a similar way >> to OpenGL and OpenDBX. For example NodeJs works across >> unixes - perhaps they have a useful cross-platform >> abstraction, boost has abstractions of IPC etc >> >> Nick >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot >> >> >> >> >> -- >> Mariano >> http://marianopeck.wordpress.com >> >> >> >> >> -- >> _,,,^..^,,,_ >> best, Eliot > >

Jan. 17, 2016

Re: [Pharo-dev] Why there is no setting to disable QA?

by Ben Coman

On Sun, Jan 17, 2016 at 7:33 PM, Yuriy Tymchuk <yuriy.tymchuk(a)me.com> wrote: > You can simply do `QANautilusPlugin uninstall`. I have not figured out how > to put it in the prefs. > > Now about the false positivesâ¦ I donât know what you are talking about, but > I suspect that it is about something like âclass not being usedâ. Now if it > is not being used itâs not a false positive, itâs just a useless > information. What we can do, is put this rules into âinformationâ category > and disable it in settings. Or check class creation time and don't display message until its a week old ??. cheers -ben > > Cheers. > Uko > > On 17 Jan 2016, at 11:47, stepharo <stepharo(a)free.fr> wrote: > > Hi nicolai > > Le 17/1/16 11:29, Nicolai Hess a Ã©crit : >> >> Hi >> >> Why there is no setting for that? >> >> Stef >> > > > You can remove the Nautilus plugin (Nautilus window menu -> plugin manager) > > Yes I saw looking around. Now I have to dive into it to script it so that I > avoid to be forced as today to redo > my screenshots (to avoid all the false positive on class creation). > Because else > > For the spotter and inspector integration, there is a setting (default > "disabled") > > >

Jan. 17, 2016

Re: [Pharo-dev] [Pharo-project] [Vm-dev] Re: Can OSProcess functionality be implemented using FFI instead of plugin?

by Thierry Goubier

Hi Eliot, I still don't understand why this platform-specific data (external datatypes sizes, constants values) has to be written as a json file and not as Smalltalk code ? If I write an interface to an external lib, I'd like to benefit from nice access to Smalltalk code to, say, test versions of the external libraries, set external types based on system information, etc, etc... instead of having to deal with all those various json files outside my normal workflow. Okay, I can write smalltalk code which runs those C tests programs, extract their output, model them in smalltalk and generate the jsons from there... but, then, I really don't understand why I have to write those jsons then every user loads those jsons instead of just running that setup code I have in my external library smalltalk support package ? Thierry (Which is going through a hardware/system capabilities description language for high performance computing adaptation meeting at the moment and where one see issues about "static" external information kept in files) Le 17/01/2016 04:40, Eliot Miranda a Ã©crit : > Hi Mariano, > > On Sat, Jan 16, 2016 at 6:25 PM, Mariano Martinez Peck > <marianopeck(a)gmail.com <mailto:marianopeck@gmail.com>> wrote: > > > > On Sat, Jan 16, 2016 at 11:02 PM, Eliot Miranda > <eliot.miranda(a)gmail.com <mailto:eliot.miranda@gmail.com>> wrote: > > > > On Sat, Jan 16, 2016 at 6:00 AM, Mariano Martinez Peck > <marianopeck(a)gmail.com <mailto:marianopeck@gmail.com>> wrote: > > > Hi all, > > Sorry for reviving an old thread but I thought it was better > to continue the discussion here because of the context. > As you may have read, the other day I released a first > approeach to a subset of OSProcess based on FFI > (posix_spwan() family of functions): > > https://github.com/marianopeck/OSSubprocess > > And with that in mind, I wanted to share a few things with > you. The main 2 problems I found with implementing this with > FFI was: > > 1) We have all already agree and discussed that fork+exec > cannot be done in separate FFI calls. So at the very min you > need either a plugin method that does the fork()+exec() OR > wrapping a lib like posix_spwan() > > 2) The other main problem, is, as you all said (and mostly > Nicolas), is the problems with the preprocessor > (constants, macros, etc). > > With all that said, I was able to get my stuff working. > However, I am still using some primitives of OSProcess > plugin because of 2). > > I read Eliot idea and what I don't like is the need of a C > compiler in the user machine. I think that's a high > constrain. Then Igor suggested that WE (developers and > maintainers of a certain tool) are the ones that compiles > the little C program to extract constant values etc and then > WE provide as part of our source code, some packages with > some SharedPool depending on the platform/OS. And Igor > approach looked a bit better to me. > > > > > You misunderstand the proposal. > > > I think I did. But let me confirm that below ;) > > The C compiler is needed /only when changing the set of > constants/, i.e. when /developing/ the interface. The C > compiler is /not/ needed when deploying. > > The idea is to > a) at development time, e.g. when a new variable is added to a > SharedPool containing platform constants, a C program is > autogenerated that outputs in some format a description of the > names and values of all the constants defined in the pool. One > convenient notation is e.g. STON. For the purposes of this > discussion let's assume we're using ston, but any format the > image an parse (or indeed a shared object the image can load on > teh current pkatform) will do. The output of the autogenerated > C program would be called something like > <SharedPoolName>.<PlatformName>.ston, e.g. > UnixConstants.MacOSX64.ston or UnixConstants.Linux32.ston. The > ston files can easily be parsed by facilities in the Smalltalk > image. > > b) when deploying the system to a set of platforms one includes > all the relevant platform-specific ston files. > > > OK. But let me ask something. Below you said "be it a plugin or a > dll doesn't matter". To autogenerate the C program, I must know > which header files to include for each platform and probably a few > others things. For example, besides exporting the value, I would > also like to export the sizeof(). At that depends how was the VM > compiled, right? So...my question is...if such a autogenerated C > code could be part of the VM building (considering all the settings > being assume when building), cannot I reuse the knowledge the VM > already has? Like which header files to include, if it was compiled > 32 bits or 64 bits, which C compiler to use, etc.. > > > I actually said that using text is easier than a dll. So I'm saying > autogenerate a C program that outputs name-value pairs in some > convenient textual representation, e.g. ston. But answering your > question... > > The knowledge in the VM as to what header files are included *applies > only to the include files the VM uses*. The VM uses a subset of the > platform. It doesn't for example include any headers that define a > database interface. It doesn't include header files that define the > interface to a UI tooklit such at GTK. Etc, etc. So in fact the VM > *doesn't* include the knowledge one needs to determine the set of > include files for an arbitrary FFI interface. And even so, the include > files that it does use are in the VM's platform source files, and that > information is not readily accessible. > > Let me summarise. No, the VM cannot be used to determine the set of > include files needed to generate constants used in an arbitrary FFI > interface. > > What I mean is if it would be easier if I take the SharedPool at VM > building time, and from there I autogenerate (and run) the C code > that would generate the output. Then, when we "deploy" the VM, we > can deploy it with relevant platform specific ston files as you said. > > > No. The VM is something that provides an FFI. It doesn't *define* an > FFI. One must be able to develop an FFI interface without needing to > rebuild the VM. So computing the values of constants should be > *separate* from building a VM. Now let me give you more of an example. > > Let's say we define a subclass of SharedPool called FFISharedPool. > FFISharedPool 's job is to manage autogenerating a C file, compiling it > for the platform, and organizing parsing the relevant output. Let's say > we use a convention like class-side pragmas to define include files, and > compiler flags. The VM provides two crucial pieces of information: > > 1. the platform name > 2. the word size > > One can't run a Mac OS VM on Linux, and one can't run a 64-bit VM on a > 32-bit operating system. So taking this information from the VM > accurately tells the current system what ABI (application binary > interface) to use, and that's what's important in generating the right > constants. > > So we use these two pieces of information to index the method pragmas > that tell us what specific files to include. > > Let's imagine we subclass FFISharedPool to add a shared pool for > constants for an SQL database. We might have a class declaration like > > FFISharedPool subclass: #MYSQLInterface > instanceVariableNames: '' > classVariableNames: 'MYSQL_DEFAULT_AUTH MYSQL_ENABLE_CLEARTEXT_PLUGIN > MYSQL_INIT_COMMAND MYSQL_OPT_BIND MYSQL_OPT_CAN_HANDLE_EXPIRED_PASSWORDS > MYSQL_OPT_COMPRESS > MYSQL_OPT_CONNECT_ATTR_DELETE MYSQL_OPT_CONNECT_ATTR_RESET' > poolDictionaries: '' > category: 'MYSQLInterface-Pools' > > The job of FFISharedPool is to compute the right values for the class > variables on every platform we want to deploy the MYSQL interface on. > > So we need to know the relevant include files and C flags for each > platform/word-size combination. A few of them might look like > > > MYSQLInterface class methods for platform information > mac32 > "I describe the include files and C flags to use when developing a > 32-bit MYSQL FFI interface on Mac OS X" > <platformName: 'Mac OS' wordSize: 4> > <cFlags: #('-m32') includeFiles: #('/opt/mysql/include32')> > ^self "all the info is in the pragmas" > > mac64 > "I describe the include files and C flags to use when developing a > 64-bit MYSQL FFI interface on Mac OS X" > <platformName: 'Mac OS' wordSize: 8> > <cFlags: #('-m64') includeFiles: #('/opt/mysql/include64')> > > The above might cause FFISharedPool to autogenerate files called > MYSQLInterface.mac32.c & MYSQLInterface.mac64.c. And these, when run, > might output ston notation to MYSQLInterface.mac32.ston & > MYSQLInterface.mac64.ston (or maybe to stdout which has to be redirected > to MYSQLInterface.mac32.ston; whatever). > > Now, you might use pragmas, or you might answer a Dictionary instance. > What ever style pleases you and seems convenient and readable. But > these methods define the necessary metadata (C flags, include paths, and > ...?) for FFISharedPool to autogenerate the C program that, when > compiled with the supplied C flags and run on the current platform, > outputs the values for the constants the shared pool wants to define. > > > You can get fancy and have FFISharedPool autogenerate the C programs > whenever one adds or removes a constant name. Or you can require the > programmer run something, e.g. MYSQLInterface generateInterfaces. It's > really nice if FFISharedPool submits the file to the C compiler > automatically, but this can only work for e.g. 32 & 64 bit versions on a > single platform. You have to compile the autogenerated program on the > relevant platform, with the necessary libraries and include files installed. > > You could imagine a set of servers for different platforms so one could > submit the autogenerated program for compilation and execution on each > platform. That's a facility I'd make it easy to implement. I could > imagine that a programmer whose company develops an FFI interface and > deploys it on a number of platforms would love to be able to automate > compiling and running the relevant autogenerated code on a set of > servers. I could imagine the Pharo community providing a set of servers > upon which lots of software is installed for precisely this purpose. > That means that people could develop FFI interfaces without even having > to have the C compiler installed on their platform. > > You could also add a C parser to FFISharedPool that parses the > post-preprocessed code and extracts function declarations. But the > important thing is autogenerating the C program so that it generates > easily parsable output containing the values for the constants. You can > extend the system in interesting ways once you ave this core > functionality implemented. > > So once the program is autogenerated and compiled for the current > platform, it is run and its output collected in a file whose name can be > recognised by FFISharedPool. > > > Now the class side of FFISharedPool might be declared as > > FFIShardPool class > instanceVariableNames: 'platformName wordSize' > > and on start-up FFIShardPool could examine its subclasses, and for each > whose platformName & wordSize do not match the current platform, search > for all the matching FOOInterface.plat.ston files, parse them and update > the subclasses' variables, and update that pool's platformName & > wordSize. It could emit a warning on the Transcript or stdout (headful > vs headless) indicating which subclasses it couldn't find the relevant > FOOInterface.plat.ston files for. > > But the end result is that > > a) providing the system is deployed with FOOInterface.plat.ston files > for each interface and platform used, a cross-platform application can > be deployed *that does not require a C compiler*. > b) providing that a system's FOOInterface files have been initialized on > the intended platform, a platform-specific application can be deployed > for a single platform *without needing the ston files*. > > Does this make more sense now? > > c) at startup the image checks its current platform. If the > platform is the same that it was saved on, no action is taken. > But if the platform as changed then the relevant ston file is > selected, parsed, and the values for the variables in the shared > pool updated to reflect the values of the current platform. > > So the C compiler is only needed when developing the interface, > not when deploying it. > > OK > > > Then Nicolas made a point that if we plan to manage all that > complexity at the image level it may become a hell too. > > So.... what if we take a simpler (probably not better) > approach and we consider the "c program that exports > constants and sizes" a VM Plugin? Let's say we have a > UnixPreprocessorPlugin (that would work for OSX, Linux and > other's Unix I imagine for the time being) which provides a > function (that is exported) which answers an array of > arrays. For each constant, we include the name of the > constant, the value, and the sizeof(). Then from image > side, we simply do one FFI call, we get the large array and > we adapt it to a SharedPool or whatever kind of object > representing that info. > > > > > This is what I suggestred in teh first place. That what is > autogenerated is a shared object (be it a plgin or a dll doesn't > matter, it is machine code generated by a C compiler form an > autogenerated C program compiled with the platform's C compiler) > that can be loaded at run-time and interrogated to fetch the > values of a set of variables > > > OK, got it. But still, it would be easier if the "platform" in this > case is the "machine where we build the VM we will then distribute" > right? i mean, I would like to put this in the CI jobs that > automatically builds the VM, and not myself building for each platform. > > > NO! For example, why would a company that has some proprietary > arithmetic package implemented in its secret labs in C or C++ and > accessed through the FFI want to have that code on the Pharo community's > build servers? > > > *I mean, my main doubt is if this job of autogenerating C code, > compile it, run it, export text file, and distribute text file with > the VM, could be done as part of the VM building. * > > > For fuck's sake. Developing an FFI is not something one does when > building a VM. It is something one does wen using the system. f you > want to do this you *use a plugin*. The FFI is a different beast. It > is to allow programers to interface to external librarys that are > *independent from teh VM*. > > I'm not going to answer this one again. OK? > > > . But I think that the textual notation suggested above is > simpler. The test files are easier to distribute and change. > Shared objects and plugins have a habit of going stale, and > there needs to be metadata in there to describe the set of > constants etc, which is tricky to generate and parse because it > is binary (pointer sizes, etc, etc). Instead a simple textual > format should be much more robust. One could even edit by hand > to add new constants. It would be easy to make the textual file > a versioned file. Etc, etc. > > > OK. Got it. And do you think using X Macros for the autogenerated C > (from the SharedPool) is a good idea? > And then I simply write a text file out of it. > > > I know that different users will need different constants. > But let's say the infrastructure (plugin etc) is already > done. And let's say I am a user that I want to build > something with FFI and I need some constants that I see are > not defined. Then I can simply add the ones I need in the > plugin, and next VM release will have those. If Cog gets > moved to Github, then this is even easier. Everybody can do > a PR with the constants he needs. And in fact, if we have > the infrastructure in place, I think that we each of us > spend half an hour, we may have almost everything we need. > > For example, I can add myself all those for signals (to use > kill() from FFI), all those from fcntl (to make none > blocking pipes), all those from wait()/waitpid() family (so > that I can do a waitpid() with WNOHANG), etc etc etc. > > I know it's not the best approach but it's something that > could be done very easily and would allow A LOT of stuff to > be moved to FFI just because we have no access to preprocess > constants or sizeof() (to know how to allocate). I also > know this won't cover macros and other stuff. But still. > > If you think this is a good idea, I can spend the time to do > it. > > Cheers, > > > > > > > > > > On Thu, May 10, 2012 at 10:09 AM, Nick Ager > <nick.ager(a)gmail.com <mailto:nick.ager@gmail.com>> wrote: > > <snip> > Well, like opendbx, maybe because opengl has quite > standard interface... > </snip> > > and > > <snip> > It's not that it's not doable, it's that we gonna > reinvent gaz plant > and it gonna be so boring... > I'd like to see a proof of concept, even if we restrict > to libc, libm, > kernel.dll, msvcrt.dll ... > </snip> > > <snip> > Is the unix style select() > ubiquitous or should I use WaitForMultipleObject() on > Windows? Are > specification of read/write streams implementation > machine independant > (bsd/sysv/others...) > </snip> > > Perhaps *a* way forward is to try to find existing > projects which have already created cross-platform > abstractions for platform specific functionality. Then > we can use FFI to access that interface in a similar way > to OpenGL and OpenDBX. For example NodeJs works across > unixes - perhaps they have a useful cross-platform > abstraction, boost has abstractions of IPC etc > > Nick > > > > > -- > Mariano > http://marianopeck.wordpress.com > > > > > -- > _,,,^..^,,,_ > best, Eliot > > > > > -- > Mariano > http://marianopeck.wordpress.com > > > > > -- > _,,,^..^,,,_ > best, Eliot

Jan. 17, 2016

Re: [Pharo-dev] [Vm-dev] Unable to read Utf8 ?

by Esteban Lorenzano

> On 17 Jan 2016, at 13:43, Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > > Le 17/01/2016 13:22, Esteban Lorenzano a Ã©crit : >> this is working in latest VM build. >> >> https://pharo.fogbugz.com/f/cases/17389/ >> >> so.. you can go back to use it :) > > Thanks! thanks to Eliot :) > > Thierry >

Jan. 17, 2016