October 2018 - Pharo-users - lists.pharo.org

Re: [Pharo-users] Installing SmaCC

by Dimitris Chloupis

About your last part on platforms, I will be providing a way to inline C code so one can you use C macros to detect the platform and generate code accordingly. Or this could happen via a pragma too, it should not be an issue. This also a reason why I previously talked about an "in place" annotation and why specially named variables was my first choice instead of pragmas. I am also not a big fan of pragmas syntax which for me at least deviates from standard smalltalk syntax style. But as I said I am not against their usage at all. Generally because this is no an afternoon project obviously, I will be relying on C code inling at first for special corner cases and then I will implement them as annotations the more the project moves forward. On Wed, Oct 17, 2018 at 1:30 PM Dimitris Chloupis <kilon.alios(a)gmail.com> wrote: > Hello Allistair > > I have used Slang only once and it was generating code that was indeed > readbale but my aim is for more finer control over the output. Lets say I > want import a specific C header file or I want a string to map to custom C > type I created etc. So yes Slang is by no mean a bad tool at all its just > is not designed with making source output that is undetectable as > autogenerated by a human. But I will have to give it a more serious try > because it may be closer than I initially thought. > > I am not against the usage of pragmas, and indeed are an excellent way to > annotate stuff , my only concern is when I may want to annotate in place > for some weird reason , but that may be doable with pragmas as well too. > > Smalltalk code that is 100% smalltalk should be able to execute , you > mention however the execution of external C functions , problem is that in > my case that code does not live in DLLs but in an executable so no I am not > amaing to that level of execution. > > Also I have an easier solution for this too, when I made the CPPBridge, > which is a Pharo library that allows the usage of C++ libraries from Pharo, > I used a shared memory bridge to communicate back to Pharo giving the > ability of both function calls and callbacks. If I really want to capture > execution I can do it like this which is the exact opposite of what you do, > instead of the VM capturing the executable it will be the executable > capturing the VM if that makes any sense. This makes things far easier. As > a matter of fact not only my CPPBridge does this it also allows to extend > the pharo image file because it uses memory mapped files for the shared > memory which like pharo image files are memory dumps. So there is a lot > potential in that department. > > However my main goal is to use Smalltalk code execution to make sure the > prototype works on a basic level, there will be a C cide in this project > obviously which will act like a runtime that will provide live coding > features. This is also a library I made in C that does this through the > usage of DLLs that rebuilds and reloads dynamically. > > So I dont really need the VM to execute my code to check that is working > cause the C compiler and the live coding runtime can handle this. I could > even hook in the Pharo debugger, I have done this with my Atlas library > that allows to use Python library from Pharo by sending the python error > back to Pharo debugger where it triggers an error and the debugger pops to > allow you to do your usual live coding magic and basically resends the code > back to python. Because of my C livecoding library I can do this with C > too. My only concern is how the C/C++ compiler reports errors because > obviously it known that it kinda sucks on this. But hey I cannot make C > better :D > > Generally speaking the tools I am making are not designed for general > consuption but designed to solve my own problems. Of course I like to share > them because there is always the chance for someone to find them useful as > it has happened with Atlas for example. Plus as happened with Atlas one can > take my code and adjust it to his or her personal needs. > > On Wed, Oct 17, 2018 at 1:04 PM Alistair Grant <akgrant0710(a)gmail.com> > wrote: > >> Hi Dimitris, >> >> As someone currently learning to use Slang (i.e. not an expert), I've >> added my 2c below... >> >> On Wed, 17 Oct 2018 at 11:06, Dimitris Chloupis <kilon.alios(a)gmail.com> >> wrote: >> > >> > Thierry you have done it !!! you just gave a very easy solution to my >> problems. >> > >> > Yeap Slang is quite close to what I am thinking, unfortunately Clement >> told me to stay away from it because the code is ugly and specially used >> for VM only. If I remember also correctly it does not generate readable C >> code either. But the idea as a concept is very close to what I imagine. >> >> I've found the C code produced to be quite readable, but that is >> probably influenced by the fact that I have read the slang first. >> >> >> > As a matter of fact you mentioning Slang made I have an epiphany that >> I dont have to create a new syntax at all, instead I could use specific >> variables or methods to provide type annotation. Thus like Slang I can use >> regular Smalltalk code that avoids changing types but without the need for >> type inference (although I am not excluding this either). >> > >> > So yes I am definetly want to move to the direction that Slang goes so >> I can fully utilise the Pharo IDE and minise code that I have to write. >> > >> > So basically I am thinking write code as you always write in Pharo and >> either >> > a) Have special dictionary variables in each method that provide static >> type annotations for the arguments of the methods, its return type and >> local variables >> >> Slang uses method pragmas to define the variables types. This seems >> to work quite well. >> >> >> > b) Have special methods that provide such dictionaries seperately. >> > >> > Or probably both. This way I can write 100% Smalltalk code and use a >> very small compiler to read those dictionary variables for the type of the >> variables and functions/structs (essentially a class will be output for a C >> struct with pointers to functions for methods and variables for instance >> variables). Why invent a whole new language when everything I need already >> Pharo provides ? >> > I could also use special variable dictionaries for all sort of things >> like generation of header files, generation of CMake files for automatic >> building. >> > >> > Also I like to use the way UFFI is doing C function signatures by using >> symbol arrays. >> > >> > So thank you all for inspiration it looks like all I need is Pharo AST >> methods (which I can from the AST packages) and SmaCC. >> > So yeap looks like Magnatar will be a new Slang afterall, I will keep >> you posted. >> > >> > Also this also opens the possibility of autowrapping the generated c >> code back to Pharo through UFFI, so one can use C code as if its Pharo >> code. I can leverage the TalkFFI project that does this already. Seems all >> the pieces have fallen in their place. >> > >> > Keep the suggestions and advice coming, you guys are inspirational :D >> >> Do you intend that the Smalltalk code can be executed? This will >> likely increase the complexity quite a bit. In the VM simulation we >> end up creating a XSimulation subclass that provides the framework for >> executing the smalltalk code, e.g simulating functions that are only >> in C. >> >> There is also the problem of platform differences. Slang doesn't >> really handle them well (pragmas can be used to indicated that methods >> should only be compiled on certain platforms, and #ifdef type code can >> be used, but it isn't enough). It would be nice to have a class that >> provides cross platform functionality, and then platform specific >> classes as required. >> >> HTH, >> Alistair >> >>

Oct. 17, 2018

Re: [Pharo-users] compiling libraries for Pharo on macOS

by Michel Onoff

Afaik, dlopen(3), which I presume UFFI uses internally to load (shared) libraries on demand (I cannot imagine anything else), looks in LD_LIBRARY_PATH (DYLD_LIBRARY_PATH on macOS), not in PATH. So let's focus on one of my question again, reworded here: What is the strategy that UFFI internally uses in its own implementation to search for a library? What if that library depends on other libraries in turn? Where would these ones be searched for? On 2018-10-17 12:05, Dimitris Chloupis wrote: > yeah I apologise for the wrong example , LibC utilises the > enviroment/system variables. > > Generally speaking all OSes have a PATH variable that defines the places > where common libraries and tools are to be found. If you are on windows > you can type in the command prompt , "PATH", in Linux and MacOS is > "$PATH" . You can add your path in Windows using the system variables > dialog window, in MacOS and Linux it usually happens via .bash-config > file that setups the path but I am not so sure. > > The easiest way is to do something like macModuleName does in the > LGitLibrary class the code is > > macModuleName > Â Â Â | pluginDir | > Â Â Â pluginDir := Smalltalk vm binary parent / 'Plugins'. > Â Â Â #('libgit2.dylib' 'libgit2.0.dylib') > Â Â Â Â Â Â detect: [ :each | (pluginDir / each) exists ] > Â Â Â Â Â Â ifFound: [ :libName | ^ libName ]. > > Â Â Â self error: 'Module not found.' > > Hope that helps > > On Wed, Oct 17, 2018 at 12:54 PM Michel Onoff <michel.onoff(a)web.de > <mailto:michel.onoff@web.de>> wrote: > > Hi Dimitris, > > thanks for the long explanation. > > As you can see from my example, I build the library with the > Â Â Â -undefined dynamic_lookup > flag to clang, so this should be compatible with your (3) option if I > understand you correctly. Compiling/linking this way should allow the > use of dlopen(3) from UFFI during the execution of Pharo code, exactly > as I try to simulate with my small test programs. > > Pharo complains about not being able to load the module (the library), > even if it appears in the Pharo main folder. It does not report whether > it cannot find it or if it cannot bind it using dlopen(3). > > Moreover, it is unclear to me how Pharo interacts with > DYLD_LIBRARY_PATH > (on macOS) or LD_LIBRARY_PATH (on Linux) environment variables if they > are set. > > In other words, Where does it search for libraries? > > > > Anyway, I'll try your script on cmake. > > > Regards > MO > > > > > for use with dlopen() > > On 2018-10-17 11:32, Dimitris Chloupis wrote: > > I advice first of all the use of CMake , you can thank me later. > > > > CMake if you are not aware is the standard build system for C/C++ > , it > > makes it super easy to build and compile projects so you dont > have to do > > what you are doing, call the compiler and pass the inifite amount of > > compiler flags that it needs. > > > > To understand how to build a library you have to understand first > what a > > library is. > > > > Libraries used by Pharo are known as either Shared Libraries on > Macos > > (dylib) and Linux (so) but Dynamically Linked Libraries on Windows > > (DLLs). They are executables designed to be called from another > > executable (this includes another library as well). > > > > Those libraries operate in 3 modes > > 1) Static linking > > 2) semi dynamic linking > > 3) dynamic linking > > > > (1) may come as suprise but yes those libraries can be linked to the > > executable and build inside the exrecutable. But in our case we done > > want that because it would mean to rebuild the VM. > > (2) is not what we want either because eventhough it builds the > library > > independently generating the usual exetension files I mentioned > above it > > includes its symbole table in the executable so you wont have to > do what > > UFFI does, wrap or map your code to those library functions. The > > equivelant for Pharo would mean that we would not need UFFI at > all but > > we do need to rebuld the VM to include ths symbol table. The symbole > > table basically includes the name of the functions, their signatures > > (type of arguments) and their corresponing adresses in memory > needed to > > access them. > > > > (3) is what UFFI does, in that cause the executable is not > affected at > > all so you dont need to to rebuild the vm. But you do need to > know where > > to find the library , to access its symbol table and make sure the > > variables your executable uses match the type and signature in the > > symbol table or else the executable wont able to locate and > correctly > > call these functions. Of course libraries can contain structs and > global > > variables as well but the concept is the same. Now this messy > part is > > handled by the UFFI although barely because it still needs from > you to > > provide the correct signature to the function. > > > > I am giving you the long version because you have to make sure > you build > > you library with (3) way and not (2). (1) can be easily avoid but > it is > > easy to mix (2) and (3). From there on its easy , locating the > library > > is provided as a string returned by a method , see how UFFI utilises > > LibC for example and from there on you just convert the > signatures to > > symbol arrays as described by the UFFI tutorials. > > > > For cmake you only need to use -> add_library(libraryname MODULE > ${SRC} > > ). You can find a CmakeList.txt file that I use here > > https://gist.github.com/kilon/ff4b973bdeb3b0c0253ac373a4b2506a > > > > If you are wondering why its so big, its because the project I link > > against is huge and because I move the DLL to its build folder > instead > > of building it in the same folder as my source code. It CMake's > why of > > keeping things seperate and organised. Observe also that I use dll > > exension for Windows and MacOS, The extensions plays no role so I > > decided to go for the same extension to save time although > usually its > > preferable go for the standard extensions. > > > > Beware that if your library uses any external code even parts of > the C > > standard library those will have to be included , usually you > need only > > the header files, which is what SET INC is doing in my example, > while > > SET SRC includes only the library files. The inclusion of headers is > > super important because they are used to build the symbol table > which is > > the most important part of the library. Without a correctly build > symbol > > table the library is useless even when used from C/C++. > > > > On Wed, Oct 17, 2018 at 11:09 AM Michel Onoff > <michel.onoff(a)web.de <mailto:michel.onoff@web.de> > > <mailto:michel.onoff@web.de <mailto:michel.onoff@web.de>>> wrote: > > > >Â Â Â Hello, > > > >Â Â Â I've made some attempts to use the Pharo FFI. I've gone > through the > >Â Â Â documentation [1] and tried some examples. > > > >Â Â Â While the documentation is quite clear about using external > libraries > >Â Â Â from Pharo, it lacks details on how to build own libraries to > be used > >Â Â Â from Pharo. > > > >Â Â Â Specifically, I'm trying to build on all supported platforms, > namely > >Â Â Â macOS, Windows and Linux. Currently I've got Linux and > Windows under > >Â Â Â control, but I cannot find a systematic way to build on macOS. > > > >Â Â Â Here's an example for a library that I can use from a small test > >Â Â Â program > >Â Â Â but not from Pharo. Unfortunately, the diagnostic on Pharo is not > >Â Â Â detailed: it simply reports that it cannot load the module. > > > > > >Â Â Â clang -I <include-path> -O3 -c <source-path> > >Â Â Â ... > >Â Â Â clang -dynamiclib <object-files> -o lib<name>.dylib -undefined > >Â Â Â dynamic_lookup > > > >Â Â Â cp lib<name>.dylib <pharo-folder> > > > > > >Â Â Â Is there good documentation about how to correctly build > libraries so > >Â Â Â that they can be used from Pharo? > > > >Â Â Â Alternatively, are there details on how Pharo FFI searches for > >Â Â Â libraries > >Â Â Â and links them in the Pharo process? > > > > > >Â Â Â Regards > >Â Â Â MO > > > >Â Â Â ---- > > > >Â Â Â [1] https://files.pharo.org/books-pdfs/booklet-uFFI/UFFIDRAFT.pdf > > > > > >

Oct. 17, 2018

Re: [Pharo-users] Installing SmaCC

by Dimitris Chloupis

Hello Allistair I have used Slang only once and it was generating code that was indeed readbale but my aim is for more finer control over the output. Lets say I want import a specific C header file or I want a string to map to custom C type I created etc. So yes Slang is by no mean a bad tool at all its just is not designed with making source output that is undetectable as autogenerated by a human. But I will have to give it a more serious try because it may be closer than I initially thought. I am not against the usage of pragmas, and indeed are an excellent way to annotate stuff , my only concern is when I may want to annotate in place for some weird reason , but that may be doable with pragmas as well too. Smalltalk code that is 100% smalltalk should be able to execute , you mention however the execution of external C functions , problem is that in my case that code does not live in DLLs but in an executable so no I am not amaing to that level of execution. Also I have an easier solution for this too, when I made the CPPBridge, which is a Pharo library that allows the usage of C++ libraries from Pharo, I used a shared memory bridge to communicate back to Pharo giving the ability of both function calls and callbacks. If I really want to capture execution I can do it like this which is the exact opposite of what you do, instead of the VM capturing the executable it will be the executable capturing the VM if that makes any sense. This makes things far easier. As a matter of fact not only my CPPBridge does this it also allows to extend the pharo image file because it uses memory mapped files for the shared memory which like pharo image files are memory dumps. So there is a lot potential in that department. However my main goal is to use Smalltalk code execution to make sure the prototype works on a basic level, there will be a C cide in this project obviously which will act like a runtime that will provide live coding features. This is also a library I made in C that does this through the usage of DLLs that rebuilds and reloads dynamically. So I dont really need the VM to execute my code to check that is working cause the C compiler and the live coding runtime can handle this. I could even hook in the Pharo debugger, I have done this with my Atlas library that allows to use Python library from Pharo by sending the python error back to Pharo debugger where it triggers an error and the debugger pops to allow you to do your usual live coding magic and basically resends the code back to python. Because of my C livecoding library I can do this with C too. My only concern is how the C/C++ compiler reports errors because obviously it known that it kinda sucks on this. But hey I cannot make C better :D Generally speaking the tools I am making are not designed for general consuption but designed to solve my own problems. Of course I like to share them because there is always the chance for someone to find them useful as it has happened with Atlas for example. Plus as happened with Atlas one can take my code and adjust it to his or her personal needs. On Wed, Oct 17, 2018 at 1:04 PM Alistair Grant <akgrant0710(a)gmail.com> wrote: > Hi Dimitris, > > As someone currently learning to use Slang (i.e. not an expert), I've > added my 2c below... > > On Wed, 17 Oct 2018 at 11:06, Dimitris Chloupis <kilon.alios(a)gmail.com> > wrote: > > > > Thierry you have done it !!! you just gave a very easy solution to my > problems. > > > > Yeap Slang is quite close to what I am thinking, unfortunately Clement > told me to stay away from it because the code is ugly and specially used > for VM only. If I remember also correctly it does not generate readable C > code either. But the idea as a concept is very close to what I imagine. > > I've found the C code produced to be quite readable, but that is > probably influenced by the fact that I have read the slang first. > > > > As a matter of fact you mentioning Slang made I have an epiphany that I > dont have to create a new syntax at all, instead I could use specific > variables or methods to provide type annotation. Thus like Slang I can use > regular Smalltalk code that avoids changing types but without the need for > type inference (although I am not excluding this either). > > > > So yes I am definetly want to move to the direction that Slang goes so I > can fully utilise the Pharo IDE and minise code that I have to write. > > > > So basically I am thinking write code as you always write in Pharo and > either > > a) Have special dictionary variables in each method that provide static > type annotations for the arguments of the methods, its return type and > local variables > > Slang uses method pragmas to define the variables types. This seems > to work quite well. > > > > b) Have special methods that provide such dictionaries seperately. > > > > Or probably both. This way I can write 100% Smalltalk code and use a > very small compiler to read those dictionary variables for the type of the > variables and functions/structs (essentially a class will be output for a C > struct with pointers to functions for methods and variables for instance > variables). Why invent a whole new language when everything I need already > Pharo provides ? > > I could also use special variable dictionaries for all sort of things > like generation of header files, generation of CMake files for automatic > building. > > > > Also I like to use the way UFFI is doing C function signatures by using > symbol arrays. > > > > So thank you all for inspiration it looks like all I need is Pharo AST > methods (which I can from the AST packages) and SmaCC. > > So yeap looks like Magnatar will be a new Slang afterall, I will keep > you posted. > > > > Also this also opens the possibility of autowrapping the generated c > code back to Pharo through UFFI, so one can use C code as if its Pharo > code. I can leverage the TalkFFI project that does this already. Seems all > the pieces have fallen in their place. > > > > Keep the suggestions and advice coming, you guys are inspirational :D > > Do you intend that the Smalltalk code can be executed? This will > likely increase the complexity quite a bit. In the VM simulation we > end up creating a XSimulation subclass that provides the framework for > executing the smalltalk code, e.g simulating functions that are only > in C. > > There is also the problem of platform differences. Slang doesn't > really handle them well (pragmas can be used to indicated that methods > should only be compiled on certain platforms, and #ifdef type code can > be used, but it isn't enough). It would be nice to have a class that > provides cross platform functionality, and then platform specific > classes as required. > > HTH, > Alistair > >

Oct. 17, 2018

Re: [Pharo-users] compiling libraries for Pharo on macOS

by Dimitris Chloupis

yeah I apologise for the wrong example , LibC utilises the enviroment/system variables. Generally speaking all OSes have a PATH variable that defines the places where common libraries and tools are to be found. If you are on windows you can type in the command prompt , "PATH", in Linux and MacOS is "$PATH" . You can add your path in Windows using the system variables dialog window, in MacOS and Linux it usually happens via .bash-config file that setups the path but I am not so sure. The easiest way is to do something like macModuleName does in the LGitLibrary class the code is macModuleName | pluginDir | pluginDir := Smalltalk vm binary parent / 'Plugins'. #('libgit2.dylib' 'libgit2.0.dylib') detect: [ :each | (pluginDir / each) exists ] ifFound: [ :libName | ^ libName ]. self error: 'Module not found.' Hope that helps On Wed, Oct 17, 2018 at 12:54 PM Michel Onoff <michel.onoff(a)web.de> wrote: > Hi Dimitris, > > thanks for the long explanation. > > As you can see from my example, I build the library with the > -undefined dynamic_lookup > flag to clang, so this should be compatible with your (3) option if I > understand you correctly. Compiling/linking this way should allow the > use of dlopen(3) from UFFI during the execution of Pharo code, exactly > as I try to simulate with my small test programs. > > Pharo complains about not being able to load the module (the library), > even if it appears in the Pharo main folder. It does not report whether > it cannot find it or if it cannot bind it using dlopen(3). > > Moreover, it is unclear to me how Pharo interacts with DYLD_LIBRARY_PATH > (on macOS) or LD_LIBRARY_PATH (on Linux) environment variables if they > are set. > > In other words, Where does it search for libraries? > > > > Anyway, I'll try your script on cmake. > > > Regards > MO > > > > > for use with dlopen() > > On 2018-10-17 11:32, Dimitris Chloupis wrote: > > I advice first of all the use of CMake , you can thank me later. > > > > CMake if you are not aware is the standard build system for C/C++ , it > > makes it super easy to build and compile projects so you dont have to do > > what you are doing, call the compiler and pass the inifite amount of > > compiler flags that it needs. > > > > To understand how to build a library you have to understand first what a > > library is. > > > > Libraries used by Pharo are known as either Shared Libraries on Macos > > (dylib) and Linux (so) but Dynamically Linked Libraries on Windows > > (DLLs). They are executables designed to be called from another > > executable (this includes another library as well). > > > > Those libraries operate in 3 modes > > 1) Static linking > > 2) semi dynamic linking > > 3) dynamic linking > > > > (1) may come as suprise but yes those libraries can be linked to the > > executable and build inside the exrecutable. But in our case we done > > want that because it would mean to rebuild the VM. > > (2) is not what we want either because eventhough it builds the library > > independently generating the usual exetension files I mentioned above it > > includes its symbole table in the executable so you wont have to do what > > UFFI does, wrap or map your code to those library functions. The > > equivelant for Pharo would mean that we would not need UFFI at all but > > we do need to rebuld the VM to include ths symbol table. The symbole > > table basically includes the name of the functions, their signatures > > (type of arguments) and their corresponing adresses in memory needed to > > access them. > > > > (3) is what UFFI does, in that cause the executable is not affected at > > all so you dont need to to rebuild the vm. But you do need to know where > > to find the library , to access its symbol table and make sure the > > variables your executable uses match the type and signature in the > > symbol table or else the executable wont able to locate and correctly > > call these functions. Of course libraries can contain structs and global > > variables as well but the concept is the same. Now this messy part is > > handled by the UFFI although barely because it still needs from you to > > provide the correct signature to the function. > > > > I am giving you the long version because you have to make sure you build > > you library with (3) way and not (2). (1) can be easily avoid but it is > > easy to mix (2) and (3). From there on its easy , locating the library > > is provided as a string returned by a method , see how UFFI utilises > > LibC for example and from there on you just convert the signatures to > > symbol arrays as described by the UFFI tutorials. > > > > For cmake you only need to use -> add_library(libraryname MODULE ${SRC} > > ). You can find a CmakeList.txt file that I use here > > https://gist.github.com/kilon/ff4b973bdeb3b0c0253ac373a4b2506a > > > > If you are wondering why its so big, its because the project I link > > against is huge and because I move the DLL to its build folder instead > > of building it in the same folder as my source code. It CMake's why of > > keeping things seperate and organised. Observe also that I use dll > > exension for Windows and MacOS, The extensions plays no role so I > > decided to go for the same extension to save time although usually its > > preferable go for the standard extensions. > > > > Beware that if your library uses any external code even parts of the C > > standard library those will have to be included , usually you need only > > the header files, which is what SET INC is doing in my example, while > > SET SRC includes only the library files. The inclusion of headers is > > super important because they are used to build the symbol table which is > > the most important part of the library. Without a correctly build symbol > > table the library is useless even when used from C/C++. > > > > On Wed, Oct 17, 2018 at 11:09 AM Michel Onoff <michel.onoff(a)web.de > > <mailto:michel.onoff@web.de>> wrote: > > > > Hello, > > > > I've made some attempts to use the Pharo FFI. I've gone through the > > documentation [1] and tried some examples. > > > > While the documentation is quite clear about using external libraries > > from Pharo, it lacks details on how to build own libraries to be used > > from Pharo. > > > > Specifically, I'm trying to build on all supported platforms, namely > > macOS, Windows and Linux. Currently I've got Linux and Windows under > > control, but I cannot find a systematic way to build on macOS. > > > > Here's an example for a library that I can use from a small test > > program > > but not from Pharo. Unfortunately, the diagnostic on Pharo is not > > detailed: it simply reports that it cannot load the module. > > > > > > clang -I <include-path> -O3 -c <source-path> > > ... > > clang -dynamiclib <object-files> -o lib<name>.dylib -undefined > > dynamic_lookup > > > > cp lib<name>.dylib <pharo-folder> > > > > > > Is there good documentation about how to correctly build libraries so > > that they can be used from Pharo? > > > > Alternatively, are there details on how Pharo FFI searches for > > libraries > > and links them in the Pharo process? > > > > > > Regards > > MO > > > > ---- > > > > [1] https://files.pharo.org/books-pdfs/booklet-uFFI/UFFIDRAFT.pdf > > > > > > >

Oct. 17, 2018

Re: [Pharo-users] Installing SmaCC

by Alistair Grant

Hi Dimitris, As someone currently learning to use Slang (i.e. not an expert), I've added my 2c below... On Wed, 17 Oct 2018 at 11:06, Dimitris Chloupis <kilon.alios(a)gmail.com> wrote: > > Thierry you have done it !!! you just gave a very easy solution to my problems. > > Yeap Slang is quite close to what I am thinking, unfortunately Clement told me to stay away from it because the code is ugly and specially used for VM only. If I remember also correctly it does not generate readable C code either. But the idea as a concept is very close to what I imagine. I've found the C code produced to be quite readable, but that is probably influenced by the fact that I have read the slang first. > As a matter of fact you mentioning Slang made I have an epiphany that I dont have to create a new syntax at all, instead I could use specific variables or methods to provide type annotation. Thus like Slang I can use regular Smalltalk code that avoids changing types but without the need for type inference (although I am not excluding this either). > > So yes I am definetly want to move to the direction that Slang goes so I can fully utilise the Pharo IDE and minise code that I have to write. > > So basically I am thinking write code as you always write in Pharo and either > a) Have special dictionary variables in each method that provide static type annotations for the arguments of the methods, its return type and local variables Slang uses method pragmas to define the variables types. This seems to work quite well. > b) Have special methods that provide such dictionaries seperately. > > Or probably both. This way I can write 100% Smalltalk code and use a very small compiler to read those dictionary variables for the type of the variables and functions/structs (essentially a class will be output for a C struct with pointers to functions for methods and variables for instance variables). Why invent a whole new language when everything I need already Pharo provides ? > I could also use special variable dictionaries for all sort of things like generation of header files, generation of CMake files for automatic building. > > Also I like to use the way UFFI is doing C function signatures by using symbol arrays. > > So thank you all for inspiration it looks like all I need is Pharo AST methods (which I can from the AST packages) and SmaCC. > So yeap looks like Magnatar will be a new Slang afterall, I will keep you posted. > > Also this also opens the possibility of autowrapping the generated c code back to Pharo through UFFI, so one can use C code as if its Pharo code. I can leverage the TalkFFI project that does this already. Seems all the pieces have fallen in their place. > > Keep the suggestions and advice coming, you guys are inspirational :D Do you intend that the Smalltalk code can be executed? This will likely increase the complexity quite a bit. In the VM simulation we end up creating a XSimulation subclass that provides the framework for executing the smalltalk code, e.g simulating functions that are only in C. There is also the problem of platform differences. Slang doesn't really handle them well (pragmas can be used to indicated that methods should only be compiled on certain platforms, and #ifdef type code can be used, but it isn't enough). It would be nice to have a class that provides cross platform functionality, and then platform specific classes as required. HTH, Alistair

Oct. 17, 2018

Re: [Pharo-users] compiling libraries for Pharo on macOS

by Michel Onoff

Hi Dimitris, thanks for the long explanation. As you can see from my example, I build the library with the -undefined dynamic_lookup flag to clang, so this should be compatible with your (3) option if I understand you correctly. Compiling/linking this way should allow the use of dlopen(3) from UFFI during the execution of Pharo code, exactly as I try to simulate with my small test programs. Pharo complains about not being able to load the module (the library), even if it appears in the Pharo main folder. It does not report whether it cannot find it or if it cannot bind it using dlopen(3). Moreover, it is unclear to me how Pharo interacts with DYLD_LIBRARY_PATH (on macOS) or LD_LIBRARY_PATH (on Linux) environment variables if they are set. In other words, Where does it search for libraries? Anyway, I'll try your script on cmake. Regards MO for use with dlopen() On 2018-10-17 11:32, Dimitris Chloupis wrote: > I advice first of all the use of CMake , you can thank me later. > > CMake if you are not aware is the standard build system for C/C++ , it > makes it super easy to build and compile projects so you dont have to do > what you are doing, call the compiler and pass the inifite amount of > compiler flags that it needs. > > To understand how to build a library you have to understand first what a > library is. > > Libraries used by Pharo are known as either Shared Libraries on Macos > (dylib) and Linux (so) but Dynamically Linked Libraries on Windows > (DLLs). They are executables designed to be called from another > executable (this includes another library as well). > > Those libraries operate in 3 modes > 1) Static linking > 2) semi dynamic linking > 3) dynamic linking > > (1) may come as suprise but yes those libraries can be linked to the > executable and build inside the exrecutable. But in our case we done > want that because it would mean to rebuild the VM. > (2) is not what we want either because eventhough it builds the library > independently generating the usual exetension files I mentioned above it > includes its symbole table in the executable so you wont have to do what > UFFI does, wrap or map your code to those library functions. The > equivelant for Pharo would mean that we would not need UFFI at all but > we do need to rebuld the VM to include ths symbol table. The symbole > table basically includes the name of the functions, their signatures > (type of arguments) and their corresponing adresses in memory needed to > access them. > > (3) is what UFFI does, in that cause the executable is not affected at > all so you dont need to to rebuild the vm. But you do need to know where > to find the library , to access its symbol table and make sure the > variables your executable uses match the type and signature in the > symbol table or else the executable wont able to locate and correctly > call these functions. Of course libraries can contain structs and global > variables as well but the concept is the same. Now this messy part is > handled by the UFFI although barely because it still needs from you to > provide the correct signature to the function. > > I am giving you the long version because you have to make sure you build > you library with (3) way and not (2). (1) can be easily avoid but it is > easy to mix (2) and (3). From there on its easy , locating the library > is provided as a string returned by a method , see how UFFI utilises > LibC for example and from there on you just convert the signatures to > symbol arrays as described by the UFFI tutorials. > > For cmake you only need to use -> add_library(libraryname MODULE ${SRC} > ). You can find a CmakeList.txt file that I use here > https://gist.github.com/kilon/ff4b973bdeb3b0c0253ac373a4b2506a > > If you are wondering why its so big, its because the project I link > against is huge and because I move the DLL to its build folder instead > of building it in the same folder as my source code. It CMake's why of > keeping things seperate and organised. Observe also that I use dll > exension for Windows and MacOS, The extensions plays no role so I > decided to go for the same extension to save time although usually its > preferable go for the standard extensions. > > Beware that if your library uses any external code even parts of the C > standard library those will have to be included , usually you need only > the header files, which is what SET INC is doing in my example, while > SET SRC includes only the library files. The inclusion of headers is > super important because they are used to build the symbol table which is > the most important part of the library. Without a correctly build symbol > table the library is useless even when used from C/C++. > > On Wed, Oct 17, 2018 at 11:09 AM Michel Onoff <michel.onoff(a)web.de > <mailto:michel.onoff@web.de>> wrote: > > Hello, > > I've made some attempts to use the Pharo FFI. I've gone through the > documentation [1] and tried some examples. > > While the documentation is quite clear about using external libraries > from Pharo, it lacks details on how to build own libraries to be used > from Pharo. > > Specifically, I'm trying to build on all supported platforms, namely > macOS, Windows and Linux. Currently I've got Linux and Windows under > control, but I cannot find a systematic way to build on macOS. > > Here's an example for a library that I can use from a small test > program > but not from Pharo. Unfortunately, the diagnostic on Pharo is not > detailed: it simply reports that it cannot load the module. > > > clang -I <include-path> -O3 -c <source-path> > ... > clang -dynamiclib <object-files> -o lib<name>.dylib -undefined > dynamic_lookup > > cp lib<name>.dylib <pharo-folder> > > > Is there good documentation about how to correctly build libraries so > that they can be used from Pharo? > > Alternatively, are there details on how Pharo FFI searches for > libraries > and links them in the Pharo process? > > > Regards > MO > > ---- > > [1] https://files.pharo.org/books-pdfs/booklet-uFFI/UFFIDRAFT.pdf > >

Oct. 17, 2018

Re: [Pharo-users] compiling libraries for Pharo on macOS

by Dimitris Chloupis

I advice first of all the use of CMake , you can thank me later. CMake if you are not aware is the standard build system for C/C++ , it makes it super easy to build and compile projects so you dont have to do what you are doing, call the compiler and pass the inifite amount of compiler flags that it needs. To understand how to build a library you have to understand first what a library is. Libraries used by Pharo are known as either Shared Libraries on Macos (dylib) and Linux (so) but Dynamically Linked Libraries on Windows (DLLs). They are executables designed to be called from another executable (this includes another library as well). Those libraries operate in 3 modes 1) Static linking 2) semi dynamic linking 3) dynamic linking (1) may come as suprise but yes those libraries can be linked to the executable and build inside the exrecutable. But in our case we done want that because it would mean to rebuild the VM. (2) is not what we want either because eventhough it builds the library independently generating the usual exetension files I mentioned above it includes its symbole table in the executable so you wont have to do what UFFI does, wrap or map your code to those library functions. The equivelant for Pharo would mean that we would not need UFFI at all but we do need to rebuld the VM to include ths symbol table. The symbole table basically includes the name of the functions, their signatures (type of arguments) and their corresponing adresses in memory needed to access them. (3) is what UFFI does, in that cause the executable is not affected at all so you dont need to to rebuild the vm. But you do need to know where to find the library , to access its symbol table and make sure the variables your executable uses match the type and signature in the symbol table or else the executable wont able to locate and correctly call these functions. Of course libraries can contain structs and global variables as well but the concept is the same. Now this messy part is handled by the UFFI although barely because it still needs from you to provide the correct signature to the function. I am giving you the long version because you have to make sure you build you library with (3) way and not (2). (1) can be easily avoid but it is easy to mix (2) and (3). From there on its easy , locating the library is provided as a string returned by a method , see how UFFI utilises LibC for example and from there on you just convert the signatures to symbol arrays as described by the UFFI tutorials. For cmake you only need to use -> add_library(libraryname MODULE ${SRC} ). You can find a CmakeList.txt file that I use here https://gist.github.com/kilon/ff4b973bdeb3b0c0253ac373a4b2506a If you are wondering why its so big, its because the project I link against is huge and because I move the DLL to its build folder instead of building it in the same folder as my source code. It CMake's why of keeping things seperate and organised. Observe also that I use dll exension for Windows and MacOS, The extensions plays no role so I decided to go for the same extension to save time although usually its preferable go for the standard extensions. Beware that if your library uses any external code even parts of the C standard library those will have to be included , usually you need only the header files, which is what SET INC is doing in my example, while SET SRC includes only the library files. The inclusion of headers is super important because they are used to build the symbol table which is the most important part of the library. Without a correctly build symbol table the library is useless even when used from C/C++. On Wed, Oct 17, 2018 at 11:09 AM Michel Onoff <michel.onoff(a)web.de> wrote: > Hello, > > I've made some attempts to use the Pharo FFI. I've gone through the > documentation [1] and tried some examples. > > While the documentation is quite clear about using external libraries > from Pharo, it lacks details on how to build own libraries to be used > from Pharo. > > Specifically, I'm trying to build on all supported platforms, namely > macOS, Windows and Linux. Currently I've got Linux and Windows under > control, but I cannot find a systematic way to build on macOS. > > Here's an example for a library that I can use from a small test program > but not from Pharo. Unfortunately, the diagnostic on Pharo is not > detailed: it simply reports that it cannot load the module. > > > clang -I <include-path> -O3 -c <source-path> > ... > clang -dynamiclib <object-files> -o lib<name>.dylib -undefined > dynamic_lookup > > cp lib<name>.dylib <pharo-folder> > > > Is there good documentation about how to correctly build libraries so > that they can be used from Pharo? > > Alternatively, are there details on how Pharo FFI searches for libraries > and links them in the Pharo process? > > > Regards > MO > > ---- > > [1] https://files.pharo.org/books-pdfs/booklet-uFFI/UFFIDRAFT.pdf > > >

Oct. 17, 2018

Re: [Pharo-users] Installing SmaCC

by Dimitris Chloupis

Thierry you have done it !!! you just gave a very easy solution to my problems. Yeap Slang is quite close to what I am thinking, unfortunately Clement told me to stay away from it because the code is ugly and specially used for VM only. If I remember also correctly it does not generate readable C code either. But the idea as a concept is very close to what I imagine. As a matter of fact you mentioning Slang made I have an epiphany that I dont have to create a new syntax at all, instead I could use specific variables or methods to provide type annotation. Thus like Slang I can use regular Smalltalk code that avoids changing types but without the need for type inference (although I am not excluding this either). So yes I am definetly want to move to the direction that Slang goes so I can fully utilise the Pharo IDE and minise code that I have to write. So basically I am thinking write code as you always write in Pharo and either a) Have special dictionary variables in each method that provide static type annotations for the arguments of the methods, its return type and local variables b) Have special methods that provide such dictionaries seperately. Or probably both. This way I can write 100% Smalltalk code and use a very small compiler to read those dictionary variables for the type of the variables and functions/structs (essentially a class will be output for a C struct with pointers to functions for methods and variables for instance variables). Why invent a whole new language when everything I need already Pharo provides ? I could also use special variable dictionaries for all sort of things like generation of header files, generation of CMake files for automatic building. Also I like to use the way UFFI is doing C function signatures by using symbol arrays. So thank you all for inspiration it looks like all I need is Pharo AST methods (which I can from the AST packages) and SmaCC. So yeap looks like Magnatar will be a new Slang afterall, I will keep you posted. Also this also opens the possibility of autowrapping the generated c code back to Pharo through UFFI, so one can use C code as if its Pharo code. I can leverage the TalkFFI project that does this already. Seems all the pieces have fallen in their place. Keep the suggestions and advice coming, you guys are inspirational :D On Wed, Oct 17, 2018 at 11:07 AM Thierry Goubier <thierry.goubier(a)gmail.com> wrote: > Le mer. 17 oct. 2018 Ã 08:44, Dimitris Chloupis > <kilon.alios(a)gmail.com> a Ã©crit : > > > > Those are interesting languages and probably better ideas than what I am > thinking. However they are not what I am talking about. > > > > The language I am making is called "Magnatar" > > > > Magnatar is not Smalltalk, but it is a Smalltalk trojan horse. > > > > My main goals are > > > > 1) Create 100% readable C code, code that it looks like it was written > by a person and not automatically generated , leaving no room for suspicion > > 2) Keep as close to Smalltalk as I can > > 3) Map to C syntax , one to one > > 4) Implement a loose variant of OOP > > 5) Powerful macro system for source manipulation without the need for > header files > > 6) CMake integration (automate building which a headache in C/C++) > > > > Essentially I am trying to make a language that can allow a smalltalker > to write C in Smalltalk and none noticing that he did. Hence why I talked > about a trojan horse. I got the live coding part covered already but I > would also like to use Pharo as the IDE of the language if I can. Also > because of (3) Magnatar will be purely statically typed not because I have > any love for static types but because it makes (3) far easier to > accomplish. So Magnatar wont be a Smalltalk implementation neither a > replacement for it. > > I believe you're trying to rewrite SLANG... > > Maybe you can reuse part of it. > > Making a better SLANG would certainly help, in more areas than one! > > Thierry > > This means also that Magnatar will be a way to extend Pharo on low > level. Writting readable C using Smalltalky syntax for extensions instead > of just writting in C and then using Pharo's UFFI. Of course this will > benefit those that want to squeeze much more performance out of Pharo. > > > Magnatar also is not another language trying to take over the world but > a language I will use to write C code for a project I am working on which > is 3d graphics and I need the full performance. So in the end it will be a > personal language. > > > > Unfortunately this means no dynamic types at all, no GC , no VM, no > bytecodes, not intepretation of any sort, no image format. The only thing I > am keeping will be live coding. Also OOP will be basically super simple, > only c structs and c functions with a small API to allow you to "compose" > objects similarly to prototype based OOP but I am considering some syntax > for basic classes also. Again all these are under the prime goal of 100% > readable idiomatic C code. The rule is simple "if it cannot easily map to C > , it wont get in". So this is a highly focused language. > > > > I have not found yet a language that compiles to readable C code > although there a lot of them that compile to unreadable C (nim, ECL, > chicken scheme, etc). > > > > On Wed, Oct 17, 2018 at 8:37 AM H. Hirzel <hannes.hirzel(a)gmail.com> > wrote: > >> > >> The successor of Ni is 'Spry' > >> > >> https://github.com/gokr/spry > >> http://sprylang.se/ > >> > >> "Spry borrows homoiconicity from Rebol and Lisp, free form syntax from > >> Forth and Rebol, the word of different types from Rebol, good data > >> structure literal support from JavaScript and the general coding > >> experience and style from Smalltalk. It also has a few ideas of its > >> own, like an interesting argument passing mechanism and a relatively > >> novel take on OO." > >> > >> --Hannes > >> > >> On 10/17/18, Ben Coman <btc(a)openinworld.com> wrote: > >> > Have you looked at Ni? (I only read about it) > >> > http://goran.krampe.se/2015/09/16/ni-a-strange-little-language/ > >> > > >> > cheers -ben > >> > > >> > On Wed, 17 Oct 2018 at 03:45, Dimitris Chloupis < > kilon.alios(a)gmail.com> > >> > wrote: > >> > > >> >> Thank you Thierry , that was exactly what i was looking for :) > >> >> > >> >> On the subject of syntax, StrongTalk looks far more advanced > compared to > >> >> what I am aiming which is basically writting C code with Smalltalk > like > >> >> syntax. I am looking at this > >> >> > >> >> http://bracha.org/nwst.html > >> >> > >> >> Which describes some really impressive features. So I am aiming only > for > >> >> source to source compiler and not implementation of complex systems > for > >> >> incremental compilations , optional type system etc. > >> >> > >> >> On parsing strange code that is not much of an issue cause the > project I > >> >> am working on has pretty reasonable code and will probably offer a > way to > >> >> inline c code in case the parser fail. In any case my goals are > small , > >> >> cause I dont have resources for complex implementations. Its also a > >> >> language that will be designed solely for my needs and be offered > open > >> >> source for anyone else that may find it useful. In any case I am > sure I > >> >> will have many questions to ask :) > >> >> > >> >> I was looking into ANTLR , since the book I am reading on language > design > >> >> is using ANTLR but I rather implement this in Pharo. I used SmaCC > when I > >> >> was working for my Python bridge and I really liked it , mostly > because > >> >> it > >> >> offers ready made syntax definitions for most popular languages. > Which > >> >> makes my life a lot easier. > >> >> > >> >> > >> >> On Tue, Oct 16, 2018 at 9:45 PM Thierry Goubier > >> >> <thierry.goubier(a)gmail.com> > >> >> wrote: > >> >> > >> >>> Hi Dimitris, > >> >>> > >> >>> Le 16/10/2018 Ã 19:39, Dimitris Chloupis a Ã©crit : > >> >>> > yes i already said that i followed the instructions in the github > repo > >> >>> > >> >>> Yes, by default that installation of SmaCC does not load all parsers > >> >>> (some of them are fairly large). However, most of them are in the > >> >>> downloaded repository, so you can load them independently. > >> >>> > >> >>> Otherwise, loading that way, should load everything: > >> >>> > >> >>> Metacello new > >> >>> baseline: 'SmaCC'; > >> >>> repository: 'github://SmaCCRefactoring/SmaCC'; > >> >>> load: #('Tools' 'Examples' 'Examples-Extra') > >> >>> > >> >>> Regarding your language question, I'd suggest two things: > >> >>> > >> >>> - Look at StrongTalk for a way to write Smalltalk with type > >> >>> declarations... > >> >>> > >> >>> - C parsers able to parse most strange C code one may encounter > takes > >> >>> some work... > >> >>> > >> >>> Regards, > >> >>> > >> >>> Thierry > >> >>> > >> >>> > On Tue, Oct 16, 2018 at 8:18 PM H. Hirzel < > hannes.hirzel(a)gmail.com > >> >>> > <mailto:hannes.hirzel@gmail.com>> wrote: > >> >>> > > >> >>> > Refers to > >> >>> > https://github.com/SmaCCRefactoring/SmaCC > >> >>> > > >> >>> > which says > >> >>> > > >> >>> > This is the port for Smalltalk/Pharo 1.3, 2, 3, 4, 5 > and 6. > >> >>> > > >> >>> > > >> >>> > Installing a Development version of Pharo for the latest Pharo > >> >>> > (with > >> >>> > no guarantees): > >> >>> > > >> >>> > Metacello new > >> >>> > baseline: 'SmaCC'; > >> >>> > repository: 'github://SmaCCRefactoring/SmaCC'; > >> >>> > load > >> >>> > > >> >>> > On 10/16/18, H. Hirzel <hannes.hirzel(a)gmail.com > >> >>> > <mailto:hannes.hirzel@gmail.com>> wrote: > >> >>> > > What about trying > >> >>> > > > >> >>> > > > >> >>> > > Metacello new > >> >>> > > baseline: 'SmaCC'; > >> >>> > > repository: 'github://ThierryGoubier/SmaCC'; > >> >>> > > load > >> >>> > > > >> >>> > > This worked in Pharo 6.1 in November 2017 > >> >>> > > > >> >>> > > On 10/16/18, Dimitris Chloupis <kilon.alios(a)gmail.com > >> >>> > <mailto:kilon.alios@gmail.com>> wrote: > >> >>> > >> thanks for the info Peter , will give it a try :) > >> >>> > >> > >> >>> > >> On Tue, Oct 16, 2018 at 7:35 PM PBKResearch > >> >>> > <peter(a)pbkresearch.co.uk <mailto:peter@pbkresearch.co.uk>> > >> >>> > >> wrote: > >> >>> > >> > >> >>> > >>> Dimitris > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> If you download the latest Moose Suite 6.1, you will have > >> >>> > Pharo > >> >>> > 6.1 with > >> >>> > >>> lots of extra packages, including SmaCC. The SmaCC > includes > >> >>> > compilers > >> >>> > >>> for > >> >>> > >>> C, Smalltalk and Java, among others, but with little or > no > >> >>> > >>> documentation. > >> >>> > >>> I > >> >>> > >>> am not a SmaCC expert, so I canât say whether it will do > what > >> >>> > you want, > >> >>> > >>> but > >> >>> > >>> at least it will give you a start. Moose also includes > >> >>> > PetitParser and > >> >>> > >>> PP2,if you want to try other parsing approaches. Of > course, > >> >>> > the > >> >>> > Windows > >> >>> > >>> version is 32-bit only, for reasons explained elsewhere > in > >> >>> > this > >> >>> > thread. > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> HTH > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> Peter Kenny > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> *From:* Pharo-users <pharo-users-bounces(a)lists.pharo.org > >> >>> > <mailto:pharo-users-bounces@lists.pharo.org>> *On Behalf Of > >> >>> > >>> *Dimitris > >> >>> > >>> Chloupis > >> >>> > >>> *Sent:* 16 October 2018 15:40 > >> >>> > >>> *To:* Any question about pharo is welcome > >> >>> > <pharo-users(a)lists.pharo.org <mailto: > pharo-users(a)lists.pharo.org>> > >> >>> > >>> *Subject:* [Pharo-users] Installing SmaCC > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> Hey guys > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> I downloaded the latest Pharo 6.1 64bit for Windows and > tried > >> >>> > to install > >> >>> > >>> SmaCC through the catalog browser but it failed > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> I did manage to install it following the instruction in > the > >> >>> > github repo > >> >>> > >>> but I see that I am missing most parser packages. > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> The languages I am interested are Smalltalk (which is > >> >>> > included) > >> >>> > and C > >> >>> > >>> (if > >> >>> > >>> possible C++ too) cause I will be creating a new language > >> >>> > which > >> >>> > will be > >> >>> > >>> a > >> >>> > >>> cross between C and Smalltalk (very similar to smalltalk > >> >>> > syntax > >> >>> > but with > >> >>> > >>> the addtion of C types and no GC and dynamic typing and > also > >> >>> > a > >> >>> > partial > >> >>> > >>> implementation of OOP that is quite diffirent). My goal > is > >> >>> > compilation > >> >>> > >>> of > >> >>> > >>> my language to readable C code so the ability to parse > also > >> >>> > existing C > >> >>> > >>> code > >> >>> > >>> is needed. > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> > >> >>> > >>> Any help is greatly appreciated , thanks :) > >> >>> > >>> > >> >>> > >> > >> >>> > > > >> >>> > > >> >>> > >> >>> > >> >>> > >> > > >> > >

Oct. 17, 2018

compiling libraries for Pharo on macOS

by Michel Onoff

Hello, I've made some attempts to use the Pharo FFI. I've gone through the documentation [1] and tried some examples. While the documentation is quite clear about using external libraries from Pharo, it lacks details on how to build own libraries to be used from Pharo. Specifically, I'm trying to build on all supported platforms, namely macOS, Windows and Linux. Currently I've got Linux and Windows under control, but I cannot find a systematic way to build on macOS. Here's an example for a library that I can use from a small test program but not from Pharo. Unfortunately, the diagnostic on Pharo is not detailed: it simply reports that it cannot load the module. clang -I <include-path> -O3 -c <source-path> ... clang -dynamiclib <object-files> -o lib<name>.dylib -undefined dynamic_lookup cp lib<name>.dylib <pharo-folder> Is there good documentation about how to correctly build libraries so that they can be used from Pharo? Alternatively, are there details on how Pharo FFI searches for libraries and links them in the Pharo process? Regards MO ---- [1] https://files.pharo.org/books-pdfs/booklet-uFFI/UFFIDRAFT.pdf

Oct. 17, 2018

Re: [Pharo-users] Installing SmaCC

by Thierry Goubier

Le mer. 17 oct. 2018 Ã 08:44, Dimitris Chloupis <kilon.alios(a)gmail.com> a Ã©crit : > > Those are interesting languages and probably better ideas than what I am thinking. However they are not what I am talking about. > > The language I am making is called "Magnatar" > > Magnatar is not Smalltalk, but it is a Smalltalk trojan horse. > > My main goals are > > 1) Create 100% readable C code, code that it looks like it was written by a person and not automatically generated , leaving no room for suspicion > 2) Keep as close to Smalltalk as I can > 3) Map to C syntax , one to one > 4) Implement a loose variant of OOP > 5) Powerful macro system for source manipulation without the need for header files > 6) CMake integration (automate building which a headache in C/C++) > > Essentially I am trying to make a language that can allow a smalltalker to write C in Smalltalk and none noticing that he did. Hence why I talked about a trojan horse. I got the live coding part covered already but I would also like to use Pharo as the IDE of the language if I can. Also because of (3) Magnatar will be purely statically typed not because I have any love for static types but because it makes (3) far easier to accomplish. So Magnatar wont be a Smalltalk implementation neither a replacement for it. I believe you're trying to rewrite SLANG... Maybe you can reuse part of it. Making a better SLANG would certainly help, in more areas than one! Thierry > This means also that Magnatar will be a way to extend Pharo on low level. Writting readable C using Smalltalky syntax for extensions instead of just writting in C and then using Pharo's UFFI. Of course this will benefit those that want to squeeze much more performance out of Pharo. > Magnatar also is not another language trying to take over the world but a language I will use to write C code for a project I am working on which is 3d graphics and I need the full performance. So in the end it will be a personal language. > > Unfortunately this means no dynamic types at all, no GC , no VM, no bytecodes, not intepretation of any sort, no image format. The only thing I am keeping will be live coding. Also OOP will be basically super simple, only c structs and c functions with a small API to allow you to "compose" objects similarly to prototype based OOP but I am considering some syntax for basic classes also. Again all these are under the prime goal of 100% readable idiomatic C code. The rule is simple "if it cannot easily map to C , it wont get in". So this is a highly focused language. > > I have not found yet a language that compiles to readable C code although there a lot of them that compile to unreadable C (nim, ECL, chicken scheme, etc). > > On Wed, Oct 17, 2018 at 8:37 AM H. Hirzel <hannes.hirzel(a)gmail.com> wrote: >> >> The successor of Ni is 'Spry' >> >> https://github.com/gokr/spry >> http://sprylang.se/ >> >> "Spry borrows homoiconicity from Rebol and Lisp, free form syntax from >> Forth and Rebol, the word of different types from Rebol, good data >> structure literal support from JavaScript and the general coding >> experience and style from Smalltalk. It also has a few ideas of its >> own, like an interesting argument passing mechanism and a relatively >> novel take on OO." >> >> --Hannes >> >> On 10/17/18, Ben Coman <btc(a)openinworld.com> wrote: >> > Have you looked at Ni? (I only read about it) >> > http://goran.krampe.se/2015/09/16/ni-a-strange-little-language/ >> > >> > cheers -ben >> > >> > On Wed, 17 Oct 2018 at 03:45, Dimitris Chloupis <kilon.alios(a)gmail.com> >> > wrote: >> > >> >> Thank you Thierry , that was exactly what i was looking for :) >> >> >> >> On the subject of syntax, StrongTalk looks far more advanced compared to >> >> what I am aiming which is basically writting C code with Smalltalk like >> >> syntax. I am looking at this >> >> >> >> http://bracha.org/nwst.html >> >> >> >> Which describes some really impressive features. So I am aiming only for >> >> source to source compiler and not implementation of complex systems for >> >> incremental compilations , optional type system etc. >> >> >> >> On parsing strange code that is not much of an issue cause the project I >> >> am working on has pretty reasonable code and will probably offer a way to >> >> inline c code in case the parser fail. In any case my goals are small , >> >> cause I dont have resources for complex implementations. Its also a >> >> language that will be designed solely for my needs and be offered open >> >> source for anyone else that may find it useful. In any case I am sure I >> >> will have many questions to ask :) >> >> >> >> I was looking into ANTLR , since the book I am reading on language design >> >> is using ANTLR but I rather implement this in Pharo. I used SmaCC when I >> >> was working for my Python bridge and I really liked it , mostly because >> >> it >> >> offers ready made syntax definitions for most popular languages. Which >> >> makes my life a lot easier. >> >> >> >> >> >> On Tue, Oct 16, 2018 at 9:45 PM Thierry Goubier >> >> <thierry.goubier(a)gmail.com> >> >> wrote: >> >> >> >>> Hi Dimitris, >> >>> >> >>> Le 16/10/2018 Ã 19:39, Dimitris Chloupis a Ã©crit : >> >>> > yes i already said that i followed the instructions in the github repo >> >>> >> >>> Yes, by default that installation of SmaCC does not load all parsers >> >>> (some of them are fairly large). However, most of them are in the >> >>> downloaded repository, so you can load them independently. >> >>> >> >>> Otherwise, loading that way, should load everything: >> >>> >> >>> Metacello new >> >>> baseline: 'SmaCC'; >> >>> repository: 'github://SmaCCRefactoring/SmaCC'; >> >>> load: #('Tools' 'Examples' 'Examples-Extra') >> >>> >> >>> Regarding your language question, I'd suggest two things: >> >>> >> >>> - Look at StrongTalk for a way to write Smalltalk with type >> >>> declarations... >> >>> >> >>> - C parsers able to parse most strange C code one may encounter takes >> >>> some work... >> >>> >> >>> Regards, >> >>> >> >>> Thierry >> >>> >> >>> > On Tue, Oct 16, 2018 at 8:18 PM H. Hirzel <hannes.hirzel(a)gmail.com >> >>> > <mailto:hannes.hirzel@gmail.com>> wrote: >> >>> > >> >>> > Refers to >> >>> > https://github.com/SmaCCRefactoring/SmaCC >> >>> > >> >>> > which says >> >>> > >> >>> > This is the port for Smalltalk/Pharo 1.3, 2, 3, 4, 5 and 6. >> >>> > >> >>> > >> >>> > Installing a Development version of Pharo for the latest Pharo >> >>> > (with >> >>> > no guarantees): >> >>> > >> >>> > Metacello new >> >>> > baseline: 'SmaCC'; >> >>> > repository: 'github://SmaCCRefactoring/SmaCC'; >> >>> > load >> >>> > >> >>> > On 10/16/18, H. Hirzel <hannes.hirzel(a)gmail.com >> >>> > <mailto:hannes.hirzel@gmail.com>> wrote: >> >>> > > What about trying >> >>> > > >> >>> > > >> >>> > > Metacello new >> >>> > > baseline: 'SmaCC'; >> >>> > > repository: 'github://ThierryGoubier/SmaCC'; >> >>> > > load >> >>> > > >> >>> > > This worked in Pharo 6.1 in November 2017 >> >>> > > >> >>> > > On 10/16/18, Dimitris Chloupis <kilon.alios(a)gmail.com >> >>> > <mailto:kilon.alios@gmail.com>> wrote: >> >>> > >> thanks for the info Peter , will give it a try :) >> >>> > >> >> >>> > >> On Tue, Oct 16, 2018 at 7:35 PM PBKResearch >> >>> > <peter(a)pbkresearch.co.uk <mailto:peter@pbkresearch.co.uk>> >> >>> > >> wrote: >> >>> > >> >> >>> > >>> Dimitris >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> If you download the latest Moose Suite 6.1, you will have >> >>> > Pharo >> >>> > 6.1 with >> >>> > >>> lots of extra packages, including SmaCC. The SmaCC includes >> >>> > compilers >> >>> > >>> for >> >>> > >>> C, Smalltalk and Java, among others, but with little or no >> >>> > >>> documentation. >> >>> > >>> I >> >>> > >>> am not a SmaCC expert, so I canât say whether it will do what >> >>> > you want, >> >>> > >>> but >> >>> > >>> at least it will give you a start. Moose also includes >> >>> > PetitParser and >> >>> > >>> PP2,if you want to try other parsing approaches. Of course, >> >>> > the >> >>> > Windows >> >>> > >>> version is 32-bit only, for reasons explained elsewhere in >> >>> > this >> >>> > thread. >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> HTH >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> Peter Kenny >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> *From:* Pharo-users <pharo-users-bounces(a)lists.pharo.org >> >>> > <mailto:pharo-users-bounces@lists.pharo.org>> *On Behalf Of >> >>> > >>> *Dimitris >> >>> > >>> Chloupis >> >>> > >>> *Sent:* 16 October 2018 15:40 >> >>> > >>> *To:* Any question about pharo is welcome >> >>> > <pharo-users(a)lists.pharo.org <mailto:pharo-users@lists.pharo.org>> >> >>> > >>> *Subject:* [Pharo-users] Installing SmaCC >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> Hey guys >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> I downloaded the latest Pharo 6.1 64bit for Windows and tried >> >>> > to install >> >>> > >>> SmaCC through the catalog browser but it failed >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> I did manage to install it following the instruction in the >> >>> > github repo >> >>> > >>> but I see that I am missing most parser packages. >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> The languages I am interested are Smalltalk (which is >> >>> > included) >> >>> > and C >> >>> > >>> (if >> >>> > >>> possible C++ too) cause I will be creating a new language >> >>> > which >> >>> > will be >> >>> > >>> a >> >>> > >>> cross between C and Smalltalk (very similar to smalltalk >> >>> > syntax >> >>> > but with >> >>> > >>> the addtion of C types and no GC and dynamic typing and also >> >>> > a >> >>> > partial >> >>> > >>> implementation of OOP that is quite diffirent). My goal is >> >>> > compilation >> >>> > >>> of >> >>> > >>> my language to readable C code so the ability to parse also >> >>> > existing C >> >>> > >>> code >> >>> > >>> is needed. >> >>> > >>> >> >>> > >>> >> >>> > >>> >> >>> > >>> Any help is greatly appreciated , thanks :) >> >>> > >>> >> >>> > >> >> >>> > > >> >>> > >> >>> >> >>> >> >>> >> > >>

Oct. 17, 2018