- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] UnifiedFFI Docs

by Todd Blanchard

Hey Esteban, Now that you have outed yourself as a maintainer of FFI, I have a question. :-) I want to call the libclang function: /** * \brief Parse the given source file and the translation unit corresponding * to that file. * * This routine is the main entry point for the Clang C API, providing the * ability to parse a source file into a translation unit that can then be * queried by other functions in the API. This routine accepts a set of * command-line arguments so that the compilation can be configured in the same * way that the compiler is configured on the command line. * * \param CIdx The index object with which the translation unit will be * associated. * * \param source_filename The name of the source file to load, or NULL if the * source file is included in \c command_line_args. * * \param command_line_args The command-line arguments that would be * passed to the \c clang executable if it were being invoked out-of-process. * These command-line options will be parsed and will affect how the translation * unit is parsed. Note that the following options are ignored: '-c', * '-emit-ast', '-fsyntax-only' (which is the default), and '-o \<output file>'. * * \param num_command_line_args The number of command-line arguments in * \c command_line_args. * * \param unsaved_files the files that have not yet been saved to disk * but may be required for parsing, including the contents of * those files. The contents and name of these files (as specified by * CXUnsavedFile) are copied when necessary, so the client only needs to * guarantee their validity until the call to this function returns. * * \param num_unsaved_files the number of unsaved file entries in \p * unsaved_files. * * \param options A bitmask of options that affects how the translation unit * is managed but not its compilation. This should be a bitwise OR of the * CXTranslationUnit_XXX flags. * * \param[out] out_TU A non-NULL pointer to store the created * \c CXTranslationUnit, describing the parsed code and containing any * diagnostics produced by the compiler. * * \returns Zero on success, otherwise returns an error code. */ enum CXErrorCode clang_parseTranslationUnit2(CXIndex CIdx, const char *source_filename, const char *const *command_line_args, int num_command_line_args, struct CXUnsavedFile *unsaved_files, unsigned num_unsaved_files, unsigned options, CXTranslationUnit *out_TU); The parts I'm not clear on are turning an array or ordered collection of strings into an argv/argc, turning an array of CXUnsavedFile's (FFIExternalStruct) into an array of pointers and count, and getting data back in an output argument as CXTranslationUnit which is an opaque pointer to a C++ object. Any tips on these mappings would be great. I've currently guessed it as: clang_parseTranslationUnit__cxIndex: cxIndex source: sourceFile arguments: argv unsavedFiles:unsaved options: opts | argc unsaved_files num_unsaved_files options error out_TU | argc := argv size. unsaved_files := unsaved ifNil: [ #() ] ifNotNil: [ unsaved ]. num_unsaved_files := unsaved_files ifNil: [ 0 ] ifNotNil: [unsaved_files size]. options := opts sum. error := self ffiCall: #(CXErrorCode clang_parseTranslationUnit2(CXIndex cxIndex, String sourceFile, String *argv, uint argc, CXUnsavedFile *unsaved_files, uint num_unsaved_files, CXTranslationUnitFlag options, CXTranslationUnit *out_TU)). (error = CXError_Success) ifTrue: [ ^out_TU ]. self error: error Thanks. > On Oct 22, 2017, at 2:42 AM, Esteban Lorenzano <estebanlm(a)gmail.com> wrote: > > Hi, > >> On 21 Oct 2017, at 21:15, Stephane Ducasse <stepharo.self(a)gmail.com> wrote: >> >> Esteban will probably reply to this thread > > yes, I will :) > >> >> >> On Thu, Oct 19, 2017 at 10:34 PM, Todd Blanchard <tblanchard(a)mac.com> wrote: >>> I have found the problem with VoidPointer3 generating accessors. > > can you report that on fogbugz ? > (along with your solution) > > thanks! > Esteban > >>> >>> No idea how to contribute back a fix but this is what I've found. >>> >>> FFITypeArrayType>>annonymousClassCreator >>> ^ String streamContents: [ :stream | >>> stream >>> nextPutAll: '(FFITypeArray ofType: '; >>> print: (self objectClass type isPointer ifTrue: [self externalTypeWithArity >>> printString] ifFalse: ['#',self objectClass type class]); >>> nextPutAll: ' size: '; >>> print: self objectClass numberOfElements; >>> nextPutAll: ')' ] >>> >>> Recalling that we are trying to come up with an accessor that can pull out a >>> void*[3], this produces >>> >>> '(FFITypeArray ofType: #FFIVoid size: 3)' >>> >>> which produces an error as FFIVoid's size is undefined. In general, this >>> will be wrong for any pointer type and will probably get the size >>> calculation wrong. >>> >>> Doing a little digging I find that >>> >>> FFITypeArray>>ofType: aType size: aSize >>> >>> delegates resolution of aType to FFIExternalType>>resolveType:aType and this >>> can take all kinds of different things including the native type name. >>> >>> It would be better if this generated the native name for pointers. >>> >>> (FFITypeArray ofType: 'void*' size: 3) >>> >>> So I changed it to: >>> >>> FFITypeArrayType>>annonymousClassCreator >>> ^ String streamContents: [ :stream | >>> stream >>> nextPutAll: '(FFITypeArray ofType: '; >>> print: (self objectClass type isPointer >>> ifTrue: [self externalTypeWithArity printString] >>> ifFalse: ['#',self objectClass type class]); >>> nextPutAll: ' size: '; >>> print: self objectClass numberOfElements; >>> nextPutAll: ')' ] >>> >>> and this seems to work fine. >>> >>> Onwards... >>> >>> On Oct 19, 2017, at 7:18 AM, Todd Blanchard <tblanchard(a)mac.com> wrote: >>> >>> Thatâs great - itâs been kind of magical but a couple things have changed >>> since you wrote it and some bugs have crept into ffi. >>> >>> Right now Iâm finding the generated accessor for the VoidPointer3 is >>> actually generating a void 3 accessor and that doesnât work. I spent all day >>> yesterday tracking it to the accessor generating code. >>> >>> Sent from the road >>> >>> On Oct 18, 2017, at 22:29, Ben Coman <btc(a)openinworld.com> wrote: >>> >>> >>> >>> On Thu, Oct 19, 2017 at 1:05 AM, Todd Blanchard <tblanchard(a)mac.com> wrote: >>>> >>>> I'm working through Ben's great blog post about playing with libclang and >>>> I am puzzled by something. >>> >>> >>> Thx Todd. Knowing someone is looking at it encourages me to expand it. >>> I'm interested in looking at your specific questions, but it might not be >>> until next week after this stint of long work days. >>> >>> cheers -ben >>> >>> >>>> >>>> >>>> invalidateSessionData >>>> handle atAllPut: 0. >>>> >>>> zero;s out the handle. Cool. However, >>>> >>>> handle isNull >>>> >>>> does not return true despite it being used in the getString method as >>>> >>>> ^ handle isNull >>>> ifTrue: ['external memory invalidated by session restart'] >>>> ifFalse:[LibClang clang_getCString__cxString: self]. >>>> >>>> Looks like there should be an isNull on ByteArray that returns true if all >>>> bytes are zero but it isn't there. Was it dropped for some reason? >>>> >>>> >>>> >>>> On Oct 18, 2017, at 2:58 AM, Dimitris Chloupis <kilon.alios(a)gmail.com> >>>> wrote: >>>> >>>> Sure the documentation could be better, that is definetly important, but >>>> is already good enough and UFFI is a very technical subject much more suited >>>> to a mailing list . Its not physical possible to cover the massive potential >>>> of UFFI. >>>> >>>> On Wed, Oct 18, 2017 at 9:32 AM Stephane Ducasse <stepharo.self(a)gmail.com> >>>> wrote: >>>>> >>>>> Please do not hesitate to do Pull Requests. >>>>> Luc told me that he wants to do a pass on it and Esteban promises that to >>>>> me >>>>> but he is super busy. >>>>> >>>>> Stef >>>>> >>>>> >>>>> On Wed, Oct 18, 2017 at 5:54 AM, Todd Blanchard <tblanchard(a)mac.com> >>>>> wrote: >>>>>> Wonderful! Thanks. >>>>>> >>>>>>> On Oct 17, 2017, at 3:45 PM, stephan <stephan(a)stack.nl> wrote: >>>>>>> >>>>>>> On 17-10-17 23:06, Todd Blanchard wrote: >>>>>>>> Anyone know what happened to this? >>>>>>>> >>>>>>>> https://ci.inria.fr/pharo-contribution/view/Books/job/PharoBookWorkInProgre… >>>>>>> >>>>>>> https://github.com/SquareBracketAssociates/Booklet-uFFI >>>>>>> >>>>>>> has a link to a bintray pdf download >>>>>>> >>>>>>> Stephan >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> >> > >

Oct. 24, 2017

Re: [Pharo-dev] latest Cryptography package integration

by henry

I am grateful for your interest. The issue is conflicting requirements: 1 - same package for Squeak and Pharo. 2 - X509 support is spurious but RSA depends on ASN.1 support 3 - Pharo [and Squeak!] both have hash functions and DigitalSecureAlgorithm that may get overwritten, leading to... 4 - Both Squeak and Pharo should have a common basis of crypto in both minimal images. 5 - Bootstrap loads a separately repositoried [in SmalltalkHub] Cryptography fork version 50, from version 36 common. Andso... 6 - we need a) common basic loaded, common support, b) common crypto-full loadable by Squeak, Pharo [and Bootstrap!] and c) removal of Undeclared. I think this sums up our situation. Would others be willing to commit to make these basis changes, in both Squeak and Pharo? Thank you, Henry On Tue, Oct 24, 2017 at 12:55, Stephane Ducasse <[stepharo.self(a)gmail.com]("mailto:stepharo.self@gmail.com")> wrote: > I would like the package to be working for Pharo. Now from your description I have problem to really understand the situation. I think that Pharo core should have minimum and packages should be able to be loaded nicely on top. > Currently, SHA1, SHA256 and MD5 all get redefined by Cryptography-zzz.111.mcz loading. Do you mean that the version in Pharo are older than the one in 111? Stef On Tue, Oct 24, 2017 at 6:17 PM, henry wrote: > I am curious as to integration of the newest Cryptography package there into > Pharo. There are two Undeclared: SoundRecorder and FillInTheBlank, though I > do not know if such would be a part of a Pharo-Squeak compatibility layer. > > More pertinent would be what to do with the HashFunctions that already > reside in the Pharo image. If the decision was to include the entire > Cryptography into base, then making the HashFunctions not go Obselete would > be the objective. > > Currently, SHA1, SHA256 and MD5 all get redefined by > Cryptography-zzz.111.mcz loading. > > What do you think about bringing Cryptography up to date for Pharo and what > would be needed. It would help to seek common ground between Squeak and > Pharo, yes? > > > - HH > > @callistohouse.club>

Oct. 24, 2017

Re: [Pharo-dev] [ANNOUNCE] ParrotTalk release/design change considerations

by henry

In my tests, at the moment. Only there. I derived the ParrotTalk-3.5 spec from erightâs.orgâs ELib implementation. Do you think I should change the spec to allow for free, anonymous connections? - HH On Tue, Oct 24, 2017 at 12:52, Stephane Ducasse <[stepharo.self(a)gmail.com]("mailto:stepharo.self@gmail.com")> wrote: > Hi henry thanks for this announce. Can you tell where such protocols are used? Stef On Tue, Oct 24, 2017 at 6:33 PM, henry wrote: > Hi all, > > I am happy to announce the release of version 3.5 of ParrotTalk, for Squeak > and Pharo, found here: > > http://www.squeaksource.com/Cryptography/ParrotTalk-zzz.2.mcz > > It follows this specification: > https://github.com/ZiroZimbarra/callistohouse/blob/master/docs/ParrotTalkFr… > > One item of note, in version 3.5, the system connecting to a server, sending > the IWant msg, must know the vatId of the system being connected to. I am > considering changing this to version 3.6 by removing one round-trip in > messaging. Therefore, these messages would be combined: IWant/GiveInfo, > IAm/ReplyInfo. I will keep ProtocolOffered and ProtocolAccepted to allow > eLindaSession to support both versions: 3.5 and 3.6. > > Thoughts please? > > - HH @callistohouse.club>

Oct. 24, 2017

Re: [Pharo-dev] [ANNOUNCE] ParrotTalk release/design change considerations

by henry

Oh! I was U.N.familiar with Parrot drones. Thatâs awesome! Well how about using encrypted communications with a Parrot drone? This is the purpose and accomplishment of ParrotTalk, encrypted communications. In 33 classes, minus 4 test classes with 16 tests (same #, 16, as eLinda!), ParrotTalk delivers 2048-bit key negotiation to encrypt communications with user-selected (through the SessionAgentMap - see tests) cryptoProtocol [AESede, DESede, DES], as well as allowing user-selected encoding [asn1der, Bytes {JSON}] of content data. I am considering changing the spec to 3.6, as I mentioned. - HH On Tue, Oct 24, 2017 at 12:44, Alexandre Bergel <[alexandre.bergel(a)me.com]("mailto:alexandre.bergel@me.com")> wrote: > Hi! What ParrotTalk is about? It does not seem related to the drone stuff (https://www.parrot.com) ? (Which I first thought). I had a look at the .pdf youâve sent, it seems to do something with networking > Cheers, Alexandre > >> On Oct 24, 2017, at 9:33 AM, henry wrote: > > Hi all, > > I am happy to announce the release of version 3.5 of ParrotTalk, for Squeak and Pharo, found here: > > http://www.squeaksource.com/Cryptography/ParrotTalk-zzz.2.mcz > > It follows this specification: > https://github.com/ZiroZimbarra/callistohouse/blob/master/docs/ParrotTalkFr… > > One item of note, in version 3.5, the system connecting to a server, sending the IWant msg, must know the vatId of the system being connected to. I am considering changing this to version 3.6 by removing one round-trip in messaging. Therefore, these messages would be combined: IWant/GiveInfo, IAm/ReplyInfo. I will keep ProtocolOffered and ProtocolAccepted to allow eLindaSession to support both versions: 3.5 and 3.6. > > Thoughts please? > > - HH @callistohouse.club>

Oct. 24, 2017

Re: [Pharo-dev] latest Cryptography package integration

by Stephane Ducasse

I would like the package to be working for Pharo. Now from your description I have problem to really understand the situation. I think that Pharo core should have minimum and packages should be able to be loaded nicely on top. > Currently, SHA1, SHA256 and MD5 all get redefined by Cryptography-zzz.111.mcz loading. Do you mean that the version in Pharo are older than the one in 111? Stef On Tue, Oct 24, 2017 at 6:17 PM, henry <henry(a)callistohouse.club> wrote: > I am curious as to integration of the newest Cryptography package there into > Pharo. There are two Undeclared: SoundRecorder and FillInTheBlank, though I > do not know if such would be a part of a Pharo-Squeak compatibility layer. > > More pertinent would be what to do with the HashFunctions that already > reside in the Pharo image. If the decision was to include the entire > Cryptography into base, then making the HashFunctions not go Obselete would > be the objective. > > Currently, SHA1, SHA256 and MD5 all get redefined by > Cryptography-zzz.111.mcz loading. > > What do you think about bringing Cryptography up to date for Pharo and what > would be needed. It would help to seek common ground between Squeak and > Pharo, yes? > > > - HH > >

Oct. 24, 2017

Re: [Pharo-dev] [ANNOUNCE] ParrotTalk release/design change considerations

by Stephane Ducasse

Hi henry thanks for this announce. Can you tell where such protocols are used? Stef On Tue, Oct 24, 2017 at 6:33 PM, henry <henry(a)callistohouse.club> wrote: > Hi all, > > I am happy to announce the release of version 3.5 of ParrotTalk, for Squeak > and Pharo, found here: > > http://www.squeaksource.com/Cryptography/ParrotTalk-zzz.2.mcz > > It follows this specification: > https://github.com/ZiroZimbarra/callistohouse/blob/master/docs/ParrotTalkFr… > > One item of note, in version 3.5, the system connecting to a server, sending > the IWant msg, must know the vatId of the system being connected to. I am > considering changing this to version 3.6 by removing one round-trip in > messaging. Therefore, these messages would be combined: IWant/GiveInfo, > IAm/ReplyInfo. I will keep ProtocolOffered and ProtocolAccepted to allow > eLindaSession to support both versions: 3.5 and 3.6. > > Thoughts please? > > - HH

Oct. 24, 2017

Re: [Pharo-dev] [ANNOUNCE] ParrotTalk release/design change considerations

by Alexandre Bergel

Hi! What ParrotTalk is about? It does not seem related to the drone stuff (https://www.parrot.com) ? (Which I first thought). I had a look at the .pdf youâve sent, it seems to do something with networking Cheers, Alexandre > On Oct 24, 2017, at 9:33 AM, henry <henry(a)callistohouse.club> wrote: > > Hi all, > > I am happy to announce the release of version 3.5 of ParrotTalk, for Squeak and Pharo, found here: > > http://www.squeaksource.com/Cryptography/ParrotTalk-zzz.2.mcz > > It follows this specification: > https://github.com/ZiroZimbarra/callistohouse/blob/master/docs/ParrotTalkFr… > > One item of note, in version 3.5, the system connecting to a server, sending the IWant msg, must know the vatId of the system being connected to. I am considering changing this to version 3.6 by removing one round-trip in messaging. Therefore, these messages would be combined: IWant/GiveInfo, IAm/ReplyInfo. I will keep ProtocolOffered and ProtocolAccepted to allow eLindaSession to support both versions: 3.5 and 3.6. > > Thoughts please? > > - HH

Oct. 24, 2017

[ANNOUNCE] ParrotTalk release/design change considerations

by henry

Hi all, I am happy to announce the release of version 3.5 of ParrotTalk, for Squeak and Pharo, found here: http://www.squeaksource.com/Cryptography/ParrotTalk-zzz.2.mcz It follows this specification: https://github.com/ZiroZimbarra/callistohouse/blob/master/docs/ParrotTalkFr… One item of note, in version 3.5, the system connecting to a server, sending the IWant msg, must know the vatId of the system being connected to. I am considering changing this to version 3.6 by removing one round-trip in messaging. Therefore, these messages would be combined: IWant/GiveInfo, IAm/ReplyInfo. I will keep ProtocolOffered and ProtocolAccepted to allow eLindaSession to support both versions: 3.5 and 3.6. Thoughts please? - HH

Oct. 24, 2017

latest Cryptography package integration

by henry

I am curious as to integration of the newest Cryptography package there into Pharo. There are two Undeclared: SoundRecorder and FillInTheBlank, though I do not know if such would be a part of a Pharo-Squeak compatibility layer. More pertinent would be what to do with the HashFunctions that already reside in the Pharo image. If the decision was to include the entire Cryptography into base, then making the HashFunctions not go Obselete would be the objective. Currently, SHA1, SHA256 and MD5 all get redefined by Cryptography-zzz.111.mcz loading. What do you think about bringing Cryptography up to date for Pharo and what would be needed. It would help to seek common ground between Squeak and Pharo, yes? - HH

Oct. 24, 2017

Re: [Pharo-dev] [ANN] sha256 checksum for Pharo6 downloads

by Marcus Denker

> On 24 Oct 2017, at 17:56, henry <henry(a)callistohouse.club> wrote: > > Hi Marcus, > > Are you using SHA256 inside Pharo to generate the checksum files? If so were you planning to use the OpenPGP by hmmosner, in the Cryptography repository for PGP signatures? > No, this just uses sha56sum on Linux. Marcus

Oct. 24, 2017