On Thu, Mar 16, 2017 at 3:05 PM <raffaello.giulietti@lifeware.ch> wrote:

Indeed, experimenting using some combination of AddDllDirectory(),
SetDllDirectory() and LoadLibraryEx() was in my plan for next week.

Thanks

On 16/03/17 11:59, Mark Bestley wrote:
> Yes the calling process can alter where these indirect DLLs are loaded
> from and so should be able to be done in UFFI and directly from Smalltalk
>
> The OS will load the DLL specified then it runs DllMain and doing that
> causes the OS to load any DLLs that it calls
>
> MSDN https://msdn.microsoft.com/en-us/library/7d83bc18.aspx says
>
> With both implicit and explicit linking, Windows first searches for
> "known DLLs", such as Kernel32.dll and User32.dll. Windows then searches
> for the DLLs in the following sequence:
> The directory where the executable module for the current process is
> located.
> The current directory.
> The Windows system directory. The GetSystemDirectory function retrieves
> the path of this directory.
> The Windows directory. The GetWindowsDirectory function retrieves the
> path of this directory.
> The directories listed in the PATH environment variable.
>
>
> This can be changed using the call LoadLibraryEx instead of LoadLibrary
> Flags can be set to change the directories used
>
> https://msdn.microsoft.com/en-us/library/windows/desktop/ms684179(v=vs.85).aspx
>
>
>
> Also see AddDllDirectory
> https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
>
>
> Mark
>
> On 15/03/2017 17:22, Raffaello Giulietti wrote:
>> Hi Dimitris,
>>
>> I don't get your point about proper C coding, C performance or Pharo
>> crashes. It is not the focus of this thread.
>>
>> How dependencies between libraries are resolved at load-time is dictated
>> by the OS. If the OS offers a working mechanism to specify folders that
>> the OS uses to search for libraries at load-time, well, I think that
>> Pharo, hence the UFFI, should offer it as well and in a more fashionable
>> format.
>> UFFI is quite elegant in many respects but I wouldn't call it a
>> minimalist design (zero hand holding, if I understand you correctly).
>> But I'm happy the extra elegance is there because it makes life more
>> comfortable.
>>
>> Again, I still have to experiment with SetDllDirectory() in the case of
>> Windows, so I'm not in a position to say something authoritative yet.
>>
>>
>>
>>
>>
>>
>>
>> On 2017-03-15 17:41, Dimitris Chloupis wrote:
>>> Please note the moment you use a FFI of any language you step to a C
>>> territory. VMs tend not to touch the code because the code is already
>>> in machine code format.
>>>
>>> There is zero reason for UFFI to worry about the dependencies of a
>>> library loaded, this must happen at the library level that UFFI does
>>> not focus on.
>>>
>>> What you describe here is standard practice because the vast majority
>>> of libraries depend on other libraries. In the end UFFI or any FFI in
>>> any language is not there to replace proper C coding but rather to
>>> allow you to utilize C libraries. You cannot hope to achieve
>>> performance comparable to C if you try to use a language that is not C
>>> to do standard things like dependency management.
>>>
>>> If the library fail to load it wont be because of UFFI it will be
>>> because the library is buggy at handling its dependencies. That means
>>> that the library will be buggy even if you use it from C directly.
>>>
>>> Even you do not know what you doing at C level, you going to crash
>>> Pharo countless times anyway and the least of your worries will be
>>> handling dependencies which far more obvious than some other nasty C
>>> bugs.
>>>
>>> I think its a great idea that UFFI does zero hand holding. This way it
>>> gives the clear message that if you do not understand C then you
>>> should not be using the UFFI in the first place.
>>>
>>> On Wed, Mar 15, 2017 at 4:51 PM Esteban Lorenzano
>>> <estebanlm@gmail.com
>>> <mailto:estebanlm@gmail.com>> wrote:
>>>
>>>
>>>�� > On 15 Mar 2017, at 15:44, Raffaello Giulietti
>>>�� <raffaello.giulietti@lifeware.ch
>>>�� <mailto:raffaello.giulietti@lifeware.ch>>
>>>�� wrote:
>>>�� >
>>>�� > On 2017-03-15 15:20, Esteban Lorenzano wrote:
>>>�� >>
>>>�� >>> On 15 Mar 2017, at 15:08, Raffaello Giulietti
>>>�� <raffaello.giulietti@lifeware.ch
>>>�� <mailto:raffaello.giulietti@lifeware.ch>>
>>>�� wrote:
>>>�� >>>
>>>�� >>> Hi Esteban,
>>>�� >>>
>>>�� >>> I understand this is the current status of the UFFI, so I can
>>>�� certainly use the workarounds discussed below.
>>>�� >>>
>>>�� >>> But I hope UFFI will soon offer a more "declarative" way to
>>>�� specify the search folders, kind of LD_LIBRARY_PATH mechanism but
>>>�� in the UFFI.
>>>�� >>
>>>�� >> that will NOT solve the problem of dependencies of libraries
>>>�� indirectly references.
>>>�� >> There is no way to do that (indirect reference solving) on an
>>>�� executing program that I know.
>>>�� >> So is not ��current status of UFFI��: it will never provide that.
>>>�� >>
>>>�� >
>>>�� > Do you mean that SetDllDirectory() has no influence when invoked
>>>�� between Pharo's start and the LoadLibrary() call done by the UFFI?
>>>
>>>�� I really have no idea. You can try to add it to your app.
>>>
>>>�� Esteban
>>>
>>>�� >
>>>�� >
>>>�� >
>>>�� >
>>>�� >> All that, unless someone come and says: "look, Esteban is like
>>>�� this��.
>>>�� >>
>>>�� >> I spent last three weeks (not full time of course) trying to do
>>>�� exactly that for our current linux vm, to force libgit2 to use the
>>>�� version of libssh2 we want (and not the version on system)�� and I
>>>�� failed. I solved some cases, but not all. So at the end I opted to
>>>�� change zeroconf to define LD_LIBRARY_PATH before calling pharo.
>>>�� Only way I found I can force the lookup paths.
>>>�� >>
>>>�� >> cheers,
>>>�� >> Esteban
>>>�� >>
>>>�� >>>
>>>�� >>>
>>>�� >>>
>>>�� >>> Greetings
>>>�� >>> Raffaello
>>>�� >>>
>>>�� >>>
>>>�� >>>
>>>�� >>>
>>>�� >>> On 2017-03-15 14:52, Esteban Lorenzano wrote:
>>>�� >>>> Hi,
>>>�� >>>>
>>>�� >>>> UFFI cannot do what you want.
>>>�� >>>> If I understand well, you have:
>>>�� >>>>
>>>�� >>>> Pharo -> libA.dll -> libB.dll
>>>�� >>>>
>>>�� >>>> Pharo does a LoadLibrary(libA.dll), but has no control on how
>>>�� libA calls libB�� and is not possible for us to influence it more
>>>�� than the predefined platform ways.
>>>�� >>>>
>>>�� >>>> One posible workaround (just possible, no idea if it will
>>>�� work) is to perform a load of libB before is required by libB.
>>>�� Then hopefully the loader will understand is same library and will
>>>�� answer same handler (but not sure, because if it discriminates by
>>>�� path�� then you��re in the same position). Anyway, if you want to
>>>�� try it, it would be something like this:
>>>�� >>>>
>>>�� >>>> DynamicLoader loadLibrary: 'path/to/libB.dll��.
>>>�� >>>> DynamicLoader loadLibrary: 'path/to/libA.dll��.
>>>�� >>>>
>>>�� >>>> then you can try to execute functions of libA��
>>>�� >>>>
>>>�� >>>> cheers,
>>>�� >>>> Esteban
>>>�� >>>>
>>>�� >>>>
>>>�� >>>>> On 15 Mar 2017, at 14:32, Raffaello Giulietti
>>>�� <raffaello.giulietti@lifeware.ch
>>>�� <mailto:raffaello.giulietti@lifeware.ch>>
>>>�� wrote:
>>>�� >>>>>
>>>�� >>>>> On 2017-03-15 13:51, Ben Coman wrote:
>>>�� >>>>>>
>>>�� >>>>>>
>>>�� >>>>>> On Wed, Mar 15, 2017 at 4:42 PM, Raffaello Giulietti
>>>�� >>>>>>
>>>�� <raffaello.giulietti@lifeware.ch
>>>�� <mailto:raffaello.giulietti@lifeware.ch>
>>>�� >>>>>>
>>>�� <mailto:raffaello.giulietti@lifeware.ch
>>> <mailto:raffaello.giulietti@lifeware.ch>>>
>>>�� wrote:
>>>�� >>>>>>
>>>�� >>>>>>�� Hi Ben,
>>>�� >>>>>>
>>>�� >>>>>>�� my understanding is that SetDllDirectory only affects the
>>>�� search
>>>�� >>>>>>�� path of libraries that are loaded at *run-time* with
>>>�� LoadLibrary.
>>>�� >>>>>>
>>>�� >>>>>>�� What I'm asking for is a mechanism that works at
>>>�� *load-time*, when a
>>>�� >>>>>>�� library I'm accessing directly depends on another one
>>>�� which I'm not
>>>�� >>>>>>�� targeting directly.
>>>�� >>>>>>
>>>�� >>>>>>
>>>�� >>>>>> I don't quite follow.�� I'm not intimate with Windows
>>>�� dynamic loading, so
>>>�� >>>>>> these questions are an opportunity for me to learn...
>>>�� >>>>>>
>>>�� >>>>>> You can't mean when Pharo loads, because it wouldn't know
>>>�� of OTHER.DLL ?
>>>�� >>>>>>
>>>�� >>>>>> Do you mean that LoadLibrary is not called explicitly by
>>>�� YOUR.DLL before
>>>�� >>>>>> its uses the function from OTHER.DLL?
>>>�� >>>>>>
>>>�� >>>>>
>>>�� >>>>> During the build of YOUR.DLL, you can specify that it should
>>>�� depend on OTHER.LIB (kind of binary which lists function headers
>>>�� and static data declarations for OTHER.DLL). Linking resolution is
>>>�� then done at load-time of YOUR.DLL, not by invoking
>>>�� LoadLibrary("OTHER.DLL").
>>>�� >>>>>
>>>�� >>>>> When YOUR.DLL is loaded by UFFI at run-time (here by means
>>>�� of LoadLibrary(), I guess), Windows (not the UFFI) also attempts
>>>�� to load OTHER.DLL because of the linking information gathered
>>>�� during the build. Notice that the linking information does *not*
>>>�� include a specific path for OTHER.DLL.
>>>�� >>>>>
>>>�� >>>>> (You don't have to mention standard *.LIBs like Kernel32.lib
>>>�� because they are included by default. But of course, Windows knows
>>>�� where to find the corresponding .dll.)
>>>�� >>>>>
>>>�� >>>>> OTHER.DLL is searched using the strategy described in the
>>>�� MSDN page you mention. My expectation is/was that UFFI has/had an
>>>�� API for specifying the search paths where to find OTHER.DLL in a
>>>�� platform-independent way. In other words, I would expect the UFFI
>>>�� to do the call of SetDllDirectory() or whatever is needed on
>>>�� Windows for me or to use other mechanisms on other platforms.
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>> Do you mean you link OTHER.DLL to YOUR.DLL at compile time?
>>>�� >>>>>> But still, YOUR.DLL it not loaded until called via Pharo
>>>�� FFI, so neither
>>>�� >>>>>> is OTHER.DLL,
>>>�� >>>>>> and Dynamic-Link Library Search Order [1] says... "If a DLL
>>>�� with the
>>>�� >>>>>> same module name
>>>�� >>>>>> is already loaded in memory, the system uses the loaded
>>>�� DLL, no matter
>>>�� >>>>>> which directory it is in.
>>>�� >>>>>> The system does not search for the DLL."
>>>�� >>>>>>
>>>�� >>>>>> So maybe before calling any of YOUR.DLL,
>>>�� >>>>>> do a SetDllDirectory followed by a LoadLibrary(OTHER.DLL)
>>>�� >>>>>> and later when you call into YOUR.DLL,
>>>�� >>>>>> OTHER.DLL is already loaded.
>>>�� >>>>>>
>>>�� >>>>>
>>>�� >>>>> Yes, this might work but is rather contrived.
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>> Otherwise it would seems(?) your options are either Manifests
>>>�� >>>>>> or Dynamic-Link Library Redirection, both done outside of
>>>�� Pharo.
>>>�� >>>>>>
>>>�� >>>>>
>>>�� >>>>> What if neither YOUR.DLL nor OTHER.DLL are under my control?
>>>�� The only thing I know is where the .dll and the corresponding .lib
>>>�� + .h are.
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>> Thanks for your interest
>>>�� >>>>> Raffaello
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>>>�� But I'll try whether SetDllDirectory also affects
>>>�� load-time searches.
>>>�� >>>>>>
>>>�� >>>>>>�� Anyway, I hope future versions of the UFFI to offer
>>>�� setting the
>>>�� >>>>>>�� search paths more easily.
>>>�� >>>>>>
>>>�� >>>>>>
>>>�� >>>>>>
>>>�� >>>>>>�� Greetings
>>>�� >>>>>>�� Raffaello
>>>�� >>>>>>
>>>�� >>>>>
>>>�� >>>>>
>>>�� >>>>
>>>�� >>>>
>>>�� >>>
>>>�� >>>
>>>�� >>
>>>�� >>
>>>�� >
>>>�� >
>>>
>>>
>>
>
>