Go Back
2 Votes

Traditional Synergy to .NET Interop Should Modify The Default Loader Policy To Include The Execution Directory


Accepted

When using the Traditional Synergy to .NET Interop Bridge the run time currently does not modify the Default Loader Policy to include the execution directory of the DBR/ELB which invoked it.

Because of this it uses the default .NET Runtime Probing rules (https://docs.microsoft.com/en-us/dotnet/framework/deployment/how-the-runtime-locates-assemblies); but uses them with respect to the runtime (dbr.exe) location which is the application which is hosting the runtime.

This creates a problem for applications which DO NOT register their assemblies into the Global Assembly Cache (GAC). The symptoms of this error are similar to the following
 

=== Pre-bind state information ===
LOG: DisplayName = System.Windows.Interactivity, PublicKeyToken=31bf3856ad364e35
(Partial)
WRN: Partial binding information was supplied for an assembly:
WRN: Assembly Name: System.Windows.Interactivity, PublicKeyToken=31bf3856ad364e35 | Domain ID: 1
WRN: A partial bind occurs when only part of the assembly display name is provided.
WRN: This might result in the binder loading an incorrect assembly.
WRN: It is recommended to provide a fully specified textual identity for the assembly,
WRN: that consists of the simple name, version, culture, and public key token.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.
LOG: Appbase = file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/
LOG: Initial PrivatePath = NULL
Calling assembly : PresentationFramework, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35.
===
LOG: This bind starts in default load context.
LOG: No application configuration file found.
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind).
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity/System.Windows.Interactivity.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity/System.Windows.Interactivity.EXE.

Computers Unlimited long has worked around this by registering EVERYTHING to the GAC; however this is not a viable long term solution as Microsoft intends to deprecate the GAC with .NET Core. Because of this several third party libraries are beginning to get lazy/sloppy with their programming, specifically by not strong naming their assemblies which prevents them from being inserted into the GAC. Because of the security rules of the GAC everything MUST be strong named (including all N-Order Dependencies). Today we have been able to muddle through but a day will come when this is no longer feasible.

I understand that this COULD be a breaking change (this really should have been addressed when this was introduced). However I would be willing to compromise and have this be an OPT-IN feature (by setting a DBLOPT).

Doing so would drastically reduce the complexity for our Internal Developers and open up the ability for our end product to reduce our reliance on the GAC.

 

15 Comments | Posted by Ace Olszowka to Synergy DBL on 4/2/2019, 8:57 PM
Ace Olszowka
This burned us today, and was the root cause of Case #099862 "IL Error TimsNetInterop.dll: Unable to resolve token [Error: 0x80131860]"

4/30/2020, 3:56 PM   0  
Steve Ives
Hi Ace. I have been discussing this more with Roger and Jeff, and we think that we already have a mechanism for you to resolve this issue, but it's possible that you are not aware of the mechanism.

Before I start, I just want to clarify our understanding of your request. We believe that what you are using the Traditional Synergy .NET Assembly API to access .NET functionality from a Traditional Synergy application, and what you are asking to do is be able to have a bunch of .NET Assemblies in a folder, that have not been registered in the GAC, and be able to have all of the dependencies between those assemblies resolved correctly as the environment is loaded.

If that is correct, then the existing product should work just fine for you, assuming that you use an environment variable to reference the location of the first or "main" assembly when you use the gennet40 tool, and ensure that the environment variable is available at runtime, pointing to the folder containing all of your assemblies.

So when using the gennet40 command, use an environment variable, like this:
gennet40 -o SRC:NetWrappers.dbl MYBIN:MyAssembly.dll
Or, if you are using the -s option and an XML input file, do something like this:
<?xml version='1.0'?>
<assemblies>
    <assembly Name="MYBIN:MyAssembly.dll">
        <class Name = "MyBarChart"/>
        <class Name = "MyLineChart"/>
    </assembly>
</assemblies>
This works because, whenever you successfully load an assembly via an environment variable, we remember the fact that you did that, and attempt to use the same mechanism to locate dependencies.

And if you do have .NET Framework dependencies, that are not in the same folder, that's OK because they will be in the GAC and loaded from there.

We hope this addresses your requirement, but if not then we must be misunderstanding something, in which case we'll set up a meeting to discuss further.
 

6/2/2020, 11:39 PM   0  
Ace Olszowka

Hi Steve,

Please clone the following Git Repository https://github.com/aolszowka/SynergySamples/tree/SRC-0870d000000bppFAAQ

Build this (either in x64 or x86) and attempt to execute, this will reproduce the issue.

The secret sauce is attempting to have WPF attempt to load a dependent assembly (in this case System.Windows.Interactivity) which causes this to fail with the following FUSLOGVW Error:
 

*** Assembly Binder Log Entry  (6/8/2020 @ 3:02:02 PM) ***

The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified.

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Running under executable  C:\Program Files (x86)\Synergex\SynergyDE\dbl\bin\dbr.exe
--- A detailed error log follows. 

=== Pre-bind state information ===
LOG: DisplayName = System.Windows.Interactivity, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
 (Fully-specified)
LOG: Appbase = file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = dbr.exe
Calling assembly : PresentationFramework, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35.
===
LOG: This bind starts in default load context.
LOG: No application configuration file found.
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Post-policy reference: System.Windows.Interactivity, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35
LOG: GAC Lookup was unsuccessful.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity/System.Windows.Interactivity.DLL.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity.EXE.
LOG: Attempting download of new URL file:///C:/Program Files (x86)/Synergex/SynergyDE/dbl/bin/System.Windows.Interactivity/System.Windows.Interactivity.EXE.
LOG: All probing URLs attempted and failed.
 

The lines in question are in UserControl1.xaml 11-12: https://github.com/aolszowka/SynergySamples/blob/de73173127bce536a1e8778c996e7c97a0095408/DotnetAssembly/CSharpInterop/UserControl1.xaml#L11-L12

In your defense attempting to load several other binaries without putting WPF into the mix appears to work as you describe, but CU's setup, as per usual, is always more exotic. I do not know enough about the underlying implementation of WPF to know how they are attempting to load the assembly in question, but my suspicion is when the Traditional Runtime (dbr.exe) hosts the CLR, it does not repoint the ApplicationBase in all scenarios.


6/8/2020, 9:11 PM   0  
Steve Ives
I made a few changes to your sample and sent you a PR.

This solution should now build and run. I made the following changes:

1. Made all three projects target your EXE: folder.
2. Removed the EXEDIR: environment variable from Common.props
3. Added a new NETSRC: environment variable in Common.props
4. Added a batch file named gennetwrappers.bat and defined the commands to execute gennet40.
5. Plugged in the execution of the batch file as a pre-build task for the DBLInterop project.
6. Declared a dependency on the CSharpInterop project from the DBLInterop project.
7. Modified CSharpInterop to have a dependency on System.Windows.Interactivity from NuGet, and modified the calculator to load a type from that assembly, just to prove that the dependency is OK.
8. Modified Program.dbl to include the .NET namespaces via the .inc file, and to display the result that comes back from C#

It is now possible to clone the source direct from GitHub, open the solution in VS, build, run, and it works. Notice the dependent assemblies are in the Bin folder.

Yes, there is a batch file involved, and yes, I had to declare a dependency on the C# project to get the build order right, but those are one time things, not too hard.

As for the numbered source files, gennet40 when used in REAL scenarios can generate some truly gargantuan source files, the numbered files and single file that includes them are by design. As is the .inc file which provides an effective way for traditional Synergy consumer code to import all of the necessary namespaces. Again, by design.
 

6/8/2020, 10:39 PM   0  
Ace Olszowka

Your PR strips the WPF Component which shows the issue in question, as noted above:

attempting to load several other binaries without putting WPF into the mix appears to work as you describe, but CU's setup, as per usual, is always more exotic. I do not know enough about the underlying implementation of WPF to know how they are attempting to load the assembly in question, but my suspicion is when the Traditional Runtime (dbr.exe) hosts the CLR, it does not repoint the ApplicationBase in all scenarios.

What you are showing here is the simple case which does work.

Lets focus on the load failure with WPF first, then loop back around for questions on the above for the simple scenario.


6/8/2020, 10:59 PM   0  
Steve Ives
OK, so modify the sample so that it fails and then we'll see your actual issue.

6/8/2020, 11:02 PM   0  
Ace Olszowka
Looking at your PR it looks like you did not pull from the linked branch (or at very least your changes are applied against master instead of the linked branch "SRC-0870d000000bppFAAQ") which contains the working example.

I have amended your patch to merge the changes from the original linked branch which shows the issue.

I have also commented out the initialization of a type from the class in question; as seen here: https://github.com/aolszowka/SynergySamples/pull/1/commits/4ea167ff57a6f6d2bd1e3aba5d14c028f472801d#diff-77c15ec4b50257ff1974b5719b27863aR7

When you do this you allow the C# Runtime to use a different loader path which avoids the issue. This is not realistic for CU which has hundreds of assemblies that would need to have initialization routines.

6/8/2020, 11:21 PM   0  
Ace Olszowka
I should also note that if you remove the .nodebug statement that is helpfully inserted you'll get the actual exception (something useful from a developer standpoint):

User-added image

Whereas with the .nodebug statement (as the generator generates by design) you end up with a terminated exception which tells you absolutely nothing:

Started debugging runtime version 10.3.3g with debug revision 2
The program '[116608] S:\GitHub\SynergySamples\DotnetAssembly\bin\DBLInteropConsumer.dbr ' has exited with code -1 (0xffffffff).

make sure to disable the code generation in the batch file otherwise it will just blow it right back in.

6/8/2020, 11:38 PM   0  
Steve Ives
Wanted to acknowledge your responses, but I'm out sick. We will respond, but it might be a few days. Cheers.

6/9/2020, 6:15 PM   0  
Ace Olszowka
Sounds fair; let me know if you need more help repo'ing this. Thank you and get well soon!

6/9/2020, 6:16 PM   0  
Steve Ives
OK, here's what we found. The assembly load behavior is as described (i.e. once an environment variable is used to refer to a location from which to load assemblies, we remember that location and automatically look in that location for other assemblies) ... BUT ... the behavior is only available at times when WE are specifically loading an assembly!!! So, you're right, it doesn't work for all dependencies, even though the dependent assembly IS in the location pointed to by the environment variable.

We plan to address this in a future release with an option that you can enable to extend the load behavior to be for the duration of the process. We don't want to just do it all the time, because this is not an issue that affects most people with their typical use cases for the .NET Assembly API.

As a temporary workaround, here is a piece of code that you could choose to drop into your C# assembly to resolve the issue until the new option is available:
 
private static System.Reflection.Assembly CurrentDomain_AssemblyResolve(object sender, ResolveEventArgs args)
{
    try
    {
        var assemblyName = args.Name;
        if (!args.Name.EndsWith(".dll"))
        {
            var postName = args.Name.IndexOf(",");
            if (postName > -1)
                assemblyName = args.Name.Substring(0, postName);

            assemblyName += ".dll";
        }
        else
            assemblyName = System.IO.Path.GetFileName(assemblyName);

        var loadFrom = Environment.GetEnvironmentVariable("EXEDIR");
        var filePath = System.IO.Path.Combine(loadFrom, assemblyName);
        if(File.Exists(filePath))
            return System.Reflection.Assembly.LoadFrom(filePath);
    }
    catch
    { }
    return null;
}
Obviously, you should modify EXEDIR in the code above to be whatever environment variable that you are using to locate your .NET Assemblies. Then use this code as an event handler for the AppDomain.CurrentDomain.AssemblyResolve event, like this:
AppDomain.CurrentDomain.AssemblyResolve += CurrentDomain_AssemblyResolve;
Thanks for bearing with us on this issue. Hopefully, the workaround will suffice while we modify the product.




 

6/12/2020, 6:26 PM   0  
Ace Olszowka
The problem with the work around is that you need to remember to apply it to all areas, the area of responsibility is the default loader policy that ships with the DBR runtime, which is why we created this idea post.

We don't want to just do it all the time, because this is not an issue that affects most people with their typical use cases for the .NET Assembly API.
This will burn anyone who uses Dependency Injection (either directly or indirectly as you can see though WPF usage). I suspect most users of this API do not see this because they are registering their dependencies in the GAC, however as time passes on this will no longer be the case.

FWIW I also believe there is also a subtle DLL loader issue as part of this bug:

Because the Default Loader Policy used by the dbr.exe runtime is using C:\Program Files (x86)\Synergex\SynergyDE\dbl\bin as its app base if one were to shove in a common dependency (for example Newtonsoft.Json.dll) into that location (C:\Program Files (x86)\Synergex\SynergyDE\dbl\bin) many DI frameworks would load that version of the dependency as opposed to the one you *thought* you were loading.

6/12/2020, 7:24 PM   0  
Ace Olszowka
Following up on this; was there any resolution?

8/6/2020, 4:33 PM   0  
Steve Ives
As previously mentioned, we do plan to address this in a future release, but you're not going to see a change in a patch release; for the core SDE runtime products we're reverting to our previous policy of not distributing new features in patch releases.

I understand that the suggested workaround requires a specific effort to implement in each case, that will not be the case with the final solution, but it's what we have to offer right now.

9/18/2020, 6:46 PM   0  
Ace Olszowka
That is fair, we have lived this long, and really today when we encounter it we ask our devs to better research the feasibility of just completely deprecating the Synergy Language in that area to avoid the problem going forward.

9/18/2020, 6:54 PM   0  
Please log in to comment on this idea.