-
Notifications
You must be signed in to change notification settings - Fork 40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ModelicaUtilities for shared object / dynamic link library #2191
Comments
Comment by beutlich on 20 Jun 2017 17:36 UTC Thus the ticket reduces to a tool issue, actually, however, the specification could be improved for clarification. |
Comment by sjoelund.se on 20 Jun 2017 17:47 UTC The case when generation an executable is simpler: just link in the tool-specific symbols so they are found by the library (for GCC, add the library after all user libraries on the command line). |
Comment by beutlich on 20 Jun 2017 18:09 UTC |
Comment by sjoelund.se on 20 Jun 2017 20:16 UTC Which would be a problem since I believe the header in the specification doesn't have any dllimport in it... |
Comment by beutlich on 20 Jun 2017 20:42 UTC |
Modified by beutlich on 21 Jun 2017 08:01 UTC |
Modified by beutlich on 6 Oct 2017 06:12 UTC |
Comment by c.schulze on 6 Oct 2017 07:38 UTC
|
Comment by hansolsson on 19 Oct 2017 08:08 UTC
Hans to test in Dymola for next meeting. |
Modified by beutlich on 19 Oct 2017 08:33 UTC |
Comment by beutlich on 20 Nov 2017 20:17 UTC |
Comment by beutlich on 20 Nov 2017 20:26 UTC
No, ModelicaExternalC.dll is tool-specific and would be part of the installation of the Modelica simulation environment.
Right, OpenModelica would use its appropriate ModelicaExternalC.dll, Dymola another one from its installation directory.
Currently, all (error) messages go to a single log window only. Different instances cannot be differentiated other by its instance name if part of the (error) message.
For static libraries (LIB) /MT for MSVC compilation is pretty established since the Dymola build scripts build with this flag. But for dynamic link libraries (DLL) you are free to use /MD as well. |
Comment by c.schulze on 20 Nov 2017 21:20 UTC
FMUs can be executed without the Modelica simulation environment. The external library using the error function and the ModelicaExternalC.dll have to be part of the FMU to make it portable.
Yet still it needs to be available. So copying it to the executable or a tool specific search algorithm with delayed loading of the dll would be possible solutions. It would be nice if exported executables could be executed on computers without an installed Modelica simulation environment.
The only solution I can think of is an implementation similar to the fmiComponent approach. This structure can be constructed with external objects. I have to think about that. No idea how to realize that with Modelica.Media.
Agree. MD might be more appropriate. |
Comment by beutlich on 20 Nov 2017 21:34 UTC
I believe we cannot force FMI import tools to provide a ModelicaExternalC.dll. Well, then it only will work out if we apply the same mechanisms from FMI also for Modelica external C interface, that is, have logging/error functions set by callbacks. |
Comment by sjoelund.se on 20 Nov 2017 23:59 UTC
Furthermore, some FMI importing tools (FMIL comes to mind) import FMUs with flags set to ignore functions found in its own DLL, preferring already loaded versions of for example ModelicaExternalC instead (meaning if you load a Dymola FMU and then an OpenModelica FMU you could possibly have the OpenModelica FMU using Dymola's version of ModelicaError). |
Comment by beutlich on 21 Nov 2017 06:57 UTC
Or in other - more radical - words: Because of the incompatibilities (w.r.t. to logging and error reporting, i.e., ModelicaUtilities) of shared objects / dynamic link libraries (DLL) used with the Modelica external C interface, we declare this interface as deprecated and recommend library developers to use the FMI instead. Of course, the FMI ist the heavier interface and we need to address issues like msl:#2389 and #1626 first (and even should guide library devs how to build an FMU), but FMI is more mature, widely supported and progressing in development. |
Comment by fcasella on 21 Nov 2017 13:46 UTC The actual external function is called by the wrapper function and returns an error string, which is empty if there is no error, or contains the error message otherwise. This is a small example demonstrating the approach
I guess this approach should be fine in most cases, without the need using FMI, which has a much broader scope and overhead. |
Comment by beutlich on 21 Nov 2017 13:55 UTC |
Modified by beutlich on 29 Nov 2017 10:35 UTC |
Comment by beutlich on 29 Nov 2017 10:40 UTC |
Modified by beutlich on 23 Mar 2018 08:05 UTC |
Based on recent experience with the ExternalMedia library and on expert input from @fedetftpolimi, I'd like to reopen the discussion and hopefully lead it to port. AnalysisExternal C functions can be called by a Modelica simulation runtime, but also by the Modelica tool itself at compile time, e.g. if the dimensions of some arrays are determined by the output of external functions that read from some data tables, so they need to be called at compile time before performing structural analysis. Proper management of anomalous situations requires function from ModelicaExternal to be available to those external functions, in particular ModelicaError, which is needed to handle exceptions properly, e.g. by re-trying with a shorter time step at runtime, or issuing an error message at compile time. Existing tools are unable to call into static libraries as part of evaluating package constants, so any solution involving static libraries solves the problem for the run time part, but not for the compile time part. This includes the previously proposed approach of having external functions call a wrapper function which is directly defined in the include annotation. It appears that the only way to make an external C function callable both by the Modelica simulation runtime and by the Modelica tool itself is to wrap it in a dynamic/shared library, which brings us to the question of how to make the symbols of ModelicaUtilities (such as ModelicaError) available to shared/dynamic libraries in a tool-independent way. All the discussion in this thread so far focused on the need to put those symbols in an additional dynamic/shared library such as ModelicaExternalC.so/ModelicaExternalC.dll library and make it available, but actually it is possible to do better for these reasons:
The solution outlined in 2. works well with both OpenModelica and Dymola on Linux, where the task is performed transparently by the dynamic linker. It also works well with OpenModelica on Windows with the solution proposed in 3., since the OpenModelica compiler executable happens by chance to already export the required symbols Unfortunately, this solution currently doesn't work with Dymola on Windows, as Dymola does not export ModelicaUtilities symbols, for the reason that this is not required by the specification. This required us to get into some ugly workarounds into ExternalMedia to pass pointers to ModelicaError to the external functions, which only work at runtime and are unnecessarily complicated. ProposalWe think that the MSL should be amended to require that the Modelica compilers and their simulation runtimes both export the symbols of all ModelicaUtilities functions either in the executable or in a runtime library, before trying to load any user-supplied dynamic/shared library. This would allow for a clean solution that works across all tools and all operating systems. We can provide a stripped-down MWE that demonstrates the concept, based on what we learned with ExternalMedia. @HansOlsson, @henrikt-ma, @gkurzbach what do you think? |
I am not the right person to analyze the proposal, but perhaps @otronarp has something to say?
It might not be the most efficient approach, but I thought a simple solution for compile time evaluation would be to compile a small external function evaluator program for evaluating any of the external functions of a library. This program would be built automatically by the tool, and the external functions can be handled similarly to when building a simulator, except that the implementation of ModelicaUtilities.h doesn't necessarily have to be the same. Since the evaluator program is built automatically on demand, I guess it would be safe to use a binary data format for passing the arguments and receiving results, and by letting the evaluator program run as a server during the model translation, one could avoid program startup overhead with each evaluation. As far as I can tell, this approach requires nothing beyond what is specified today. |
The following comment is written from the perspective of a library writer who is using the Modelica external functions interface for a number of research projects, including ExternalMedia. Moreover, static libraries bring their set of hassles too. For example, ExternalMedia depends on CoolProp that in turn depends on libstdc++. How to tell the Modelica tool that, when linking the static library libExternalMedia.a it needs to link with libstdc++ as well? Of course libstdc++ is just an example, another static library may need to link with an arbitrary set of other libraries and that set is not encoded in the static library itself (unlike with dynamic libraries), it's usually hardcoded by the library developer in their build system. The cool thing with dynamic libraries isn't just that they can be loaded at runtime into a running program without the need for a software infrastructure involving calling the compiler and launching a separate program and adding some sort of IPC, but also that they contain enough information to load all their dependencies in a way that is transparent to the tool that loads them. |
I'm also attaching a minimum working example with a model Test that calls a foo function at translation time and a bar function at runtime, both of which can be made to fail by calling ModelicaError. I only tested it with OpenModelica on Linux, but it uses the same importer code of ExternalMedia so it should also work on Windows. Note: the C/C++ code is only provided in source form, you need to build it yourself to try the example. |
See further discussion in modelica/ModelicaStandardLibrary#4476. There are two proposals to fix this issue:
I would suggest to put this on the agenda of MAP-Lang so we can get this done soon. |
Modified by beutlich on 29 Nov 2017 10:35 UTC
It is currently not (easily) possible that an external shared object / dynamic link library depends on ModelicaUtilities (e.g., for ModelicaFormatMessage or ModelicaFormatError), since ModelicaUtilities is tool-dependent, but shared object / dynamic link libraries are executable (opposed to static or import libraries).
This leads to trouble for library developers of tool-generic Modelica libraries, especially for the two tools Dymola and SimulationX on Windows, see e.g.,
with the following workarounds:
Both workarounds have their drawbacks and are error-prone. One possible solution would be an extension of the Modelica external function interface by providing optional callback functions for libraries. At least, this cleans up the current mess, however I am not sure if new problems are likely.
Modified by beutlich on 19 Oct 2017 08:33 UTC
It is currently not (easily) possible that an external shared object / dynamic link library depends on ModelicaUtilities (e.g., for ModelicaFormatMessage or ModelicaFormatError), since ModelicaUtilities is tool-dependent, but shared object / dynamic link libraries are (opposed to) static or import libraries executable.
This leads to trouble for library developers of tool-generic Modelica libraries, especially for the two tools Dymola and SimulationX on Windows, see e.g.,
with the following workarounds:
Both workarounds have their drawbacks and are error-prone. One possible solution would be an extension of the Modelica external function interface by providing optional callback functions for libraries. At least, this cleans up the current mess, however I am not sure if new problems are likely.
Modified by beutlich on 21 Jun 2017 08:01 UTC
It is currently not (easily) possible that an external shared object / dynamic link library depends on ModelicaUtitlities (e.g., for ModelicaFormatMessage or ModelicaFormatError), since ModelicaUtilities is tool-dependent, but shared object / dynamic link libraries are (opposed to) static or import libraries executable.
This leads to trouble for library developers of tool-generic Modelica libraries, especially for the two tools Dymola and SimulationX on Windows, see e.g.,
with the following workarounds:
Both workarounds have their drawbacks and are error-prone. One possible solution would be an extension of the Modelica external function interface by providing optional callback functions for libraries. At least, this cleans up the current mess, however I am not sure if new problems are likely.
Reported by beutlich on 20 Jun 2017 13:11 UTC
It is currently not (easily) possible that an external shared object / dynamic link library depends on ModelicaUtitlities (e.g., for ModelicaFormatMessage or ModelicaFormatError), since ModelicaUtilities is tool-dependent, but shared object / dynamic link libraries are (opposed to) static or import libraries executable.
This leads to trouble for library developers of tool-generic Modelica libraries, especially for the two tools Dymola and SimulationX on Windows, see e.g.,
with the following workarounds:
Both workarounds have their drawbacks and are error-prone. One possible solution would be an extension of the Modelica external function interface by providing optional callback functions for libraries. At least, this cleans up the current mess, however I am not sure if new problems are likely.
Migrated-From: https://trac.modelica.org/Modelica/ticket/2191
The text was updated successfully, but these errors were encountered: