In OpenFOAM, the solvers and numerical models are separated. Most of the numerical models are located in the
$FOAM_SRCpath, and being compiled to a dynamic library “.so” binary file, which can be loaded either by linking it during the solver compiling process or loading it by specify the “.so” file in the case controlDict. For example, we develop a new combustion model, named
PaSR, and compile it to a “libPaSR.so”. When we run a case, we just need to add a line
controlDict. Then, we can see and use this model in the
How does the “libPaSR.so” file is loaded, and why the solver knows that a new model “PaSR” is available? This blog will explain it.
Before digging into OpenFOAM, let’s see how does the dynamic library is loaded in C++. There are three famous function named
dlclose, which are used to open, find function address and close library, in your operating system.
I wrote a short code to demonstrate how does it work, see https://github.com/ZmengXu/dynamicLibloading/tree/master/ch01
First, we define two models WSR and PaSR (which are two simple combustion model) in two
.C files. Each file has only one simple function, named
get_library_name. Compiling these two
.C files to two libraries,
// In WSR.C
// In PaSR.C
Second, we write a solver named
solverFoam to call
get_library_name function in
In this code, we get the librarypath from the solver argument, and try to find the dynamic library, load the library to computer memory and get a handle libhandle using dlopen. We use dlsym to find the function
_Z16get_library_namev in this library and call this function to print its information.
you can use Allrun in https://github.com/ZmengXu/dynamicLibloading/blob/master/Allrun to compile and run it.
For example, typing the above commands after compiling the library and solver. It will find the “libWSR.so” in
$LD_LIBRARY_PATH and printing the following results.
libWSR.so is found.
Here, we have two things need to be explained:
typedef char* (*funcType)(void);is an alias, define a function pointer type. This function has a return value of
voidargument. We call this kind of function as
funcType. Now you can understand that Line 32:
funcType funcPtr = (funcType)voidPtr;means that we are about to convert
voidPtrto a function pointer and assign it to a new
funcTypetype function pointer
funcPtr. Thus, it can be used as
funcPtr()in Line 34.
You must be curious that the function name we defined in
PaSR.Cis “get_library_name”, but why we are looking for the function name “_Z16get_library_namev”. This is due to the name mangling (also called name decoration) in C++ compiler. We can use
objdump -tT $FOAM_USER_LIBBIN/libPaSR.soor
nm -D $FOAM_USER_LIBBIN/libPaSR.soto see the function list in the
This is an example for
nm -D $FOAM_USER_LIBBIN/libPaSR.so result, you can find
_Z16get_library_namev in the optput.
0000000000201040 B __bss_start
The name mangling is a common issue when you want to use Hybrid Programming in OpenFOAM, like “C” and “C++”, or “fortune” and “C++”. You can use the keyword
extern C to avoid the compiler converting function name. For example, we can change WSR.C to:
compile it and use
nm -D $FOAM_USER_LIBBIN/libPaSR.so you can get this. The function name is not decorated anymore.
0000000000201020 B __bss_start
In OpenFOAM, the dynamic library are loaded and unloaded implicitly in the
dlLibraryTable class, see
$FOAM_SRC/OpenFOAM/db/dynamicLibrary/dlLibraryTable. I created a tutorial code to demonstrate how does it work, see https://github.com/ZmengXu/dynamicLibloading/tree/master/ch02
In the dlLibraryTable class, it has a constructor using a specific dictionary to look through all the libraries, load the libraries to computer memory and store the library names and pointers to libNames_ and libPtrs_ list. When the dlLibraryTable object is deleted, it will call the deconstructor function to unload these libraries.
// * * * * * * * * * * * * * * * * Constructors * * * * * * * * * * * * * * //
In the solverFoam, we create a dictionary named controlDict, which will read a dictionary file controlDict, and providing the “libs” as the libsEntry. Then the libraies in the controlDict.libs will be loaded and unloaded automatically. This work is done in the
Time class in
$FOAM_SRC/OpenFOAM/db/Time and be called from
#include "createTime.H" by each CFD solver. This is the reason why the libraries we wrote in the “libs” of controlDict will be loaded automaticlly.
In terms of the libraries, a base class combustionModelBase and a derived class WSR in the combustionModel folder are compiled to libcombustModel.so file.
In the base class, a static HashTable pointer member WordConstructorTablePtr_ is created. A template class addWordConstructorToTable is defined to operate this pointer. In the addWordConstructorToTable constructor, the static HashTable pointer member WordConstructorTablePtr_ is created and insert a “debug value - typeName” pair into the table. Once the constructor is called, it will print “Try to insert typeName” in the constructor, if the typeName exist in the table, it will print “Duplicate entry typeName in runtime selection table”.
template< class Type >
WSR is inherited from the base class, so it also has the template class addWordConstructorToTable, in the WSR.C, an object addWSRWordConstructorTocombustionModelBaseTable_ is defined, which will call the constructor, and get the above output. We will see when the table is created. This is very important.
Apart from that, a static function “New” is defined in combustionModelNew.C to search the typeName from this table.
// static Factory Method (selector)
In the same way, PaSR is defined, the difference is that it is compiled separately to libPaSR.so file. The libcombustModel.so is written in the solverFoam’s Make/options and linked to the solverFoam during the compiling stage. while the libPaSR.so file is loaded when we run the solverFoam. You can use Allrun in https://github.com/ZmengXu/dynamicLibloading/blob/master/Allrun to compile and run it. Before the libPaSR.so is created, we can not find it in the
$LD_LIBRARY_PATH, thus only one model
WSR is available in the WordConstructorTable. After libPaSR.so is created, you will see the following outputs.
Try to insert WSR into the WordConstructorTablePtr_
It is interesting that WSR is inserted into the WordConstructorTablePtr_ before entring main() function, PaSR is inserted into the WordConstructorTablePtr_ after the libPaSR.so is loaded. This is because that libcombust.so is loaded before the main() function, while libPaSR.so is loaded during the run time. Once the dynamic library is loaded, the objects addWSRWordConstructorTocombustionModelBaseTable_ and addPaSRWordConstructorTocombustionModelBaseTable_ are constructed automatically in the computer memory as global variables. If we load one library twice, the insert function will reject the inserting. This is why we see the Duplicate entry before the OpenFOAM output header.