mjhjmtu

Operating System Concepts says

8.1.4 Dynamic Loading

In our discussion so far, it has been necessary for the entire program
and all data of a process to be in physical memory for the process to
execute. The size of a process has thus been limited to the size of
physical memory. To obtain better memory-space utilization, we can use
dynamic loading. With dynamic loading, a routine is not loaded until
it is called. All routines are kept on disk in a relocatable load
format. The main program is loaded into memory and is executed. When a
routine needs to call another routine, the calling routine first checks
to see whether the other routine has been loaded. If it has not, the
relocatable linking loader is called to load the desired routine into
memory and to update the program’s address tables to reflect this
change. Then control is passed to the newly loaded routine.

The advantage of dynamic loading is that a routine is loaded only when
it is needed. This method is particularly useful when large amounts of
code are needed to handle infrequently occurring cases, such as error
routines. In this case, although the total program size may be large,
the portion that is used (and hence loaded) may be much smaller.

Dynamic loading does not require special support from the operating
system. It is the responsibility of the users to design their programs
to take advantage of such a method. Operating systems may help the
programmer, however, by providing library routines to implement
dynamic loading.

8.1.5 Dynamic Linking and Shared Libraries

In "With dynamic loading, a routine is not loaded until it is called", is such a routine provided in a shared library or can it be provided by some other way than a shared library?

In "All routines are kept on disk in a relocatable load format", do .so shared libraries have a relocatable load format? Do .o object files? Do executable files?

I ask this because the book doesn't mention anything yet about loading or linking a library to a user program, until section 8.1.5.

section 8.1.4 seems to give me the impression that dynamic loading is not necessarily loading a library into a user program, but may load something else. Is that true?

The last paragraph of section 8.1.4 seems to say that the OS doesn't implicitly perform dynamic loading, but need programmers to specify that explicitly. Does it refers to dlopen()? But dlopen() only accepts a .so shared library as an argument, and nothing else. Can dlopen() dynamically load a .o object file or an executable ELF file?

Thanks.

A shared library does not really need to be relocatable. AT&T used non-relocatable shared libraries in SYSvr3 around 1987.

The method used by AT&T was based on fixed addresses in the shared lib and the shared lib was installed in system memory (usable by any process) by a special system tool at startup. The programs that used such libraries have been linked against the fixed addresses that were used at install time. This method is very limited as there is a need for a global administrator who defines load addresses for various libs.

The method that is used today is based on research at Sun Microsystems in 1987 and was published in December 1987.

It is based on the mmap() facility in the kernel. Each program contains special startup code that maps the needed libraries that use relocatable code.

Each library contains a special table that contains offsets for various symbols in the library. Each program conains a table with function names and the binary does a Jump Subroutine to the table entry related to all functions needed.

That table entry contains a jump instruction to the runtime linker code and when it is used the first time, the runtime linker replaces that jump by a jump to the funtion in the shared library.

All this is handled by system software and not subject of the users program design.

BTW: the book claims that no program may become larger than the physical memory. This was true in the 1960s. IBM changed that in the early 1970s for their mainframes and BSD changed that in 1979 for UNIX with virtual memory management.

dlopen() may be used to load dynamic libraries "manually" at run time and thus allows to make that depend on what happened before. A sound playing program may e.g. decide to use dlopen to load a mp3 decoder after it discovers that a file argument refers to a sound file that uses mp3 encoding.

If you use dlopen, linking to functions in the loaded shared library does not happen automatically. You need to use dlsym() to get the address from the function name and then call the function by a pointer dereferencing.

You cannot dlopen a .o file as there is a need to have the _PROCEDURE_LINKAGE_TABLE_ and that is created by the special linking process that is needed to create a .so. file.

The method used by Microsoft is derived from the manual method that is based on dlopen. Microsoft however automatically creates code that works the way you do manually with dlopen. The related code is inside the static library with the same name as the so called dll. This static library is linked statically to the program and contains the trmapolin code that loads the library, retrives the function address and then calls the function pointer.

The Microsoft compiler always creates relocatable code, so even fully static libraries compiled with that compiler use relocatable code.

The claim that a routine is not loaded until referenced is wrong in the general case, it only applies to the dlopen case.

Libraries managed by the ELF runtime system may be listed via ldd,
libraries managed manally by dlopen cannot be listed since their
names may be a result of run time string processing.