4.3.2 Loading files, active code and threads

Traditionally, Prolog environments allow for reloading files holding currently active code. In particular, the following sequence is a valid use of the development environment:

• Trace a goal
• Find unexpected behaviour of a predicate
• Enter a break using the b command
• Fix the sources and reload them using make/0
• Exit the break, retry using the r command

Goals running during the reload keep running on the old definition, while new goals use the reloaded definition, which is why the retry must be used after the reload. This implies that clauses of predicates that are active during the reload cannot be reclaimed. Normally a small amount of dead clauses should not be an issue during development. Such clauses can be reclaimed with garbage_collect_clauses/0.

garbage_collect_clauses

Clean up all dirty predicates, where dirty predicates are defined to be predicates that have both old and new definitions due to reloading a source file while the predicate was active. Of course, predicates that are active using garbage_collect_clauses/0 cannot be reclaimed and remain dirty. Predicates are, like atoms, shared resources and therefore all threads are suspended during the execution of this predicate.

4.3.2.1 Compilation of mutually dependent code

Large programs are generally split into multiple files. If file A accesses predicates from file B which accesses predicates from file A, we consider this a mutual or circular dependency. If traditional load predicates (e.g., consult/1) are used to include file B from A and A from B, loading either file results in a loop. This is because consult/1 is mapped to load_files/2 using the option if(true)(if(true)) Such programs are typically loaded using a load file that consults all required (non-module) files. If modules are used, the dependencies are made explicit using use_module/1 statements. The use_module/1 predicate, however, maps to load_files/2 with the option if(not_loaded)(if(not_loaded)) A use_module/1 on an already loaded file merely makes the public predicates of the used module available.

Summarizing, mutual dependency of source files is fully supported with no precautions when using modules. Modules can use each other in an arbitrary dependency graph. When using consult/1, predicate dependencies between loaded files can still be arbitrary, but the consult relations between files must be a proper tree.

4.3.2.2 Compilation with multiple threads

This section discusses compiling files for the first time. For reloading, see section 4.3.2.3.

In older versions, compilation was thread-safe due to a global lock in load_files/2 and the code dealing with autoloading (see section 2.13). Besides unnecessary stalling when multiple threads trap unrelated undefined predicates, this easily leads to deadlocks, notably if threads are started from an initialization/1 directive.^{35Although such goals are started after loading the file in which they appear, the calling thread is still likely to hold the `load' lock because it is compiling the file from which the file holding the directive is loaded.}

Starting with version 5.11.27, the autoloader is no longer locked and multiple threads can compile files concurrently. This requires special precautions only if multiple threads wish to load the same file at the same time. Therefore, load_files/2 checks automatically whether some other thread is already loading the file. If not, it starts loading the file. If another thread is already loading the file, the thread blocks until the other thread finishes loading the file. After waiting, and if the file is a module file, it will make the public predicates available.

Note that this schema does not prevent deadlocks under all situations. Consider two mutually dependent (see section 4.3.2.1) module files A and B, where thread 1 starts loading A and thread 2 starts loading B at the same time. Both threads will deadlock when trying to load the used module.

The current implementation does not detect such cases and the involved threads will freeze. This problem can be avoided if a mutually dependent collection of files is always loaded from the same start file.

4.3.2.3 Reloading running code

This section discusses not re-loading of code. Initial loading of code is discussed in section 4.3.2.2.

As of version 5.5.30, there is basic thread-safety for reloading source files while other threads are executing code defined in these source files. Reloading a file freezes all threads after marking the active predicates originating from the file being reloaded. The threads are resumed after the file has been loaded. In addition, after completing loading the outermost file, the system runs garbage_collect_clauses/0.

What does that mean? Unfortunately it does not mean we can `hot-swap' modules. Consider the case where thread A is executing the recursive predicate P. We `fix' P and reload. The already running goals for P continue to run the old definition, but new recursive calls will use the new definition! Many similar cases can be constructed with dependent predicates.

It provides some basic security for reloading files in multithreaded applications during development. In the above scenario the system does not crash uncontrolled, but behaves like any broken program: it may return the wrong bindings, wrong truth value or raise an exception.

Future versions may have an `update now' facility. Such a facility can be implemented on top of the logical update view. It would allow threads to do a controlled update between processing independent jobs.