|Did you know ...||Search Documentation:|
|Reloading files, active code and threads|
Traditionally, Prolog environments allow for reloading files holding currently active code. In particular, the following sequence is a valid use of the development environment:
Reloading a previously loaded file is safe, both in the debug scenario above and when the code is being executed by another thread. Executing threads switch atomically to the new definition of modified predicates, while clauses that belong to the old definition are (eventually) reclaimed by garbage_collect_clauses/0.49As of version 7.3.12. Older versions wipe all clauses originating from the file before loading the new clauses. This causes threads that executes the code to (typically) die with an undefined predicate exception. Below we describe the steps taken for reloading a file to help understanding the limitations of the process.
meta_predicateare in part applied immediately and in part during the fixup process after the file completes loading. Currently,
thread_localare applied immediately.
The above generally ensures that changes to the content of source files can typically be activated safely using make/0. Global changes such as operator changes, changes of module names, changes to multi-file predicates, etc. sometimes require a restart. In almost all cases, the need for restart is indicated by permission or syntax errors during the reload or existence errors while running the program.
In some cases the content of a source file refers `to itself'. This
is notably the case if local rules for goal_expansion/2
are defined or goals are executed using
directives.50Note that initialization/1
directives are executed after loading the file. SWI-Prolog
allows for directives that are executed while loading the file
:- Goal. or initialization/2.
Up to version 7.5.12 it was typically needed to reload the file twice,
once for updating the code that was used for compiling the remainder of
the file and once to effectuate this. As of version 7.5.13, conventional transaction
semantics apply. This implies that for the thread performing the
reload the file's content is first wiped and gradually rebuilt, while
other threads see an atomic update from the old file content to
the new.51This feature was
implemented by Keri Harris.
The clause garbage collector (CGC) scans the environment stacks of all threads for referenced dirty predicates and at which generation this reference accesses the predicate. It then removes the references for clauses that have been retracted before the oldest access generation from the clause list as well as the secondary clauses indexes of the predicate. If the clause list is not being scanned, the clause references and ultimately the clause itself is reclaimed.
The clause garbage collector is called under three conditions, (1) after reloading a source file, (2) if the memory occupied by retracted but not yet reclaimed clauses exceeds 12.5% of the program store, or (3) if skipping dead clauses in the clause lists becomes too costly. The cost of clause garbage collection is proportional with the total size of the local stack of all threads (the scanning phase) and the number of clauses in all `dirty' predicates (the reclaiming phase).
Large programs are generally split into multiple files. If file A
accesses predicates from file B which accesses predicates
A, we consider this a mutual or circular dependency. If
traditional load predicates (e.g., consult/1)
are used to include file B from A and A
from B, loading either file results in a loop. This is
is mapped to load_files/2
using the option
if(true)(if(true)) Such programs are
typically loaded using a load file that consults all required
(non-module) files. If modules are used, the dependencies are made
explicit using use_module/1
predicate, however, maps to load_files/2
with the option
if(not_loaded)(if(not_loaded)) A use_module/1
on an already loaded file merely makes the public predicates of the used
Summarizing, mutual dependency of source files is fully supported with no precautions when using modules. Modules can use each other in an arbitrary dependency graph. When using consult/1, predicate dependencies between loaded files can still be arbitrary, but the consult relations between files must be a proper tree.
This section discusses compiling files for the first time. For reloading, see section 4.3.2.
In older versions, compilation was thread-safe due to a global lock in load_files/2 and the code dealing with autoloading (see section 2.13). Besides unnecessary stalling when multiple threads trap unrelated undefined predicates, this easily leads to deadlocks, notably if threads are started from an initialization/1 directive.53Although such goals are started after loading the file in which they appear, the calling thread is still likely to hold the `load' lock because it is compiling the file from which the file holding the directive is loaded.
Starting with version 5.11.27, the autoloader is no longer locked and multiple threads can compile files concurrently. This requires special precautions only if multiple threads wish to load the same file at the same time. Therefore, load_files/2 checks automatically whether some other thread is already loading the file. If not, it starts loading the file. If another thread is already loading the file, the thread blocks until the other thread finishes loading the file. After waiting, and if the file is a module file, it will make the public predicates available.
Note that this schema does not prevent deadlocks under all situations. Consider two mutually dependent (see section 22.214.171.124) module files A and B, where thread 1 starts loading A and thread 2 starts loading B at the same time. Both threads will deadlock when trying to load the used module.
The current implementation does not detect such cases and the involved threads will freeze. This problem can be avoided if a mutually dependent collection of files is always loaded from the same start file.