Minimizing Cache Contention: the --random Option

SCons User Guide 0.98.0
Prev | Chapter 22. Caching Built Files | Next

# 22.6. Minimizing Cache Contention: the `--random` Option

If you allow multiple builds to update the shared cache directory simultaneously, two builds that occur at the same time can sometimes start "racing" with one another to build the same files in the same order. If, for example, you are linking multiple files into an executable program:

           Program('prog',
                   ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])

SCons will normally build the input object files on which the program depends in their normal, sorted order:

          % scons -Q
          cc -o f1.o -c f1.c
          cc -o f2.o -c f2.c
          cc -o f3.o -c f3.c
          cc -o f4.o -c f4.c
          cc -o f5.o -c f5.c
          cc -o prog f1.o f2.o f3.o f4.o f5.o

But if two such builds take place simultaneously, they may each look in the cache at nearly the same time and both decide that `f1.o` must be rebuilt and pushed into the shared cache directory, then both decide that `f2.o` must be rebuilt (and pushed into the shared cache directory), then both decide that `f3.o` must be rebuilt... This won't cause any actual build problems--both builds will succeed, generate correct output files, and populate the cache--but it does represent wasted effort.

To alleviate such contention for the cache, you can use the `--random` command-line option to tell SCons to build dependencies in a random order:

          % scons -Q --random
          cc -o f3.o -c f3.c
          cc -o f1.o -c f1.c
          cc -o f5.o -c f5.c
          cc -o f2.o -c f2.c
          cc -o f4.o -c f4.c
          cc -o prog f1.o f2.o f3.o f4.o f5.o

Multiple builds using the `--random` option will usually build their dependencies in different, random orders, which minimizes the chances for a lot of contention for same-named files in the shared cache directory. Multiple simultaneous builds might still race to try to build the same target file on occasion, but long sequences of inefficient contention should be rare.

Note, of course, the `--random` option will cause the output that SCons prints to be inconsistent from invocation to invocation, which may be an issue when trying to compare output from different build runs.

If you want to make sure dependencies will be built in a random order without having to specify the `--random` on very command line, you can use the `SetOption` function to set the `random` option within any `SConscript` file:

           Program('prog',
                   ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])

           SetOption('random', 1)
           Program('prog',
                   ['f1.c', 'f2.c', 'f3.c', 'f4.c', 'f5.c'])

Prev | Home | Next
Populating a Shared Cache With Already-Built Files | Up | Alias Targets