Caching¶

Depending on the type of node you are using, you have different caching options.

On all nodes you can directly interact with the underlying cache object store via regular build commands that we outline below. In that case, you can store data under specific keys that you can read in subsequent builds or nodes that run later on.

Besides on auto-setup nodes, you can define the directories that we should cache and Scrutinizer will automatically create caches that are scoped to the node/branch combination (falling back to the main branch if a new branch does not yet have a cache). This cache is restored at the beginning of a build and automatically stored at the end of a build.

Defining Cached Directories (Auto-Setup Nodes only)¶

On auto-setup nodes, you can simply list the directories that should be cached. Some directories like those of popular package managers are automatically added to this list.

build:
  nodes:
    some-node:
      cache:
        directories:
          - ~/.npm
          - vendor/   # <- this relative to ~/build where your repository is checked out

Using Explicit Commands (All Nodes)¶

Available Commands¶

Storing files and directories in the cache¶

For storing, the command usage is as follows:

# Storing a compressed tar archive with the given files/directories
store-in-cache <Scope: repository|execution> <key> <file-or-dir> <...more-files-or-dirs-or-a-tar-option>

# Storing piped input from any command
tar cf - some-dir/ | gzip | store-in-cache <Scope: repository|execution> <key> "-"

In the first case, all arguments after <key> are passed to the tar command. So, you can freely use any other options like --exclude that tar supports to further customize what is cached.

Restoring files and directories from the cache¶

For restoring, the command usage is as follows:

# Restoring a compressed tar archive in the current directory
restore-from-cache <Scope: repository|execution> <key>

# Piping data to another process
restore-from-cache <Scope: repository|execution> <key> - | gunzip | tar xf

All files and directories are unpacked relative to the directory you are currently in. Typically you would want to run store-in-cache and the corresponding restore-from-cache from the same directory.

If a file does not yet exist in the cache, restoring will still exit successfully and print a message that the key was not found in storage. If you specifically would like to check if a file exists, please see below.

Checking if a key exists in the cache¶

For checking the existence of a cache entry, the command usage is as follows:

exists-in-cache <Scope: repository|execution> <key>

If a file exists, the command will exit with a zero exit code. If it does not exist, it will exit with a non-zero exit code instead.

Scope¶

There are two cache scopes repository and execution. A repository-scoped cache entry is visible to all builds of the repository. In contrast, a execution-scoped cache entry is only visible to nodes that run later in the same build. Execution-scoped entries for example allow you to share build artifacts that might be needed later on a node that deploy these artifacts.

Examples¶

build:
  nodes:
    # In this example, we are using a minimal node.
    some-node:
      commands:
        # Store folder vendor/ with key "dependencies". This means all nodes use the same
        # cache for dependencies across all branches.
        - store-in-cache repository "dependencies" vendor/
        - restore-from-cache repository "dependencies"
        - exists-in-cache repository "dependencies"

        # If you would like to use a different cache per branch, you can make the branch
        # name part of the cache key like this.
        - store-in-cache repository "dependencies-$SCRUTINIZER_BRANCH" vendor/
        - restore-from-cache repository "dependencies-$SCRUTINIZER_BRANCH"
        - exists-in-cache repository "dependencies-$SCRUTINIZER_BRANCH"

        # If you further would like to use a different cache per branch and per node, all
        # we need to do is to add the node name to the cache key, too.
        - store-in-cache repository "dependencies-$SCRUTINIZER_BRANCH-$SCRUTINIZER_NODE_NAME" vendor/
        - restore-from-cache repository "dependencies-$SCRUTINIZER_BRANCH-$SCRUTINIZER_NODE_NAME"
        - exists-in-cache repository "dependencies-$SCRUTINIZER_BRANCH-$SCRUTINIZER_NODE_NAME"

        # The storing command is a regular system command, so you can use any environment
        # variables or the output of other commands to make up the cache key. For example,
        # if you wanted to automatically invalidate the cache if a certain file like a lock file
        # changes, this could look like this:
        - store-in-cache repository "dependencies-$(sha1sum some.lock | awk '{print $1}')" vendor/
        - restore-from-cache repository "dependencies-$(sha1sum some.lock | awk '{print $1}')"
        - exists-in-cache repository "dependencies-$(sha1sum some.lock | awk '{print $1}')"