For many projects creating the Python environment and installing dependencies takes a lot of time. For local development environment this happens occasionally and is usually not a big problem. However, in your CI, this happens for every build and might be a significant part of the build time. GH Actions does have a caching mechanism which is used in various ways.
actions/setup-python already does some caching to avoid downloading with the
- uses: actions/setup-python@v4 with: python-version: 3.9 cache: pip
From its documentation it’s easy to get the impression that the packages are cached in their installed form and that they magically appear in your Python environment when there is a cache hit. However, this is not the case. This action only caches the downloading of the packages from PYPI. pip itself caches the downloads by default on the system level and this action allows you to use the functionality in the GHA environment where workers start from scratch each time by default. Arguably, you’re just replacing the download from PYPI with the download from GitHub Actions cache. There is quite a discussion about this in the actions/setup-python repository
The other slow steps of pip installing a lot of dependencies are: resolving dependencies, unzipping and moving them to
the right locations within the Python environment and compiling extensions if needed.
actions/setup-python@v4 does not
cache any of these steps.
The good news is that it’s possible to use and cache the whole virtual environment in your GitHub Action workflows.
It is possible to craft this manually using
actions/cache, however it’s quite error-prone, and you have to be careful to
get everything right. Luckily, the syphar/restore-virtualenv custom action does all the heavy lifting for you.
restore-virtualenv is a simple 1-liner that
In its simplest form you have to put this after your setup-python and before your pip install:
- uses: syphar/restore-virtualenv@v1
You can also modify your pip install command to avoid running it altogether if the virtualenv is restored:
- run: pip install -r requirements.txt if: steps.restore-virtualenv.outputs.cache-hit != 'true'
We’ve been using this technique for the development of testmon.net and we never had any corrupted virtual environment or any other problem.
To learn about more options how to use this action, head to its documentation