Caching and optimization

Introduction

Definition of caching

Benefits of using caching

  • Also useful for increasing reliability of external services

  • Can reduce cost (VM running time), especially if the VM is expensive as you're normally charged for runtime and not how much CPU it uses

When not to cache

Setting up pipeline caching

Knowing your intermediate build artifacts

  • How to handle different types of files and assets that need to be cached, such as build artifacts, dependencies, and test results

  • Some artifacts do and don't benefit from compression, and some compression algorithms might be more efficient for certain types of data, you'd have to experiment with them

  • Whether symlinks have to be preserved or modification dates (tar might be good in this case)

  • Consider decompression algorithms that can be streamed, so you can decompress the data while downloading

  • Some files may be large, others might be a set of small files. This will impact how the files are stored and if any preprocessing is required. For example, one tar file will probably be better to compact the data, but tarring a large file might not provide any benefits (other than to provide a container.) Some very large files may have to be downloaded quickly, so a distributed cache system could be better.

  • Some artifacts can be quickly regenerated, some might be more complex. Consider this when prioritizing the caching strategies.

  • If you're building a dependency that is used by multiple teams, then it could instead be built once and then included as a dependency (e.g., through a CDN or as-is as an artifact) and then could be cached more widely. This also means it could exist on a more permanent storage medium which could improve the performance characteristics.

Creating a cache

Uploading and downloading a cache

Configuring cache timeout and size

Debugging and Troubleshooting

Cache metrics

  • Hit rate (is my cache being used?)

  • Miss rate (how often can it be used?)

  • Usage/size (is it too big/small?)

  • Performance/latency (is it taking forever to download?)

  • Eviction rate (are items just being cached never to be used again and evicted immediately? This might not show up in the miss rate) Does the eviction policy have to change? Is there enough disk space to hold the cache?

Incorrect or missing cache key

  • If cache string is too long, just hash it

  • The cache key determines which cache should be used, so if the key is incorrect or missing, the cache won't be used or the wrong cache will be used. This can lead to slower build times or incorrect build results.

Caching too many or too few files

  • Caching too many files can lead to slow cache retrieval times and can also cause issues when different builds use different versions of the same file. Caching too few files can cause a slower build time because files that should be cached have to be rebuilt.

Using an old or non-existent cache

  • If the cache is not updated or invalidated when it should be, old or non-existent files can be used, leading to incorrect build results or slow build times.

  • Preventing vendor lock-in

  • When you construct your workflows, you want to make sure that you can run them locally. This is important because:

  • Fast feedback loop for debugging or adding steps to the build process. This is because some parts of the CI/CD workflow are proprietary and must be run on the CI server itself. This means that one may resort to manually updating the workflow file to update it, resulting in a very slow and frustrating experience.

  • An understanding of what is happening behind the scenes when your workflow is running, and to prevent "magic". It's important to have a good understanding of the build process and what processes do what, because debugging requires extensive knowledge of the system. It is also important to ensure its correctness, because if you do not understand what the desired state is or what the program is doing, then it is not possible to verify it.

  • When you run/test software locally, then it depends on a specific environment. If your CI/CD system is too complicated, then it might mean that it is not possible to run it within any reasonable approximation locally. This means that it can be difficult to know if your software is working as intended, because the two environments are different and may introduce subtle bugs.

  • If there is too much vendor lock-in, then it might be difficult to move to a new platform in the future because it would cause the existing workflows to have to be rewritten, verified, and require additional staff training. This means that your business requirements are partially dependent on what the vendor seeks to offer, which may or may not be aligned with your business model. Therefore, it is theoretically possible to be constrained by outside limitations on which you do not have control over.

  • Some things are difficult to replicate locally, but are not impossible. For example, caching actions usually upload to a vendor-specific location that is encapsulated within a vendor's proprietary action.

  • Even if actions/workflows are open-source, ultimately they depend on the infrastructure and idioms of the infrastructure that they are implemented within.

  • Other things

  • By key

  • Writing safe cache keys

  • Dependencies on package-lock.json, OS, and node and npm versions

  • Make sure to add a delimiter that isn't used by any scripts, so that values are not erroneously concatenated together and create a new cache key that may already exist. For example "3" and "39" or "33" and "9". If you use dashes then it becomes 3-39 or 33-9 but they cannot be mixed up.

  • Use a monotonically increasing number that is incremented when you want the cache to be reset

  • npm scripts may cause node_modules to not be cacheable because it can mutate it depending on the source code

  • Also, npm scripts may cause the node_modules not to be cacheable if software is installed outside of node_modules (for example, npm_config_binroot scripts | npm Docs.)

  • How consistent does it need to be?

  • For example, npm caches don't have to match the packages that are being installed, because it will backfill with items from the external registry. However, if the items are only being fetched from the cache, then there is a risk it could be out of date. Check the ETags of resources.

  • Checking the hash of the downloaded file can still help (even if you have to re-download it) because the downloaded file might in and of itself be an installer, so this would save on CPU time re-installing it

  • Advanced auto-expiring cache rules (TTL)

  • Expire after date

  • Expire if file matches hash

  • Expire after end of day, end of week, end of month

  • Expire after day of the week

  • Expire after X days (use X cache keys with +1 added to each of them?)

  • Expire if size of folder is too large

  • Algebra with keys (ORing, ANDing, XORing, etc.) ORing would be a cartesian product

Cost and Resource Management

GitHub Actions charges for compute time (minutes) and storage (artifacts, packages). These tips help keep both under control without sacrificing developer experience.

Workflow Duration and Timeouts

  • Monitor how long your workflows take. Duration typically grows with the codebase — sudden increases indicate a problem.
  • Lower the default 6-hour timeout to match your actual workflow duration plus a reasonable buffer (e.g., if your workflow usually takes 20 minutes, set a timeout of 40–60 minutes).
  • Use Meercode (free tier available) or GitHub's own workflow analytics to visualize duration trends.

Concurrency Control

Limit concurrent builds per pull request to one. Auto-cancel older runs when a new commit is pushed:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

For deployment workflows, set cancel-in-progress: false instead — you don't want two deploys racing each other.

Branch and Trigger Selection

  • Avoid running CI on branches that have no open pull request (except main/master).
  • Don't trigger expensive workflows on documentation-only changes:
on:
  push:
    paths-ignore:
      - '**.md'
      - 'docs/**'

Artifact Storage

  • Compress artifacts before uploading. Be explicit about what files you include.
  • Set retention periods intentionally: PR/dev builds 1–3 days; release artifacts 30–90 days.
  • Exclude files not needed for deployment or debugging (intermediate object files, test fixtures).

Test and Analysis Efficiency

  • Fast fail first: Run the quickest-failing tests first in a separate early job.
  • Selective testing: For monorepos, use path filters or tools like nx affected to run only tests relevant to changed code.
  • Static analysis: Only run static analysis tools if someone is actively looking at and acting on the results.

Dependabot Cost Considerations

Dependabot PRs trigger CI just like developer PRs. Limit open Dependabot PRs with open-pull-requests-limit in dependabot.yml:

version: 2
updates:
  - package-ecosystem: "npm"
    directory: "/"
    schedule:
      interval: "weekly"
    open-pull-requests-limit: 5

Use grouped updates to batch multiple dependency bumps into a single PR.

Appendix