OCaml dependencies lower-bounds CI

When submitting OCaml packages to the opam package repository, opam-ci runs extensive checks on the package submitted by the pull request. Most of these checks are very standard, involving building and testing the package on various OCaml versions, Linux distributions and OCaml compiler variants. In addition to all of that, there are checks called “lower-bounds”.

The purpose of these unique jobs is to check whether the lower bounds of the package’s dependencies (i.e. their minimal versions), if declared at all, are right. That is, does the package actually compile when the oldest allowed versions of its dependencies are installed. All the other checks follow the default behavior of the opam package manager and install the newest allowed dependencies, essentially checking the upper bounds (at the time of submission at least).

The lower-bounds check works by first installing the dependencies of the package normally. The dependency constraint solver of opam is then reconfigured to instead downgrade and remove as many packages as possible (while still satisfying your package’s lower bounds). Having installed the up-to-date versions first, this step nicely shows the version ranges being downgraded, which makes related issues easier to debug.

Problem

These lower-bounds jobs can be quite annoying when trying to submit a package to opam, because package developers usually don’t test for that. You only find missing or too relaxed lower bounds after doing a release of the package and submitting a PR to opam-repository, just to find out it fails on their extensive CI.

There are two main ways to fix these issues:

1. Tighten the lower bound for a particular dependency (or add a lower bound if it doesn’t have one).
2. If possible, change the usage of a dependency to not require features it only introduced in newer versions.

If you follow the recommendations of dune-release, then after fixing the lower bound, instead of re-releasing the exact same version number of the package (and replacing the archive in-place), you release a new patch version of it. This might go on for a while: you release a patched version, submit that to opam-repository, see another lower-bounds failure, fix that – rinse and repeat.

Solution

It would be much quicker and less hassle, if you could somehow run a similar lower-bounds job on your own GitHub repository’s Actions. In fact, you can by using opam-0install and its --prefer-oldest argument to downgrade the dependencies to their lower bounds¹. The complete GitHub Actions workflow using setup-ocaml is the following (replace MY_PACKAGE with your package name)²:

on:
  push:
  pull_request:

jobs:
  lower-bounds:
    strategy:
      matrix:
        os:
          - ubuntu-latest
        ocaml-compiler:
          - 4.13.1

    runs-on: ${{ matrix.os }}

    env:
      OPAMCONFIRMLEVEL: unsafe-yes # allow opam depext to yes package manager prompts

    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up OCaml ${{ matrix.ocaml-compiler }}
        uses: ocaml/setup-ocaml@v2
        with:
          ocaml-compiler: ${{ matrix.ocaml-compiler }}

      - name: Install dependencies
        run: opam install . --deps-only --with-test

      - name: Install opam-0install
        run: opam install opam-0install

      - name: Downgrade dependencies
        # Option 1: allow OCaml version downgrade
        run: opam install --unlock-base $(opam exec -- opam-0install --prefer-oldest --with-test MY_PACKAGE)
        # Option 2: forbid OCaml version downgrade (specify ocaml-base-compiler again to prevent it from being downgraded)
        # run: opam install $(opam exec -- opam-0install --prefer-oldest --with-test MY_PACKAGE ocaml-base-compiler.${{ matrix.ocaml-compiler }})

      - name: Build
        run: opam exec -- dune build

      - name: Test
        run: opam exec -- dune runtest