Flux release procedures

Flux release procedures documentation.

This document provides an overview of the release procedures for each component type in the Flux project. It is intended for project maintainers, but may also be useful for contributors who want to understand the release process.

If you have any questions, please reach out to another maintainer for clarification.

Table of contents

General rules

Signing releases

To ensure the integrity and authenticity of releases, all releases must be signed with a GPG key. The public key must be uploaded to the GitHub account of the maintainer, and the private key must be kept secure.

In addition, we recommend the following practices:

  1. Safeguard your GPG private key, preferably using a hardware security key like a YubiKey.
  2. Use a subkey dedicated to signing releases, set an expiration date for added security, and keep the master key offline. Refer to this guide for detailed instructions.

We understand that this may seem overwhelming. If you are not comfortable with the process, don’t hesitate to seek assistance from another maintainer who has experience with signing releases.

Please note that SSH signatures are not supported at this time due to limited availability of SSH signature verification outside the git CLI.

Component types

Shared packages

To release a package as a project maintainer, follow these steps:

  1. Tag the main branch using SemVer.
  2. Sign the tag according to the general rules.
  3. Push the signed tag to the upstream repository.

Use the following commands as an example:

git clone https://github.com/fluxcd/pkg.git
git switch main
git tag -s -m "<module>/<semver>" "<module>/<semver>"
git push origin "<module>/<semver>"

In the commands above, <module> represents the relative path to the directory containing the go.mod file, and <semver> refers to the SemVer version of the release. For instance, runtime/v1.0.0 for fluxcd/pkg/runtime, or http/fetch/v0.1.0 for fluxcd/pkg/http/fetch.

Before cutting a release candidate, ensure that the package’s tests pass successfully.

Here’s an example of releasing a candidate from a feature branch:

git switch <feature-branch>
git tag -s -m "<module>/<semver>-<feature>.1" "<module>/<semver>-<feature>.1"
git push origin "<module>/<semver>-<feature>.1"

Controllers

To release a controller as a project maintainer, follow the steps below. Note that the release procedure differs depending on the type of release.

Controllers: minor releases

  1. Checkout the main branch and pull changes from the remote repository.

  2. Create a “release series” branch for the next minor SemVer range, e.g., release/v1.2.x, and push it to the remote repository.

    git switch -c release/v1.2.x main

  3. From the release series branch, create a release preparation branch for the specific release.

    git switch -c release-v1.2.0 release/v1.2.x

  4. Add an entry to CHANGELOG.md for the new release and commit the changes.

    vim CHANGELOG.md
git add CHANGELOG.md
git commit -s -m "Add changelog entry for v1.2.0"

  5. Update github.com/fluxcd/<name>-controller/api version in go.mod and newTag value in config/manager/kustomization.yaml to the target SemVer (e.g., v1.2.0). Commit and push the changes.

    vim go.mod
vim config/manager/kustomization.yaml
git add go.mod config/manager/kustomization.yaml
git commit -s -m "Release v1.2.0"
git push origin release-v1.2.0

  6. Create a pull request for the release branch and merge it into the release series branch (e.g., release/v1.2.x).

  7. Create api/<next semver> and <next semver> tags for the merge commit in release/v1.2.x. Ensure the tags are signed according to the general rules, and push them to the remote repository.

    git switch release/v1.2.x
git pull origin release/v1.2.x
git tag -s -m "api/v1.2.0" api/v1.2.0
git push origin api/v1.2.0
git tag -s -m "v1.2.0" v1.2.0
git push origin v1.2.0

  8. Confirm that the CI builds and releases the newly tagged version.

  9. Create a pull request for the release series branch and merge it into main.

  10. Add the backport/v1.2.x label to .github/labels.yaml and create a pull request against main.

Controllers: patch releases

  1. Ensure everything to be included in the release is backported to the “release series” branch (e.g., release/v1.2.x). If not, refer to the backporting section.

  2. From the release series branch, create a release preparation branch for the specific patch release.

    git pull origin release/v1.2.x
git switch -c release-v1.2.1 release/v1.2.x

  3. Add an entry to CHANGELOG.md for the new release and commit the changes.

    vim CHANGELOG.md
git add CHANGELOG.md
git commit -s -m "Add changelog entry for v1.2.1"

  4. Update github.com/fluxcd/<name>-controller/api version in go.mod and newTag value in config/manager/kustomization.yaml to the target SemVer (e.g., v1.2.1). Commit and push the changes.

    vim go.mod
vim config/manager/kustomization.yaml
git add go.mod config/manager/kustomization.yaml
git commit -s -m "Release v1.2.1"
git push origin release-v1.2.1

  5. Create a pull request for the release branch and merge it into the release series branch (e.g., release/v1.2.x).

  6. Create api/<next semver> and <next semver> tags for the merge commit in release/v1.2.x. Ensure the tags are signed according to the general rules, and push them to the remote repository.

    git switch release/v1.2.x
git pull origin release/v1.2.x
git tag -s -m "api/v1.2.1" api/v1.2.1
git push origin api/v1.2.1
git tag -s -m "v1.2.1" v1.2.1
git push origin v1.2.1

  7. Confirm that the CI builds and releases the newly tagged version.

  8. Cherry-pick the changelog entry from the release series branch and create a pull request to merge it into main.

    git pull origin main
git switch -c pick-changelog-v1.2.1 main
git cherry-pick -x <commit hash>
git push origin pick-changelog-v1.2.1

Controllers: release candidates

In some cases, it may be necessary to release a release candidate of a controller.

To create a first release candidate, follow the steps to create a minor release, but use the rc.X suffix for SemVer version to release (e.g., v1.2.0-rc.1).

Once the release series branch is created, subsequent release candidates and the final (non-RC) release should follow the procedure for patch controller releases.

Controllers: preview releases

In some cases, it may be necessary to release a preview of a controller. A preview release is a release that is not intended for production use, but rather to allow users to quickly test new features or an intended bug fix, and provide feedback.

Preview releases are made by triggering the release GitHub Actions workflow on a specific Git branch. This workflow will build the container image, sign it using Cosign, and push it to the registry. But it does not require a Git tag, GitHub release, or a changelog entry.

To create a preview release, follow the steps below.

  1. Navigate to https://github.com/fluxcd/<name>-controller/actions/workflows/release.yml.

  2. Click the Run workflow button.

  3. Select the branch to release from the Branch dropdown.

  4. The default values for the image tag (preview) should be correct, but can be changed if necessary.

  5. Click the green Run workflow button.

  6. The workflow will now run, and the container image will be pushed to the registry. Once the workflow has completed, the image reference will be available in the logs, and can be shared in the relevant issue or pull request.

Distribution

To release a Flux distribution as a project maintainer, follow the steps below.

Note that the Flux distribution contains multiple components, and you may need to release controllers before releasing the distribution. Automation is in place to automatically create a pull request to update the version in the main branch when a new controller version is released.

Distribution: minor releases

  1. Ensure everything to be included in the release is merged into the main branch, including any controller releases you wish to include in the release.

  2. Create a “release series” branch for the next minor SemVer range, e.g., release/v2.2.x, and push it to the remote repository.

    git switch -c release/v2.2.x main

  3. Prepare the required release notes for this release, see release notes for more information.

  4. Tag the release series branch with the SemVer version of the release, e.g., v1.2.0. Ensure the tag is signed according to the general rules, and push it to the remote repository.

    git tag -s -m "v2.2.0" v2.2.0
git push origin v2.2.0

  5. Confirm that the CI builds and releases the newly tagged version.

  6. Once the release is complete, update the release notes on GitHub with the release notes prepared in step 3.

  7. Post a message in the #flux CNCF Slack channel announcing the release, and pin it.

  8. Add the backport/v2.2.x label to .github/labels.yaml and create a pull request against main.

Distribution: minor release website

  1. Go to https://app.netlify.com/sites/fluxcd/configuration/deploys#branches-and-deploy-contexts

  2. Click on “Configure”

  3. In “Additional branches” add the “v2-N” branch, e.g. “v2-1” for the 2.2.0 release (the documentation for the 2.2.0 release itself is built from main).

  4. Click “Save”

  5. Create the “v2-N” branch (e.g. “v2-1”) in the website repo from main:

    git checkout main
git pull
git checkout -b v2-1
git push origin HEAD:v2-1

  6. In the v2-1 branch edit the following fields in hugo.yaml:

    1. Set params.archived_version to true
    2. Add the following entry to params.versions:
      - version: "v2.2"
  url: https://fluxcd.io
    3. Edit the existing params.versions entry pointing to https://fluxcd.io to point to https://v2-1.docs.fluxcd.io
    4. Commit the changes and create a PR for the v2-1 branch.

  7. In the main branch edit the following fields in hugo.yaml:

    1. Set params.version to the latest version (e.g. “2.2”)
    2. Add the following entry to params.versions:
      - version: "v2.2"
  url: https://fluxcd.io
    3. Edit the existing params.versions entry pointing to https://fluxcd.io to point to https://v2-1.docs.fluxcd.io
    4. Commit the changes and create a PR for the main branch.

Distribution: patch releases

  1. Ensure everything to be included in the release is backported to the “release series” branch (e.g., release/v2.2.x). If not, refer to the backporting section.

  2. Prepare the required release notes for this release, see release notes for more information.

  3. Tag the release series branch with the SemVer version of the release, e.g., v2.2.1. Ensure the tag is signed according to the general rules, and push it to the remote repository.

    git tag -s -m "v2.2.1" v2.2.1
git push origin v2.2.1

  4. Confirm that the CI builds and releases the newly tagged version.

  5. Once the release is complete, update the release notes on GitHub with the release notes prepared in step 2.

  6. Post a message in the #flux CNCF Slack channel

Distribution: release candidates

In some cases, it may be necessary to release a release candidate of the distribution.

To create a first release candidate, follow the steps to create a minor release, but use the rc.X suffix for SemVer version to release (e.g., v2.2.0-rc.1).

Once the release series branch is created, subsequent release candidates and the final (non-RC) release should follow the procedure for patch releases.

Distribution: release notes

The release notes template for Flux distributions is available in the release-notes-template.md file.

Distribution: Release and EOL notice update

Release and support cadence information is published at https://endoflife.date/flux. The source of truth for that page is maintained in this Git repository. Patch releases should automatically be captured by the automation in place there in a matter of roughly one day after release.

New minor releases need to be added manually to the flux.md page. Create a PR updating the file accordingly.

Backport changes for patch releases

The Flux projects have a backport bot that automates the process of backporting changes from main to the release series branches. The bot is configured to backport pull requests that are labeled with backport:<release series> (e.g., backport:release/v2.1.x) and have been merged into main.

The label(s) are preferably added to the pull request before it is merged into main. If the pull request is merged into main without labeling, they can still be added to the pull request after it has been merged.

The backport bot will automatically backport the pull request to the release series branch and create a pull request for the backport. It will comment on the pull request with a link to the backport pull request.

If the backport bot is unable to backport a pull request automatically (for example, due to conflicts), it will comment on the pull request with instructions on how to backport the pull request manually.

Manual backporting

In some cases, the backport bot may not be suitable for backporting a pull request. For example, if the pull request contains a series of changes of which a subset should be backported. In these cases, the pull request should be backported manually.

To backport a pull request manually, create a new branch from the release series branch (e.g., release/v2.1.x) and cherry-pick the commits from the pull request into the new branch. Push the new branch to the remote repository and create a pull request to merge it into the release series branch.

git pull origin release/v2.1.x
git switch -c backport-<pull request number>-to-v2.1.x release/v2.1.x
git cherry-pick -x <commit hash>
# Repeat the cherry-pick command for each commit to backport
git push origin backport-<pull request number>-to-v2.1.x
Last modified 2023-11-10: Document website changes for minor releases (bd4cb3d)