Tool container release process

Within PhenoMeNal we have adopted a versioning strategy for containers and tools which, among other things, mandates how releases need to be performed. This page summarizes how to follow this procedure to make a new release for your tool.

To begin, you'll need to set specific container labels with the tool and the container versions; you'll also have to use specific branch names in the container's PhenoMeNal GitHub repository. These are explained in the Dockerfile creation guide.

Once you've set the various container labels and your code is organized in the recognized branches, making a new release of your container is simply a matter of merging the current develop branch to the master branch — but don't do this yet! Our CI will then build the new head of the master branch and will tag the new docker image with a release tag that uses the current version. This strategy allows us to discriminate between development builds (stemming from the develop branch, with a dev_ prefix on the docker image tag) and release/production builds (stemming from the master branch, with no dev_ prefix). In PhenoMeNal we use develop branch for everyday work/coding (and hopefully feature branches that merge to develop) and master for release versions of the containers.

We'll describe this procedure in detail in the text below.

Pre-requisites for making a release

• The container is adequately versioned and is compliant with the Dockerfile creation guide.
  • Among other things, this compliancy means that the git repo has both master and develop branches.
• The container has been tested and verified to work correctly (test it with data!). Generally this means that you have configured a test job here to run the runTest1.sh (or equivalent) script that you provisioned in your container. The relevant Jenkins jobs shouldn't be failing.
• The container works with Galaxy or another workflow runtime environment that we support.
• If the tool is part of a workflow, then the workflow has been tested, at least manually and the tool has been verified to work in that setting.

Making the release

Start by choosing a release name, which should be the current version tag for your container, without the branch prefix and build number. For instance, if this would the latest development build of my tool which complies with all pre-requisites, then release name should be v0.2_cv1.0; note that we omit the dev_ prefix and patch number for the container version (the third number after _cv) as this varies when we commit.

Please note that, if since the last release neither the major or minor version number for the container version have changed, your new release tag will be the same as the previous release tag, which git won't allow. If in this case and there has been no change for the version numbers, please bump the minor version number for the container to avoid failure.

Then, there are basically two ways of making a release — namely, merging the develop branch into master:

• Make a git flow release: this is the recommended way, as it will handle the branching pattern for you and create useful tags for later reference.
• Manually merge to master: alternative if you don't want to use git flow, but it might require more manual steps and making sure that develop is completely ready for merging.

Either way, make sure that your local git repository is up-to-date with the remote repository at GitHub, both at the master and develop branches. You can verify this with git remote -v show <origin>.

Output should look like this.

remote origin
  Fetch URL: https://github.com/phnmnl/container-abcd.git
  Push  URL: https://github.com/phnmnl/container-abcd.git
  HEAD branch: master
  Remote branches:
    develop tracked
    master  tracked
  Local branches configured for 'git pull':
    develop merges with remote develop
    master  merges with remote master
  Local refs configured for 'git push':
    develop pushes to develop (up to date)
    master  pushes to master  (up to date)

Git flow based release

The git flow release process is nicely explained diagrammatically in the git flow cheatsheet. It can be done both through command line calls to git flow or through the many git GUI clients — such as Atlassian SourceTree, Git Kraken and many others. Basically, the idea is that you (1) start a release

git flow release start <release-name>

creating a new release/<release-name> branch. You then make any necessary adjustments on this new branch (e.g., make sure that README.md and container version labels are in sync with the version numbers, make sure that the galaxy wrapper has the correct version on it, etc.), although it might be perfectly normal that you don't need to do any changes. If any, commit the changes. Finally, regardless of whether you made changes or not, finish the release with:

git flow release finish <release-name>

This will trigger a merge from the release branch to master, master will be tagged with the <release-name> and the same merge from the release branch will be done to the develop branch. Push the changes to your local master and develop branches to GitHub:

git checkout master
git push
git checkout develop
git push

The CI server will see the changes and build a new version from the master branch, with an appropiate tag.

On Atlassian SourceTree, this process can be triggered by using the menus on the GUI:

• Repository -> Git flow / Hg flow -> Start release
• Make any changes if needed, commit, and then to finish
• Repository -> Git flow / Hg flow -> Finish release
  • GUI might ask if you want to delete the the release branch (say no) and if you want to push straight away (choose no as well). This is to simplify matters if something goes wrong in the process (for instance, merge conflicts with master).
• Push both master and develop branches.

Manually

Supposing that you're standing on the develop branch:

• Make any release-specific changes, such as setting version numbers; check that README.md and labels in Dockerfile file are in sync, etc. Commit those changes locally to develop.
• Merge develop into master and push changes:

git checkout master
git merge --no-ff develop
git tag -a <release-name>
git push
git push --tags
git checkout develop
git push

Note that on the above commands, we are not using a transitory release branch. You could also use a temporary release/<release-name> branch in this manual approach, but you would need to adapt the steps above: merge from the release branch into both develop and master, push both branches to GitHub, and finally delete that feature branch, if desired.