We encourage developers to use branches when developing the model. Do you find version control, branching and merging a bit difficult, try the NorESM svn branch/merge tutorial to understand branching and merging. In 30 minutes you can become an svn wizard
Follow this link to find the tutorial: SVN Branch/Merge tutorial
Note: As of November 13th 2015, NorESM uses git as version control system. The rules and guidelines for merging/branching, tags and branch names are still valid in the new system
A branch is your own version of the repository. Sometimes it is nice to work on a feature “in private” without having to relate to other developers' codes. And sometimes you want to work on something which takes some time before it is finished.
In these cases you can create a branch. You can work on it on your own or in a team of colleques. When you are happy with the the content of the branch, you can merge it back to the trunk.
The trunk is considered a safe repository. The code in the trunk should always have passed a set of tests to make sure it is stable.(https://wiki.met.no/noresm/testlist)
Branches let you take advantage of the svn features, but without having to worry every day that your code passes the tests.
Create a branch with (http://svnbook.red-bean.com/en/1.7/svn.branchmerge.using.html)
svn copy http://svn.example.com/repos/calc/trunk \
http://svn.example.com/repos/calc/branches/my-calc-branch \
-m "Creating a private branch of /calc/trunk."
or specifically for the NorESM repository with
svn copy https://svn.met.no/NorESM/noresm/tags/trunk2.0-1 \
https://svn.met.no/NorESM/noresm/branches/privateMYPORJECT_trunk2.0-1 \
-m "Creating a project branch of tags/trunk2.0-1."
Then check out the branch using
svn checkout $BRANCHURL nameOfBranchOnMyPC
In git: First create your new branch locally and then make the remote aware of the new branch like so:
git checkout -b my_branch_name git push -u origin my_branch_name
..and make sure your .gitconfig-file is configured for doing a merge (for example):
[merge]
tool = vimdiff
[diff]
tool = vimdiff
In most cases the answer is “always”. However look at it this way:
If you work directly on the trunk you have to check that your code passes the tests every time you change the code. This is cumbersome and takes a lot of time. However if you are on a branch, you can work peacefully together with your small team taking full advantage of the version control system. In the end, when you and your teammates are happy with the changes, you perform the tests and merge back to trunk.
You should obviously test your code also before you commit to a branch, but for a branch the test does not include running and analyzing long simulations.
The NorESM branches have the following naming convension : {PurposeOfLife}_{ParentTagName}. This means that a branch name should state why it is created and where it is created from.
PurposeOfLife is a text string (allowed characters are a-z, A-Z, 0-9 and “.”) that must start with feature, release, project or private, where
ParentTagName is the name of an existing noresm tag.
Examples of valid branch names are:
featureIceActivation_trunk2.0-1 (development of ice activation feature, created from tag trunk2.0-1)release2.0.0_trunk2.0-19 (release branch created from tag trunk2.0-19) release2.0.1_release2.0.0-15 (release branch created from tag release2.0.0-15) privateHiatusStudy_release2.0.1-3 (private branch for Ingo's hiatus study, created from tag release2.0.1-3)projectEXPECT_cmip5-r143-1 (branch created for use in project named EXPECT, created from tag cmip5-r143-1)Note that in this scheme a new branch is never branched off directly from trunk or another branch. A tag MUST be created before creating a branch (see section on tags below).
Note that purposeOfLife of release-branches contains version numbers. “release2.0.0” and “release2.0.1” are “purposeOfLife”. When creating release-branches, please agree on numbering with Mats.Bentsen@uni.no. The version numbers in release-branches' “purposeOfLife” should not be confused with “increasingVersionNumber” used to make tag-names unique.
Consider the following example: A small group decides to have a branch of their own. The group defines their branch's purpose of life as “featureLandSurfaceModeling”.
They can now branch off from trunk and create the branch featureLandSurfaceModeling_trunk2.0-1. They will work happily on their branch until one day they have completed their feature. In the mean time they have tagged their branch a couple of times as featureLandSurfaceModeling-1, featureLandSurfaceModeling-2.
After the feature is completed, the branch is merged back to trunk, and following svn recommendations, the branch should now be considered dead.
The team still want their branch on which they cooperate well. They should now re-generate their branch from trunk, for example featureLandSurfaceModeling_trunk2.0-40. This is OK. The repository should now have two branches, but by inspection of branch names it is easy to see that featureLandSurfaceModeling_trunk2.0-40 is the recent one and the other is a dead end.
While developing on featureLandSurfaceModeling_trunk2.0-40, the team decides to tag their model twice, creating the tags featureLandSurfaceModeling-3 and featureLandSurfaceModeling-4.
The repository now has four tags: featureLandSurfaceModeling-1, featureLandSurfaceModeling-2, featureLandSurfaceModeling-3 and featureLandSurfaceModeling-4. They originate from two different branches. However this is OK in the NorESM naming convention scheme!
The PurposeOfLife string of a release branch should begin with “release” followed by <ModelMajorVersion>.<ModelMinorVersion>.<ReleaseMinorVersion> (e.g., release2.0.1).
The numbers <ModelMajorVersion> and <ModelMinorVersion> are inherited from the tag from which the branch is created. The number <ReleaseMinorVersion> is set to 0 in the special case that the parent is a trunk tag (e.g., release2.0.0_trunk2.0-19) and augmented if the release branch is created from an existing release branch tag (e.g., release2.0.1_release2.0.0-5).
For more information on <ModelMajorVersion> and <ModelMinorVersion> see section “How should I name the tag”.
You can decide this for yourself. The main hypothesis is that the trunk is always stable and working, so you will not harm your branch by merging from trunk. If you are working on something which you will finally merge back to the trunk, you can merge often. Then the final merge will be easier.
Just use svn merge ^/noresm/trunk (The “^” means “the URL of the repository's root directory“ in newer versions of svn. In older versions you need to give full URL)
You can merge to trunk after your changed code has passed a list of tests. The tests are available here: https://wiki.met.no/noresm/testlist
If your code does not pass the tests, you can not merge your code back to the trunk
Note that in svn, you can only merge ONE time from your branch to the trunk, or you risk making a mess of the system! (See http://svnbook.red-bean.com/en/1.7/svn.branchmerge.basicmerging.html =⇒ “Reintegrating a branch”, note the statement Once a –reintegrate merge is done from branch to trunk, the branch is no longer usable for further work”). N.B. There is a workaround to this, described in http://svnbook.red-bean.com/en/1.7/svn.branchmerge.advanced.html#svn.branchmerge.advanced.reintegratetwice
The merge command (from trunk) will be something like (http://svnbook.red-bean.com/en/1.7/svn.branchmerge.basicmerging.html)
svn merge --reintegrate $BRANCHURL
Using git, just use
git merge branchNameIWantToMergeWith
A tag is a version of the model which should be considered “frozen”. It makes sense to make a new “tag” for a production system, for example a specific version which should be used for many runs.
In svn notation, there is no difference between a branch and a tag. It is just a question of naming convension. Recommended convension is that tags are branches saved under the “tags” subdirectory and a branch is saved under the “branches” directory.
In NorESM tags are considered frozen, and people are not allowed to do developement on tagged versions.
Create the tag with a command like (http://svnbook.red-bean.com/en/1.6/svn.branchmerge.tags.html)
svn copy http://svn.example.com/repos/calc/trunk \
http://svn.example.com/repos/calc/tags/release-1.0 \
-m "Tagging the 1.0 release of the 'calc' project."
or specifically for the NorESM repository with
svn copy https://svn.met.no/NorESM/noresm/trunk \
https://svn.met.no/NorESM/noresm/tags/trunk2.0-2 \
-m "Creating new tag of trunk."
You should create a tag in the following cases:
All tags are in https://svn.met.no/viewvc/noresm/noresm/tags/. Have a look there to find out which tags exist before you create a new tag. In order not to break the naming convension scheme, you need to know which tags exist already!
Tags created from trunk have the naming convention:
trunk{MainModelVersion}.{MinorModelVersion}-{IncreasingVersionNumber}
Tags created from branches have the naming convention:
{BranchPurposeOfLife}-{IncreasingVersionNumber}
Every time a release branch is created from trunk, then MinorModelVersion is increased and IncreasingVersionNumber reset to 1. If no new release branch is created, then MinorModelVersion stays the same and IncreasingVersionNumber is increased.
Important: If a user wants to create a tag from a freshly created branch, then IncreasingVersionNumber should be set to 0 (e.g., featureMicomDevelopment-0). Be aware, however, that creating a tag from a freshly created branch results in tag duplication, e.g., if the branch is featureMicomDevelopment_trunk2.0-19 then featureMicomDevelopment-0 will be identical to trunk2.0-19.
MainModelVersion and MinorModelVersion are global counters while IncreasingVersionNumber is local to the trunk or a specific branch.
Examples are:
trunk2.0-1trunk2.0-2featureIceActivation-1release2.0.0-1release2.0.0-2release2.0.1-1 (associated branch created from tag release2.0.0-2)privateHiatusStudy-1 (owned by Ingo)privateKatlaStudy-1 (owned by Øyvind)Note how this is consistent with the branch naming scheme. You actually need to create a tag in order to give your branch a proper name!
NorESM0 and NorESM1 had a quite random naming convension for branches where branch names involved “noresm” and “revision numbers”. For example a NorESM1 branch names is “noresm-ver1_cmip5-r112/”. When tagging these, we need to know what is “purposeOfLife” of this branch. People tend to talk about these branches as the “112-version” or the branch “noresm-ver1_cmip5-r143/” as the “143 version”.
Therefore, even though it is confusing it is proposed here to use “cmip5-r112” or “cmip5-r143” as purposeOfLife for these branches. So tags created from these branches would be called “cmip5-r112-1”, “cmip5-r112-2”, “cmip5-r143-1”, “cmip5-r143-2” etc.
purposeOfLife of NorESM1 branches is therefore whatever follows after “noresm-verX-” in the branch name
A user has performed a control simulation using a tagged NorESM version. He/she wants to use this control simulation as baseline for several new implementations. How should the user create/update working branches without having to worry about changes in trunk that can affect the result?
This is straight forward. The user creates several branches based on the original tag. Following the naming convension scheme, they all have a different “PurposeOfLife” but they have the same ParentTagName. The combination is a unique branch name for all the new implementations.
Importantly, “merge from trunk” - which is required to be able to reintegrate the branch into trunk - has to be postponed until after evaluation experiments for the new implementations have been performed.
A user group wants to develop one model component without constantly having to worry about changes in other model components.
A solution is to bundle new implementations for a specific component by creating a super branch (e.g., featureMicomDevelopment_trunk2.0-19).
The super branch can then be used to create ordinary feature branches for the individual implementations (e.g., featureNewMixingScheme_featureMicomDevelopment-0).
Reintegration into trunk is done in two steps: First, feature branches from the individual implementations are reintegrated into the super branch. Last, the super branch is reintegrated into trunk.
A tagged NorESM version has been used for the production of a certain simulation. Part of the simulations has to be rerun with extended diagnostic capability (e.g., with U10 output). How can I commit the extended diagnostic capability to svn?
One can imagine that something like this happened:
When the update is needed there are two possibilities:
This is the normal (and simplest) case:
Update the “release2.0.0_trunk2.0-45” branch with the changes needed. After the development is done on “release2.0.0_trunk2.0-45” brand, tag this as “release2.0.0-4” and do the experiments.
In the mean time, someone has done a lot of bugfixing in “release2.0.0_trunk2.0-45”, so the latest version of “release2.0.0_trunk2.0-45” is significantly different from “release2.0.0-3” and you are afraid including the bugfixes will change the original results. You can not include all the bugfixes before doing the extra simulations so using the latest version of “release2.0.0_trunk2.0-45” is not an option.
In this case, create a new branch: “release2.0.1_release2.0.0-3”, update it and tag it “release2.0.1-1”. Note that in this case, you have created a new branch with a different “purposeOfLife”: “release2.0.1” is different from “release2.0.0”.
Always state the JIRA issue associated with the work! This makes the code changes pop up in JIRA!: For example “svn commit -m “NE-100: These are the fixes needed to solve this issue” ” (see https://wiki.met.no/noresm/usingtheissuetracker)
The commit message should include a concise description of the committed changes.
Furthermore, it should indicate whether the changes preserve bit-reproducibility. If this information is omitted, then one should assume that bit-identity is broken.
Examples 1:
Added new compset COMPSETNAME, bit identical compared to revision r
Examples 2:
Added new diagnostics in cloud.F90, bit identical compared to revision r
A more elaborated “purpose of life” than indicated by the branch name can be provide as svn message when creating the branch. If a corresponding jira issue exists, then the text provided to the issue track can be reused here.
Ideally, the commit message of a tag should provide a complete change history relative to the last tag. This information can be easily extracted from the svn viewer.
Example 1:
Change history: trunk-2.0 -> trunk-2.1 (r190 -> r198) Revision 198 changes made in r198... Revision 193 changes made in r193... Revision 190 changes made in r190...
Example 2:
Change history: trunk-2.3 -> featureMicomDevelopment-1 (r200 -> r205) Revision 205 changes made in r205 to branch featureMicomDevelopment... Revision 202 changes made in r202 to branch featureMicomDevelopment... Revision 200 changes made in r200 to branch featureMicomDevelopment...