+-----------------+ + Release process + +-----------------+ Key management and associated commits: -------------------------------------- Replicant releases use several different set of keys: - The images being built are at the end signed with a gpg key. That gpg key needs to be defined in the RELEASE_KEY variable in the releasevars.sh file. Users are expected to manually verify at least the recovery with the associated public key as part of the installation instructions. - In addition, during the build process, additional keys and certificates are generated. These are for instance used to sign the Android applications that are built as part of the Replicant images. They are also used by the recovery to check if the zip being installed is built with the same keys. If not it will simply refuse to install it. As the users already verified the gpg signature of the recovery, this can also be used to simplify the installation instructions by making users check the signature only for the recovery. When the developer doing the release changes, none of these keys are passed around to the new developer: - Having the developer's personal GPG keys enable people to more easily verify the developer's key as people can check that directly instead of checking the keys of the people who signed a Replicant release key. In addition, there is more security as the key don't need to be transferred from person to person. - As for the keys and certificates generated build process, they don't need to be passed around either. So when a developer is doing a release, that developer gpg public key is published automatically as part of the release process, and the installation instructions already contain instructions on how to download and use that public key, so no extra steps are needed during the release. However the key (or subkey) signing the release needs to be kept valid (not expired) for as long as it is relevant for less technical users to install any of the releases signed by that key. As for the keys generated during the build, the release instructions already take care of making users install the recovery associated with a given release, so that part is being taken care of. However if the keys signing system application changed between one release and another, users won't be able to upgrade to the new release without wiping their data. As this would be extremely inconvenient for users (which potentially includes yourself as well), we can generate a shell script that will take care of the migration from the old set of keys to the new one during the first run of the new Replicant image. In vendor/replicant-scripts/images/gen_key_migration_script, there is a python script (gen_key_migration_script.py) that can generate that shell script. Once generated, that shell script needs to be copied in vendor/replicant/prebuilt/common/bin/key-migration.sh and a commit needs to be made in vendor/replicant to add it to the Replicant source code. Once the commit is made, it doesn't necessarily need to be pushed to a branch: if it is in the Replicant source code that will be tagged on later, it will then be pushed to the tag. The next sections have more more details on that. Alternatively, as the generated keys are in vendor/replicant-security/, it is also possible to keep the same keys from one release to the next. The downside of keeping the same keys is that it slightly increase the security risk as these keys are also used by the recovery to check the zip. Still, since Replicant 6.0 0004, the release version is now part of the image and can be found in Settings->About phone to mitigate the fact that the same set of keys could also end up signing images that are not part of a release. In another hand, changing the keys at each release would increase the number of public keys, which will make the migration script slower and slower over time. As at the time of writing, it check if keys need to be changed at each boot, and that already takes a significant amount of time (about 5 seconds) so if we were to change the generated keys at each release, we would also need to redesign part of that script. Adding the last commits before the release: -------------------------------------------- Before a release, we need to make some additional changes in several repositories. In this repository: - We need to edit releasevars.sh and change the RELEASE field to reflect the release version we will have, as it is used by the scripts. Additionally we might want to fix bugs and/or improve things along the way. In the vendor/replicant repository: - We need need to modify REPLICANT_VERSION in config/common.mk and RELEASE in sign-build.sh to add the new release version. - We might also need to modify the ChangeLog or not depending on the type of release. In the manifest repository: - We need to change all the revisions to use the new tag. However it's best not to push these vendor/replicant and manifest changes yet. Since Replicant 4.2, every repository used to produce the Replicant images being released are tagged with the specific version of that release (like replicant-6.0-0004-rc2). In addition it's also possible to build the latest version of a major Replicant version by using a branch instead (like replicant-6.0). As the branches need to be buildable at any moment, if we push the manifest and vendor/replicant modifications described above in the branch of a major Replicant version (like replicant-6.0), we will end up with images that will indicate that they are coming from a release (like replicant-6.0-0003-rc2) instead. In some cases, this could become a serious issue: If you are using keys that have been or will be used in a release, and build an image from a branch, if that branch has the Replicant version of a release, people could be (accidentally) mislead to think that they are installing a release while they are not. In turn this could lead to a huge number of hours being lost on trying to track down regressions due to making the wrong assumption about which image is being run. As we will need to tag all the source code later on, we can just add the commits that have the modifications described above in the source code, without pushing them: they will then be picked up by the tagging process and be pushed in the tag. Preparing the source code for tagging: -------------------------------------- First you need to make sure that all the commits with the changes mentioned above are in the replicant directory before proceeding because otherwise the tagging script will tag revisions that don't contain any of these commits. You also need to make sure that you have no other local modifications that aren't intended to go in the release for the same reason. Here's a procedure you can follow to do that very fast: - save the commits you have on top, like the "last commits before the release" which were discussed previously. - move the .repo directory outside of the replicant directory - delete the replicant directory - re-create the replicant directory - move the .repo directory in it - re-run the repo init and repo sync commands. This will re-use the .repo cache which will speed up a lot the download time. - re-import all the commits you need to add on top of the existing source code. Make sure that the manifest commits are added in the git repository that is checked out in manifest/ as the tagging script will use that to push the tags for the manifest. If you also want to check if all the source code fetches fine (for instance if you recently migrated some repositories to the Replicant mirrors system) you can also re-download all the source code from scratch instead. Theses are probably not the only ways to do it. If you have other ideas, feel free to send patches to add them in this README as well. Tagging the source code: ------------------------ Once this is done we can tag the various repositories with releasetag.sh. This script will tag all the repositories but the manifest, so it expects all repositories to be mirrored on the Replicant git server. If you build Replicant on a machine that cannot push the code, for instance because it doesn't have access to your ssh keys, you can simply export the Replicant source directory with sshfs to your main computer which has all the access and run the releasetag.sh from it. Building the source: -------------------- Once the tagging is done, you need to re-download the source code again from the new manifest tag as tools like release.sh expect the manifest to have the release tag. Doing it from scratch will make sure that the repositories are fetchable and that nothing went wrong. Once this is done, it would be a good idea to make sure that you have enough space for building the release: For a release, you will need to build Replicant for all the supported devices, so it can take more space than usual. As a point of comparison, for the Replicant 6.0 0004 RC2 release which supported 10 targets (espresso3g, espressowifi, i9100, i9300, i9305, maguro, n5100, n5110, n7000, n7100) after the build the replicant-6.0 directory took about 219 GiB (228773716 bytes). After that, you can finally start building all the images with build_release.sh. The build process can many hours on an older computer, so it's a good idea to plan that ahead. For instance, if your build computer is noisy and that its noise can prevent you from sleeping, you might want considering launching the build as soon as you wake up the morning to have it finished before you need to go to sleep, or if it's possible you might also want to consider moving that computer before starting the build. At the time of writing, around the end of the build of the first target, it will ask you some information for a certificate, so make sure to be around as that part is interactive. Alternatively you can try to make it fill-in the defaults and make the build proceed by pressing enter multiple times right after having launched the build. Then it will proceed for many hours uninterrupted if everything goes fine. You can checks the build logs in logs/ to see if the build has finished. Releasing the images: --------------------- When the build is done, you can finally use the the release.sh script to populate the final directory with all the files to release. For instance for releasing Replicant 6.0 0004 RC2 I used the following: $ release.sh replicant-6.0 replicant-6.0-0004-rc2 The replicant-6.0 directory contained the Replicant source code and replicant-6.0-0004-rc2 was an empty directory. This script can take a long time if the connection to your builder is slow, as this downloads all the images. You need to sign the images with the release.sh script. I used the following for the replicant-6.0-0004-rc2 release: $ release.sh replicant-6.0 replicant-6.0-0004-rc2 signatures The signature is done as a separate process to enable use cases where the connection to the build server is slow and gpg is setup to be run interactively (for instance by typing a password). You can then push that directory to the FTP server and run the FTP server trigger script when all the files have been copied. Announcing the release: ----------------------- As releases also need to be announced, it might be a good idea to work on the release text in parallel. We typically inform people through both the mailing list and the blog system we use. At the time of writing, we use WordPress for our blog, which isn't very adapted to our needs as the drafts are not public. However we wrote a script to make it easier to review drafts and to publish them on the mailing list. To use the script, you need to save the html page that contains the blog post, and use the release_notes.py on it. If the blog post is a draft, you can simply preview the draft, and save the resulting html page. In addition to the blog post and the mailing list: - We might also need to modify the Replicant IRC channel(s) topic(s) to indicate the last Replicant version. - We also need to push a patch to the replicant website (website.git) to do the same: the last Replicant version is mentioned on the main page of the replicant.us website. Copyright: ---------- This README is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.