path: root/README.txt

+-----------------+
+ Release process +
+-----------------+

Key management and associated commits:
--------------------------------------
Replicant releases use several different set of keys:
- The images being built are at the end signed with a gpg key. That gpg
  key needs to be defined in the RELEASE_KEY variable in the releasevars.sh
  file. Users are expected to manually verify at least the recovery with the
  associated public key as part of the installation instructions.
- In addition, during the build process, additional keys and certificates are
  generated. These are for instance used to sign the Android applications that
  are built as part of the Replicant images. They are also used by the recovery
  to check if the zip being installed is built with the same keys. If not it
  will simply refuse to install it. As the users already verified the gpg
  signature of the recovery, this can also be used to simplify the installation
  instructions by making users check the signature only for the recovery.

When the developer doing the release changes, none of these keys are passed
around to the new developer:
- Having the developer's personal GPG keys enable people to more easily verify
  the developer's key as people can check that directly instead of checking
  the keys of the people who signed a Replicant release key. In addition, there
  is more security as the key don't need to be transferred from person to person.
- As for the keys and certificates generated build process, they don't need to
  be passed around either.

So when a developer is doing a release, that developer gpg public key is
published automatically as part of the release process, and the installation
instructions already contain instructions on how to download and use that public
key, so no extra steps are needed during the release.

However the key (or subkey) signing the release needs to be kept valid (not
expired) for as long as it is relevant for less technical users to install
any of the releases signed by that key.

As for the keys generated during the build, the release instructions already
take care of making users install the recovery associated with a given release,
so that part is being taken care of.

However if the keys signing system application changed between one release
and another, users won't be able to upgrade to the new release without wiping
their data. As this would be extremely inconvenient for users (which potentially
includes yourself as well), we can generate a shell script that will take care of
the migration from the old set of keys to the new one during the first run of the
new Replicant image.

In vendor/replicant-scripts/images/gen_key_migration_script, there is a python
script (gen_key_migration_script.py) that can generate that shell script.

Once generated, that shell script needs to be copied in
vendor/replicant/prebuilt/common/bin/key-migration.sh and a commit needs to be
made in vendor/replicant to add it to the Replicant source code.

Once the commit is made, it doesn't necessarily need to be pushed to a branch:
if it is in the Replicant source code that will be tagged on later, it will then
be pushed to the tag. The next sections have more more details on that.

Alternatively, as the generated keys are in vendor/replicant-security/, it is
also possible to keep the same keys from one release to the next.

The downside of keeping the same keys is that it slightly increase the security
risk as these keys are also used by the recovery to check the zip. Still, since
Replicant 6.0 0004, the release version is now part of the image and can be found
in Settings->About phone to mitigate the fact that the same set of keys could
also end up signing images that are not part of a release.

In another hand, changing the keys at each release would increase the number of
public keys, which will make the migration script slower and slower over time.
As at the time of writing, it check if keys need to be changed at each boot, and
that already takes a significant amount of time (about 5 seconds) so if we were
to change the generated keys at each release, we would also need to redesign part
of that script.

Adding the last commits before the release:
--------------------------------------------
Before a release, we need to make some additional changes in several
repositories.

In this repository:
- We need to edit releasevars.sh and change the RELEASE field to reflect the
  release version we will have, as it is used by the scripts. Additionally we
  might want to fix bugs and/or improve things along the way.

In the vendor/replicant repository:
- We need need to modify REPLICANT_VERSION in config/common.mk and RELEASE
  in sign-build.sh to add the new release version.
- We might also need to modify the ChangeLog or not depending on the type
  of release.

In the manifest repository:
- We need to change all the revisions to use the new tag.

However it's best not to push these vendor/replicant and manifest changes
yet. Since Replicant 4.2, every repository used to produce the Replicant
images being released are tagged with the specific version of that release
(like replicant-6.0-0004-rc2). In addition it's also possible to build the
latest version of a major Replicant version by using a branch instead
(like replicant-6.0).

As the branches need to be buildable at any moment, if we push the manifest
and vendor/replicant modifications described above in the branch of a major
Replicant version (like replicant-6.0), we will end up with images that will
indicate that they are coming from a release (like replicant-6.0-0003-rc2)
instead.

In some cases, this could become a serious issue: If you are using keys that
have been or will be used in a release, and build an image from a branch,
if that branch has the Replicant version of a release, people could be
(accidentally) mislead to think that they are installing a release while they
are not.

In turn this could lead to a huge number of hours being lost on trying to
track down regressions due to making the wrong assumption about which image
is being run.

As we will need to tag all the source code later on, we can just add the
commits that have the modifications described above in the source code, without
pushing them: they will then be picked up by the tagging process and be pushed
in the tag.

Preparing the source code for tagging:
--------------------------------------
First you need to make sure that all the commits with the changes mentioned
above are in the replicant directory before proceeding because otherwise the
tagging script will tag revisions that don't contain any of these commits.

You also need to make sure that you have no other local modifications that
aren't intended to go in the release for the same reason.

Here's a procedure you can follow to do that very fast:
- save the commits you have on top, like the "last commits before the release"
  which were discussed previously.
- move the .repo directory outside of the replicant directory
- delete the replicant directory
- re-create the replicant directory
- move the .repo directory in it
- re-run the repo init and repo sync commands. This will re-use the .repo cache
  which will speed up a lot the download time.
- re-import all the commits you need to add on top of the existing source code.

Make sure that the manifest commits are added in the git repository that is
checked out in manifest/ as the tagging script will use that to push the tags
for the manifest.

If you also want to check if all the source code fetches fine (for instance if
you recently migrated some repositories to the Replicant mirrors system) you can
also re-download all the source code from scratch instead.

Theses are probably not the only ways to do it. If you have other ideas, feel
free to send patches to add them in this README as well.

Tagging the source code:
------------------------
Once this is done we can tag the various repositories with releasetag.sh.

This script will tag all the repositories but the manifest, so it expects
all repositories to be mirrored on the Replicant git server.

If you build Replicant on a machine that cannot push the code, for instance
because it doesn't have access to your ssh keys, you can simply export the
Replicant source directory with sshfs to your main computer which has all the
access and run the releasetag.sh from it.

Building the source:
--------------------
Once the tagging is done, you need to re-download the source code again from the
new manifest tag as tools like release.sh expect the manifest to have the release
tag.
Doing it from scratch will make sure that the repositories are fetchable and
that nothing went wrong.

Once this is done, it would be a good idea to make sure that you have enough
space for building the release: For a release, you will need to build Replicant
for all the supported devices, so it can take more space than usual.

As a point of comparison, for the Replicant 6.0 0004 RC2 release which supported
10 targets (espresso3g, espressowifi, i9100, i9300, i9305, maguro, n5100, n5110,
n7000, n7100) after the build the replicant-6.0 directory took about 219 GiB
(228773716 bytes).

After that, you can finally start building all the images with build_release.sh.
The build process can many hours on an older computer, so it's a good idea to
plan that ahead. For instance, if your build computer is noisy and that its
noise can prevent you from sleeping, you might want considering launching the
build as soon as you wake up the morning to have it finished before you need to
go to sleep, or if it's possible you might also want to consider moving that
computer before starting the build.

At the time of writing, around the end of the build of the first target, it
will ask you some information for a certificate, so make sure to be around
as that part is interactive. Alternatively you can try to make it fill-in the
defaults and make the build proceed by pressing enter multiple times right after
having launched the build.

Then it will proceed for many hours uninterrupted if everything goes fine.

You can checks the build logs in logs/ to see if the build has finished.

Releasing the images:
---------------------
When the build is done, you can finally use the the release.sh script to
populate the final directory with  all the files to release.

For instance for releasing Replicant 6.0 0004 RC2 I used the following:
$ release.sh replicant-6.0 replicant-6.0-0004-rc2

The replicant-6.0 directory contained the Replicant source code and
replicant-6.0-0004-rc2 was an empty directory.

This script can take a long time if the connection to your builder is slow, as
this downloads all the images.

You need to sign the images with the release.sh script. I used the following
for the replicant-6.0-0004-rc2 release:
$ release.sh replicant-6.0 replicant-6.0-0004-rc2 signatures

The signature is done as a separate process to enable use cases where the
connection to the build server is slow and gpg is setup to be run interactively
(for instance by typing a password).

You can then push that directory to the FTP server and run the FTP server
trigger script when all the files have been copied.

Announcing the release:
-----------------------
As releases also need to be announced, it might be a good idea to work on the
release text in parallel.

We typically inform people through both the mailing list and the blog system we
use. At the time of writing, we use WordPress for our blog, which isn't very
adapted to our needs as the drafts are not public. However we wrote a script to
make it easier to review drafts and to publish them on the mailing list.

To use the script, you need to save the html page that contains the blog post,
and use the release_notes.py on it. If the blog post is a draft, you can simply
preview the draft, and save the resulting html page.

In addition to the blog post and the mailing list:
- We might also need to modify the Replicant IRC channel(s) topic(s) to indicate
  the last Replicant version.
- We also need to push a patch to the replicant website (website.git) to do the
  same: the last Replicant version is mentioned on the main page of the
  replicant.us website.

Copyright:
----------
This README is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.