aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* builder-patches.sh: Add support for transitions releasesHEADmasterDenis 'GNUtoo' Carikli2022-01-242-5/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In the Replicant 6.0 0004 transition release, we have the following patches on top of the vendor/replicant master branch: 53ab2743 Release Replicant 6.0 0004 transition release 6c2772be Re-run the key migration at each boot this is because for transition releases, we need two apply two patches instead of one in vendor/replicant in the builder source code. The goal of the transition releases is to enable users running images signed with older keys to migrate to new images that are signed with newly generated keys (which are stored in vendor/replicant-security), without having to erase all their data. This requires an extra patch to enable the service that runs the key migration script at each boot. After booting a transition release, users are expected to install a regular release like the Replicant 6.0 0004 in order not to have the key migration run at each boot (if the power runs out precisely when the migration script runs, users could loose all their data). The second patch is similar to what we have in all other releases: it just sets the release versions and potentially adds release notes to the changelog. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release.sh: metadata: remove blocking pagerDenis 'GNUtoo' Carikli2022-01-181-2/+2
| | | | | | | | | | | | Without that fix, when releasing the metadata, the release.sh script blocks with a pager with the following content: Saved manifest to [...]/metadata/manifest.xml If the user press 'q', the pager exit and the release proceed. This removes this pager and makes the release proceed without human intervention anymore. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release.sh: Fix missing zip files in the releaseDenis 'GNUtoo' Carikli2022-01-181-1/+1
| | | | | | | | | | Without that fix, we end up with no zip files in the release, so we can't install Replicant. This was broken by commit 10c741bd9ff53e86e6a5035f09af159f7116b6c8 (releasevars.sh: generate RELEASE_IMAGES from RELEASE_DEVICES). Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasetag.sh: Add force option to force git tag and git pushDenis 'GNUtoo' Carikli2021-12-301-6/+10
| | | | | | | | | | The releases are now built from a source tarball that is then officially released and signed as part of the release process. So having to force push to correct mistakes during a release is still possible until that tarball is signed and officially released. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasetag.sh: fix helpDenis 'GNUtoo' Carikli2021-12-301-1/+1
| | | | | | | | | The square brackets ([]) are most of the time used for optional arguments, but here the REPLICANT_DIR argument is mandatory, so we use the Less-than and more-than signs (<>) instead as they are typically used for mandatory arguments. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* make_source_tarball.sh: fix replicant source directory pathDenis 'GNUtoo' Carikli2021-12-301-1/+1
| | | | | | | | | | | | | | | | | | | Without that fix, if the output directory is not the current directory (. or ./), making the tarball fails: $ ./make_source_tarball.sh replicant-6.0-0004 releases Fetching projects: 100% (502/502), done. Checking out projects: 100% (502/502), done. repo sync has finished successfully. tar: releases/replicant-6.0-0004: Cannot stat: No such file or directory tar: Exiting with failure status due to previous errors lzip: replicant-6.0-0004.tar: Can't open input file: No such file or directory This is because the path that referred to the Replicant source directory was relative to the directory that the user was in when launching make_source_tarball.sh, instead of being relative to the current directory, in the script, when running the tar command. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add script to copy the release patches to a remote build machineDenis 'GNUtoo' Carikli2021-12-301-0/+262
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This script makes a patch out of the latest commit from the following repositories: - ../vendor_replicant-release-scripts - ../vendor_replicant - ../manifest it then apply them inside the respective directories inside the replicant directory on another machine, with SSH and Rsync. When doing a release we typically need to change the Replicant version, and so that requires to add 1 patch to each of the repositories mentioned above. However in some cases, the Replicant directory is on another computer. This is for instance the case for people working on their laptop and building Replicant on another machine. The release process already takes care of that situation but it required to manually apply patches on the remote builder before tagging the source code. And the Replicant 6.0 0004 RC6 release failed because some patches were missing in the released (and signed) source code. So it's better to automate things to avoid forgetting patches for the next releases. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* make_source_tarball: switch to lzip for long term archivingDenis 'GNUtoo' Carikli2021-11-102-21/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The xz format is not well adapted for long term archiving[1] as the format has a lot of room for incompatible implementations. In addition, the format is very fragile which makes it harder to implement robust and easy to use recovery tools. In contrast the lzip format is designed to make it easy to recover data from corrupted archives and a tool (lziprecover[2]) already exists for that. There are also several downsides of using lzip over other compression formats (like xz, gzip or bzip2): - The lzip package usually needs to be installed, and there is no 'unlzip' command (users are expected to use 'lzip -d' instead), and for progression you need two -v instead of one but at least once lzip is installed, tar commands work out of the box[3]. - There is no option to use more than one thread in the most popular lzip implementation (GNU lzip). In addition, for the previous Replicant 6.0 source releases tarball, xz -9e compresses more than lzip -9. Here the size in bytes (retrieved with du -b <path>) of the replicant-6.0-0001.tar source release: 13415884800 replicant-6.0-0001.tar 5732651960 replicant-6.0-0001.tar.xz 5927113267 replicant-6.0-0001.tar.lz And for replicant-6.0-0004-rc5: 15629813760 replicant-6.0-0004-rc5.tar 6662406932 replicant-6.0-0004-rc5.tar.xz 6969502733 replicant-6.0-0004-rc5.tar.lz [1]https://www.nongnu.org/lzip/xz_inadequate.html [2]https://www.nongnu.org/lzip/lziprecover.html [3]For instance 'tar xf replicant-6.0-0004-rc5.tar.lz' or 'tar tf replicant-6.0-0004-rc5.tar.lz' work out of the box. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Enable source to work from any directoryDenis 'GNUtoo' Carikli2021-11-085-5/+5
| | | | | | | | | Without that fix, running releasetag.sh in the replicant root directory fails: $ ../releasetag.sh ./ ../releasetag.sh: line 26: releasevars.sh: No such file or directory Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add support for building Replicant 11Denis 'GNUtoo' Carikli2021-10-071-3/+6
| | | | | | | | | | | | | | | | | | | If we don't source envsetup.sh, we have the following when calling 'make clobber' in Replicant 11: build/make/core/main.mk:2: Calling make directly is no longer supported. build/make/core/main.mk:3: Either use 'envsetup.sh; m' or 'build/soong/soong_ui.bash --make-mode' build/make/core/main.mk:4: *** done. Stop. and sourcing envsetup.sh should also work on Replicant 6.0. As Replicant 11 doesn't have a script to build a toolchain, we don't run ./vendor/replicant/build-toolchain with Replicant 11. As for using $_dir instead of $dir, on Replicant 11, $dir is set to 'product' after sourcing build/envsetup.sh so we can't use it. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add Replicant 11 RELEASE_DEVICESDenis 'GNUtoo' Carikli2021-10-072-7/+41
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasevars.sh: generate RELEASE_IMAGES from RELEASE_DEVICESDenis 'GNUtoo' Carikli2021-10-071-22/+5
| | | | | | This avoid having to touch RELEASE_IMAGES when adding new devices. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Move set_color in common.shDenis 'GNUtoo' Carikli2021-10-072-12/+28
| | | | | | | | | | | The set_color function seem to be in releasevars.sh only because releasevars.sh is sourced from other scripts. However the file name of releasevars.sh implies that only release variables related code or variables would be found in it. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasevars.sh: convert remaining spaces to tabsDenis 'GNUtoo' Carikli2021-10-071-15/+20
| | | | | | | | In this repository, in all the files, there are more tabs than spaces, and in releasevars.sh the spaces seem to have been used to avoid lines over 80 characters. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasevars.sh: order devices and images alphabeticallyDenis 'GNUtoo' Carikli2021-10-071-23/+23
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* README: disconnect the builder from the Internet before buildingDenis 'GNUtoo' Carikli2021-04-301-0/+21
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* README: document how to provide source codeDenis 'GNUtoo' Carikli2021-04-301-12/+25
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* make_source_tarball.sh: explain why we git reset --hard the manifestDenis 'GNUtoo' Carikli2021-04-301-0/+2
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release_notes.py: Add blank line between the elements of a listDenis 'GNUtoo' Carikli2021-04-261-0/+19
| | | | | | | | | | | | Without that, emacs will break the lists when using fill-paragraph on them: they will end up in a compact single paragraph with * in the middle of the text. Since the width limit doesn't seem to work well at least with recent versions of python and/or beatifulsoup, it's good not to break the ability to fix it with emacs. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release_notes: Fix the text alignmentDenis 'GNUtoo' Carikli2021-04-261-0/+9
| | | | | | The text is unnecessarily shifted toward the right by two spaces. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release_notes: only process the article contentDenis 'GNUtoo' Carikli2021-04-261-1/+3
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release_notes: Handle reference links without names like [1]Denis 'GNUtoo' Carikli2021-04-261-0/+41
| | | | | | | | | | | | | Some links in WordPress have text like [1], [2], etc. When they get converted into text the "[1]" is kept and a reference is added so we have something like "[[1]][2]" in the final text. The fix is to completely remove the [[1]] part as it is not necessary anymore. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release_notes: cleanup the codeDenis 'GNUtoo' Carikli2021-04-261-28/+39
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* README: Document the new key migration processDenis 'GNUtoo' Carikli2021-04-261-13/+36
| | | | | | | | Applications need to be signed by a set of keys that are generated during the build. When they change, if they is no key migration done, they can prevent the device from booting. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* make_source_tarball.sh: Make compression threads and fetch jobs configurableDenis 'GNUtoo' Carikli2021-03-291-7/+38
| | | | | | | | | | | | | | | | | | | | | This should improve speed a lot: - Increasing the number of fetch jobs speeds up a lot the downloads when resuming from a previously interrupted download session as checking for new commits can have a big latency, so checking many as repositories as possible at the same time speeds up things a lot. With Replicant 4.2 0004, using "-j 100" worked well for me, however it didn't seem to have reached 100 parallel download jobs, so using more would not improve performance in my case. - As for the number of compression threads, one has to consider it carefully: xz -9e uses quite a big amount of RAM when compressing, so using too much threads can result in xz being killed because it used too much RAM. In another hand using 1 thread per core seems to divide the amount of time spent compressing by the number of threads. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* make_source_tarball.sh: add support for git:// for Replicant <= 4.2-0004Denis 'GNUtoo' Carikli2021-03-291-0/+19
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add script to generate source code tarballDenis 'GNUtoo' Carikli2021-03-051-0/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The following text is part of the GPLv2: If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. Since we currently host images (which contains many executable or object code) on ftp.osuosl.org, it would be best to also ship the corresponding source code on the exact same server. This way we could be pretty sure that the corresponding source code is available at the same place. This script can generate the corresponding source code tarballs for all the Replicant 4.2 and 6.0 versions. For that it simply downloads the Replicant source code of a given tag (which corresponds to a release). To save space it doesn't download the history (it uses --depth=1) and it compresses the results with xz -9e, but we still kept the git and repo information in case the build system uses it somehow. Keeping the repo and git information also makes it compatible with the current build instructions on the wiki. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Rename releasebuild.sh to build_release.shDenis 'GNUtoo' Carikli2021-02-253-2/+2
| | | | | | | | The releasebuild name could make people think that this script will release a build. Instead it builds a release. So it's better to rename it to reflect that. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release_notes.py: add support for more recent libraries versionsDenis 'GNUtoo' Carikli2021-01-091-3/+29
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Release procedure: Add information about key handlingDenis 'GNUtoo' Carikli2020-10-242-1/+81
| | | | | | | | | | Before, during and after a release, we need to manage signing keys: - We might need to take care of keeping the gpg signing key still valid (not expired) for some time after the release. - We also may need to migrate from one set of keys generated during the build process to a new one. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasevars.sh: guard against wrong RELEASE variableDenis 'GNUtoo' Carikli2020-10-241-1/+13
| | | | | | | | | This will prevent developers from accidentally pushing tags for a wrong version on new repositories that weren't tagged yet by the previous releases, and also prevent from pushing tags with the same name than branches. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Remove VERSION to be able to use release.sh on branches tooDenis 'GNUtoo' Carikli2020-10-243-17/+18
| | | | | | | | | | | | | | | | | | | | | | To use these scripts, users and developers are expected to define the VERSION and RELEASE variables. For instance: VERSION="6.0" RELEASE="0004-rc2" However if you want to build Replicant 6.0 not as part of an official release process, all the scripts expect a minor version, which is not relevant for a branch. This could for instance be used by anyone, including people that don't have the ability to push tags to release their own builds of Replicant along with information about the source code used to produce these images. So instead of the VERSION and RELEASE example above we could have: RELEASE="6.0-0004-rc2" Or if we build a branch: RELEASE="6.0" Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Release procedure: Also suggest to verify if all repositories fetch fine.Denis 'GNUtoo' Carikli2020-10-241-2/+6
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Release procedure: warn against pushing Replicant versions to branches.Denis 'GNUtoo' Carikli2020-10-241-0/+28
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Release procedure: certificates generation: add workaround to avoid waiting ↵Denis 'GNUtoo' Carikli2020-08-111-1/+3
| | | | | | for input Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Release procedure: mention approximate size requirementsDenis 'GNUtoo' Carikli2020-08-111-7/+16
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add a README documenting the release processDenis 'GNUtoo' Carikli2020-07-261-0/+138
| | | | | | | | This procedure depends on the following commit: 04f0014 releasetag.sh: tag all repositories 04f00147fefd40f18c6b6e7bfa612255124e2dcf Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasetag.sh: tag all repositoriesDenis 'GNUtoo' Carikli2020-07-201-7/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Here is the rationale for tagging all repositories: - In Replicant 6.0 we now have mirrored all the repositories we use, so we can now tag them all. - For libsamsung-ipc, as the tag as replicant-<version> in its name, it cannot conflict with future tags for other distributions, or future Replicant versions. - Even if libsamsung-ipc is meant to be shared across different distributions, having a record of which Replicant versions use which libsamsung-ipc versions in libsamsung-ipc is very important as libsamsung-ipc is heavily tested with Replicant and users reporting bugs in Replicant typically know the Replicant version not the exact libsamsung-ipc commit used. In addition, we need to tag the manifest otherwise we have an error like that: $ ./release.sh replicant-6.0 replicant-6.0-0004-rc2 all Releasing metadata fatal: ambiguous argument 'replicant-6.0-0004-rc2': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' And it is better to have it done automatically, in the same way than for the other repositories. To do that, we will document the release procedure in a subsequent commit to make sure that no step is missed. It will make sure that .repo/manifests has the required tag before being used by release.sh by: - Making sure that the people doing the release don't forget to add patches in the manifest/ directory. Patches are needed to change the revision to the release tag. - Making sure that the tag pushed by releasetag.sh. - Making sure that people doing the release will re-download the source for building, so the tag will be in .repo/manifests. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release.sh: Add more release types to improve debuggingDenis 'GNUtoo' Carikli2020-07-201-0/+52
| | | | | | | | | | Even if people that want to debug this release.sh script will most likely have to modify it anyway, to add some set -x for instance, already having the ability to re-run only a specific failing function can save time if it's already implemented. This change is also relatively non-intrusive, so it doesn't affect people not debugging the script. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Bump RELEASE revision for Replicant 6.0 0004 rc2Denis 'GNUtoo' Carikli2020-07-181-1/+1
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add Replicant 6.0 0004 rc1 supportDenis 'GNUtoo' Carikli2020-01-192-46/+15
| | | | | | | | | | | | | | | | This accomodates the following changes since last release: - The images paths changed to something like replicant-6.0-0004-rc1.zip - There is now a wrapper to build images in vendor/replicant/build.sh - Wolfgang's key has been replaced by my key This change doesn't take that into account but it should still be sufficent for adding the replicant-6.0-0004 tag to the various git repositories. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* releasevars: Document RELEASE_IMAGES and cosmetics improvementsDenis 'GNUtoo' Carikli2020-01-191-2/+35
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release.sh: fix relative pathsDenis 'GNUtoo' Carikli2020-01-191-12/+18
| | | | | | | | | | | | | | | | | | | | | In release_metadata_git, after doing a cd to $REPLICANT_DIR, it tries to write to "$METADATA/git-tags.txt" and it fails like this: $ ./release.sh replicant-6.0 images ./release.sh: line 184: images/metadata/git-tags.txt: No such file or directory ./release.sh: line 184: images/metadata/git-tags.txt: No such file or directory [...] This is because $METADATA is relative to "$RELEASE_DIR": METADATA="$RELEASE_DIR/metadata" And RELEASE_DIR is defined from user input: RELEASE_DIR=$2 So if users use a relative path, then the "$METADATA/git-tags.txt" will be relative to $REPLICANT_DIR. To fix that, the input paths were converted to absolute paths. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* release.sh: Also handle the case where there are no prebuilt applications.Denis 'GNUtoo' Carikli2020-01-191-4/+6
| | | | | | | | | | | | | In Replicant 6.0 0004: - F-Droid was removed due to freedom issues. - RepWifi was removed because it has been replaced by patches enabling to use the stock WiFi settings. This leaves the source with no more prebuilts applications. Though other prebuilts are still used in the toolchain. Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add a script to handle release notes from blog postsDenis 'GNUtoo' Carikli2020-01-182-0/+710
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Add license fileDenis 'GNUtoo' Carikli2020-01-171-0/+674
| | | | Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
* Replicant 6.0 0003 images releaseWolfgang Wiedmeyer2017-12-101-1/+1
| | | | Signed-off-by: Wolfgang Wiedmeyer <wolfgit@wiedmeyer.de>
* Replicant 6.0 0002 images releaseWolfgang Wiedmeyer2017-09-101-3/+3
| | | | | | adds newly supported devices Signed-off-by: Wolfgang Wiedmeyer <wolfgit@wiedmeyer.de>
* release: don't suppress errors with manifest repo for tag collectionWolfgang Wiedmeyer2017-09-101-1/+1
| | | | Signed-off-by: Wolfgang Wiedmeyer <wolfgit@wiedmeyer.de>
* releasebuild: sign all images at once after all of them are builtWolfgang Wiedmeyer2017-09-101-2/+22
| | | | | | This is more convenient if key passwords need to be entered. Signed-off-by: Wolfgang Wiedmeyer <wolfgit@wiedmeyer.de>