diff options
author | Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> | 2021-11-09 18:02:14 +0100 |
---|---|---|
committer | Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> | 2021-11-10 01:53:04 +0100 |
commit | 82bfee48f074c496347e77e277d63cd1c7c1155a (patch) | |
tree | 7136db4a20e64ba48cb1a632dbc99afcd54a4927 | |
parent | ace7e3c1fcc5842f9c1baf7887a23be96da0fd40 (diff) | |
download | vendor_replicant-release-scripts-82bfee48f074c496347e77e277d63cd1c7c1155a.tar.gz vendor_replicant-release-scripts-82bfee48f074c496347e77e277d63cd1c7c1155a.tar.bz2 vendor_replicant-release-scripts-82bfee48f074c496347e77e277d63cd1c7c1155a.zip |
make_source_tarball: switch to lzip for long term archiving
The xz format is not well adapted for long term archiving[1]
as the format has a lot of room for incompatible
implementations. In addition, the format is very fragile
which makes it harder to implement robust and easy to use
recovery tools.
In contrast the lzip format is designed to make it easy to
recover data from corrupted archives and a tool
(lziprecover[2]) already exists for that.
There are also several downsides of using lzip over other
compression formats (like xz, gzip or bzip2):
- The lzip package usually needs to be installed, and there
is no 'unlzip' command (users are expected to use
'lzip -d' instead), and for progression you need two -v
instead of one but at least once lzip is installed,
tar commands work out of the box[3].
- There is no option to use more than one thread in the most
popular lzip implementation (GNU lzip).
In addition, for the previous Replicant 6.0 source releases
tarball, xz -9e compresses more than lzip -9.
Here the size in bytes (retrieved with du -b <path>) of the
replicant-6.0-0001.tar source release:
13415884800 replicant-6.0-0001.tar
5732651960 replicant-6.0-0001.tar.xz
5927113267 replicant-6.0-0001.tar.lz
And for replicant-6.0-0004-rc5:
15629813760 replicant-6.0-0004-rc5.tar
6662406932 replicant-6.0-0004-rc5.tar.xz
6969502733 replicant-6.0-0004-rc5.tar.lz
[1]https://www.nongnu.org/lzip/xz_inadequate.html
[2]https://www.nongnu.org/lzip/lziprecover.html
[3]For instance 'tar xf replicant-6.0-0004-rc5.tar.lz' or
'tar tf replicant-6.0-0004-rc5.tar.lz' work out of the
box.
Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
-rw-r--r-- | README.txt | 4 | ||||
-rwxr-xr-x | make_source_tarball.sh | 24 |
2 files changed, 7 insertions, 21 deletions
@@ -204,8 +204,8 @@ make_source_tarball.sh will have created. This way we can simply publish that tarball along with the images to make sure that we are providing complete and corresponding source code in the same location than the images. -For Replicant 6.0 0004 RC5, the replicant-6.0-0004-rc5.tar.xz tarball is about -6.3GiB (6662406932 bytes) and once the images for all the supported devices are +For Replicant 6.0 0004 RC5, the replicant-6.0-0004-rc5.tar.lz tarball is about +6.5GiB (6969502733 bytes) and once the images for all the supported devices are built, the replicant-6.0-0004-rc5 directory takes about 211GiB (223219395971 bytes). diff --git a/make_source_tarball.sh b/make_source_tarball.sh index 3843eed..a8c6701 100755 --- a/make_source_tarball.sh +++ b/make_source_tarball.sh @@ -19,43 +19,30 @@ set -e usage() { - echo "$0 <tag> <output directory> [-j N] [-T N]" + echo "$0 <tag> <output directory> [-j N]" echo " -j N use at most N download jobs to run in parallel;" echo " the default is 4;" - echo " -T N use at most N threads to compress the tarball;" - echo " the default is 1; set to 0 to use as many threads" - echo " as there are processor cores" - exit 1 } jobs=4 -threads=1 get_option() { option="$1" value="$2" - if [ "${option}" = "-T" ] ; then - threads="${value}" - elif [ "${option}" = "-j" ] ; then + if [ "${option}" = "-j" ] ; then jobs="${value}" else usage fi } -if [ $# -ne 2 -a $# -ne 4 -a $# -ne 6 ] ; then +if [ $# -ne 2 -a $# -ne 4 ] ; then usage elif [ $# -eq 4 ] ; then get_option "$3" "$4" -elif [ $# -eq 6 ] ; then - if [ "$3" = "$5" ] ; then - usage - fi - get_option "$3" "$4" - get_option "$5" "$6" fi tag="$1" @@ -98,10 +85,9 @@ fi cd .. tarball="${tag}.tar" -if [ ! -f "${tarball}" ] && [ ! -f "${tarball}.xz" ] ; then +if [ ! -f "${tarball}" ] && [ ! -f "${tarball}.lz" ] ; then tar cf "${tarball}" "${replicant_dir}" || rm -f "${tarball}" fi # TODO: Make the tarball reproducible by removing metadata - -xz -T "${threads}" -9e --verbose "${tarball}" +lzip -vv -9 "${tarball}" |