diff options
| author | Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> | 2022-10-29 16:42:57 +0200 |
|---|---|---|
| committer | Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org> | 2022-10-30 20:07:07 +0100 |
| commit | 606e5fa891ebeb07be573c252cf98968bfd4160d (patch) | |
| tree | 52293550b705c49831d28d710ea6cdaee6db242b /NLnet/porting_replicant_to_android9 | |
| parent | 92e11031b3a27bea84b6e30ca7e8466b3a92d89b (diff) | |
| download | documents-606e5fa891ebeb07be573c252cf98968bfd4160d.tar.gz documents-606e5fa891ebeb07be573c252cf98968bfd4160d.tar.bz2 documents-606e5fa891ebeb07be573c252cf98968bfd4160d.zip | |
Add document for the October 2022 RFP
This monday (31 October) is the deadline for the request for payments,
and it's faster to reuse porting_replicant_to_android9.tex as all the
information about the tasks that needed to be done is already inside.
Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Diffstat (limited to 'NLnet/porting_replicant_to_android9')
| -rw-r--r-- | NLnet/porting_replicant_to_android9/Makefile | 6 | ||||
| -rw-r--r-- | NLnet/porting_replicant_to_android9/rfp-10-2022.txt | 303 | ||||
| -rw-r--r-- | NLnet/porting_replicant_to_android9/rfp-october-2022.tex | 636 |
3 files changed, 944 insertions, 1 deletions
diff --git a/NLnet/porting_replicant_to_android9/Makefile b/NLnet/porting_replicant_to_android9/Makefile index 6613511..bc88091 100644 --- a/NLnet/porting_replicant_to_android9/Makefile +++ b/NLnet/porting_replicant_to_android9/Makefile @@ -1,7 +1,7 @@ TEXFLAGS ?= -halt-on-error -output-format pdf -output-directory output TITLE = porting_replicant_to_android9 -all: $(TITLE)_en.pdf +all: $(TITLE)_en.pdf rfp-october-2022.pdf output: mkdir output @@ -10,6 +10,10 @@ $(TITLE)_en.pdf: output/10037B00000023D0000005804A3C085245797F4E.png pdflatex $(TEXFLAGS) '$(TITLE).tex' pdflatex $(TEXFLAGS) '$(TITLE).tex' +rfp-october-2022.pdf: output + pdflatex $(TEXFLAGS) rfp-october-2022.tex + pdflatex $(TEXFLAGS) rfp-october-2022.tex + output/10037B00000023D0000005804A3C085245797F4E.png: data/10037B00000023D0000005804A3C085245797F4E.svg output convert data/10037B00000023D0000005804A3C085245797F4E.svg $@ diff --git a/NLnet/porting_replicant_to_android9/rfp-10-2022.txt b/NLnet/porting_replicant_to_android9/rfp-10-2022.txt new file mode 100644 index 0000000..21c0ea8 --- /dev/null +++ b/NLnet/porting_replicant_to_android9/rfp-10-2022.txt @@ -0,0 +1,303 @@ ++----------+ +| Details: | ++----------+ + +3.a: +---- +The code is available at[1]. The code works and was tested in both +Android and GNU/Linux with the ipc-modem tool in[2]. + +3.b: +---- +The base code was available here: https://github.com/fourkbomb/ +linux/commits/modem. + +And the cleaned up code is in[1]. + +The work on cleaning up the driver started: + +- We have a commit doing some cleanups: + 12a435ec8234fa05eef3a85a12a916c04780 + +- The devicetree documentation was added for the modem driver(this is + also something required when upstreaming drivers). + +But the work wasn't completed: the sipc modem driver still doesn’t +pass checkpatch.pl tests so a lot of the work still need to be done. + +However when working on porting libsamsung-ipc to Replicant 9, I found +out that its code quality was so problematic that it was even slowing +down the porting work too much. + +So instead work went into cleaning up libsamsung-ipc, and it can be +found in the many ”switch to Linux code style” commits that have been +merged in libsamsung-ipc upstream[2] and in various other commits +doing code cleanup commits at the same place. + +This is consistent with this task because submiting these driver in +Linux will require kernel maintainers to read libsamsung-ipc code to +understand how this modem protocol work because that is the only +public source of documentation we have on it. + +In addition, nowadays the upstream kernel requires FLOSS userspace +code to be able to test the driver (https://lwn.net/Articles/821817/), +so they need to be able to read and understand the userspace code. + +In addition kernels developers are more used to kernel code than +libsamsung-ipc which implements a protocol unknown to them, so making +the code readable in libsamsung-ipc is very important. + +3.c: +---- +- The port of libsamsung-ipc port is done: It was tested (with + ipc-modem) in both Android and GNU/Linux and it works. Source code: + [2]. + +- The libsamsung-ril port was not finished: while it can boot the + modem, Android doesn’t detect the SIM card. So either it was only + partially converted to the newer RIL daemon or it contains bugs + that need to be fixed. Source code [4]. + +4.a: +---- +- No work done + +4.b: +---- +I only imported a very basic HAL which is hardcoded to speaker out. We +still need something that supports all audio inputs and outputs +(headphones, bluetooth, modem, etc). Source code: [5]. + +4.c: +---- + +The WiFi stack works fine: we can get Internet through it. However +Either there is a bug in the upstream kernel or the upstream driver +lacks a proper calibration file that needs to be written from scratch +for each WiFi hardware variant (some have different analog +components). + +The result is that the range seem extremely poor. For the tests I +tested it between a Replicant 11 phone and a Replicant 6 phone in +access point mode. The two phones were on the same table and were +almost touch- ing eath other. PostmarketOS has the exact same issue. + +The code is in the replicant-11 branch here: +https://git.replicant.us/replicant-next/frameworks_opt_net_wifi/ + +It is based on another existing HAL. When doing the Memorandum of +Understanding, I expected the task to be just configuration as I +assumed that a generic WiFi HAL did exist, but it didn’t. + +So I took one of the HALs in Android upstream and modified it to work +with an upstream Linux. The configuration is in[5]. It is done there +to share the configuration between various devices (GT-I9300 and +GT-I9305). + +4.d: +---- +No work was done. + +5.a: +---- +A bootlogo was added with the following modifications: +- In [5] there is code to add a boot logo (bootanimation.zip). + +- The code to generate the zip is in the main branch of + https://git.replicant.us/replicant/vendor_replicant_artwork + +- A patch was contributed to mat2 to make cleaning up zip + reproducible: + https://0xacab.org/jvoisin/mat2/-/commit/1703ed6ebbe2552990fb847d5fd5dde1d0382 + +It is done in this way because I found no easy way to make Android +build the zip, so the next best alternative was to make the zip +reproducible and to include it as-is. The kernel build system I +adapted from LineageOS or the way mesa generates nir files could not +be adapted easily for building zip files in a way that would have been +easily maintainable over time. + +The name of the distribution and its version are set in [6]. + +5.b: +---- +- The work to find the licenses was not done. I wanted to look into + automatic tools for it but never got the time to try them. + +- For removing nonfree software, it was done by hand using a method + that is known to work well. + +The work of removing nonfree software is mainly in the manifest +repository in the replicant-11-dev branch in +https://git.replicant.us/replicant-next/manifest in commits like +"Remove external/u-boot", that also document where was the nonfree +software (here in u-boot). + +The work has been done with commands like "repo grep" and by reusing +my knowledge of where nonfree software is typically found (I’ve +contributed to many distributions certified by the FSF) and for many +hardware I know what nonfree software they need to work. + +This method was very succesful with finding nonfree software that +slipped through in the Parabola GNU/Linux distribution, so I simply +reused it here and adapted it for Android. + +For removing nonfree software in linux I used the the 5.10.70 Debian +kenrel as a base and I pused it in the debian/v5.10.70 branch. + +And on top I rebased the patches and pused that in the replicant-11 +branch and pused there. The repository is here[1]. + +Before that I attempted to use linux-libre for that but adding back +the ability to load nonfree firmwares (that are not shipped by +Replicant) but it turned out to be too complex because everything is +untertwined together in linux-libre and it doesn’t separate the +removal of nonfree software with blocking the loading of nonfree +firmwares. + +5c: +--- + +During the time I contributed to Replicant (I created the project in +2009, but there were times were other maintainers took charge) all the +privacy issues found came from either LineageOS or CyanogenMod, not +from the official AOSP (Google’s Android Open Source Project) source +code. + +For instance in 2017 we found a very serious issue where the dialer +application sent private data to phone number reverse lookup +providers: https: //redmine.replicant.us/issues/1827 + +This was fixed in Replicant 6.0 (the fix is not part of this MOU): +https://blog.replicant.us/2022/06/new-replicant-6-0-0004-release-and-replicant-11-status/ + +Where that is relevant is that this task was made preciely to be able +to find issues like the one just mentioned because the bug was +confirmed with a network analysis. + +If we look at the ASOP version of the dialer (this can be done with +”git log -p | grep def forward lookup” inside this git tree: +https://android.googlesource.com/platform/ packages/apps/Dialer then +we find nothing at all. This mean that in the master branch commits +adding this malicious functionality are not present at all. + +Here since the Replicant project switched to AOSP (this work was +mostly done by Joonas Kylmälä), so we don’t have this risk level +anymore. It also brings a lot more advantages like git tags to refer +to, or having the ability to use the official Android documentation +for porting work. + +So instead of keeping being based on LineageOS doing this search, I +agreed with Joonas Kylmälä to move to ASOP and instead I added back +fundamental functionalities that were lost during that transition. + +Without my work there would be no [6] repository for Replicant 11. + +It is based on a stripped down version of LineageOS’s vendor lineageos +and the LineageOS history has been kept to not loose the commit on +which the work is based. + +Basically without this work, the maintenance would be a nightmare: +- It enables to build a kernel in Replicant 11 (AOSP uses kernel + binaries instead of building kernels), and that the resulting binaries + match the source code (manual kernel builds are too error prone). +- It needed for branding to work. +- It is needed to be able to ship applications to all supported + devices without having to add the application to each device + repository (for instance in vendor replicant i9300). +- It is also needed to be able to reuse the same build instructions + than Replicant 6.0. + +5d: +--- + +work is not complete as we still need to make it check the signatures +of the zip file. It is still useful as-is because we can (and need to) +reformat the partitions for installing Replicant 11. + +We use the standard Android recovery. A lot of work went into making +it fit into the recovery partition of 8MiB. A very serious bug +(https://github.com/ Benjamin-Dobell/Heimdall/issues/494) was also +reported along the way in heimdall (the tool that is used to install +recoveries for many Samsung devices) and upstream fixed it. + +The code to make the kernel smaller is available at [1]. + +More specifically we have now a replicant recovery defconfig specially +for the recovery. The code to actually build the recovery and its +con- figuration is in [5] (for instance the ”Ship full recovery image +in system” commit). + +5e: +--- +Before our installation instructions had one big page +per device: +- +https://redmine.replicant.us/projects/replicant/wiki/GalaxyS2I9100Installation +- +https://redmine.replicant.us/projects/replicant/wiki/GalaxyS3I9300Installation +- +https://redmine.replicant.us/projects/replicant/wiki/GalaxyTab2Px1xxInstallation +- etc.. + +We now have the same installation instructions for all the devices and +they are at: +https://redmine.replicant.us/projects/replicant/wiki/Installation + +They are already in use for Replicant 6.0 and they were also tested by +less technical people that barely know the command line. + +The information gathered during the tests also helped improving the +instructions. + +They were also improved a lot along the way: there is now information +about backuping important partitions like the modem data partition, +information about the Replicant status, warning about common pitafalls +(like making sure it’s the device is really the right variant of the +phone or tablet model), etc. + +Since these instructions are now modular, they are now used in other +part of the documetation like for explaining how to do upgrades for +instance: +https://redmine.replicant.us/projects/replicant/wiki#Replicant-installation-and-update + +To check the details it is possible to view the history of each page: +There is a button with 3 dots ”...”, clicking on it brings a menu with +the history with the date and author of the modifications. It should +bring a page like that: +https://redmine.replicant.us/projects/replicant/wiki/Installation/history?per_page=50 + +As for supporting Replicant 11 releases, there should be no +differences in the installation instructions (when we'll have a +release, users will be expected to install them in the same way than +before). This is for instance reflected in the recovery task where +verifying the signatures is required. + +References: +----------- +[1]https://git.replicant.us/replicant-next/kernel_replicant_linux/ in +the replicant-11 branch. +[2]https://git.replicant.us/replicant-next/hardware_replicant_libsamsung-ipc + in the replicant-11 branch. +[3]https://git.replicant.us/replicant/hardware_replicant_libsamsung-ipc + in the main branch. +[4]https://git.replicant.us/replicant/hardware_replicant_libsamsung-ril + in the main branch. +[5]https://git.replicant.us/replicant-next/device_samsung_midas_common/ + in the replicant-11 branch. +[6]https://git.replicant.us/replicant-next/vendor_lineage in the + replicant-11 branch. Once checked out with repo, this repository + ends up in vendor/replicant and the lineage name was kept for + compatibility reasons. + +== Build instructions == +The official build instructions[1] should work at the time of writing, +however the work on Replicant will likely not stop after this project +has ended. + +To make it easier for anybody to check the work in the future, I've +also created a 11-nlnet-eu-grant-825310-replicant-update git tag in +the various repositories. The tag can also be used to retrieve all the +source code as documented in the official build instructions[1]. + +[1]https://redmine.replicant.us/projects/replicant/wiki/PortingToAndroid11 diff --git a/NLnet/porting_replicant_to_android9/rfp-october-2022.tex b/NLnet/porting_replicant_to_android9/rfp-october-2022.tex new file mode 100644 index 0000000..25475af --- /dev/null +++ b/NLnet/porting_replicant_to_android9/rfp-october-2022.tex @@ -0,0 +1,636 @@ +\documentclass[a4paper,titlepage]{article} +\usepackage[english]{babel} +\usepackage[a4paper]{geometry} + +\usepackage[gen]{eurosym} +\usepackage{hyperref} +\usepackage[utf8]{inputenc} +\usepackage{listings} +\usepackage{longtable} +\usepackage{multirow} + +\usepackage{graphicx} + +\author{Denis 'GNUtoo' Carikli \and NLnet} +\date{\today} + +\input{costs.tex} + +\begin{document} +\begin{titlepage} + \centering + \vspace{1.5cm} + {\huge\bfseries Request for payment October 2022\par} + \vspace{1.5cm} {\huge\bfseries "Finish porting Replicant to a + newer Android version" Project\par} \vfill +\end{titlepage} + +\section{ + Tasks already payed +} +\begin{itemize} +\item Boot a device under Android 9 without known nonfree libraries +\item Easy Linux upstreaming work +\item Conferences +\end{itemize} + +\section{ + Remaining tasks +} + +\subsection{ + Modem work +} + +\begin{longtable}[!t]{|p{1cm}|p{2cm}|p{6cm}|p{6cm}|} + \hline + Cost estimation & Advancement & Task & Comments \\ + \hline + + \hline + + \workdays{1} & 100\% & + \begin{itemize} + \item{ + Port the the Galaxy SIII (i9300) modem Linux driver from 4.16 to 5.0 + } + \item{ + The driver to port is available at + \url{https://github.com/fourkbomb/linux/} in the modem branch. + } + \end{itemize} & + The code is available at + \url{https://git.replicant.us/replicant-next/kernel_replicant_linux/} + in the replicant-11 branch. + + The code was tested in both Android and GNU/Linux with the ipc-modem + tool available in + \url{https://git.replicant.us/replicant-next/hardware_replicant_libsamsung-ipc/} + + Thanks to this work there was a lot of code cleanup in libsamsung-ipc. + + Before this work, libsamsung-ipc had no automatic testing at + all. Thanks to it, it is now possible to do automatic testing both + at build time and at runtime (in GNU/Linux and/or Android). + + In GNU/Linux we can run test and get all the logs (kernel and + libsamsung-ipc) redirected to logging systems like journald so that + can help tracking failures over time. + + And in both Android and GNU/Linux we can now script sending an SMS + that is then received by ipc-modem and if the content match the test + is succesful. This can help tracking regression much more easily. + + \\ + + \hline + + \workdays{10} & 100\% & + \begin{itemize} + \item{ + Cleanup the the Galaxy SIII (i9300) modem Linux driver. + } + \item { + The code quality of the modem driver for Linux 4.16 is bad. + When doing that kind of work, having bad code quality at the + beginning is normal. It then needs to be cleaned up afterward. As + Simon Shields didn't finish his work, the code still needs to be + cleaned up and completed. Upstreaming the code will probably need + to be done in a later time as it requires to re-architecture part + of the code.} + \end{itemize} & + The base code was available here: + \url{https://github.com/fourkbomb/linux/commits/modem}. + + And the cleaned up code is in + \url{https://git.replicant.us/replicant-next/kernel_replicant_linux}. + + The work to clean the driver started, for instance: + \begin{itemize} + \item{ + We have a commit doing some cleanups: + \url{https://git.replicant.us/replicant-next/kernel_replicant_linux/commit/?h=replicant-11&id=12a435ec8234fa05eef3a85a12a916c047801433}. + } + \item{The devicetree documentation was added for the modem driver + (this is also something required when upstreaming drivers).} + \end{itemize} + + The sipc modem driver still doesn't pass checkpatch.pl tests so a + lot of the work still need to be done. + + However when working on porting libsamsung-ipc to Replicant 9, I + found out that the code quality was extremely problematic to the + point that it was even slowing down the porting work too much. + + The work of cleaning up libsamsung-ipc can be found in the many + "switch to Linux code style" commits that have been merged in + libsamsung-ipc upstream and in various other commits doing code + cleanup (in + \url{https://git.replicant.us/replicant/hardware_replicant_libsamsung-ipc/} + in the main branch). + + So I fixed the libsamsung-ipc code quality instead because submiting + these driver in Linux will require kernel maintainers to read + libsamsung-ipc code to understand how this modem protocol work + because that is the only public source of documentation we have on + it. In addition, nowadays the upstream kernel requires FLOSS + userspace code to be able to test the driver (reference: + \url{https://lwn.net/Articles/821817/}), so they probably read the + userspace code too. + + In addition kernels developers are more used to kernel code than + libsamsung-ipc which implements a protocol unknown to them, so making + the code readable in libsamsung-ipc is very important. + + To cope with bad code quality in kernel drivers, and other issues, + there is a staging area in the Linux kernel to help cleanup drivers + step by step. + + \\ + + \hline + + \workmonths{1} & 90\% & + \begin{itemize} + \item{Port libsamsung-ril and libsamsung-ipc to Android 9} + \item{Make the modem driver and libsamsung-ipc work together} + \item{ + The port of libsamsung-ipc and libsamsung-ril will be attempted + with a compatibility layer at first. Devices manufacturer shipping + new devices with Android 9 are probably required not to use an + approach like that\cite{hal-types}\cite{HIDL}. + This issue will need to be looked in more details at later. + + This will be done to be able to focus more on the modem driver + port to be able to test it. + + Then the compatibility layer will be removed and patches will be merged step + by step in libsamsung-ril and libsamsung-ipc. + + Until the modem driver is upstreamed, we will probably need to keep some of + the patches in a branch, as the kernel API will most probably change as the + driver probably need to be converted to use the network device API. + + Later (which is not part of this deliverable), we will then need + to look if the approach chosen is already using a "binderized HAL" + and if not, what are the consequences of converting the code to use + it, because even if libsamsung-ipc has already been used used in + GNU/Linux, we still want to enable GNU/Linux distributions to easily + use libsamsung-ril as well.} + \end{itemize} & + \begin{itemize} + \item{The port of libsamsung-ipc port is done. It was tested (with + ipc-modem) in both Android and GNU/Linux and it works} + \item{ + The libsamsung-ril port was not finished: while it can boot + the modem, Android doesn't detect the SIM card. So either it was + only partially converted to the newer RIL daemon or it contains + bugs that need to be fixed.} + \end{itemize} + + The code is available in the replicant-11 branch of the following + repositories: + \begin{itemize} + \item + \url{https://git.replicant.us/replicant-next/hardware_replicant_libsamsung-ipc/} + \item + \url{https://git.replicant.us/replicant-next/hardware_replicant_libsamsung-ril/} + \end{itemize} + + \\ + + \hline + + \hline + + \multicolumn{4}{|c|}{ + Total: + \euro{189} + + \euro{1890} + + \euro{4266} x 0.9 = \euro{5918.4} + } \\ + + \hline + +\end{longtable} + +\subsection{ + Other hardware support +} + +\begin{longtable}[!t]{|p{1cm}|p{2cm}|p{6cm}|p{6cm}|} + \hline + Cost estimation & Advancement & Task & Comments \\ + \hline + + \hline + + \workdays{6} & 0\% & + \begin{itemize} + \item{port the sensors libraries and other device specific libraries} + \end{itemize} & The work was not started. \\ + + \hline + + \workdays{4} & 10\% & + \begin{itemize} + \item{Add support for Audio with the upstream kernel driver} + \item{Might be faster, depending on what Android 9 uses.} + \end{itemize} & + I only imported a very basic HAL which is hardcoded to speaker + out. We still need something that supports all audio inputs and + outputs (headphones, bluetooth, modem, etc). + + The code is in the replicant-11 branch of + device\_samsung\_midas\_common: + \url{https://git.replicant.us/replicant-next/device_samsung_midas_common/} \\ + + \hline + + \workdays{1} & 100\% & + \begin{itemize} + \item{Add internal WiFi support and validate the functionality} + \item{This assumes that no big issues are encountered and that Android 9 + already supports upstream Linux standard WiFi interfaces like + nl80211 or Wext. + Note that the internal WiFi don't work without nonfree firmwares. + The Replicant project doesn't ship the firmwares nor advises or + instructs users on how to install them. + This however don't prevent users to install them by their own.} + \end{itemize} & + The WiFi stack works fine: we can get Internet through it. However + Either there is a bug in the upstream kernel or the upstream driver + lacks a proper calibration file that needs to be written from + scratch for each WiFi component variant (some have different analog + components). The result is that the range seem extremely poor. For + the tests I tested it between a Replicant 11 phone and a Replicant 6 + phone in access point mode. The two phones were on the same table + and were almost touching eath other. PostmarketOS has the exact same + issue. + + The code is here: + \url{https://git.replicant.us/replicant-next/frameworks_opt_net_wifi/} + in the replicant-11 branch. It is based on another existing + HAL. When doing the Memorandum of Understanding, I expected the task + to be just configuration as I assumed that a generic WiFi HAL did + exist, but it didn't. So I took one of the HALs upstream and + modified it to work with upstream Linux. The configuration is in the + device\_samsung\_midas\_common repository in the replicant-11 + branch. It is done this way to share the configuration between + various devices (GT-I9300 and GT-I9305). The commits can be viewed + online at this URL: + \url{https://git.replicant.us/replicant-next/device_samsung_midas_common/log/?h=replicant-11} + \\ + + \hline + + \workdays{3} & 0\% & + \begin{itemize} + \item{Add external WiFi support and validate the functionality} + \item{This might be very tricky. This is known to work on Replicant + 6 on some of the supported devices, but it has reliability issues, + which seem to depend a lot on the device used and the charge of + the battery. This is most probably due to the fact that some + devices aren't able to provide enough current to the external + dongle. More research will be needed later (which is not part of + this deliverable) to better understand and characterize the issue + for some of the devices. If no big software issues are + encountered, I will try to make it work as well as on Replicant 6. + This is required for basic usage as the Internal WiFi doesn't work + without nonfree firmwares.} + \end{itemize} & + The work was not started. I didn't even try to integrate USB host + yet.\\ + + \hline + + \hline + + \multicolumn{4}{|c|}{ + Total: \euro{189} + \euro{756} x 0.1 = \euro{202.4} + } \\ + + \hline + +\end{longtable} + +\subsection{ + Release an image +} + +\begin{longtable}[!t]{|p{1cm}|p{1cm}|p{6cm}|p{8cm}|} + \hline + Cost estimation & Advancement & Task & Comments \\ + \hline + + \hline + + \workdays{5} & 100\% & + \begin{itemize} + \item{rebrand LineageOS as Replicant} + \item{ The branding was done extensively in Replicant 6 and + vendor/lineageOS was forked to vendor/replicant with almost all + mention of LineageOS being changed, including in the build system. + This is tricky to do. I tried to rebase the Replicant 6 patches + for that on top of Replicant 6, as merges were used but this + doesn't look simple as the patches contains merges from Replicant + 4.2, and the build system extensively depends on LineageOS + strings. But even if this takes too much time, it's still + possible to do basic branding such as changing the boot screen and + some of the text.} + \end{itemize} & + The code can be found in the following repositories: + \begin{itemize} + \item{The device\_samsung\_midas\_common repository (in the + replicant-11 branch) has code to add a boot logo + (bootanimation.zip). I didn't find any way to generate that zip + file in Android but at least the zip is reproducible: + \begin{itemize} + \item { + The code to generate the zip is in vendor\_replicant\_artwork in + the main branch. The zip is reproducible. + URL: \url{https://git.replicant.us/replicant/vendor_replicant_artwork}} + \item{ A patch was contributed to mat2 to make cleaning up zip + reproducible: + URL: \url{https://0xacab.org/jvoisin/mat2/-/commit/1703ed6ebbe2552990fb847d5fd5dde1d0382a37} + } + \end{itemize} + As for the bootanimation.zip, here's the URL: + URL: \url{https://git.replicant.us/replicant-next/device_samsung_midas_common/log/?h=replicant-11} + } + \item {The name of the distribution and its version are set in the + vendor\_lineage repository (which ends up in the vendor/replicant + directory once the replicant source code is checked out with repo). + + The code is in the replicant-11 branch and the repository URL is at: + \url{https://git.replicant.us/replicant-next/vendor_lineage/}. + } + \end{itemize} + + It is done in this way because I didn't find any way to make Android + build the zip, so the next best alternative was to make the zip + reproducible and to include it as-is. + + \\ + + \hline + + \workdays{10} & 50\% & + \begin{itemize} + \item{Find the repositories global licenses, remove and document proprietary + software} + \item{ + I started working on a script to do list repositories + licenses for Replicant 6 that will then be adapted to Replicant + 9. Android doesn't use packages like GNU/Linux, and instead has + all its source code composed of many git repositories that are + checked out in a single directory. Because of that we need to + write a script to identify the licenses of each repositories. It + will not be as fine grained than GNU/Linux but it's a very good + start and it will most probably find some nonfree code that we + will be able to remove.} + \end{itemize} & + \begin{itemize} + \item{The work to find the licenses was not done. I wanted to look + into automatic tools for it but never got the time to try them.} + \item{For removing nonfree software, it was done by hand using + a method that is known to work well.} + \end{itemize} + + The work of removing nonfree software is mainly in the manifest + repository in thereplicant-11-dev branch: + \url{https://git.replicant.us/replicant-next/manifest/}. Commits + like "Remove external/u-boot" document where is the nonfree software + in u-boot and remove the offending repository. + + The work has been done with commands like "repo grep" and by reusing + my knowledge of where nonfree software is typically found (I've + contributed to many distributions certified by the FSF) and for many + hardware I know what nonfree software they need to work. This method + was very succesful with finding nonfree software that slipped + through in the Parabola GNU/Linux distribution, so I simply reused + it here and adapted it for Android. + + For removing nonfree software in linux I based the work in the + 5.10.70 Debian kenrel, and I pused that in the debian/v5.10.70 + branch. And on top I rebased the patches and pused that in the + replicant-11 branch. The repository is here: + \url{https://git.replicant.us/replicant-next/kernel_replicant_linux/}. + + Before that I attempted to use linux-libre for that but adding back + the ability to load nonfree firmwares (that are not shipped by + Replicant) turned out to be too complex because everything is + untertwined together in linux-libre and it doesn't separate the + removal of nonfree software with blocking the loading of nonfree + firmwares. + \\ + + \hline + + \workdays{5} & 100\% & + \begin{itemize} + \item{Find, remove and document privacy issues in the source code.} + \item{ This task consist in looking for privacy issues. This can be + done for instance by: + \begin{itemize} + \item{ + Looking if issues we found in the past are still present in LineageOS 16 + } + \item{Searching online for known issues in LineageOS 16} + \item{ Doing some network analysis of the traffic, for suspicious + activities like: } + \begin{itemize} + \item {At the first start} + \item {When calling someone} + \item {When using the predictive keyboard} + \end{itemize} + \end{itemize} + It may not find every privacy issues, but it's already better than + what we did before as it's proactive instead of being reactive. } + \end{itemize} & + During the time I contributed to Replicant (I created the project in + 2009, but there were times were other maintainers took charge) all + the privacy issues found came from either LineageOS or CyanogenMod, + not from the official AOSP (Google's Android Open Source Project) + source code. + + For instance in 2017 we found a very serious issue where the dialer + application sent private data to phone number reverse lookup + providers: \url{https://redmine.replicant.us/issues/1827} + + This was fixed in Replicant 6.0 (the fix is not part of this MOU): + \url{https://blog.replicant.us/2022/06/new-replicant-6-0-0004-release-and-replicant-11-status/} + + Where that is relevant is that this task was made preciely to be + able to find issues like the one just mentioned because the bug was + confirmed with a network analysis. + + But when looking at the ASOP version of the dialer (this can be done + with "git log -p | grep def\_forward\_lookup" inside this git tree: + \url{https://android.googlesource.com/platform/packages/apps/Dialer} + then we find nothing at all. This mean that in the master branch + commits adding this malicious functionality are not present at all. + + Here since the Replicant project switched to AOSP (with a lot of + help from Joonas Kylmälä) we don't have this risk level anymore. + + So instead of doing this search I agreed with Joonas Kylmälä to move + to ASOP and instead I added back fundamental functionalities that + were lost during that transition. Without my work there would be no + vendor\_lineageos (that ends up in vendor/replicant when the source + is checked out, the repository name is for compatibility reasons). + + The work is in the replicant-11 branch at + \url{https://git.replicant.us/replicant/vendor_lineageos}. + + It is based on a stripped down version of LineageOS's + vendor\_lineageos and the LineageOS history has been kept to not + loose the commit on which the work is based. + + Basically without this work, the maintenance would be a nightmare: + \begin{itemize} + \item{It enables to build a kernel in Replicant 11 (AOSP uses kernel + binaries instead of building kernels), and that the resulting + binaries match the source code (manual kernel builds are too error + prone).} + \item{It needed for branding to work.} + \item{It is needed to be able to ship applications to all supported + devices without having to add the application to each device + repository (for instance in vendor\_replicant\_i9300).} + \item{It is also needed to reuse the same build instructions than + Replicant 6.0.} + \end{itemize} + + \\ + + \hline + + \workdays{3} & 50\% & + \begin{itemize} + \item{Create a recovery} + \end{itemize} & + While we now have a recovery and that it works, the work is not + complete as we still need to make it check the signatures of the zip + file. It is still useful as-is because we can (and need to) reformat + the partitions for installing Replicant 11. + + We use the standard Android recovery. A lot of work went into making + it fit into the recovery partition of 8MiB. A very serious bug + (\url{https://github.com/Benjamin-Dobell/Heimdall/issues/494}) was + also reported along the way in heimdall (the tool that is used to + install recoveries for many Samsung devices) and upstream fixed it. + The code to make the kernel smaller is available at + \url{https://git.replicant.us/replicant-next/kernel_replicant_linux/} + in the replicant-11 branch. More specifically we have now a + replicant\_recovery\_defconfig specially for the recovery. + + The code to actually build the recovery and its configuration is in + device\_samsung\_midas\_common in the replicant-11 branch: + \url{https://git.replicant.us/replicant-next/device_samsung_midas_common/} + (for instance the "Ship full recovery image in system" commit). + + \\ + + \hline + + \workdays{5} & 100\% & + \begin{itemize} + \item{Create new install and upgrade instructions in a more modular way} + \item{ + The instructions will need to be made modular in order to be + easily changed. More specifically, to make a change that + applied to all the devices, you needed to change + each device-specific instructions, which highly discourage + contributing to the instructions. + The instructions will also need to be as simple as + possible to enable users with very basic command line + usage to use them. + } + \end{itemize} & + Before our installation instructions had one big page per device: + \begin{itemize} + \item{\url{https://redmine.replicant.us/projects/replicant/wiki/GalaxyS2I9100Installation}} + \item{\url{https://redmine.replicant.us/projects/replicant/wiki/GalaxyS3I9300Installation}} + \item{\url{https://redmine.replicant.us/projects/replicant/wiki/GalaxyTab2Px1xxInstallation}} + \item etc.. + \end{itemize} + We now have the same installation instructions for all the devices + and they are at: + \url{https://redmine.replicant.us/projects/replicant/wiki/Installation} + They are already in use for Replicant 6.0 and they were also + tested by less technical people that barely know the command + line. The information gathered during the tests also helped + improving the instructions. + + They were also improved a lot along the way: there is now + information about backuping important partitions like the modem + data partition, information about the Replicant status, warning + about common pitafalls (like making sure it's the device is really + the right variant of the phone or tablet model), etc. + + Since these instructions are now modular, they are now used in other + part of the documetation like for explaining how to do upgrades + for instance: + \url{https://redmine.replicant.us/projects/replicant/wiki#Replicant-installation-and-update} + + To check the details it is possible to view the history of each + page: There is a button with 3 dots "...", clicking on it brings a + menu with the history with the date and author of the + modifications. It should bring a page like that: + \url{https://redmine.replicant.us/projects/replicant/wiki/Installation/history?per_page=50} + + As for supporting Replicant 11 releases, there should be no + differences in the installation instructions (users are expected + to install them in the same way than before). + + \\ + + \hline + + \hline + + \multicolumn{4}{|c|}{ + Total: + \euro{945} + + \euro{1890} x 0.5 + + \euro{945} + + \euro{567} x 0.5 + + 945 = \euro{4063.5} + } \\ + + \hline +\end{longtable} + + +\section{ + Total +} + +The total for this request for payment is \euro{10184.3}. + +(5918.4 + 202.4 + 4063.5). + +\section{ + How to test +} + +The instructions for building and running Replicant 11 are at this +URL: +\url{https://redmine.replicant.us/projects/replicant/wiki/PortingToAndroid11}. + +Instructions to test the modem will be added there as well. + +Additional instructions to test the WiFi will be added in the RFP +itself. + +\begin{thebibliography}{99} + +\bibitem{hal-types} + \url{https://source.android.com/devices/architecture/hal-types} + +\bibitem{HIDL} + \url{https://source.android.com/devices/architecture/hidl/} + +\end{thebibliography} + +\end{document} |
