Bionic libs and the dynamic linker are bind mounted

This change makes the bionic libs and the dynamic linker from the
runtime APEX (com.android.runtime) available to all processes started
after apexd finishes activating APEXes.

Specifically, the device has two sets of bionic libs and the dynamic
linker: one in the system partition for pre-apexd processes and another
in the runtime APEX for post-apexd processes. The former is referred as
the 'bootstrap' bionic and are located at
/system/lib/{libc|libdl|libm}.so and /system/bin/linker. The latter is
referred as the 'runtime' bionic and are located at
/apex/com.android.runtime/lib/bionic/{libc|libdl|libm}.so and
/apex/com.android.runtime/bin/linker.

Although the two sets are located in different directories, at runtime,
they are accessed via the same path: /system/lib/* and
/system/bin/linker ... for both pre/post-apexd processes. This is done
by bind-mounting the bootstrap or the runtime bionic to the same path.
Keeping the same path is necessary because there are many modules and
apps that explicitly or implicitly depend on the fact that bionic libs
are located in /system/lib and are loaded into the default linker
namespace (which has /system/lib in its search paths).

Before the apexd is started, init executes a built-in action
'prepare_bootstrap_bionic' that bind-mounts the bootstrap bionic to the
mount points. Processes started during this time are provided with the
bootstrap bionic. Then after the apexd is finished, init executes
another built-in action 'setup_runtime_bionic' which again mounts the
runtime bionic to the same mount points, thus hiding the previous mounts
that target the bootstrap bionic. The mounting of the runtime bionic
(which is only for post-apexd processes) is hidden from pre-apexd
processes by changing propagation type of the mount points to 'private'
and execute the pre-apexd processes with a new mount namespace using
unshare(2). If a pre-apexd process crashes and re-launched after the
apexd is on, the process still gets the bootstrap bionic by unmounting
the runtime bionic which effectively un-hides the previous bind-mounts
targeting the bootstrap bionic.

Bug: 120266448
Test: device boots
Test: cat /proc/`pidof zygote`/mountinfo shows that
/system/lib/{libc|libdl|libm}.so and /system/bin/linker are from the
runtime APEX
Test: cat /proc/'pidof vold`/mountinfo shows that the same mount points
are from system partition.

Change-Id: I7ca67755dc0656c0f0c834ba94bf23ba9b1aca68

1 files changed, 76 insertions, 0 deletions

diff --git a/init/service.cpp b/init/service.cpp
index 272809f7c..645c34fcb 100644
--- a/init/service.cpp
+++ b/init/service.cpp
@@ -140,6 +140,43 @@ Result<Success> Service::SetUpMountNamespace() const {
     return Success();
 }
 
+Result<Success> Service::SetUpPreApexdMounts() const {
+    // If a pre-apexd service is 're' launched after the runtime APEX is
+    // available, unmount the linker and bionic libs which are currently
+    // bind mounted to the files in the runtime APEX. This will reveal
+    // the hidden mount points (targetting the bootstrap ones in the
+    // system partition) which were setup before the runtime APEX was
+    // started. Note that these unmounts are done in a separate mount namespace
+    // for the process. It does not affect other processes including the init.
+    if (pre_apexd_ && ServiceList::GetInstance().IsRuntimeAvailable()) {
+        if (access(kLinkerMountPoint, F_OK) == 0) {
+            if (umount(kLinkerMountPoint) == -1) {
+                return ErrnoError() << "Could not umount " << kLinkerMountPoint;
+            }
+            for (const auto& libname : kBionicLibFileNames) {
+                std::string mount_point = kBionicLibsMountPointDir + libname;
+                if (umount(mount_point.c_str()) == -1) {
+                    return ErrnoError() << "Could not umount " << mount_point;
+                }
+            }
+        }
+
+        if (access(kLinkerMountPoint64, F_OK) == 0) {
+            if (umount(kLinkerMountPoint64) == -1) {
+                return ErrnoError() << "Could not umount " << kLinkerMountPoint64;
+            }
+            for (const auto& libname : kBionicLibFileNames) {
+                std::string mount_point = kBionicLibsMountPointDir64 + libname;
+                std::string source = kBootstrapBionicLibsDir64 + libname;
+                if (umount(mount_point.c_str()) == -1) {
+                    return ErrnoError() << "Could not umount " << mount_point;
+                }
+            }
+        }
+    }
+    return Success();
+}
+
 Result<Success> Service::SetUpPidNamespace() const {
     if (prctl(PR_SET_NAME, name_.c_str()) == -1) {
         return ErrnoError() << "Could not set name";
@@ -929,6 +966,14 @@ Result<Success> Service::Start() {
         scon = *result;
     }
 
+    if (!ServiceList::GetInstance().IsRuntimeAvailable() && !pre_apexd_) {
+        // If this service is started before the runtime APEX gets available,
+        // mark it as pre-apexd one. Note that this marking is permanent. So
+        // for example, if the service is re-launched (e.g., due to crash),
+        // it is still recognized as pre-apexd... for consistency.
+        pre_apexd_ = true;
+    }
+
     LOG(INFO) << "starting service '" << name_ << "'...";
 
     pid_t pid = -1;
@@ -945,6 +990,26 @@ Result<Success> Service::Start() {
             LOG(FATAL) << "Service '" << name_ << "' could not enter namespaces: " << result.error();
         }
 
+        if (pre_apexd_) {
+            // pre-apexd process gets a private copy of the mount namespace.
+            // However, this does not mean that mount/unmount events are not
+            // shared across pre-apexd processes and post-apexd processes.
+            // *Most* of the events are still shared because the propagation
+            // type of / is set to 'shared'. (see `mount rootfs rootfs /shared
+            // rec` in init.rc)
+            //
+            // This unsharing is required to not propagate the mount events
+            // under /system/lib/{libc|libdl|libm}.so and /system/bin/linker(64)
+            // whose propagation type is set to private. With this,
+            // bind-mounting the bionic libs and the dynamic linker from the
+            // runtime APEX to the mount points does not affect pre-apexd
+            // processes which should use the bootstrap ones.
+            if (unshare(CLONE_NEWNS) != 0) {
+                LOG(FATAL) << "Creating a new mount namespace for service"
+                           << " '" << name_ << "' failed: " << strerror(errno);
+            }
+        }
+
         if (namespace_flags_ & CLONE_NEWNS) {
             if (auto result = SetUpMountNamespace(); !result) {
                 LOG(FATAL) << "Service '" << name_
@@ -952,6 +1017,13 @@ Result<Success> Service::Start() {
             }
         }
 
+        if (pre_apexd_ && ServiceList::GetInstance().IsRuntimeAvailable()) {
+            if (auto result = SetUpPreApexdMounts(); !result) {
+                LOG(FATAL) << "Pre-apexd service '" << name_
+                           << "' could not setup the mount points: " << result.error();
+            }
+        }
+
         if (namespace_flags_ & CLONE_NEWPID) {
             // This will fork again to run an init process inside the PID
             // namespace.
@@ -1324,6 +1396,10 @@ void ServiceList::MarkServicesUpdate() {
     delayed_service_names_.clear();
 }
 
+void ServiceList::MarkRuntimeAvailable() {
+    runtime_available_ = true;
+}
+
 void ServiceList::DelayService(const Service& service) {
     if (services_update_finished_) {
         LOG(ERROR) << "Cannot delay the start of service '" << service.name()