soong_jar: Parallel compression

This compresses multiple files in parallel, and will split up larger
files (5MB+) into smaller chunks (1MB) to compress in parallel.

There is a small size overhead to recombine the chunks, but it's only a
few bytes per chunk, so for a 1MB chunk, it's minimal.

Rough numbers, with everything in the page cache, this can compress
~4GB (1000 files) down to 1GB in 6.5 seconds, instead of 120 seconds with
the non-parallel soong_jar and 150 seconds with zip.

Go's DEFLATE algorithm is still a bit worse than zip's -- about 3.5%
larger file sizes, but for most of our "dist" targets that is fine.

Change-Id: Ie4886c7d0f954ace46e599156e35fea7e74d6dd7

1 files changed, 2 insertions, 0 deletions

diff --git a/Android.bp b/Android.bp
index 523bec16..e287dabe 100644
--- a/Android.bp
+++ b/Android.bp
@@ -206,8 +206,10 @@ blueprint_go_binary {
 
 blueprint_go_binary {
     name: "soong_jar",
+    deps: ["android-archive-zip"],
     srcs: [
         "cmd/soong_jar/soong_jar.go",
+        "cmd/soong_jar/rate_limit.go",
     ],
 }