aboutsummaryrefslogtreecommitdiffstats
path: root/cmd
Commit message (Collapse)AuthorAgeFilesLines
* Add integration testing infrastructureColin Cross2017-07-141-1/+2
| | | | | | | | | | | | | | | | | | | | | Fix mutator registration for tests to allow different tests in the same package to register different mutators. Allow tests to track the resulting ModuleBuildParams objects to use in assertions, and provide helpers for getting them. For example: config := android.TestConfig(buildDir) ctx := android.NewTestContext() ctx.RegisterModuleType(...) ctx.MockFileSystem(...) ctx.ParseBlueprintsFile("Android.bp") ctx.PrepareBuildActions(config) ctx.Register() // Get the Inputs value passed to the javac rule for the foo module inputs := ctx.ModuleForTests("foo".Rule("javac").Inputs Test: java_test.go Change-Id: I10c82967f5f3586d2c176f169906b571ed82fc73
* microfactory: Sort dependenciesDan Willemsen2017-07-131-0/+1
| | | | | | | | This was causing android/soong/ui/build to rebuild sometimes when it didn't need to. Test: `m -j nothing; grep "B compile" out/.soong_ui.trace` repeatedly Change-Id: I5d33271a7b037f53674a0d312d8da1746eed8eaa
* Add microfactory tracingDan Willemsen2017-07-133-5/+52
| | | | | | Test: m clean; load out/build.trace.gz in chrome://tracing Test: m -j nothing; load again Change-Id: I67d4e006a4bdde593b54c20d6d93a48278fe696d
* Support running soong_ui.bash from anywhere in the treeDan Willemsen2017-07-121-1/+4
| | | | | | | | | | | It will cd to $TOP before running soong_ui itself, so that soong_ui still runs from the top of the tree. ORIGINAL_PWD is saved so that we can reference that later (for example, to move 'mma' implementation into Go). Test: cd system; ../build/soong/soong_ui.bash Test: Set absolute / relative OUT_DIR or OUT_DIR_COMMON_BASE and repeat Change-Id: Icb67a3ee6a3358cca50300755c8953419fc19437
* Make '**' implicitColin Cross2017-06-232-3/+34
| | | | | | | Default to copying all entries in the input zip to the output zip. Test: zip2zip_test Change-Id: I0d70620c621bfeee2b7fe2a64b350b4866ad2877
* Add jar sorting to zip2zipColin Cross2017-06-232-10/+79
| | | | | | | | | | Jars have a strange sorting order; the META-INF/ directory should come first, then META-INF/MANIFEST.MF, then any other files in META-INF/, and then any files outside META-INF. Add a -j argument to zip2zip that sorts using jar ordering. Test: zip2zip_test Change-Id: I80e2bc7e284ef74f6561c26cb6541298834db1bc
* zip2zip: Support sorting globbed arguments, '**'Dan Willemsen2017-06-223-42/+272
| | | | | | | | | | | | | | | When '-s' is passed, any globbed arguments will have their results sorted. When there are multiple arguments, the files will still be inserted in argument order. A bare '**' is now special cased to mean every file in the input zip. This allows zip2zip to sort entire zip files efficiently by using `zip2zip -s -i <> -o <> '**'`. This can be useful if your original zip program used filesystem ordering which was not reproducible. Test: m -j blueprint_tools (new tests pass) Change-Id: Ic3512c5fe14c94c6f3e134296905121d2ff8b58a
* Make absolute OUT_DIR work with sboxJeff Gaston2017-06-121-5/+37
| | | | | | | | Test: make OUT_DIR=/tmp/abspath Test: make OUT_DIR=relative-out Bug: 35562758 Change-Id: I688d5f6117b194440c5d01c1040033b5671187c2
* Easier debugging of sboxJeff Gaston2017-06-091-4/+24
| | | | | | | | | | Cleanup of some failure messages Also, this leaves the temp directory untouched if a declared output was not created Bug: 35562758 Test: make Change-Id: I8ef1315af80eb327752501f12a331dbdf52ba3e9
* Have Soong try to enforce that genrules declare all their outputs.Jeff Gaston2017-06-092-0/+154
| | | | | | | | | | | This causes Soong to put the outputs of each genrule into a temporary location and copy the declared outputs back to the output directory. This gets the process closer to having an actual sandbox. Bug: 35562758 Test: make Change-Id: I8048fbf1a3899a86fb99d71b60669b6633b07b3e
* Add pom2mk, a maven repo -> Android.mk toolDan Willemsen2017-06-072-0/+273
| | | | | | | | | This is still fairly simplistic, but good enough to test some build system changes. Bug: 33381544 Test: run, inspect output Change-Id: Ia5c19570493116dca01cb65605cdf20becf8c1d0
* Prepare multiproduct_kati to run on the build serversDan Willemsen2017-05-221-11/+60
| | | | | | | | | | | | | Adds a -dist argument that will use DIST_DIR to save logs. Also writes a summary of each std.log to stderr on errors, so that the error is more likely to show up in the error reporting. This output is prefixed with "> " to differentiate it from the progress reports from multiproduct_kati itself. Test: multiproduct_kati -only-config Test: DIST_DIR=dist build/soong/build_test.bash -dist (introducing errors) Change-Id: I5005b5f3f200c876bc004dd9b0e01e7b6edf5be2
* Add build_test.sh, split common parts of soong_ui.bashDan Willemsen2017-05-151-0/+88
| | | | | | | | | | Move common microfactory functions to cmd/microfactory/microfactory.bash so that they can be used to build both soong_ui for normal build and multiproduct_kati for build system tests. Test: m -j Test: build/soong/build_test.bash Change-Id: I9512642d846ce54d05a027b6d33a2b3029b3f90b
* Allow specifying a build variantDan Willemsen2017-05-151-1/+3
| | | | | Test: multiproduct_kati --variant userdebug Change-Id: I6548889caf868cad6270110c038753d029247572
* Improve multiproduct_kati outputDan Willemsen2017-05-151-17/+83
| | | | | | | | | It now uses the same output style as ninja, overwriting status lines in smart terminals. Test: multiproduct_kati Test: multiproduct_kati | cat Change-Id: I8db5198ffdc5ebc5503241ac492379753d92978e
* Supported minor features in soong_zipNan Zhang2017-05-021-47/+103
| | | | | | | | | | 1. Added ability to keep the mixed "-f"/"-l" order as same as command-line flags order. 2. Added "-s" flag to specify which target file within zip is stored uncompressed. Test: manual Change-Id: I338b25a7bd6bf1b7e9cc29ad3324575167630fb7
* Don't write output during testsDan Willemsen2017-04-202-6/+7
| | | | | | | Removes the "\nPASS\n" print (since we only strip "PASS\n") Test: m -j blueprint_tools Change-Id: I31abd8474d92af29e1fa4c1ae5a940f6a588336d
* Convert soong_javac_filter to a wrapperColin Cross2017-04-183-22/+126
| | | | | | | | | | Piping the output of javac through a filter makes it hard to capture the exit status. Convert it to a wrapper that executes javac and propagates the exit status. Bug: 36666657 Test: javac_wrapper_test Change-Id: I9b56cc3794023aabc9328138a68830e26e980f97
* Merge "multiproduct_kati: better directory names."Treehugger Robot2017-04-041-4/+6
|\
| * multiproduct_kati: better directory names.Steven Moreland2017-03-291-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Naming directories according to current date. This makes them easier to parse when using the tool multiple time. Also considered: millis := time.Now().UnixNano() / 1000000 name := fmt.Sprintf("multiproduct-%d", millis) Also considered putting separators in the number, but it makes for too long of a directory name. Test: multiproduct_kati Change-Id: I10ac6754094653abf5bf11b04efc3c44905d3c8d
* | Microfactory support for transitive link dependenciesJeff Gaston2017-03-312-19/+51
| | | | | | | | | | | | Bug: 36816202 Test: make -j Change-Id: Idbdf81002ed4c363a2b725905d91ffc204ef3d0e
* | fileslist: hash the content of symlink, not the file it points to.Sen Jiang2017-03-291-11/+19
| | | | | | | | | | | | | | | | | | | | | | | | IsDir() doesn't handle the case where the file is a symlink to a directory, which cause fileslist to crash. The hash is used to validate whether system image is the same, so hashing the content of symlink makes more sense. Bug: 36274890 Test: joule builds Change-Id: I6359418a5b28f8da13f85b01a30a72228fecf4ce
* | Add fileslist.go to calculate hashes in parallelMakoto Onuki2017-03-242-0/+185
|/ | | | | | | Bug: 36274890 Test: Manual Change-Id: I548da5607cf3b993ad21cbb04a57fcbd5bfb7f51
* Added functionality in soong_zip that supports adding root prefix.Nan Zhang2017-03-201-63/+79
| | | | | | | | | | | | | | | Added -P flag in soong_zip: It provided a path prefix which will be added in front after -f/-l was applied with -C. eg. ./soong_zip -o /path1/../ -P a/c -C d/e -f d/e/test.txt zip_dest: a/c/test.txt Bug: b/31676493 Test: go build -a to build a real soong_zip, and test it with real files. Change-Id: Ic924ba252e963ca5847c370f733058888426b696
* Refactor compressWholeFile to take a zipEntryandroid-n-mr2-preview-2Dan Willemsen2017-03-151-18/+5
| | | | | | Test: m -j vts; compare android-vts.zip Test: soong_zip -C ./ -f short_test.sh -f log -o test.zip Change-Id: I9a843c6f8436ecfacc7948915e50f25c04497057
* Preserve executable bit in soong_zipDan Willemsen2017-03-151-2/+10
| | | | | | Bug: 36292596 Test: m -j vts; zipinfo out/host/linux-x86/vts/android-vts.zip Change-Id: If595641544cc387019d93b99e0a7890e0b7a9453
* Fix `dist` if $DIST_DIR/logs doesn't existDan Willemsen2017-03-011-3/+4
| | | | | | Bug: 35886901 Test: rm -rf out; m -j dist Change-Id: I1008d9eef884d60766e35a543e72df5df01d61de
* Updates for the new ckati dropDan Willemsen2017-02-241-2/+1
| | | | | | | | | | | | | We can start removing out directories again in multiproduct_kati, since the opendir bug has been fixed. Add --color_warnings to the Kati command line. Since this is different from Make, take this opportunity to reorder the command line to make more sense. This wasn't done before because kati forces a regen whenever the command line changes. Test: USE_SOONG_UI=true m -j blueprint_tools Change-Id: I5ad03359fbc16db482722946202297c1ae0f2b90
* Don't use runtime.Version() to find the current go versionDan Willemsen2017-02-221-1/+15
| | | | | | | | | That will be the go version at compile time. So read $GOROOT/VERSION, or fall back to executing `$GOROOT/bin/go version` to find the go version currently in GOROOT. Test: Ensure everything rebuilds when switching between go1.8rc2 and go1.8 Change-Id: I8738a7aa249a088b1e0668af260fa3974844dab7
* increase javac_filter line length limit to 2MBColin Cross2017-02-191-0/+4
| | | | | | | | | Some javac wrappers output the entire list of java files being compiled on a single line, which can be very large, set the maximum buffer size to 2MB. Test: pipe the grok build output through soong_javac_wrapper Change-Id: Ib208461dca02ced4959433b7695ec760dc134468
* Add soong_javac_filter toolColin Cross2017-02-093-0/+202
| | | | | | | | | | | | | | | | The soong_javac_filter tool will take the output of the javac tool on stdin and produce a colorized version similar to what clang produces on stdout. It also strips the useless "warning there are warnings" messages: Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. Modules that want the warnings can pass the appropriate -Xlint options. Test: javac_filter_test Change-Id: Ic8d337d95ae6c48146771f2982fd1316cb7d82be
* Ship the build log and trace on the build serversDan Willemsen2017-02-072-4/+11
| | | | | Test: m -j; m -j dist Change-Id: I4718e7a0cda6719bbab4243038ea2666c212cf19
* Add build tracingDan Willemsen2017-02-064-11/+32
| | | | | | | | | | | This creates a rotating build.trace.gz in the out directory that can be loaded with chrome://tracing. It'll include start and end timings for make/soong/kati/ninja, and it will import and time-correct the ninja log files. Test: m -j; load out/build.trace.gz in chrome://tracing Test: multiproduct_kati -keep; load out/multiproduct*/build.trace.gz Change-Id: Ic060fa9515eb88d95dbe16712479dae9dffcf626
* Add multiproduct_katiDan Willemsen2017-02-063-8/+218
| | | | | | | | | | | | | | | This is a replacement for build/tools/kati_all_products.sh using the new Soong ui/build package. It doesn't even attempt to run ninja, and it can be configured to run only the product config, or only the product config and Soong. For AOSP on my machine: -only-config 1.4s -only-soong 1m20s <none> 13m Test: multiproduct_kati Change-Id: Ie3e6e7bdf692e46a8b8eb828f437190f8003500b
* Add microfactory to bootstrap a go program with minimal overheadDan Willemsen2017-02-063-0/+971
| | | | | | | | | | | | | | | | | | | | | | | | | microfactory is a tool to incrementally compile a go program. It's similar to `go install`, but doesn't require a GOPATH. A package->path mapping can be specified as command line options. All input files are hashed, and if any change, the necessary packages will be rebuilt. microfactory can (re)build itself as necessary, so combined with a shell script that runs `go run microfactory.go` the first time, it can bootstrap a go program entirely from sources with just a working goroot. Time to build soong_ui only using source & GOROOT: first time no-change incremental microfactory 1400ms 15ms go install 670ms 130ms While microfactory takes longer the first time, almost half of that time is from `go run` and building microfactory for use later. If microfactory only has to build soong_ui, it's about 580ms. Test: USE_SOONG_UI=true m -j blueprint_tools Test: go test -bench . build/soong/cmd/microfactory/microfactory_test.go Change-Id: I4d2b9825788144fa10042bbd804482e44f459a54
* Add a Go replacement for our top-level Make wrapperDan Willemsen2017-02-062-0/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Right now this mostly just copies what Make is doing in build/core/ninja.mk and build/core/soong.mk. The only major feature it adds is a rotating log file with some verbose logging. There is one major functional difference -- you cannot override random Make variables during the Make phase anymore. The environment variable is set, and if Make uses ?= or the equivalent, it can still use those variables. We already made this change for Kati, which also loads all of the same code and actually does the build, so it has been half-removed for a while. The only "UI" this implements is what I'll call "Make Emulation" mode -- it's expected that current command lines will continue working, and we'll explore alternate user interfaces later. We're still using Make as a wrapper, but all it does is call into this single Go program, it won't even load the product configuration. Once this is default, we can start moving individual users over to using this directly (still in Make emulation mode), skipping the Make wrapper. Ideas for the future: * Generating trace files showing time spent in Make/Kati/Soong/Ninja (also importing ninja traces into the same stream). I had this working in a previous version of this patch, but removed it to keep the size down and focus on the current features. * More intelligent SIGALRM handling, once we fully remove the Make wrapper (which hides the SIGALRM) * Reading the experimental binary output stream from Ninja, so that we can always save the verbose log even if we're not printing it out to the console Test: USE_SOONG_UI=true m -j blueprint_tools Change-Id: I884327b9a8ae24499eb6c56f6e1ad26df1cfa4e4
* Move globbing to BlueprintColin Cross2016-11-042-100/+0
| | | | | | | | | | | | Move Soong's globbing-with-dependencies support into Blueprint so it can be used for subdirs= lines in Android.bp files. Blueprint has a slight change in behavior around subname= lines, it now always uses the subname and doesn't fall back to Blueprints. To support the Blueprints files in build/blueprint, use them directly with build=. Test: build, add source file that matches glob, rebuild Change-Id: Ifd0b0d3bc061aae0a16d6c7ca9a1cd8672656b4d
* Move registration into android packageandroid-n-mr1-preview-2android-n-mr1-preview-1Colin Cross2016-10-121-3/+1
| | | | | | | | Mutator registration is tightly coupled with the android package, move all registration from the soong package to the android package. Test: build.ninja identical Change-Id: Ie183d0b52cc7431c9e05b231934d189208ef1efe
* Move Android.bp definitions into subdirsDan Willemsen2016-08-255-0/+120
| | | | | Test: m blueprint_tools still produces the same tools Change-Id: Ia1e2e43c62cb6035616bef9fbef56417b46cf3a4
* Rename soong_jar to soong_zipDan Willemsen2016-08-112-5/+5
| | | | | | | This is a general purpose tool that happens to contain some jar specific features. Change-Id: I05f4654d4517c245ad7a3c15492e0d2368bbf64f
* soong_jar: Parallel compressionDan Willemsen2016-08-112-37/+522
| | | | | | | | | | | | | | | | | This compresses multiple files in parallel, and will split up larger files (5MB+) into smaller chunks (1MB) to compress in parallel. There is a small size overhead to recombine the chunks, but it's only a few bytes per chunk, so for a 1MB chunk, it's minimal. Rough numbers, with everything in the page cache, this can compress ~4GB (1000 files) down to 1GB in 6.5 seconds, instead of 120 seconds with the non-parallel soong_jar and 150 seconds with zip. Go's DEFLATE algorithm is still a bit worse than zip's -- about 3.5% larger file sizes, but for most of our "dist" targets that is fine. Change-Id: Ie4886c7d0f954ace46e599156e35fea7e74d6dd7
* Add zip2zip tool to copy zip entries from one file to anotherDan Willemsen2016-08-101-0/+128
| | | | | | | | | | | | | | | | This doesn't do any decompression / recompression, but just copies over the already compressed contents. So it's similar to zip -U, but allows rewriting of the paths. The first expected usecase is to replace img_from_target_files during the build, since it does the equivalent of this: zip2zip -i <target-files.zip> -o <img.zip> OTA/android-info.txt:android-info.txt IMAGES/*:. Except it decompresses and recompresses the images, which takes over a minute instead of a few seconds. Change-Id: I88d0df188635088783223873f78e193272dbdf1c
* soong_jar: Hardcode timeDan Willemsen2016-08-051-1/+1
| | | | | | We want reproducible builds, so timestamps are not useful. Change-Id: I5144bc62ac37c3c9d7e0638f3b3648a60d887702
* soong_jar: Implement symlinks, fix directoriesDan Willemsen2016-08-051-5/+45
| | | | | | | | | | | | | | | Ensure that we only create one directory entry, not one with a / at the end and another without. Clean incoming paths to prevent other similar duplicates. When adding directories specified directly as inputs, use the relative path just like files, not the full path. Instead of traversing symlinks, addg them to the zip file directly. Present an error instead of crashing when an input does not exist. Change-Id: Id6d6561fe12c28398f6096bf9bcae602bb7aa491
* Fix govet issueColin Cross2016-05-301-1/+1
| | | | Change-Id: I67705c5c1240bea6a21b7d62fd26d296d49735d4
* Rename common to androidColin Cross2016-05-181-2/+2
| | | | | | | | | | | | | | | | Rename the "common" package to "android", because common is too generic. Also removes all android.Android naming stutter. Ran: gomvpkg -from 'android/soong/common' -to 'android/soong/android' gorename -from '"android/soong/android".AndroidModuleContext' -to 'ModuleContext' gorename -from '"android/soong/android".AndroidBaseContext' -to 'BaseContext' gorename -from '"android/soong/android".AndroidModuleBase' -to 'ModuleBase' gorename -from '"android/soong/android".AndroidBottomUpMutatorContext' -to 'BottomUpMutatorContext' gorename -from '"android/soong/android".AndroidTopDownMutatorContext' -to 'TopDownMutatorContext' gorename -from '"android/soong/android".AndroidModule' -to 'Module' Change-Id: I3b23590b8ce7c8a1ea1139411d84a53163288da7
* Delay dependency errors to ninja time for unbundled buildsColin Cross2015-12-181-0/+2
| | | | | | | | | Unbundled builds may use a subset of the tree, which can bring in unused modules but not their dependencies. Delay handling of dependency errors for unbundled builds to ninja time, which will prevent errors if only modules with satisified dependencies are built. Change-Id: Ib93bae93fcfa0b55df500a30d8e35231ffb0987c
* Use SRCDIR as a working directoryDan Willemsen2015-09-171-2/+2
| | | | | | | | | | | The existing behavior of using the build directory as the working directory is useful if you want to move/copy the output directory around and SRCDIR still refers the the source. But, it's more useful to have the source directory be the working directory. Tools like cpp(__FILE__) and other debug prints embed relative paths from the working directory. We also have tools that expect the working directory to be $TOP. Change-Id: Ia0f1d3c6b7df72d61cf5628efa2baa98bd19775b
* Use the blueprint plugin infrastructureDan Willemsen2015-09-151-5/+0
| | | | | | | | Instead of using this hardcoded list, use the blueprint plugin infrastructure to find them at build time and load them. Bug: 23567252 Change-Id: Iae87e8c9bf020f4beb42c418b9a6bf8b0fe8b601
* Read product variables from soong.variablesColin Cross2015-09-141-1/+1
| | | | | | | Refactor the soong.config loading code to support reading in product variables from soong.variables. Change-Id: I389e6bb5c501b53167267d5f5d0d25557811cf72
generated by cgit v1.2.3 (git 2.25.1) at 2024-07-04 23:01:10 +0000