aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Add deprecated / obsolete variable supportDan Willemsen2017-10-0315-5/+222
| | | | | | | | | | | | | | | | | By calling the custom KATI_deprecated_var / KATI_obsolete_var functions, variables may be marked as deprecated or obsolete. When accessed or assigned, deprecated variables will print a warning. When accessed or assigned, obsolete variables will print an error and stop. Variables do not need to be set before calling the functions, and will persist the deprecation warning through a reassignment. This way we can easily mark variables that are sometimes passed via the environment as deprecated. Change-Id: Id04c974c446f471a18cc173f817760f4a02b9239
* Merge pull request #124 from danw/bionicDan Willemsen2017-09-293-5/+3
|\ | | | | Bionic support and build system updates
| * Fix build and test using bionic as a libcDan Willemsen2017-09-292-0/+3
| | | | | | | | | | | | There's a flag to Soong in order to use bionic as a libc instead of glibc, removing dependencies on your host system. This fixes a single include what you use, and some error message differences.
| * Remove default libsDan Willemsen2017-09-291-5/+0
|/ | | | -lrt and -lpthread are automatically added to linux host builds now.
* Merge pull request #121 from danw/implicit_outputsShinichiro Hamaji2017-08-165-17/+114
|\ | | | | Include implicit outputs in the DepNode graph
| * Include implicit outputs in the DepNode graphDan Willemsen2017-08-155-17/+114
| | | | | | | | | | | | | | | | | | | | | | The last patch was good enough for simple use cases, but if a dependency was added to the implicit output, Kati would create a phony rule in addition to the implicit output. This patch will cause the rules to be combined into a single DepNode, combining any variables and dependencies. Change-Id: I92c9e5b6fdb8bfe49d0cef0a5fa22def4acd6d99
* | Merge pull request #122 from danw/asan_default_optionsShinichiro Hamaji2017-08-161-0/+5
|\ \ | |/ |/| Turn off LeakSanitizer by default
| * Turn off LeakSanitizer by defaultDan Willemsen2017-08-151-0/+5
|/ | | | | | | We know that Kati leaks some memory. Turn off the LeakSanitizer by default so that users don't need to. Change-Id: I1ad71ba28af64f5244c51c783cff2f7d6eb86b6f
* Merge pull request #120 from danw/implicit_outputsShinichiro Hamaji2017-08-115-3/+66
|\ | | | | Support Ninja implicit outputs
| * Travis: switch to trusty, ninja 1.7Dan Willemsen2017-08-101-2/+6
| | | | | | | | Change-Id: I9c3ef7fcc465768c3be81c4cbc255e5abc5bd9d8
| * Support Ninja implicit outputsDan Willemsen2017-08-094-1/+60
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add .KATI_IMPLICIT_OUTPUTS target-specific variable to add extra implicit outputs to the build line in the ninja file. This doesn't actually affect the node graph within Kati, but combined with --gen_all_targets should be usable. This allows us to properly define the build graph in ninja when one rule creates multiple files. There are a number of ways that people attempt to do this in make, all of which have problems: a b: touch a b Define multiple outputs with the same rule. This looks correct to most people, but actually runs the rule multiple times -- once for every output file. Most of the time this works, but runs into very strange concurrency issues when the two rules happen to run at the same time. a: touch a b Only define one of the files in the graph. Then users need to depend on a file that they don't use, and the unlisted file won't be noticed if it's updated separately (or even removed). b: a a: touch a b Define dependencies between the multiple files. This doesn't actually know how to update `b` if `b` is older than `a`, or doesn't exist. b: a touch b a: touch a b A variant of the previous approach, which can "fix" the timestamp if the original rule happened to create `b` before `a`. This can be even more dangerous, since it may create / update `b` improperly. But since ninja supports a graph where there can be multiple outputs from a single edge, with this patch we can write: a: .KATI_IMPLICIT_OUTPUTS := b a: touch a b which translates to: rule rule1 command = touch a b build a | b: rule1 This ninja will allow users to depend on either `a` or `b`, and the rule will be run (once) as appropriate. Change-Id: Ieb9ced7eb4d212b0aa4de5ceb1deb9ba48ada132
* Merge pull request #119 from danw/findShinichiro Hamaji2017-07-263-12/+113
|\ | | | | Fix a FindEmulator bug
| * Fix (cd <symlink>; find .)Dan Willemsen2017-07-262-10/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | These two find commands return different results: find <symlink-to-dir> find <symlink-to-dir>/. So normalizing the path by calling ConcatDir(<chdir>, <finddir>) doesn't produce the correct result. Instead, save the result from the first FindDir and use it again without normalization. Since this removes the path normalization, we need to handle the "./<child>" case in DirentDirNode::FindDir.
| * Add some simple find unit testsDan Willemsen2017-07-262-2/+93
|/ | | | | | | | | | | | | Example failing output: Failed to match `find top/C/.`: Native: Emulated: top/C/. top/C/. top/C/./D top/C/./b top/C/./B top/C/./B/z ------------------------------------------
* Merge pull request #117 from danw/WerrorShinichiro Hamaji2017-05-236-16/+124
|\ | | | | Add --werror_find_emulator, --werror_overriding_commands
| * Add --werror_find_emulator, --werror_overriding_commandsDan Willemsen2017-05-226-16/+124
|/ | | | | | | | | | | | | | | | | | | For Android builds, we'd like to start removing some of the default warnings and turn them into errors so that they can't come back. For find emulator, we could attempt to check for errors, or silence every find command in the tree, but that doesn't particularly scale, especially when new code gets added with warnings. We've gone through and fixed many of these, but they keep coming back, so add --werror_find_emulator so that when we fix them all we can prevent them from coming back. Overriding commands is similar -- we really don't want multiple rules defining a single output file. In ninja we've turned on -w dupbuild=err, but if the paths happen to be identical the makefile overriding logic kicks in first and presents a warning instead of an error. So add --werror_overriding_commands in order to turn the make warning into an error.
* Merge pull request #116 from danw/find_eaccesShinichiro Hamaji2017-05-131-1/+1
|\ | | | | Ignore EACCES during find emulator opendir
| * Ignore EACCES during find emulator opendirDan Willemsen2017-05-121-1/+1
|/ | | | | | | | | | | In addition to the race conditions that led me to whitelist ENOENT when we're initializing the find emulator, also whitelist EACCES. The reported usecase was when two users are using the same source directory, but compiling into two out directories under the same source directory. The permissions were set up so that they didn't have access to each others out directories, so kati would get permission denied errors. Test: mkdir -p out2/a; sudo chown nobody:nobody out2/a; <run>
* Merge pull request #112 from danw/random_affinityDan Willemsen2017-02-231-4/+18
|\ | | | | Randomize cpu affinity
| * Randomize cpu affinityDan Willemsen2017-02-231-4/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Continue to lock kati to one or two CPUs, but pick one CPU at random, and pick another CPU next to it. This dramatically speeds up cases where more than one Kati instance is running at a time. There's a multiproduct_build tool in Android that attempts to run Kati on every build configuration present in the tree. My machine has 48 (including hyperthreading) cpu cores, with 64GB of ram. Before this change, it would take ~12 minutes to build all 59 configurations, 12 at a time. After this change it takes 3 minutes. There doesn't seem to be any significant change in timings for standalone builds. It looks like there was 1-2% difference with my previous change that chose two completely random CPUs, but choosing two that are likely hyperthreaded, or at least on the same chip helps.
* | Merge pull request #113 from danw/resilient_findShinichiro Hamaji2017-02-241-3/+20
|\ \ | | | | | | Be more resilient to directories disappearing during startup
| * | Be more resilient to directories disappearing during startupDan Willemsen2017-02-231-3/+20
| |/ | | | | | | | | | | | | | | | | | | | | In multiproduct_build, we run a lot of kati instances in the same source tree, but pointing to different out folders. I'm removing the out folders of successful runs so that too much disk space isn't used (unless the user requests it). But I often see a kati PERROR about opendir failing, since it's trying to set up the find emulator while a different configuration's output directory is being removed. This was also hit by #109 in similar circumstances.
* | Merge pull request #111 from danw/color_warningsDan Willemsen2017-02-2319-80/+166
|\ \ | |/ |/| Improve and color warnings (with --color_warnings)
| * Pass a Loc into FindEmulator for better warningsDan Willemsen2017-02-224-27/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Before this change, we'd only get a warning from FindEmulator, with no idea which makefile caused it: FindEmulator: find: `tests': No such file or directory With this change, we'll get a better idea of which line triggered that problem: cts/tests/tests/content/Android.mk:43: FindEmulator: find: `test': No such file or directory And it will be colorized like any other location-based warning with the previous patch if --color_warnings is turned on.
| * Add --color_warnings to make warnings/errors like clangDan Willemsen2017-02-2216-53/+139
|/ | | | | | | | | | This adds new (WARN|KATI_WARN|ERROR)_LOC log macro variants that take a location as the first argument, and will prefix that location information to the warning/error lines. When --color_warnings is enabled, it reformats them to have a standard warning:/error: infix, and adds colors in order to match the warnings/errors produced by clang.
* Merge pull request #108 from zchee/authors-glogShinichiro Hamaji2016-12-021-1/+1
|\ | | | | nit: [AUTHORS] Fix kati instead of glog
| * [AUTHORS] Fix kati instead of glogKoichi Shiraishi2016-12-021-1/+1
|/ | | | Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
* Merge pull request #106 from zchee/go-gettableShinichiro Hamaji2016-11-299-4/+16
|\ | | | | [go] Fix can't the build kati binary
| * Add Koichi Shiraishi to AUTHORS/CONTRIBUTORSKoichi Shiraishi2016-11-292-0/+2
| | | | | | | | Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
| * [C++] add +build ignore magic comment for go buildKoichi Shiraishi2016-11-295-0/+10
| | | | | | | | Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
| * [go] fix ldflags foramt to add '=' for -X flagKoichi Shiraishi2016-11-291-1/+1
| | | | | | | | | | | | - go has been changed ldflags parse format. Must be -X key=value. Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
| * [go] fix stat.Dev type to uint64Koichi Shiraishi2016-11-291-3/+3
|/ | | | Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
* Merge pull request #105 from danw/kati_poolShinichiro Hamaji2016-11-064-2/+56
|\ | | | | Allow rules to specify custom ninja pools
| * Allow rules to specify custom ninja poolsDan Willemsen2016-11-054-2/+56
|/ | | | | | | | Setting .KATI_NINJA_POOL as a rule variable will set the corresponding pool variable in the ninja file. There's no way to define custom pools in Kati, Android is planning on scaling the pool depth in a parent ninja file without re-running Kati.
* Merge pull request #104 from danw/clang-tidyShinichiro Hamaji2016-11-026-5/+10
|\ | | | | Fix some possible performance issues found by clang-tidy
| * Fix some possible performance issues found by clang-tidyDan Willemsen2016-11-016-5/+10
|/ | | | | No obvious time differences when building AOSP, but these all seem like reasonable changes.
* Merge pull request #103 from google/danw-androidbp-updateShinichiro Hamaji2016-10-211-3/+3
|\ | | | | Update Android.bp to shrink executables
| * Update Android.bp to shrink executablesDan Willemsen2016-10-201-3/+3
|/ | | Keep whole_static_libs for the main executable, but use static_libs for the rest. My goal is to check ckati_stamp_dump into Android as a prebuilt, and it goes from 2.4M->1.2M for the ASAN version, and 352K->11K for the normal version.
* Merge pull request #102 from danw/typoDan Willemsen2016-10-201-1/+1
|\ | | | | Fix typo in regen_dump.cc
| * Fix typo in regen_dump.ccDan Willemsen2016-10-201-1/+1
|/
* Merge pull request #101 from danw/findleavesShinichiro Hamaji2016-10-214-24/+92
|\ | | | | Fix a few findleaves issues
| * Optimize findleaves regen checkDan Willemsen2016-10-194-20/+90
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we've found all possible files in a leaf directory, we don't need to re-run findleaves every time the leaf directory's timestamp changes, we just need to make sure the file(s) that we found still exist. This saves a few seconds every time an atomic write is done in a source directory next to an Android.mk file. (Atomic writes use renames, so they always change the directory's timestamp) With the last commit that finds out/Android.mk and out/CleanSpec.mk, it turns out that the output directory's timestamp was changing every build, causing the global Android.mk & CleanSpec.mk findleaves.py command to be executed every regen check. TEST_FIND_EMULATOR still passes after this change.
| * Remove "out" special case handling from FindEmulatorDan Willemsen2016-10-191-4/+2
|/ | | | | | | | | | | | | | We removed the special casing of $OUT_DIR from our findleaves commands to fix problems where $OUT_DIR was named the same as a real directory. But now TEST_FIND_EMULATOR (and regen) is broken, since the real command finds out/Android.mk, but the emulated version does not. Since we're already traversing output directories that aren't called "out", just remove the "out" special casing from the find emulator. This raises the time to init the find emulator from 0.85s to 1.12s on my machine with one product built. But this only happens when you're about to read all of the makefiles anyways, not during regen. The node count goes from 683196 to 894396.
* Merge pull request #100 from danw/stamp_dump_toolShinichiro Hamaji2016-10-182-20/+84
|\ | | | | Add a tool to export a kati stamp file to other tools
| * Add a tool to export a kati stamp file to other toolsDan Willemsen2016-10-172-20/+84
| | | | | | | | | | | | | | | | | | | | | | | | | | | | There's a desire to understand which files are used by a build so that the automated builds can make better guesses at which builds should be run for a certain change. Instead of having them read a stamp file directly, which would prevent us from updating the stamp format in the future, build a tool that can be used to export the contents in a stable and more portable format. Right now, this just means exporting the file list to stdout with newlines as delimiters. An idea for the future is to define a protobuf or similar format that would contain the glob and shell information as well (if it's useful).
* | Merge pull request #87 from nlewycky/masterDan Willemsen2016-10-171-2/+4
|\ \ | |/ |/| Include details in error message
| * Fix typo in comment.Nick Lewycky2016-06-221-1/+1
| |
| * Turn CHECK failure that depends on file system state into an error messageNick Lewycky2016-06-221-1/+3
| | | | | | | | and include what failed in the error.
* | Merge pull request #98 from danw/regen_side_effectsDan Willemsen2016-10-041-4/+4
|\ \ | | | | | | Run regen commands sequentially
| * | Run regen commands sequentiallyDan Willemsen2016-10-031-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When I remove --ignore-dirty=$(OUT_DIR)/% from an Android build, it turns out that we have $(shell) commands that have side effects and cannot be reordered in relationship to one another. The trivial case is a sequence of 'rm', 'write', 'write'..., 'cmp'. The 'rm' and 'write' commands can be collapsed into a single $(file) function call, but it still needs to be ordered for the 'cmp' function to work properly. I've been making changes to prevent this from slowing down regen too much, but it's still a 2-3x slowdown overall (0.3s -> 0.8s for aosp_arm64-eng on aosp master).
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-28 15:25:39 +0000