| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |\
| |
| |
| |
| |
| |
| |
| | |
4124b98e2f
am: cfcb823d2f
Change-Id: I4f773ef61816d1b79950264cb88fb9be1454e5cd
|
| | |\
| | |
| | |
| | |
| | |
| | | |
am: 4124b98e2f
Change-Id: I2e4ef9e03c99b335629063655f231c0ce18a53f5
|
| | |/|
| | |
| | |
| | |
| | |
| | | |
am: 341a2ceccb
Change-Id: I4aae5b2b678f9a9a35dc2e91bb53664fa3238c7b
|
| | | |\
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
* aosp/upstream:
Randomize cpu affinity
Be more resilient to directories disappearing during startup
Pass a Loc into FindEmulator for better warnings
Add --color_warnings to make warnings/errors like clang
[AUTHORS] Fix kati instead of glog
Add Koichi Shiraishi to AUTHORS/CONTRIBUTORS
[C++] add +build ignore magic comment for go build
[go] fix ldflags foramt to add '=' for -X flag
[go] fix stat.Dev type to uint64
Allow rules to specify custom ninja pools
Fix some possible performance issues found by clang-tidy
Test: kati tests
Test: multiproduct_build on aosp
Change-Id: I4c427dde2bb9d1a0970e97d1e822e093bd269dfe
|
| | | | |\
| | | | |
| | | | | |
Randomize cpu affinity
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Continue to lock kati to one or two CPUs, but pick one CPU at random,
and pick another CPU next to it. This dramatically speeds up cases where
more than one Kati instance is running at a time.
There's a multiproduct_build tool in Android that attempts to run Kati
on every build configuration present in the tree. My machine has 48
(including hyperthreading) cpu cores, with 64GB of ram. Before this
change, it would take ~12 minutes to build all 59 configurations, 12 at
a time. After this change it takes 3 minutes.
There doesn't seem to be any significant change in timings for
standalone builds. It looks like there was 1-2% difference with my
previous change that chose two completely random CPUs, but choosing two
that are likely hyperthreaded, or at least on the same chip helps.
|
| | | | |\ \
| | | | | |
| | | | | | |
Be more resilient to directories disappearing during startup
|
| | | | | |/
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
In multiproduct_build, we run a lot of kati instances in the same source
tree, but pointing to different out folders. I'm removing the out
folders of successful runs so that too much disk space isn't used
(unless the user requests it). But I often see a kati PERROR about
opendir failing, since it's trying to set up the find emulator while a
different configuration's output directory is being removed.
This was also hit by #109 in similar circumstances.
|
| | | | |\ \
| | | | |/
| | | |/| |
Improve and color warnings (with --color_warnings)
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Before this change, we'd only get a warning from FindEmulator, with no
idea which makefile caused it:
FindEmulator: find: `tests': No such file or directory
With this change, we'll get a better idea of which line triggered that
problem:
cts/tests/tests/content/Android.mk:43: FindEmulator: find: `test': No such file or directory
And it will be colorized like any other location-based warning with the
previous patch if --color_warnings is turned on.
|
| | | | |/
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This adds new (WARN|KATI_WARN|ERROR)_LOC log macro variants that take a
location as the first argument, and will prefix that location
information to the warning/error lines.
When --color_warnings is enabled, it reformats them to have a standard
warning:/error: infix, and adds colors in order to match the
warnings/errors produced by clang.
|
| | | | |\
| | | | |
| | | | | |
nit: [AUTHORS] Fix kati instead of glog
|
| | | | |/
| | | |
| | | |
| | | | |
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
|
| | | | |\
| | | | |
| | | | | |
[go] Fix can't the build kati binary
|
| | | | | |
| | | | |
| | | | |
| | | | | |
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
|
| | | | | |
| | | | |
| | | | |
| | | | | |
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
- go has been changed ldflags parse format. Must be -X key=value.
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
|
| | | | |/
| | | |
| | | |
| | | | |
Signed-off-by: Koichi Shiraishi <zchee.io@gmail.com>
|
| | | | |\
| | | | |
| | | | | |
Allow rules to specify custom ninja pools
|
| | | | |/
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Setting .KATI_NINJA_POOL as a rule variable will set the corresponding
pool variable in the ninja file.
There's no way to define custom pools in Kati, Android is planning on
scaling the pool depth in a parent ninja file without re-running Kati.
|
| | | | |\
| | | | |
| | | | | |
Fix some possible performance issues found by clang-tidy
|
| | | | |/
| | | |
| | | |
| | | |
| | | | |
No obvious time differences when building AOSP, but these all seem like
reasonable changes.
|
| |\| | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
dd026a6d01
am: 2ae2ac713b
Change-Id: I900ea28a1aeb4b93803472bfc3f1222228d74f7f
|
| | |\| |
| | | |
| | | |
| | | |
| | | |
| | | | |
am: dd026a6d01
Change-Id: Ib2a64949e1f735b3c707cbd5781718b34d1e3384
|
| | | |\ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
am: 7e2a7c776b
Change-Id: I4f70289cdfb25d8d42ce0424c0473df8eea5260a
|
| | | | |\|
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
* aosp/upstream:
Update Android.bp to shrink executables
Test: mma -j
Change-Id: I0b4b8daabee1d91ffdfef1945fb08b47d53a404e
|
| | | | | |\
| | | | | |
| | | | | | |
Update Android.bp to shrink executables
|
| | | | | |/
| | | | |
| | | | | |
Keep whole_static_libs for the main executable, but use static_libs for the rest. My goal is to check ckati_stamp_dump into Android as a prebuilt, and it goes from 2.4M->1.2M for the ASAN version, and 352K->11K for the normal version.
|
| |\| | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
fa51372f9f
am: 7dda54cd2b
Change-Id: I08964e5ec9cee50898d496e860ff0ef81ca00da4
|
| | |\| | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
am: fa51372f9f
Change-Id: Icad68eb2c13880711c1f8f7396b79ede6dad1262
|
| | |/| | |
| | |/ /
| | | |
| | | |
| | | |
| | | | |
am: c4cebd6972
Change-Id: Ie788813f5dcb1196f08db80df90b44efbb73e039
|
| | | |\|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
* aosp/upstream:
Fix typo in regen_dump.cc
Optimize findleaves regen check
Remove "out" special case handling from FindEmulator
Add a tool to export a kati stamp file to other tools
Fix typo in comment.
Turn CHECK failure that depends on file system state into an error message and include what failed in the error.
Test: Upstream tests pass
Change-Id: I00479b65666d3b5fb6dd923f74d39eeea8462d08
|
| | | | |\
| | | | |
| | | | | |
Fix typo in regen_dump.cc
|
| | | | |/ |
|
| | | | |\
| | | | |
| | | | | |
Fix a few findleaves issues
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
If we've found all possible files in a leaf directory, we don't need to
re-run findleaves every time the leaf directory's timestamp changes, we
just need to make sure the file(s) that we found still exist. This
saves a few seconds every time an atomic write is done in a source
directory next to an Android.mk file. (Atomic writes use renames, so
they always change the directory's timestamp)
With the last commit that finds out/Android.mk and out/CleanSpec.mk, it
turns out that the output directory's timestamp was changing every
build, causing the global Android.mk & CleanSpec.mk findleaves.py
command to be executed every regen check.
TEST_FIND_EMULATOR still passes after this change.
|
| | | | |/
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
We removed the special casing of $OUT_DIR from our findleaves commands
to fix problems where $OUT_DIR was named the same as a real directory.
But now TEST_FIND_EMULATOR (and regen) is broken, since the real command
finds out/Android.mk, but the emulated version does not. Since we're
already traversing output directories that aren't called "out", just
remove the "out" special casing from the find emulator.
This raises the time to init the find emulator from 0.85s to 1.12s on my
machine with one product built. But this only happens when you're about
to read all of the makefiles anyways, not during regen. The node count
goes from 683196 to 894396.
|
| | | | |\
| | | | |
| | | | | |
Add a tool to export a kati stamp file to other tools
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
There's a desire to understand which files are used by a build so that
the automated builds can make better guesses at which builds should be
run for a certain change. Instead of having them read a stamp file
directly, which would prevent us from updating the stamp format in the
future, build a tool that can be used to export the contents in a stable
and more portable format.
Right now, this just means exporting the file list to stdout with
newlines as delimiters. An idea for the future is to define a protobuf
or similar format that would contain the glob and shell information as
well (if it's useful).
|
| | | | |\ \
| | | | |/
| | | |/| |
Include details in error message
|
| | | | | | |
|
| | | | | |
| | | | |
| | | | |
| | | | | |
and include what failed in the error.
|
| |\| | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
f80f337f16 am: 1f45e0859e
am: 2f288fecb4
Change-Id: Ia09e84d1573234ccfb1799b237f7bfb62c306e70
|
| | |\ \ \ \
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
f80f337f16
am: 1f45e0859e
Change-Id: I23815513d619124aca677d36edf16d2e22730297
|
| | | |\ \ \ \
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
am: f80f337f16
Change-Id: Iebfa9275bcc902841b2414ecc88b56b2bb959e19
|
| | | | |\| | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
am: 1b68243cf2
Change-Id: I4d96ed0b16c5e92b0d5dde19f098560dc0c81b75
|
| | | | | |\| |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
* aosp/upstream:
Implement the `file` function to read and write files
Run regen commands sequentially
Optimize RunCommand by removing /bin/sh wrapper when possible
Add simple benchmark for RunCommand
Support marking variables as readonly
Bug: 30947985
Test: Passes upstream kati test suite, manual android testing
|
| | | | | | |\ \
| | | | | | | |
| | | | | | | | |
Run regen commands sequentially
|
| | | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
When I remove --ignore-dirty=$(OUT_DIR)/% from an Android build, it
turns out that we have $(shell) commands that have side effects and
cannot be reordered in relationship to one another. The trivial case is
a sequence of 'rm', 'write', 'write'..., 'cmp'. The 'rm' and 'write'
commands can be collapsed into a single $(file) function call, but it
still needs to be ordered for the 'cmp' function to work properly.
I've been making changes to prevent this from slowing down regen too
much, but it's still a 2-3x slowdown overall (0.3s -> 0.8s for
aosp_arm64-eng on aosp master).
|
| | | | | | |\ \ \
| | | | | | | | |
| | | | | | | | | |
Implement the `file` function to read and write files
|
