Commit Graph

378 Commits

Author SHA1 Message Date
Colin Cross d0be2109cf Add CUJ tests
Add a cuj_tests binary that runs through a sequence of critical
user journey builds and collects metrics.

Bug: 145688720
Test: cuj_tests
Change-Id: I6bb1a3fc4a458fd8cea7b7c924f565d0cf3e6df3
2019-12-05 11:11:37 -08:00
Treehugger Robot dc0e8fd131 Merge changes Id261b5a3,Ice6f34d0
* changes:
  Fix errorProtoLog error messages
  Write BUILD_DATETIME_FILE after SetupOutDir
2019-12-05 09:10:13 +00:00
Colin Cross ff27ce4cc0 Fix errorProtoLog error messages
Use Printf instead of Println for formatted strings, and pass
the filename instead of the proto.

Test: cuj_tests
Change-Id: Id261b5a34304a0caa61faa1f3bbc388aacdd25a6
2019-11-26 16:21:58 -08:00
Colin Cross 28f527c3da Write BUILD_DATETIME_FILE after SetupOutDir
Delay writing the BUILD_DATETIME_FILE until after the out directory
has been created.

Test: cuj_tests
Change-Id: Ice6f34d003f93c26b5d2d0b64f92b11efe16c2d4
2019-11-26 16:21:58 -08:00
Dan Shi 53f1a190ff Clean vts-core in host/out to ensure old package to be removed.
Bug: None
Test: m installclean
Change-Id: I5aa451d87dc377e62dcf1b88baa58539fa661feb
2019-11-26 10:02:53 -08:00
Dan Willemsen 1c504d9c7e Remove BASH_ENV from the build environment
This runs an arbitrary shell script for all non-interactive bash sessions, and we've got lots of those.

It was causing problems with Fedora 30, which triggers the `manpath` executable, which isn't available in our $PATH:
https://groups.google.com/d/msg/android-building/WqEyZQuxYtY/zDXl9irOCAAJ

Change-Id: If2f63bbbcc3accb8d239f93fa06823d301d69488
2019-11-18 19:13:56 +00:00
Elliott Hughes 7e4452511c Merge "Switch to our own zipinfo(1)." 2019-11-16 00:31:45 +00:00
Elliott Hughes 3efacee1fd Switch to our own zipinfo(1).
Test: treehugger
Change-Id: If71e87e45a52959be9a316810fe79250771eb9d4
2019-11-14 20:09:24 -08:00
Colin Cross 9016b91290 Set default pool when RBE or goma is enabled
Pass --default_pool=local_pool to kati when RBE or goma is enabled
to put most rules into the local_pool.  Specific rules that support
RBE or goma will set .KATI_NINJA_POOL := none to remove themselves
from the local_pool.  Passing --default_pool will also disable the
hack in kati that sets the pool based on the presence of the string
"/gomacc" in the command line.

Fixes: 143938974
Test: inspect pools in build-${TARGET-PRODUCT}.ninja for m USE_RBE=true
Test: inspect pools in build-${TARGET-PRODUCT}.ninja for m USE_GOMA=true
Change-Id: I839b2488383fcd63fffd613e25b0b9abcb72b567
2019-11-11 16:08:42 -08:00
Pete Gillin a7a3d64a36 Remove the EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN option.
Previously, this setting environment variable to false reverted to an
OpenJDK 9 toolchain. This change removes that option.

Test: m core-all-system-modules
Test: zcat out/verbose.log.gz | grep 'prebuilts/jdk/jdk[0-9]*'
Test: EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=false m core-all-system-modules (fails)
Bug: 131683177
Change-Id: I0d679648e1236925ce3ef7e1652379127e846b00
2019-11-11 11:03:58 +00:00
Pete Gillin abbcddaaf4 Switch to an OpenJDK 11 toolchain by default.
Prior to this change, the default was to use and OpenJDK 9 toolchain,
with the EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=true environment
variable available to opt into OpenJDK 11. After this change, the
default is to use OpenJDK 11, with
EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=false available to opt out.

This change affects:
 - The version of javac used.
 - The version of javadoc used.
 - The version of other tools used during the build process (e.g. jar,
   jmod, and jlink).
 - For Java bytecode executed on the host, the version of the java
   executable used, and so the versions of the JVM and the OpenJDK
   libraries in the system module (or bootclasspath).

This change does not affect:
 - The Java language level, i.e. the version of Java expected in
   source code (the -source option to javac) or the version of Java
   bytecode produced (the -target option to javac).
 - Anything to do with code execution on the device.

Bug: 131683177
Test: make java droid docs cts tests
Test: zcat out/verbose.log.gz | grep 'prebuilts/jdk/jdk[0-9]*'
Test: make RunBluetoothRoboTests RunCarSettingsLibRoboTests
Test: cts-tradefed help
Test: atest CtsHostTzDataTests
Test: atest CtsLibcoreTestCases
Change-Id: I09dc22f1af4d1f2d7d3b85c08cb0ed9a1105aaca
Merged-In: Iaae89ef25c92ec099575c026fc50e41da4e143e8
2019-11-05 11:33:58 +00:00
Colin Cross 8cc19911b1 smart_status: move channel write outside lock to avoid deadlock
The smartStatusOutput.done channel is read from a goroutine that
locks smartStatusOutput.lock.  If the goroutine that writes to
smartStatusOutput.done is holding smartStatusOutput.lock it can
lead to an AB-BA deadlock.  Call stopActionTableTick from outside
the lock.

Fixes: 143558785
Test: none
Change-Id: I93a10ef9ff16c3953a1c5ccb102b024158358fe4
2019-10-29 15:27:32 -07:00
Elliott Hughes e23c63de21 Merge "Switch to toybox find for darwin and linux." 2019-10-23 14:52:28 +00:00
Inseob Kim 822fdcab1f Add BUILD_BROKEN_TREBLE_SYSPROP_NEVERALLOW
A new sysprop neverallow rules are mandatory only for devices launching
with R or later. For devices already launched, neverallow rules can be
relaxed with adding following line to BoardConfig.mk:

BUILD_BROKEN_TREBLE_SYSPROP_NEVERALLOW := true

Bug: 131162102
Test: Set PRODUCT_SHIPPING_API_LEVEL := 30 and try building with
changing some system_public_prop to system_internal_prop
Test: m cts sepolicy_tests

Change-Id: I5e1640f7b43fd47863bc7bd163c2f43b01fa3326
Merged-In: I5e1640f7b43fd47863bc7bd163c2f43b01fa3326
(cherry picked from commit e36f52754f)
2019-10-15 09:22:42 +09:00
Elliott Hughes e04b73f100 Merge "Go back to the native mac xargs for now." 2019-10-10 21:18:59 +00:00
Elliott Hughes 27cc0a8f32 Go back to the native mac xargs for now.
Should unbreak the mac build while I work on toybox xargs some more.

Test: suck it and see (since there's no mac presubmit)
Change-Id: I48954f801fc980a1174f93e474d73a24ad12ce66
2019-10-09 18:12:33 -07:00
Treehugger Robot 9026090263 Merge "Add $PRODUCT_OUT/test_harness_ramdisk to installclean" 2019-10-09 23:49:46 +00:00
Pete Gillin 1f52e93db2 Add experimental option to use OpenJDK 11 toolchain.
With this change, setting the environment variable
EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN=true switches from the OpenJDK 9
toolchain to the OpenJDK 11 one (prebuilts/jdk/jdk11).

Since the version of the java.base module has to match the version of
the jlink tool which consumes it, the --module-version argument to the
jmod create call is also switched.

Test: make
Test: EXPERIMENTAL_USE_OPENJDK11_TOOLCHAIN make
Bug: 131683177
Change-Id: I606d22538165f309fe5537bd67a26390b1dd5771
2019-10-09 18:38:14 +01:00
Bowgo Tsai 5145c2c11f Add $PRODUCT_OUT/test_harness_ramdisk to installclean
Also removes $PRODUCT_OUT/debug_ramdisk.

Bug: 140036184
Test: `make installclean` removes those directories
Change-Id: I442753817bdda4293b98ac1a7fd0173820dc2a2f
2019-10-09 01:59:51 +00:00
Dan Willemsen b6699a1414 Restore previous behavior of `get_build_var TMPDIR`
My previous change makes TMPDIR a temporary directory that's removed
before we exit. So instead of starting ckati to get this value to a
directory that no longer exists, return the semi-stable value that we
use during a normal build.

Fixes: 142277335
Test: get_build_var TMPDIR
Change-Id: I37d1219b1fda09f131dee17ef7b91e475d3e39ad
2019-10-07 15:29:23 -07:00
Dan Willemsen 4e2456bc09 Fix dumpvars $PATH / $TMPDIR
Instead of just using the host $PATH during dumpvars, use our path
configuration from the regular build. But instead of creating a ton of
symlinks to the interposer, just use a small directory of symlinks. This
only takes ~3ms (vs ~300ms), at the expense of error logging. Since we
do just about the same product configuration at the start of the build,
we can just rely on the logging there.

This fixes warnings printed by the Mac build, since we using the host
`date` instead of the toybox version:

usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]

TMPDIR was being shared with the main build, but we're allowed to run in
parallel. If a build is started while we're running, our TMPDIR may be
cleared. Instead, create our own temporary directory, which doesn't need
to be a stable value (as we never do caching here).

Fixes: 141893400
Test: (on mac) source build/envsetup.sh; lunch
Change-Id: I0ff536e71dc5649cae26daa0087eb80861704021
2019-10-04 12:53:51 -07:00
Logan Chien 65ab468452 Merge "Add BUILD_BROKEN_PREBUILT_ELF_FILES to dumpvars.go" 2019-10-03 15:51:09 +00:00
Colin Cross 1f6faeb50f Use ctx.Fatalln instead of log.Fatalln for absolute path errors
log.Fatalln calls os.Exit, which skips deferred functions and
leaves the console with the cursor disabled.  Use ctx.Fatalln
instead, which uses a panic to exit and calls the deferred
function to restore the cursor.

Test: DIST_DIR=/tmp/\test\ foo m dist
Change-Id: Ie92297075e37c171d5ba48848a0ddb19652b051c
2019-09-23 15:55:54 -07:00
Colin Cross c0b9f6b9d2 Force dumpvars into dumb terminal mode and move log files
Force soong_ui in dumpvars mode to use a dumb terminal so that it
doesn't print an empty status table.  Also move the logs to files
that are prefixed with "dumpvars-" so that they don't rotate the
real build logs.

Fixes: 139307849
Test: lunch
Change-Id: If92421af07731fedc9ca42670f309e95922a2224
2019-09-23 14:24:57 -07:00
Colin Cross f0b987ecba status table: don't write newlines in non-scrolling terminals
Multiple terminals have had issues with writing newlines into the
non-scrolling region, just set the cursor to the beginning of the
next line instead.

Test: m nothing in JediTerm
Change-Id: I2e434f4cc263ca13b82889a79d6a8bb48d084cb3
2019-09-20 15:03:31 -07:00
Colin Cross bf8f57e2f6 status table: ANSI code rows and columns are 1-based
Don't pass 0 to setCursor or setScrollingMargins.  Passing 0 hasn't
caused any known issues, but 1 is correct.

Test: m nothing
Change-Id: I8735032eb3f1d46df4c41c4ea9ef95b5fd842824
2019-09-20 15:03:11 -07:00
Logan Chien f9cf9acd7a Add BUILD_BROKEN_PREBUILT_ELF_FILES to dumpvars.go
This commit adds BUILD_BROKEN_PREBUILT_ELF_FILES to
`ui/build/dumpvars.go` so that its value can be printed to
`out/soong.log`.

Bug: 141176116
Test: lunch aosp_crosshatch-userdebug && make && \
      grep BUILD_BROKEN_PREBUILT_ELF_FILES out/soong.log

Change-Id: I99600a5b540be2bb93647ee8175b9f9afd1a6eb3
2019-09-20 11:40:49 -07:00
Roland Levillain 9c27df7d4d Merge "Add `$ANDROID_HOST_OUT/apex` to the file set removed by target `installclean`." 2019-09-18 10:47:06 +00:00
Elliott Hughes 26bce340e3 Track switch to toybox date for macOS.
Bug: http://b/139450866
Test: builds
Change-Id: I5ab0d20d761bcd03f3bee18b8343a2f8aa5e2275
2019-09-12 15:05:13 -07:00
Roland Levillain e5f9ee56b5 Add `$ANDROID_HOST_OUT/apex` to the file set removed by target `installclean`.
Before this change, host APEX (`.zipapex`) packages would still be
there after `m installclean`.

Test: Run `m installclean` and check that host APEX packages are removed.
Change-Id: I446286ce700f12666be5aa90fbe3bbc648abd280
2019-09-11 14:50:08 +01:00
Elliott Hughes 34b49d1402 Use toybox md5sum instead of md5 on macOS.
Test: builds locally
Change-Id: I04aa52de96e0525a44331cb0c057a8e18a95891d
2019-09-06 14:42:24 -07:00
Elliott Hughes ef19e01f96 Switch to toybox stat on macOS too.
Bug: http://b/139450866
Test: builds
Change-Id: I97af607a311aea5f97544bd3ce1d532d999126ff
2019-09-05 10:01:37 -07:00
Elliott Hughes 5cb822b2cf Switch to toybox find for darwin and linux.
Test: treehugger
Change-Id: Id311b403f50a76e75ed92c2a9e7cc7aa025f2448
2019-08-27 21:47:05 -07:00
Elliott Hughes f1ff226cd4 Switch most of the darwin tools to toybox.
Bug: http://b/139450866
Test: treehugger
Test: "works for me on my machine" (last week)
Change-Id: I5dd00e6fda134b06967651007afb7565ce4f9e24
2019-08-27 15:17:32 -07:00
Treehugger Robot 2569cc5026 Merge "We have md5sum/sha1sum/sha256sum/sha512sum on Darwin now too." 2019-08-27 03:17:07 +00:00
Steven Moreland 0aabb11dd4 Add $ANDROID_PRODUCT_OUT/apex to installclean.
When removing dependencies from APEXes, the lib files were still there
even after installclean.

Bug: 139016109
Test: run installclean, see apex directories go away :D
Change-Id: I4dad8e3aa0042cdd5bbb76bd6fdd01b025b98495
2019-08-26 18:35:55 +00:00
Elliott Hughes 556a37585a We have md5sum/sha1sum/sha256sum/sha512sum on Darwin now too.
Bug: http://b/139450866
Test: treehugger
Change-Id: I654f706a18949d29c8688b1abe85d574b5315808
2019-08-26 11:15:15 -07:00
Dan Willemsen 70c1ff8dac Always set ASAN_SYMBOLIZER_PATH appropriately
Otherwise tools built with ASAN can't symbolize their dumps, as
llvm-symbolizer is not allowed to be used from $PATH.

Fixes: 139825736
Test: cherry-pick https://android-review.googlesource.com/c/platform/system/tools/aidl/+/1106532
  mmma ASAN_OPTIONS="" SANITIZE_HOST=address system/tools/aidl
Change-Id: Ib1eeec15ce6694d7bde54c7201b3280e38df3a83
2019-08-21 22:34:11 +00:00
Dan Willemsen ce41e943be Remove support for ONE_SHOT_MAKEFILE
Test: treehugger
Change-Id: Icdbe62bcb7bc4717228c5d974962b939d8eafee6
2019-07-31 18:11:27 -07:00
Elliott Hughes 521602194b Merge "Switch to toybox egrep(1) and grep(1)." 2019-07-31 16:55:43 +00:00
Dan Willemsen 1eee154c3b Default to the status table output
For smart terminals, default to using the status table, using 1/4 of the
terminal height (with a min of 1 and a max of 10).

This behavior can still be overriden to a specific height with
SOONG_UI_TABLE_HEIGHT, and turned off by setting SOONG_UI_TABLE_HEIGHT
to 0.

Test: m   <adjust terminal height, see it shrink when it gets too short>
Test: SOONG_UI_TABLE_HEIGHT=20 m
Test: SOONG_UI_TABLE_HEIGHT=0 m
Change-Id: I224348a29a6e07f168c92cf5514a94bd27e32618
2019-07-30 13:44:03 -07:00
Dan Willemsen a2a8ecb330 Remove reference to PRODUCT-*
The code still needs to exist, but for buildspec.mk files, not the
PRODUCT-* arguments.

Test: none
Change-Id: I60c6044ab2d1627af7e6e2e8831b77d9db12aa55
2019-07-29 15:14:11 -07:00
Kousik Kumar 61afb56e84 Add dist_dir to goma trigger
This is done so that prototype tooling can write logs inside the DIST_DIR.

Test: I wrote out all the environment variables being passed to
goma_ctl.py and the output is https://paste.googleplex.com/6663469954760704. It contains both DIST_DIR and other variables.

Change-Id: Ia5c70e58aa1b87db9777c59cf3e5df1c9e20f08c
2019-07-23 16:16:15 -04:00
Elliott Hughes 7ceddbae8e Merge "Switch to prebuilt bzip2." 2019-07-23 19:28:02 +00:00
Elliott Hughes 2ee8b33edc Switch to prebuilt bzip2.
Test: treehugger
Change-Id: I4953ff6f9ead5ad79519ac7fa71b557807f25851
2019-07-23 15:08:58 +00:00
Ramy Medhat 2b78fda705 Merge "Add USE_RBE support to soong." 2019-07-23 11:49:10 +00:00
Treehugger Robot 1ecfa940d1 Merge "Reland "Switch to toybox tar."" 2019-07-23 02:08:40 +00:00
Ramy Medhat bbf2567cc1 Add USE_RBE support to soong.
Test: Built aosp_arm-user with and without USE_RBE. USE_RBE uses
a proxy script in place of rewrapper.

Change-Id: I5bf008a940513872d70b5b215bd6209f759826ae
2019-07-17 18:46:47 +00:00
Patrice Arruda 19bd53e39c soong_ui: Do not clean the OutDir path multiple times.
If OUT_DIR has been specified through the command line arguments,
it is sanitized first and NewConfig sets the OUT_DIR in the build
environment. When calling OutDir from Config instance, the out
was being sanitized again which is unnecessary.

Bug: b/118730755
Test: Ran unit test cases, "lunch" followed by "m" command.
Change-Id: Ieb4ffa1026c802c76c7369e8cdb3923a4c70f4b3
2019-07-16 11:22:22 -07:00
Patrice Arruda 0dcf27f639 soong_ui: Construct the proper MODULES-IN-<dir name> for m* build commands.
Previously, any of the build commands (mma, mmma, etc...) would traverse
up the directory path until an Android build file has been found. That
directory path would eventually become the MODULES-IN-<dir name> build
target. The directory where the m* command was executed may contain a
build file in a sub directory. For that case, the build target is the
MODULES-IN-<dir name> where dir is the directory of the executed m*
command since the target has been constructed by the core makefile.

Bug: b/118730755
Test: Wrote and ran unit test cases. Ran "lunch", "m". Went to a
      directory in AOSP that had build files in sub directories and
      executed mma. Went to a directory that had only build files in
      parent directories and executed mma.

Change-Id: I9458cbc4bf0fc25bfb0848e96e92d261442d5cb0
2019-07-16 11:19:05 -07:00