Commit Graph

25091 Commits

Author SHA1 Message Date
Colin Cross d888b6b4fc Add stats from ninja subprocesses to build.trace.gz
Propagate the subprocess stats from ninja to build.trace.gz.

There is too much data here to put into separate counters for each
statistic, so put them into args on each duration event for now.

We would like to track max RSS for each subprocess, but the Linux
kernel inherits the max RSS of the ninja process in each subprocess,
which sets a lower bound on the measurable max RSS to the size of
the ninja process.  The ninja process is large due to the multi-GB
build.ninja files.

Bug: 170701554
Test: examine build.trace.gz
Change-Id: I8aaaafe627a57f1a500af098c097c6381c583ba5
2020-10-15 13:46:32 -07:00
Colin Cross ce4c7cd55d Add odm_dlkm and vendor_dlkm to the installclean list
Clean the odm_dlkm and vendor_dlkm staging directories in installclean
to get consistent results in incremental builds on the build servers.

Test: treehugger
Change-Id: Id431d7457d85e0be68437a2034fd544d74cc3709
2020-10-14 11:32:23 -07:00
Colin Cross 6ddb81b0db Merge changes Idc985c52,Ibbbde323,I51417cf6,I9886498d,I195d99c9
* changes:
  Move LLNDK and NDK versionSelectorMutator special cases into versionedInterface
  use version mutator for CRT
  Use version mutator for NDK
  Reuse more of apex stubs implementation for llndk stubs
  Use libraryInterface instead of concrete type asserts for stubs
2020-10-14 00:20:05 +00:00
Colin Cross d4620701ad Merge "Start using Providers instead of direct module access" 2020-10-13 21:48:26 +00:00
Treehugger Robot 585d7d4c2e Merge "Collect the hostname of the machine as a metric." 2020-10-13 18:07:51 +00:00
Evgenii Stepanov 00e07c9779 Merge "Support asan/hwasan versions of prebuilts." 2020-10-13 17:51:58 +00:00
Steven Moreland 88ed5d9353 Merge "Zero C++ heap by default." 2020-10-13 16:49:16 +00:00
Anton Hansson ff98103505 Merge "Remove jdiff support from droiddoc" 2020-10-13 07:08:13 +00:00
Treehugger Robot c8821b9451 Merge "Expose android.filesToInstall" 2020-10-13 04:52:36 +00:00
Treehugger Robot 0807abacf3 Merge "Add additional documentation in metrics proto regen.sh script." 2020-10-13 04:20:29 +00:00
Evgenii Stepanov 2080bfe79a Support asan/hwasan versions of prebuilts.
In apex_set and cc_prebuilt_library_*, provide a way to specify an
alternative source to use when build with sanitizers.

Test: prebuilt_test, apex_test
Change-Id: I1ab8091bf696d94da3547cf5248853df489bdee6
2020-10-13 03:37:32 +00:00
Colin Cross 9ebc22cf2d Merge changes Ieeca3c39,Iddeea2d0,I8d66a5d3
* changes:
  Add jni_libs to host java binaries
  Make java_binary common variant a dependency
  Replace jniDependencyTag with a value
2020-10-13 03:33:10 +00:00
Colin Cross 3572cf74f9 Move LLNDK and NDK versionSelectorMutator special cases into versionedInterface
Implement stubsVersions on *llndkStubDecorator and *stubDecorator to
handle the special cases in versionSelectorMutator.

Test: m checkbuild
Change-Id: Idc985c52f91450df42c0275b2b2acef3f2ed8868
2020-10-12 16:55:49 -07:00
Colin Cross bbc941b0d0 use version mutator for CRT
Move the CRT objects into the version mutator and retire the
ndk_api mutator.

Test: no change to build.ninja or Android-${TARGET_PRODUCT}.mk
Change-Id: Ibbbde323e3e0e8e4702dda4f3828a49786280118
2020-10-12 16:55:49 -07:00
Colin Cross 5ec407b594 Use version mutator for NDK
The ndk_api mutator is similar to the version mutator.  Move the
ndk_library ndk_api variations into the version mutator instead,
which will help later when consolidating the stubs handling
between NDK, LLDNK and Apex libraries.

Test: No change to build.ninja or Android-${TARGET_PRODUCT}.mk
Change-Id: I51417cf669265762c15f7289e1dc186d017ef4a9
2020-10-12 16:55:49 -07:00
Colin Cross 8e21aa54eb Reuse more of apex stubs implementation for llndk stubs
Use the linker script and exported flags support from apex stubs
for llndk stubs.

Test: no change to build.ninja or Android-${TARGET_PRODUCT}.mk
Change-Id: I9886498dcac7419958d290de99cf5f39f5fdedee
2020-10-12 16:55:49 -07:00
Colin Cross c88c272298 Use libraryInterface instead of concrete type asserts for stubs
Start consolidating stubs, llndk stubs, and ndk stubs by replacing
all of the concrete type asserts in the stubs methods with
libraryInterface.

Test: no change to build.ninja, Android-${TARGET_PRODUCT}.mk
Change-Id: I195d99c91e5f7afda5bcb489d20b524b3300c1b1
2020-10-12 16:55:49 -07:00
Colin Cross 0de8a1e17b Start using Providers instead of direct module access
Export information about static libraries, shared libraries and
exported flags through Providers instead of accessing the module
directly.  Much more is left to be converted, but this significantly
simplifies the dependencies on libraries with stubs by making it easy
for a module to masquerade as another by simply exporting the
providers from the other module.  Instead of depending on all the
versions of a library and then picking which one to use later, it
can depend only on the implementation variant and then select the
right SharedLibraryInfo from the variant.

Test: m checkbuild
Test: only expected changes to build.ninja
Change-Id: I1fd9eb4d251cf96ed8398d586efc3e0817663c76
2020-10-12 16:55:47 -07:00
Treehugger Robot ff8838cb86 Merge "Add jetifier support to android_library_import" 2020-10-12 23:38:06 +00:00
Liz Kammer 9244491f17 Merge "Enable defaults in soong_config_module_type props" 2020-10-12 23:18:28 +00:00
Patrice Arruda 1e2cad95c8 Add additional documentation in metrics proto regen.sh script.
Added additional instruction in the regen.sh script to build the
blueprint_tools and aprotoc in case the regen.sh script fails.

Bug: b/169395325
Test: ./regen.sh
Change-Id: Icb38c907fb47febc17eef1067d06d0588dd0b7ba
2020-10-12 22:58:27 +00:00
Patrice Arruda 4fb8adcc4c Collect the hostname of the machine as a metric.
The hostname is collected during a build to determine where the user
is build from (from office workstation, cloudtop, etc...). This is a
useful metrics for RBE related builds.

Bug: b/169395325
Test: m nothing, ran printproto to verify that soong_metrics has the
      hostname.
Change-Id: I3e5d64f948ae43c5dda60de1d4a52b187e36dff8
2020-10-12 22:44:27 +00:00
Jaewoong Jung c1c415d0f7 Merge "Apply PRODUCT_ENFORCE_RRO_TARGETS to dependencies." 2020-10-12 21:45:33 +00:00
Treehugger Robot 79bb3d4784 Merge "Remove out/host/common/obj/PACKAGING during installclean" 2020-10-12 20:48:09 +00:00
Colin Cross 56b6de5847 Merge changes I056c49e6,I994402cb
* changes:
  Fix OutputPath.InSameDir example
  Fix go vet error
2020-10-12 20:01:27 +00:00
Steven Moreland d134201537 Zero C++ heap by default.
Let the flakes, and the crashes, and the deadlocks, and the leaks, and
the undefined behavior which steals away so many nights find their
numbers decimated in the homogeneous conditions we now rest our data.

To give some intuition why this is not so underperformant, zeroing
memory is one way of priming caches.

This change is actually a no-op, because build/make/core/soong_config.mk
always overrides this setting, but it reflects the spirit and future
direction of this change.

Bug: 131355925
Test: basic simple perf comparisons
Change-Id: I8254c36373de52091839561973c9741e8f85aa09
2020-10-12 18:41:03 +00:00
Saeid Farivar Asanjan f043696fb9 Add jetifier support to android_library_import
Bug: 170242653
Test: manual
Change-Id: I9b49ea9ed18041c5509d4d53a3ce8fdf58c8c537
2020-10-12 17:51:38 +00:00
Jaewoong Jung c779cd403f Apply PRODUCT_ENFORCE_RRO_TARGETS to dependencies.
With this change, users don't need to figure out which libraries
actually hold the resources to be overlaid when targetting apps with a
core lib dependency (e.g. Settings, SystemUI).

Fixes: 169898727
Test: app_test.go
Change-Id: I3c3b9dc0a377b1828db1199858a73d080a173205
2020-10-12 10:34:36 -07:00
Jaewoong Jung 19bc60abee Merge "Add config.AndroidFirstDeviceTarget." 2020-10-12 17:30:18 +00:00
Steven Moreland 73a9e603d5 Merge "build/soong/scripts/update-apex-allowed-deps.sh" 2020-10-12 16:59:24 +00:00
Anton Hansson 53781d55bc Merge "Make hiddenapi flag generation use new artifact" 2020-10-12 16:08:11 +00:00
Treehugger Robot f43eee4b9b Merge "Add warning about out-directory usage with RBE builds" 2020-10-12 15:25:22 +00:00
Jaewoong Jung 642916f20a Add config.AndroidFirstDeviceTarget.
Targets[Android][0] is not always the first, preferred target. Add a new
field to config to provide the correct target and begin to use it.

Test: m nothing
Test: build/soong/scripts/build-ndk-prebuilts.sh
Fixes: 156980228
Change-Id: Ib56f52424ebf47f3eee6ac62b9fd9eca2919d63a
2020-10-12 14:15:19 +00:00
Anton Hansson 858a8cabf9 Merge "Add a Impl_only_libs prop for sdk_library" 2020-10-12 09:00:13 +00:00
Treehugger Robot 6aa958902a Merge "android/: Rename Plat->SystemExt*SepolicyDirs" 2020-10-12 03:49:24 +00:00
Colin Cross 41b4676f44 Fix OutputPath.InSameDir example
The OutputPath.InSameDir example was called FileInSameDir.

Test: go vet ./android
Change-Id: I056c49e6b39f6124b0fa32ebc3498a2fe42c8ddc
2020-10-10 03:04:09 +00:00
Colin Cross ff0278b32a Fix go vet error
stderr is a bytes.Buffer, but the String() method has a pointer
receiver, so stderr does not satisify the Stringer interface
and can't be used for a %s argument.  Make stderr a *bytes.Buffer
instead.

Test: go vet ./android
Change-Id: I994402cb954946279375c9d447ad3854380381cc
2020-10-10 03:03:54 +00:00
Colin Cross 3e6f67a68b Remove out/host/common/obj/PACKAGING during installclean
Tradefed packages are created in out/host/common/obj/PACKAGING
and should be removed during installclean so that they get an
accurate embedded build number.

Fixes: 170308394
Test: treehugger
Change-Id: Ib7a57377b5c9512584bc93f10da316eadf786979
2020-10-10 03:03:38 +00:00
Colin Cross 89226d9ef9 Add jni_libs to host java binaries
Add a property to support dependencies on JNI libraries for host
java binaries.

Fixes: 170389375
Test: TestBinary
Change-Id: Ieeca3c3997615f0b17ae1f058b94e6c9ba929cab
2020-10-09 19:27:34 -07:00
Colin Cross c179ea6812 Make java_binary common variant a dependency
ctx.PrimaryModule() is wrong in the case of a java_binary that
supports both host and device, use an explicit dependency instead.
Once the dependency exists there is no need to manually request
the jar be installed, it will automatically be installed by the
host installation rules for dependencies.

Test: TestBinary
Change-Id: Iddeea2d08bc574c79d42139020558cd70d718ca1
2020-10-09 19:27:34 -07:00
Colin Cross de78d138a1 Replace jniDependencyTag with a value
Support GetDirectDepsWithTag on JNI deps by replacing the
jniDependencyTag type with a jniLibTag value.

Test: app_test.go
Change-Id: I8d66a5d3f433562e131a1fbafce75891d1b094dd
2020-10-09 19:24:05 -07:00
Colin Cross 405af07859 Revert "Make lots of tests run in parallel"
This reverts commit 323dc60712.

Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests

Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
2020-10-09 18:34:24 -07:00
Liz Kammer dbd4809b06 Enable defaults in soong_config_module_type props
Test: go tests
Test: m
Change-Id: I8fe8c17320086034f4745b2da974a2c73c949eaf
2020-10-09 14:28:34 -07:00
Treehugger Robot 6682ef42a4 Merge "Global ThinLTO: opt out vndk binaries as a workaround" 2020-10-09 19:00:42 +00:00
Treehugger Robot 38cfe29597 Merge "Revert "Implement vts_config module"" 2020-10-09 17:25:35 +00:00
Christopher Parsons c8f84809ad Merge "Mixed bazel/soong build prototype for genrule" 2020-10-09 14:27:04 +00:00
Yi Kong 134161f7e5 Global ThinLTO: opt out vndk binaries as a workaround
With global ThinLTO enabled, vndk version has different symbol ordering
from the platform version. Opt out any binaries that has vndk enabled as
a temporary workaround.

Test: GLOBAL_THINLTO=true m
Bug: 169217596
Change-Id: I75b060cbe6c74421d283c6dfbd669af20f466d1f
2020-10-09 22:08:11 +08:00
Anton Hansson 7f66efa10c Add a Impl_only_libs prop for sdk_library
Similar to Stubs_only_libs, this allows being a bit more specific
about what libs compile against what. In my usecase, it allows
removing a platform dependency from the appsearch stubs, which allows
the platform to depends on the stubs.

Bug: 169304493
Test: m
Change-Id: Ie997462819cd5266a761b2d415e81806c7877967
Merged-In: Ie997462819cd5266a761b2d415e81806c7877967
2020-10-09 13:49:30 +01:00
Thiébaud Weksteen 43977e8d0e Merge "rust: refactor tests setup" 2020-10-09 12:13:15 +00:00
Chris Parsons f3c96efea4 Mixed bazel/soong build prototype for genrule
With this change, bazel_module is a specifiable property on
genrule module definitions. With bazel-enabled mode, soong_build will
defer to Bazel for information on these modules.

source build/soong/bazelenv.sh to enter bazel-enabled mode.

Test: Manually verified on bionic/libc genrules using aosp_cf_x86_phone-userdebug
Change-Id: I3619848186d50be7273a5eba31c79989b981d408
2020-10-08 22:46:23 -04:00