Commit Graph

21292 Commits

Author SHA1 Message Date
Joel Galenson 6945e9b650 Migrate away from removed option
Upstream removed the -Z no-landing-pads option in
https://github.com/rust-lang/rust/pull/70175, which we picked up in
our latest toolchain update.

Fixes: 166317885
Test: Build and TreeHugger
Change-Id: I51c0ef671bf6a334b2b7223da42cab547e4407d3
2020-08-25 15:27:15 -07:00
Treehugger Robot 2d078b814a Merge "Bazel overlay: rename 'deps' to 'module_deps'." 2020-08-25 21:40:50 +00:00
Jingwen Chen ce3d46fe05 Bazel overlay: rename 'deps' to 'module_deps'.
'deps' is a module prop used by some module types, like
prebuilt_build_tool:

https://cs.android.com/android/platform/superproject/+/master:build/soong/android/prebuilt_build_tool.go;l=25-26;drc=751ae87ed29c67629f068a8590af80023a457294

Use `module_deps` to prevent conflicts in the generated overlay
attributes.

Bug: 162720644
Test: m bazel_overlay && cd out/soong/bazel_overlay && bazel cquery
//...
Test: soong_build tests

Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: Ia075a860983960b2d9121d30ead8f808be7e0c5b
2020-08-25 06:11:53 +00:00
Jingwen Chen dc4df2dd4a Merge "Surface module properties as Bazel BUILD target attributes in the Bazel overlay" am: 81a0b66d93
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393254

Change-Id: Iaaf6ce5bd5e9aacc2ec577bbb60fe2544ad945ba
2020-08-24 23:40:11 +00:00
Jingwen Chen 81a0b66d93 Merge "Surface module properties as Bazel BUILD target attributes in the Bazel overlay" 2020-08-24 22:59:11 +00:00
Treehugger Robot 1e4fbc23bf Merge "Support ninja rsp files in soong_zip" am: 4c2e4f381a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405368

Change-Id: I99276f84d12144624385012f8f5593c02d990d60
2020-08-24 19:33:37 +00:00
Treehugger Robot 4c2e4f381a Merge "Support ninja rsp files in soong_zip" 2020-08-24 18:40:56 +00:00
Treehugger Robot a5c0b11970 Merge "Propagate sanitize.never flag in SDK snapshots." am: e819bec240
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1406917

Change-Id: Ia8a02f8570ca0619c1b4696aa72c2dd1a2a393ee
2020-08-24 17:31:15 +00:00
Liz Kammer 380c67920e Merge "Add imports_sdk_version to prebuilt_apis" am: 9c07d24477
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1397557

Change-Id: Ic536dfe4c2f3a3a2e610520266cdca76f0d9ecfd
2020-08-24 17:30:51 +00:00
Treehugger Robot e819bec240 Merge "Propagate sanitize.never flag in SDK snapshots." 2020-08-24 17:20:58 +00:00
Liz Kammer 9c07d24477 Merge "Add imports_sdk_version to prebuilt_apis" 2020-08-24 16:50:51 +00:00
Colin Cross 6419a88214 Update go.mod to go 1.15 am: 42f135b7b0
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405909

Change-Id: Ic3bfd0cf94cd477d8a0f8830cb9b9bb3764968f0
2020-08-22 00:51:33 +00:00
Colin Cross 42f135b7b0 Update go.mod to go 1.15
The version in go.mod is used by `go build` to select the version
of the Go language to target.  If it is older than the version of Go
in the tree then command line tools like `go test` will give different
results from inside the build.  Update the version to match the version
of Go in the tree.

Test: go build ./java
Change-Id: I418f401580be7696e2829c970938c7c82715b932
2020-08-21 11:13:21 -07:00
Martin Stjernholm fbb486fc7a Propagate sanitize.never flag in SDK snapshots.
This is necessary to not get dependencies on libclang_rt sanitizer libs
for the CRT objects which lead to cyclic dependencies in sanitizer
builds.

Test: `lunch aosp_blueline_hwasan-userdebug && m nothing`
  in a tree that has the prebuilts created and unzipped from
  `build/soong/scripts/build-aml-prebuilts.sh runtime-module-{sdk,host-exports}`
Bug: 151303681
Change-Id: I3f848a084280bdc3ade4b74df03e981d8cc61222
2020-08-21 18:43:51 +01:00
Treehugger Robot 82448f2d2d Merge "Add extra_test_configs option" am: ec7a91f9fa
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1402168

Change-Id: I6de14462095a64c3ccb3c501de7e4feaa04394c9
2020-08-21 16:50:54 +00:00
Treehugger Robot ec7a91f9fa Merge "Add extra_test_configs option" 2020-08-21 16:22:43 +00:00
Jingwen Chen 69d4cbeb04 Surface module properties as Bazel BUILD target attributes in the Bazel overlay
This patchset changes bazel_overlay to generate soong_module as a macro,
instead of a rule, and generate module properties in the BUILD files as
kwargs to the soong_module macro.

Here's a sample of the new BUILD files with module properties:

bionic/libdl/BUILD.bazel:
https://paste.googleplex.com/6484466996346880?raw

art/build/apex/BUILD.bazel:
https://paste.googleplex.com/5461276001042432?raw

bionic/apex/BUILD.bazel:
https://paste.googleplex.com/4932795173437440?raw

soong_module is now a macro that conditionally expands to underlying
soong_<module type> rules with statically defined attributes. In this
CL, we are starting with a hardcoded filegroup rule definition to
demonstrate the conditional rule loading within the soong_module macro.
If the module_type matches an existing Bazel rule, soong_module forwards
the entire **kwargs into the rule, which Bazel typechecks.

Non-filegroup module types will be expanded into generic_soong_module,
but with the kwargs dropped.

This approach allows us to:

1) Programmtically generate soong_<module type> rules for all module
types available in Soong, together with the statically defined attribute
types in `attrs`.
2) Incrementally migrate and test individual module types from
generic_soong_module to their module rule shims.
3) Swap out the module rule shims to the actual Bazel rules (e.g
cc_library, java_library) and perform attribute manipulation in Starlark
itself.

Example of querying against the 'srcs' attribute in soong_filegroup:

```
$ bazel cquery 'kind(soong_filegroup, //...)' | wc -l
590

$ bazel cquery --output=build 'attr(srcs, "linker.cpp",
kind(soong_filegroup, //bionic/...))'
INFO: Analyzed 3907 targets (0 packages loaded, 0 targets configured).
INFO: Found 3907 targets...
/usr/local/google/home/jingwen/aosp/out/soong/bazel_overlay/bionic/linker/BUILD.bazel:4144:13
soong_filegroup(
  name = "linker_sources",
  generator_name = "linker_sources",
  generator_function = "soong_module",
  generator_location = "bionic/linker/BUILD.bazel:4144:13",
  srcs = ["dlfcn.cpp", "linker.cpp", "linker_block_allocator.cpp",
"linker_dlwarning.cpp", "linker_cfi.cpp", "linker_config.cpp",
"linker_debug.cpp", "linker_gdb_support.cpp", "linker_globals.cpp",
"linker_libc_support.c", "linker_libcxx_support.cpp",
"linker_namespaces.cpp", "linker_logger.cpp",
"linker_mapped_file_fragment.cpp", "linker_phdr.cpp",
"linker_relocate.cpp", "linker_sdk_versions.cpp", "linker_soinfo.cpp",
"linker_tls.cpp", "linker_utils.cpp", "rt.cpp"],
  deps = [],
)
/usr/local/google/home/jingwen/aosp/out/soong/bazel_overlay/soong_module.bzl:32:23
in <toplevel>
```

This CL is known to be lacking the following features, and will be looked at in follow up CLs:

1) Pretty printing reflect.Interface properties, like arch, multilib and
dists.
2) Generating module Bazel rule shims for all module types, instead of
hardcoding them like `soong_filegroup`.

Bug: 162720644
Test: bazel_overlay_test.go (soong build test)
Test: m bazel_overlay && cd out/soong/bazel_overlay && bazel cquery //...
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: Ic1e448887eb540ed15a55bc4090cf75a4d832d41
2020-08-21 11:45:46 +00:00
Treehugger Robot bc9a8dfc01 Merge "Remove duplicated use of UsesLibraryDependency" am: f5a2b8a641
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405677

Change-Id: I6e095d9426d3c83f361293ecda441d6983fa5275
2020-08-21 10:06:53 +00:00
Martin Stjernholm b5326f2d1c Merge "Resolve prebuilt module explicitly for dex2oat tool dependencies." am: 4ef4d55944
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1404636

Change-Id: I712c2e6b3a19ed65e6624d1e712b2b11790aa411
2020-08-21 10:06:46 +00:00
Ulyana Trafimovich d972866eb5 Merge "Use common helper functions for getting sorted map keys." am: 49b1e9b6ce
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1404519

Change-Id: I2ad29eeb5a9b7a2b69854f945f8ed8f912655eff
2020-08-21 10:06:32 +00:00
Ulyana Trafimovich df14a1b9c2 Merge "Propagate transitive <uses-library> dependencies through static libraries." am: bde92fc6be
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1404787

Change-Id: I03a83e7a2cab6cdb5d8e74df979440bb243d6ec3
2020-08-21 10:04:32 +00:00
Treehugger Robot f5a2b8a641 Merge "Remove duplicated use of UsesLibraryDependency" 2020-08-21 10:01:58 +00:00
Martin Stjernholm 4ef4d55944 Merge "Resolve prebuilt module explicitly for dex2oat tool dependencies." 2020-08-21 09:57:40 +00:00
Ulyana Trafimovich 49b1e9b6ce Merge "Use common helper functions for getting sorted map keys." 2020-08-21 09:36:30 +00:00
Ulyana Trafimovich bde92fc6be Merge "Propagate transitive <uses-library> dependencies through static libraries." 2020-08-21 09:36:10 +00:00
Treehugger Robot a3d1c127a2 Merge "Reland: Deduplicate APEX variants that would build identically" am: 68bfe074c6
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1405087

Change-Id: I3839e80972dbc025b504f4593538f589be079389
2020-08-21 07:21:53 +00:00
Treehugger Robot 68bfe074c6 Merge "Reland: Deduplicate APEX variants that would build identically" 2020-08-21 06:46:51 +00:00
Treehugger Robot 0d1812df60 Merge "Add Goma deprecation PSA to soong." am: c70043aec2
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1403640

Change-Id: I828f1770c3ad4f3d6a5c561dadd93735e413581f
2020-08-21 04:39:42 +00:00
Treehugger Robot c70043aec2 Merge "Add Goma deprecation PSA to soong." 2020-08-21 04:26:11 +00:00
Treehugger Robot 305935590d Merge "Capture list of unused methods when shrinking in R8" am: c23d6031b0
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1401848

Change-Id: Iff923f6fe8ba1d4cd0d93cc182ad6aaaa7d7e6bd
2020-08-21 00:36:49 +00:00
Treehugger Robot c23d6031b0 Merge "Capture list of unused methods when shrinking in R8" 2020-08-21 00:28:24 +00:00
Liz Kammer 9b7ecd76f5 Remove duplicated use of UsesLibraryDependency
This was resulting in a soong java test failure:
./java.go:994:80: ambiguous selector dep.DexJarBuildPath
./java.go:994:103: ambiguous selector dep.DexJarInstallPath
./java.go:2595:58: ambiguous selector dep.DexJarBuildPath
./java.go:2595:81: ambiguous selector dep.DexJarInstallPath
./sdk_library.go:879:2: duplicate method DexJarBuildPath
./sdk_library.go:879:2: duplicate method DexJarInstallPath

Test: go test java tests
Test: m
Change-Id: I34465907f917d94be643be7322215fc35fc5fbe6
2020-08-20 13:50:42 -07:00
Treehugger Robot b8638926f4 Merge "Allow globally disabling some clang-tidy checks." am: 66790d8943
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1403127

Change-Id: Ia2351d00d6952fea7f82268e4d345306e6118bff
2020-08-20 20:13:44 +00:00
Treehugger Robot 66790d8943 Merge "Allow globally disabling some clang-tidy checks." 2020-08-20 19:51:31 +00:00
Jeffrey Vander Stoep 507f3f0c0f Merge "Rust: use 1.45.2 prebuilts" am: 2e8a617bca
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1398476

Change-Id: I2d5e9e8d887f1bf3ed1c0de2dd34370b28bd372f
2020-08-20 17:23:46 +00:00
Jeffrey Vander Stoep 2e8a617bca Merge "Rust: use 1.45.2 prebuilts" 2020-08-20 17:14:19 +00:00
Liz Kammer 2d2fd859bb Add imports_sdk_version to prebuilt_apis
This allows setting the prebuilt sdk's generated java_imports
sdk_version to "none".

Test: go tests
Test: m
Bug: 160455085
Change-Id: Id24aca811f0f09692971e63418da6685d8351737
2020-08-20 09:48:50 -07:00
Martin Stjernholm c004862309 Resolve prebuilt module explicitly for dex2oat tool dependencies.
Test: m nothing
Bug: 145934348
Change-Id: If27984be26976a3abe8cb5e21cf174d5e71beb8a
2020-08-20 16:05:45 +01:00
Ulya Trafimovich b8063c6a86 Use common helper functions for getting sorted map keys.
Add a new helper SortedIntKeys similar to SortedStringKeys.

Test: lunch aosp_cf_x86_phone-userdebug && m
Change-Id: I08a43ec2cae7d1a82531295aca1a0658e3a0dd6f
2020-08-20 12:37:03 +01:00
Ulya Trafimovich 4b6d4c12cc Propagate transitive <uses-library> dependencies through static libraries.
Bug: 163037089
Test: lunch aosp_cf_x86-userdebug && m nothing
Test: added testcase in Soong
Change-Id: I943d497d779ca218a16e7208029189a1a69086c1
2020-08-20 12:34:42 +01:00
Ulyana Trafimovich 9b67bbd11c Merge "Collect paths to transitive SDK Java library dependencies." am: 9ce2221791
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1402668

Change-Id: I2f12e92ce08bd75d19073a0ba19aca2b07142422
2020-08-20 11:02:10 +00:00
Ulyana Trafimovich 9ce2221791 Merge "Collect paths to transitive SDK Java library dependencies." 2020-08-20 10:37:58 +00:00
Dan Shi 95d19422a6 Add extra_test_configs option
Bug: 163344047
Test: m -j HelloWorldHostTest hello_world_test HelloWorldTests
Change-Id: I237e3aa3f40df1f6387ba8ff6a16e56535cdeae9
2020-08-19 15:27:03 -07:00
Colin Cross 053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Colin Cross cb6143a142 Capture list of unused methods when shrinking in R8
Use the -printusage flag in R8 to output a list of the unused
methods.  Some of the files can be large (2MB for DocumentsUI,
87MB for all of AOSP), so immediately zip them and remove the
originals.  The zipped files will be merged and disted.

Bug: 151857441
Test: m TARGET_BUILD_APPS=DocumentsUI dist
Change-Id: I780e84e80eba7fe4d4fa15fec0f461890afd900b
2020-08-19 11:19:14 -07:00
Colin Cross aede88c1c7 Reland: Deduplicate APEX variants that would build identically
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories.  This causes unnecessary
build time and disk space for duplicated work.

Deduplicate APEX variants that would build identically.  Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.

This has one significant change in behavior.  Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs.  After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.

In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs.  However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.

This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.

This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.

Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
2020-08-19 10:21:17 -07:00
Ulya Trafimovich 31e444e101 Collect paths to transitive SDK Java library dependencies.
Previously only the names were collected, and later used in the
manifest_fixer to add missing <uses-library> entries to the manifest.
Now we also need to collect build-time and on-device paths, to be used
in class loader context for dexpreopt. This commit only collects paths,
but does not pass them to dexpreopt yet.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I34b229ee68f16ba215ba03770feadb4d890ec2bf
2020-08-19 13:51:21 +01:00
Treehugger Robot 2a3d0ade4c Merge "Add AIDL compiler to allowed Rust paths" am: 2d815963ba
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1394307

Change-Id: Ib892f2d01237bf56592356fdefa32859d13a6378
2020-08-18 22:12:44 +00:00
Treehugger Robot 2d815963ba Merge "Add AIDL compiler to allowed Rust paths" 2020-08-18 22:00:42 +00:00
Ramy Medhat f668917c25 Add Goma deprecation PSA to soong.
Test: build with Goma
Change-Id: Ie7e146285afd40bc0a1dc17a0b898960f57a118b
2020-08-18 17:29:48 -04:00