Commit Graph

21294 Commits

Author SHA1 Message Date
Treehugger Robot 8b073fc7f1 Merge "Reland "Support per-module MakeVars"" 2020-08-07 00:50:24 +00:00
Jingwen Chen a9af2cefaf Merge "Create a AOSP Bazel overlay workspace with Soong" am: 2106bf8458
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1359245

Change-Id: I167ee06ac23afebcf38676b490d6ce75773b2639
2020-08-07 00:10:07 +00:00
Jingwen Chen 2106bf8458 Merge "Create a AOSP Bazel overlay workspace with Soong" 2020-08-06 23:56:54 +00:00
Dan Willemsen 751ae87ed2 Reland "Add prebuilt_build_tool to allow genrules to use prebuilt tools"
Includes fixes for out dirs at absolute paths, and symlinks to absolute
paths. Also squashes support for exporting make vars, which previously
happened in a later patch (which got moved first, since it's mostly
infrastructure changes).

Original message:

cc_prebuilt_binary doesn't work well for host tools because they'll
often be using their own versions of shared libraries that may not be
compatible with what we build.

So add a module type that allows genrules to use one of these prebuilts
as a tool. Like other prebuilts, we'll use the source module if we have
it, or the prebuilt otherwise.

It supports adding extra dependencies for shared libraries or other data
files that are necessary to run the tool. Any genrules using the tool
will be rerun if any of the dependencies change.

Bug: 128690776
Test: treehugger
Test: build one-true-awk using genrule with bison
Test: with OUT_DIR=$PWD/out
Test: with OUT_DIR pointing to an absolute symlink
Change-Id: I662296cb55503b6a10ec5e5465b138e9edeb71f3
2020-08-06 15:56:38 -07:00
Dan Willemsen 6a6478d49e Reland "Support per-module MakeVars"
This allows setting per-module make variables earlier in the build,
particularly for prebuilt_build_tool users like LEX/M4/BISON (which will
come in a later CL). I moved filegroup over because it's a simpler
common interface, but it doesn't strictly need it.

Test: Inspect out/soong/make_vars-*.mk, out/soong/late-*.mk
Test: treehugger
Change-Id: I38631129d82b0aad2438e83610bb6a97568bf932
2020-08-06 15:56:09 -07:00
Andrei Homescu c7767922e0 Export Rust SourceProvider types and methods
The AIDL compiler now uses SourceProvider to compiler the generated Rust
code from system/tools/aidl/build/aidl_interface.go using its own
SourceProvider object, which needs access to baseSourceProvider and all
methods of SourceProvider.

Test: mmma system/tools/aidl with 1357705 applied
Change-Id: I226609a7fccca2e7e1bfbad5d69d1821d37e43a1
2020-08-06 15:50:27 -07:00
Yifan Hong 42bef8d0cc Add BoardKernelModuleInterfaceVersions.
This is a list of KMI versions for this board. Only gki_apex modules
with matching kmi_version field is enabled.

Test: build GKI apexes
Bug: 162888350
Change-Id: Ieb2116b79e3985ba15155a3cb8a2c340b46473c7
2020-08-06 15:47:58 -07:00
Martin Stjernholm 9e9bb7f223 Only output make targets for uninstallable static libs in the APEX
unavailable-to-platform case.

This removes the special case added in https://r.android.com/1274763
from SkipInstall(), so that it doesn't cause conflicting AndroidMk
entries when a cc_prebuilt_library_static module has prefer:true and
the corresponding source module exists.

Test: `m` in a tree with a snapshot created from art-module-sdk where
  the libartimagevalues module has prefer:true
Bug: 151303681
Change-Id: I651ae325753b707296892adb4cae80daaddb6af2
2020-08-06 22:52:58 +01:00
Matthew Maurer 86fdf15721 Merge "[rust] Escape flags for bindgen" am: 217a1e406e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1388348

Change-Id: I70a49bf440a2286c7c562538069eda56252993dd
2020-08-06 21:44:50 +00:00
Mitch Phillips 0ae16213d9 Merge "Added write only sanitizer for ASAN and HWASAN" am: 8dc2af873b
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1372178

Change-Id: I23e8b419658f01af16337a4ba2674a9422dacad4
2020-08-06 21:44:29 +00:00
Matthew Maurer 217a1e406e Merge "[rust] Escape flags for bindgen" 2020-08-06 21:33:49 +00:00
Mitch Phillips 8dc2af873b Merge "Added write only sanitizer for ASAN and HWASAN" 2020-08-06 21:17:12 +00:00
Kelvin Zhang 45e2f1405d Fix zip64 reader when file size is < 4GB and 32 bit fields are -1
When CopyFrom writes a zipentry, it strips extra fields and generates
data descriptors. When writing data descriptors, it only writes 64 bit
values if the relevant sizes are >4GB. In some cases, the sizes are <4GB
but 32 bit sizes are set to -1. In this situation, CopyFrom will write
incorrect local file header, resulting in a zip file that can't be
parsed by standard zip tools.

Test: Unit Tests
Bug: 161922066
Change-Id: I64319a80647013eaf7693cf8bf5c6120016913a3
2020-08-06 15:48:04 -04:00
Inseob Kim c42f2f2e9d Add cfi static libraries to vendor snapshot
CFI modules can't link against non-CFI static libraries, and vice versa.
So without capturing both CFI and non-CFI static libraries, vendor
modules won't be able to use CFI, which will be a critical security
hole.

This captures both CFI and non-CFI variants of all static libraries for
vendor snapshot, except for those whose cfi are explicitly disabled.

For example, suppose that "libfoo" is defined as follows.

cc_library_static {
    name: "libfoo",
    vendor_available: true,
}

As it doesn't have cfi disabled, two libraries "libfoo.a" and
"libfoo.cfi.a" will be captured. When installed, vendor snapshot module
for "libfoo" will look like:

vendor_snapshot_static {
    name: "libfoo",
    src: "libfoo.a",
    cfi: {
        src: "libfoo.cfi.a",
    },
}

The build system will recognize the "cfi" property, and will create both
CFI and non-CFI variant, allowing any modules to link against "libfoo"
safely, no matter whether CFI is enabled or not.

Two clarification:

1) The reason why we don't create separate modules is that DepsMutator
runs before sanitize mutators. CFI and non-CFI variant of a library
should exist in a single module.

2) We can't capture CFI variant if the source module explicitly disables
cfi variant by specifying the following.

sanitize: {
    cfi: false,
}

In this case, only non-CFI variant will be created for the vendor
snapshot module.

Bug: 65377115
Test: m dist vendor-snapshot && install && build against snapshot
Change-Id: Idbf3e3205d581800d6093c8d6cf6152374129ba4
2020-08-06 19:47:57 +00:00
Inseob Kim 2d34ad94f9 Refactor vendor snapshot modules
They have too many duplicated fields and functions.

Bug: 65377115
Test: manual
Change-Id: If3e2d67f02f3db7fa836c9550c978073ab7190e0
2020-08-06 19:47:43 +00:00
Steven Moreland 34fabfeffa Merge "allow empty phony modules" am: bd60c83271
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1390950

Change-Id: Id575e4e89f2dee5d727dc99d7beb9b4ecc5b2c67
2020-08-06 17:40:47 +00:00
Steven Moreland bd60c83271 Merge "allow empty phony modules" 2020-08-06 17:22:11 +00:00
Ulyana Trafimovich 434e708101 Merge "Add structured representation for colon-separated jar lists." am: 910eb70891
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1354005

Change-Id: I46ca0273cda33ca194400744fc19583d21118ffc
2020-08-06 15:24:29 +00:00
Ulyana Trafimovich 910eb70891 Merge "Add structured representation for colon-separated jar lists." 2020-08-06 15:07:27 +00:00
Nicolas Geoffray 484464e4ba Merge "Add platform-mainline-test-exports in list of modules to build." am: 1f109cd376
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1391356

Change-Id: Ie30b12b1d3ea5e9e67f716c75e67c65d402cd525
2020-08-06 14:02:15 +00:00
Nicolas Geoffray 1f109cd376 Merge "Add platform-mainline-test-exports in list of modules to build." 2020-08-06 13:49:25 +00:00
Thiébaud Weksteen 83ee52f4cd rust: validate existence of library source
While rust_bindgen modules may be interpreted as libraries, they do not
have a reference to the generated source until bindgen has been
executed. For now, ignore these modules.

Update the unit tests to cover the rust_bindgen case.

Test: SOONG_GEN_RUST_PROJECT=1 m nothing
Bug: 162881856
Change-Id: I329d0fe3d94b77d395c3684f55ab01544ff7c18f
2020-08-06 15:17:46 +02:00
Thiébaud Weksteen 2f628baeae rust: handle modules with same crate_name
crate_name do not need to be unique. A library may depend on two
different versions of the same crate. Use the module name instead of the
crate name when indexing the modules for rust-project.json.

Test: SOONG_GEN_RUST_PROJECT=1 m nothing
Bug: 162896400
Change-Id: I8af9115b106fda2bed5b9fe96059cc29a33ac247
2020-08-06 15:17:42 +02:00
Thiébaud Weksteen 2cb2bf7134 Revert "rust: validate existence of library source" am: c810510898
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393037

Change-Id: I50fc8f00dcbd528e80dc9551364250b1620d371a
2020-08-06 12:53:33 +00:00
Thiébaud Weksteen 4e62bf5357 Revert "rust: handle modules with same crate_name" am: 246e69c586
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1393036

Change-Id: Ieb7b7048d7f619e35d9d8a152754a7374c8fa288
2020-08-06 12:53:30 +00:00
Thiébaud Weksteen c810510898 Revert "rust: validate existence of library source"
Revert submission 1391076

Reason for revert: Broken downstream Darwin build (b/162975597)
Reverted Changes:
I275f04639:rust: handle modules with same crate_name
Ie736d7ebb:rust: validate existence of library source

Change-Id: I1732d50a08546864f17cd826c8af574c1064c670
2020-08-06 12:28:08 +00:00
Thiébaud Weksteen 246e69c586 Revert "rust: handle modules with same crate_name"
Revert submission 1391076

Reason for revert: Broken downstream Darwin build (b/162975597)
Reverted Changes:
I275f04639:rust: handle modules with same crate_name
Ie736d7ebb:rust: validate existence of library source

Change-Id: I995923153c11db26b4af985f2eabe94912fb04d3
2020-08-06 12:27:56 +00:00
Jingwen Chen 5ba7e479d1 Create a AOSP Bazel overlay workspace with Soong
The Bazel overlay is a directory at out/soong/bazel_overlay that
replicates the layout of the AOSP Soong module tree, but as a Bazel
workspace. Each Soong module variant is represented as a BUILD target
created with the `soong_module` rule.

To create this overlay, run `m bazel_overlay`.

A `soong_module` target can depend on other `soong_module` targets.
These dependencies replicate each module's `directDeps` in the Blueprint
graph, just before `PrepareBuildActions`.

This enables users to use bazel query as a way to introspect the Soong
module graph. For example,

- Direct reverse dependencies of //bionic/libc:generated_android_ids in
//bionic/libc/...:

$ bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'
//bionic/libc:libc_bionic_ndk--android_recovery_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_ramdisk_arm_armv7-a-neon_static
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static_com.android.runtime
//bionic/libc:libc_bionic_ndk--android_arm_armv7-a-neon_static
//bionic/libc:generated_android_ids

- Why does com.android.runtime depend on lzma?

$ bazel query
'somepath(//bionic/apex:com.android.runtime--android_common_com.android.runtime_image,
//external/lzma/...)'
//bionic/apex:com.android.runtime--android_common_com.android.runtime_image
//bionic/libc/malloc_debug:libc_malloc_debug--android_arm_armv7-a-neon_shared_com.android.runtime
//system/core/libunwindstack:libunwindstack--android_arm_armv7-a-neon_shared_com.android.runtime
//external/lzma/C:liblzma--android_arm_armv7-a-neon_shared_com.android.runtime

- What does the dep graph of //bionic/libc:crtbegin_so look like?

$ bazel query
'deps(//bionic/libc:crtbegin_so--android_arm_armv7-a-neon)'
--output=graph > graph.in && dot -Tpng < graph.in > graph.png

https://photos.app.goo.gl/DfsdoFRNsRjGwTmy8

Test:  croot && m bazel_overlay && cd out/soong/bazel_overlay && bazel
query //... && bazel query 'rdeps(//bionic/libc/...,
//bionic/libc:generated_android_ids, 1)'

Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I3bf40309bfb2d963bb8a688706385a57ee304c37#
2020-08-06 10:24:21 +00:00
Yo Chiang eb912dcce8 Improve abidump generating logic for non-REL build
Optimize build by removing redundant edges from the dependency graph.
After this change abidump files (.sdump, .lsdump, .abidiff) are built
only if (1) building phony target `findlsdumps` or (2) reference dump
is present under prebuilts/, which is the case for REL branches.

This improves the build time by 1.5 minutes on my machine.

Also remove HEADER_ABI_DIFFS because it has no user anymore.

Bug: 162301547
Test: (on master) m && .sdump and .lsdump is not generated
Test: (on REL br) m && .sdump, .lsdump and .abidiff are generated
Change-Id: Ie040371d8fd57f91ec6f0cd4d37483167ef9c445
2020-08-06 09:04:45 +00:00
Treehugger Robot d044e8452f Merge changes from topic "soong_sanitize_fix" am: 8c69770ff0
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1385288

Change-Id: Ib9bacd5827c479fc9bc2e01d081ab6c4ddab4634
2020-08-06 01:08:08 +00:00
Inseob Kim 9803127f58 Move image mutator things to image.go am: e498dd9ac3
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1387116

Change-Id: I634265c598700fc1f7c6c91bf59669bbd5444625
2020-08-06 01:08:06 +00:00
Treehugger Robot 8c69770ff0 Merge changes from topic "soong_sanitize_fix"
* changes:
  Separate lists of soong sanitize modules
  Move image mutator things to image.go
2020-08-06 00:48:16 +00:00
Colin Cross f6c453a3fd Merge "Add libraryDependencyTag to track dependencies on static and shared libraries" am: 2eddd06879
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1378332

Change-Id: I9277e0586e43230bc7af5366f3883f5c8d3d31e7
2020-08-06 00:16:56 +00:00
Colin Cross 2eddd06879 Merge "Add libraryDependencyTag to track dependencies on static and shared libraries" 2020-08-06 00:06:28 +00:00
Elliott Hughes 37e860e37f Merge "Switch to toybox patch(1)." am: 325d73f260
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/870109

Change-Id: I9dcb3f2a2e7c879acddb2958034faaa62f2b0bdc
2020-08-05 23:33:09 +00:00
Treehugger Robot 0c5157f2e6 Merge "[cc/coverage] Pass directory to -fprofile-instr-generate" am: cf187bcbbc
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1391163

Change-Id: I976ca4ae19e517616a481dc936dd33b637bcfa85
2020-08-05 23:32:33 +00:00
Elliott Hughes 325d73f260 Merge "Switch to toybox patch(1)." 2020-08-05 23:30:14 +00:00
Treehugger Robot cf187bcbbc Merge "[cc/coverage] Pass directory to -fprofile-instr-generate" 2020-08-05 23:10:00 +00:00
Matthew Maurer b32f2b3412 Merge "Make Rust test harness optional for test binaries" am: 53bba5de8a
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1295711

Change-Id: I097ee109f599c8386e0c72459155b352192edd13
2020-08-05 21:44:28 +00:00
Matthew Maurer 53bba5de8a Merge "Make Rust test harness optional for test binaries" 2020-08-05 21:26:45 +00:00
Ivan Lozano 89d57368b9 Merge "Make source_stem required for Rust SourceProviders" am: d13b308960
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1390938

Change-Id: Ib5f8f63e06743dfd95ca2d2360e81e5eeed921e5
2020-08-05 19:48:14 +00:00
Ivan Lozano d13b308960 Merge "Make source_stem required for Rust SourceProviders" 2020-08-05 19:38:19 +00:00
Steven Moreland 9cd45782e5 allow empty phony modules
In order to allow reservation of names in the build system.

Bug: 162275959
Test: build only
Change-Id: I8f1622bdbebe794dafd5c341d861da5468cd52bd
2020-08-05 19:31:54 +00:00
Ivan Lozano 66b577a9de Merge "Add support for custom bindgen binaries." am: 80668b07d2
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1391296

Change-Id: I3d58c5be64084895f967b2688a20729a6c6a98b7
2020-08-05 19:12:14 +00:00
Liz Kammer a4278eba24 Merge "Add property api_levels_jar_filename to droidstubs" am: 9c55b0efa2
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1388544

Change-Id: I4da722343ed53baedf40ad7661fc1f017051a70d
2020-08-05 19:12:02 +00:00
Ivan Lozano 80668b07d2 Merge "Add support for custom bindgen binaries." 2020-08-05 19:08:20 +00:00
Liz Kammer 9c55b0efa2 Merge "Add property api_levels_jar_filename to droidstubs" 2020-08-05 19:05:13 +00:00
Thiébaud Weksteen 285318aa0f Merge changes I275f0463,Ie736d7eb am: 6090575f63
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1391076

Change-Id: I9310ba5e7945f7f2a34f7aebe1ba98b35b699723
2020-08-05 19:02:42 +00:00
Thiébaud Weksteen 6090575f63 Merge changes I275f0463,Ie736d7eb
* changes:
  rust: handle modules with same crate_name
  rust: validate existence of library source
2020-08-05 18:50:06 +00:00
Bill Peckham 25bd47b9fa Merge "Adds 'kernel/' to vendor proprietary dirs." am: 6ed589039c
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1387788

Change-Id: I5ef08b2230ed2dbc78f3dde62bd3cca8705789e9
2020-08-05 18:30:07 +00:00