Commit Graph

219 Commits

Author SHA1 Message Date
sophiez d3e2742ad2 NDK API coverage
Update OWNERS file under build/soong/scripts to include coverage team as owner of related scripts.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only
Change-Id: I6150d5bda0d59828a985cd2ab5dd0aa38a7987c3
2021-03-29 20:29:35 +00:00
Treehugger Robot 7ebe464748 Merge "Build statsd-module-sdk-for-art" 2021-02-24 01:08:28 +00:00
David Srbecky bb96b56b0d Merge "Set block-size for XZ mini-debug-info compression." 2021-02-24 00:43:12 +00:00
Eric Holk ff0529d209 Build statsd-module-sdk-for-art
Bug: 178236337
Test: build-mainline-modules.sh
Change-Id: I51dab13416be8ea1cd65234d560d84ca89b53e13
2021-02-23 22:49:57 +00:00
Treehugger Robot 7eacdbf8a6 Merge "Add non-fatal mode for verify_uses_libraries check." 2021-02-23 20:45:42 +00:00
Paul Duffin ece454400d Merge changes from topic "hiddenapi_additional_annotations"
* changes:
  Sort hiddenapi monolithic files by signature
  Remove duplicates in monolithic hidden API files
  Remove implicit dependency from <x> -> <x>-hiddenapi
2021-02-23 12:48:24 +00:00
Ulya Trafimovich 8c35fcf4d5 Add non-fatal mode for verify_uses_libraries check.
The new mode is enabled with environment variable
RELAX_USES_LIBRARY_CHECK. If the variable is set to true, then a
verify_uses_libraries check failure does not fail the build, instead it
sets a special compiler filter "extract" for dexpreopt, which means that
the DEX file will be extracted, but it won't be compiled to native code.
Class loader context will be set to empty in this case (not &, as it is
going to be deprecated soon).

If the variable RELAX_USES_LIBRARY_CHECK is unset or set to something
other than "true", then the old behaviour of the verify_uses_libraries
check is preserved.

The intended use case for this flag is to have a smoother migration path
for the Java modules that need to add <uses-library> information in
the build files. The flag allows to quickly silence build errors. This
flag should be used with caution and only as a temporary measure, as it
masks real errors and affects performance.

verify_uses_libraries check is reworked so that it writes the error
message to a status file (which is used instead of the former timestamp
file). Currently the stored error message is not used, but it may be
used later to produce a warning. Dexpreopt command checks if the status
file exists and is nonempty; if that is the case, then compiler filter
is set to "extract".
Bug: 132357300

Test: Manually add some mismatch between the libraries in the Android.bp
      and Android.mk files for dexpreopted apps, build with
      RELAX_USES_LIBRARY_CHECK=true and obsserve that the build doesn't
      fail and they are compiled with compiler-filter "extract".
      Unset RELAX_USES_LIBRARY_CHECK and observe that the build fails.

Change-Id: Ibb5d993a25b1df1d2e70b7d5aafc6997f9d64e67
2021-02-22 15:41:03 +00:00
Anton Hansson a6bf4a189c Merge "Disallow using error-prone merge strategies" 2021-02-19 15:27:29 +00:00
Anton Hansson 771fd524f2 Disallow using error-prone merge strategies
DO NOT MERGE / RESTRICT AUTOMERGE and its cousin DO NOT MERGE
ANYWHERE are almost always misused. Make it an error to upload
commits with those merge strategies.

Bug: 156896646
Test: repo upload -n (with DNM and RM commits)
Change-Id: Iae3540fa35dbd7b87a14b53a20be0cd23a206f3a
2021-02-19 13:05:54 +00:00
Treehugger Robot 999314fd98 Merge "Add bloaty_merger" 2021-02-19 09:45:39 +00:00
Eric Holk 277b0e7f41 Make statsd apex available as a prebuilt
Bug: 178236337
Test: build-mainline-modules.sh
Change-Id: I807eca7c5296227f4b593fc9f52d29bd20e78645
2021-02-18 17:39:51 +00:00
Thiébaud Weksteen 713db480cd Add bloaty_merger
This script will be used to merge the csv outputs from bloaty into a
single proto file.

Bug: 172339795
Test: atest --host bloaty_merger_test
Change-Id: I9a3c12d7b0c4b5e347fec04e6758bcc1fbc8ab40
2021-02-18 10:25:25 +01:00
Bob Badour 1bf6765def Add license to hiddenapi
Test: m all
Change-Id: Ic61a74d94bea0e4de2a47be7e15fcf874fb397c3
2021-02-17 09:46:26 -08:00
Paul Duffin 2c36f24082 Sort hiddenapi monolithic files by signature
Adds a new --key_field option to merge_csv.py which specifies the name
of the field that should be used to sort the input. If specified it
causes that field to be the first in each row and performs the merge
operation of a merge sort on the input files. That assumes that each
input file is already sorted into the same order.

Modifies the rules that use merge_csv.py to pass in:
    --key_field signature
to sort the rows by signature.

Bug: 180387396
Test: Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change other than
      changing the order.
Change-Id: Idcd5f0fea373b520b604889e1c280f21ed495660
2021-02-16 17:38:08 +00:00
Paul Duffin 031d8693b3 Allow explicitly specified additional annotations for hiddenapi
Adds the hiddenapi_additional_annotations to allow a library to list
the libraries that provided additional hiddenapi related annotations
for a library.

Modifies merge_csv.py so it can process multiple zip files at the same
time and uses that to merge the embedded .uau files from a module and
those it depends upon.

Bug: 180102243
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: I796520021c7357398a9e2a09f1029e4a578b05b3
2021-02-16 13:28:26 +00:00
Eric Holk 7529f06707 Add statsd exports to be consumed by unbundled modules
This is needed first for ART to enable statsd metrics reporting, but
this will be needed by more modules as they become unbundled.

Test: build-mainline-modules.sh
Bug: 178500083
Change-Id: I5af0b8db7a24c86ffd3f34732ac747a4fc6a7a55
2021-02-11 19:02:40 +00:00
Treehugger Robot b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Paul Duffin fdada68320 Move hiddenapi tools used by build/soong from frameworks/base
Also, creates a python_binary_host module for generate_hiddenapi_lists
and uses that when constructing the build rule rather than using the
file directly.

Bug: 177317659
Test: m droid
      Verified that hiddenapi files (both aggregated ones and for the
      individual modules) are not affected by this change.
Change-Id: Ia11bb203ce5a74740d35f1b7e86716e15aad336e
2021-02-09 10:46:36 +00:00
Colin Cross 69f0a24762 Pass list of module libraries to gen_ndk_backed_by_apex.sh
Running find on the entire module directory can lead to flaky failures
when a directory is modified by another rule while find is traversing
it.  Pass a list of libraries in the module into the script instead
of finding them on disk.

Fixes: 178509901
Test: TARGET_BUILD_APPS=com.android.runtime dist
Change-Id: Ie4bf288f0e3d99cbbed5ca8770687a9bbe5377e4
2021-02-08 16:52:07 -08:00
Bob Badour 02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
David Srbecky 412322f2a2 Set block-size for XZ mini-debug-info compression.
This allows the compression to be done on multiple threads
and more importantly, it allows libunwindstack to decompress
only the needed blocks, which reduces unwind time and memory.

Bug: 110133331
Test: verify lazy decompression in libunwindstack
Change-Id: Ia3f597928edcbc85acdf05cb5c83ac7fced39148
2021-02-03 20:48:50 +00:00
Martin Stjernholm bd396b6d8a Force ART modules to be built from source here, regardless of default.
When we start switching vanilla platform builds to use ART prebuilts,
this script should still build from sources.

Test: TH
Bug: 172480615
Change-Id: I04cb61b484855d22f22fd9b82f6abb30eb5665c3
2021-02-02 22:24:00 +00:00
Pete Bentley 9436be4321 Merge "Revert "Delete build-aml-prebuilts.sh"" 2021-01-29 16:19:57 +00:00
Pete Bentley 237de41b38 Revert "Delete build-aml-prebuilts.sh"
Revert submission 1535006-bap

Reason for revert: Looks like it breaks the build as aapt2 depends on the build_number.txt this script was generating.  Error log below.
Reverted Changes:
I0f6b34310:Delete build-aml-prebuilts.sh
I4eae4153a:Replace use of build-aml-prebuilts

Change-Id: I7f5f1b3047f3d71eb8ac7bec95c5916425d5d8f2
FAILED: ninja: 'out/aml/soong/build_number.txt', needed by 'out/aml/soong/.intermediates/frameworks/base/tools/aapt2/aapt2/linux_glibc_x86_64/unstripped/aapt2', missing and no known rule to make it
2021-01-29 15:14:42 +00:00
Anton Hansson 30d6f68b1a Merge "Delete build-aml-prebuilts.sh" 2021-01-29 09:11:16 +00:00
Sophie Zheng 9b6596bb60 Merge "Revert "Revert "Automated NDK API coverage used by Mainline modules build integration.""" 2021-01-26 00:13:46 +00:00
sophiez 6bde0b5f9d Revert "Revert "Automated NDK API coverage used by Mainline modules build integration.""
This reverts commit 556b2ad77e.

Original change-id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15

Reason for revert: Add excutable permission to the gen_ndk_backedby_apex.sh.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Change-Id: Ib587ba200cd7f2f61d478452b43c329d72b2de06
2021-01-25 19:52:10 +00:00
Riddle Hsu f17b07fc52 Merge "Revert "Automated NDK API coverage used by Mainline modules build integration."" 2021-01-25 06:15:29 +00:00
Riddle Hsu 556b2ad77e Revert "Automated NDK API coverage used by Mainline modules build integration."
This reverts commit 87ff51db84.

Reason for revert: build breakage b/178327796

Change-Id: I51c56938540794097eda579ebe47fa5c0a4d8c12
2021-01-25 01:44:34 +00:00
Sophie Zheng 0bf77198b5 Merge "Automated NDK API coverage used by Mainline modules build integration." 2021-01-22 22:15:21 +00:00
sophiez 87ff51db84 Automated NDK API coverage used by Mainline modules build integration.
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Change-Id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15
2021-01-21 18:08:50 +00:00
Paul Duffin fe6147eb26 Merge "Export dex implementation jars from prebuilt_apex" 2021-01-19 14:34:29 +00:00
Paul Duffin 064b70c918 Export dex implementation jars from prebuilt_apex
Dexpreopt and boot jars package check all require access to dex
implementation jars created for java_library and java_sdk_library. They
were available when building from source but not when building from
prebuilts, even though they are embedded within the .apex files that
are referenced from prebuilt_apex.

This changes adds support to prebuilt_apex to export the dex
implementation jars and updates java_import to use those exported dex
implementation jars.

In a source build dexpreopt/boot jars package check access the apex (or
platform) specific variant of a java_library, e.g. core-oj, from which
it retrieves the dex implementation jar path.

After this change in a prebuilt build dexpreopt/boot jars package check
behave in the same way except in this case they retrieve the dex
implementation jar path from the apex (or platform) specific variant of
the java_import, e.g. core-oj.

The work to export files from a `.apex` file for use by other modules
is performed by a new `deapexer` module type. It is not used directly
in an `Android.bp` file but instead is created implicitly by
`prebuilt_apex`,

In order to do that this contains the following changes:
* Adds a new `dexapexer` module type to handle the exporting of files
  from the `.apex` file.
* Adds an exported_java_libs property to prebuilt_apex to specify the
  set of libraries whose dex implementation jars need exporting.
* Creates apex specific variants of the libraries listed in the
  exported_java_libs property.
* Adds the set of exported files to the ApexInfo to make them available
  to the apex specific variants.
* Prevents the prebuilt_apex variants from being merged together as
  they will not be compatible.
* Modifies java_import to use the exported file for variants of a
  prebuilt_apex.
* Adds a ninja rule to unpack (using deapexer) the contents of the
  prebuilt_apex's apex file, verify that the required files are present
  and make them available as outputs for other rules to use.
* Some minor refactorings to support these changes.
* Adds tests to cover prebuilt only, prebuilt with source preferred,
  and prebuilt preferred with source.

Test: m nothing
Bug: 171061220
Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
2021-01-15 18:14:10 +00:00
Colin Cross 0dd19125eb Merge "manifest_fixer: Set targetSdkVersion to '16' for libraries" 2021-01-15 17:13:31 +00:00
Anton Hansson 35e3fd0666 Delete build-aml-prebuilts.sh
Its usecase can now be supported natively by soong_ui via the
--skip-kati flag + the mainline_sdk product. Migrate
build-mainline-modules.sh to use this new approach, and delete the
script.

Bug: 174315599
Test: build/soong/scripts/build-mainline-modules.sh
Change-Id: I0f6b34310df51ddf060b2912d43487f4d37dd01b
2021-01-15 10:23:24 +00:00
Andrew Wheeler 18dcd046ad manifest_fixer: Set targetSdkVersion to '16' for libraries
manifest_fixer.py sets a lib's targetSdkVersion to '15' if it is
not explicitly set.  But ManifestMerger will add dangerous
permissions CALL_LOG_READ/CALL_LOG_WRITE if the following
conditions are met:

    - The app's targetSdkVersion is >16
    - A linked lib's targetSdkVersion is <16
    - The app has CONTACTS_READ/CONTACTS_WRITE permissions

This condition is triggered for some (OEM) app builds (of
Settings).  Update manifest_fixer.py to use '16' as its default
targetSdkVersion so that manifest_merge doesn't add these
dangerous permissions.

Change-Id: Id5f41f7da98c190c8d145fceba0085cf473f4c7a
2021-01-14 14:11:39 -06:00
Nicolas Geoffray c1b1663576 Add tzdata-module-test-exports to the list of targets.
Test: build-mainline-modules.sh
Bug: 142935992
Change-Id: I2e765737a971efee837daefb5af0c272f561b36d
2021-01-11 09:45:57 +00:00
Martin Stjernholm 2a9013ee17 Set the Soong config variable to keep building ART from source.
Without this Soong may run into inconsistencies due to stale prebuilts.
This amends https://r.android.com/1529658.

Test: TH build of NDK
Bug: 175918603
Bug: 172480615
Change-Id: I14680e4547d7d5f560d31ef61b71af8d159c4c53
2020-12-21 13:37:22 +00:00
Martin Stjernholm 56b584c4b2 Merge "Change default for source_build config variable to true." 2020-12-17 22:31:04 +00:00
Martin Stjernholm ed623eaed8 Change default for source_build config variable to true.
To reflect the new (temporary) true default in make.

Test: m
Bug: 172480615
Change-Id: I9a32578554650776c1d6b3920fa233c5613a05e3
2020-12-15 03:58:07 +00:00
Ulyana Trafimovich 7f487d4a86 Merge "Add MissingUsesLibraries to Soong vars for scripts that use --skip-make." 2020-12-14 22:44:17 +00:00
Martin Stjernholm 6f8fecd2e3 Stopgap fix to avoid aapt2 build errors.
Should fix build errors on mainline_modules on aosp-master. This script
still does not propagate all options to aapt2 correctly, but that will
be fixed by removing the script altogether (b/174315599).

Test: build/soong/scripts/build-aml-prebuilts.sh art-module-sdk \
      art-module-test-exports conscrypt-module-host-exports \
      conscrypt-module-sdk conscrypt-module-test-exports \
      i18n-module-host-exports i18n-module-sdk \
      i18n-module-test-exports platform-mainline-sdk \
      platform-mainline-test-exports runtime-module-host-exports \
      runtime-module-sdk
  (the failing command in
  build/soong/scripts/build-mainline-modules.sh)
Bug: 175386777
Change-Id: I81aa20b1abd177f0d2886ab9b04831437da72f2a
2020-12-14 15:12:23 +00:00
Ulya Trafimovich 37eb6c9448 Add MissingUsesLibraries to Soong vars for scripts that use --skip-make.
In platform builds missing libraries are defined with make variable
INTERNAL_PLATFORM_MISSING_USES_LIBRARIES via soong_config.mk. When Soong
is invoked with "--skip-make" parameter,
INTERNAL_PLATFORM_MISSING_USES_LIBRARIES is not picked up. As a result
the build fails because of missing dependencies.

This CL uses get_build_var to get the value of
INTERNAL_PLATFORM_MISSING_USES_LIBRARIES, then converts it to JSON list.

In the future would be better to migrate both scripts to use --skip-kati
introduced in https://r.android.com/1512613.

Bug: 175286760
Bug: 132357300
Test: forrest build for target "ndk" branch "aosp-master".
Test: Patch build-aml-prebuilts.sh and build-ndk-prebuilts.sh to dump
  their soong.variables config, observe that MissingUsesLibraries is
  coherent with INTERNAL_PLATFORM_MISSING_USES_LIBRARIES. Start running
  each script and observe that they don't fail early due to missing
  dependencies.
Change-Id: I5dbc1d3ea7a64de1e8be6332003acc940c2e6a76
2020-12-14 15:03:40 +00:00
Martin Stjernholm 4c13f58729 Add i18n-module-host-exports to the SDK list.
Necessary only when platform start using i18n prebuilts, but this helps
CI testing.

Test: build/soong/scripts/build-mainline-modules.sh
Bug: 172480615
Change-Id: Ic24f934ad0b4b7ac7c38d791f80df33315b936e9
2020-12-11 19:20:47 +00:00
Martin Stjernholm f371381cac Propagate the Soong config variable for enabling ART Module building.
It is set in art/build/build-art-module.sh.

Test: art/build/build-art-module.sh
Bug: 172480617
Change-Id: Idbb3478667a2265ba115922207cb6d9aa6c4c91d
2020-12-08 22:30:03 +00:00
Sophie Zheng 8baa8b50a8 Merge "Automated NDK API coverage used by Mainline modules build integration." 2020-12-03 23:30:21 +00:00
Kiyoung Kim 4ee686ddca Add 'append' command for conv_linker_config
Add 'append' command for conv_linker_config so any item can be appended
to linker.config.pb file.

Bug: 174646355
Test: buildbot_build.sh succeeded
Change-Id: Ic3f12ee5e5c0e7ca7c27d4a8cff347e93061eb5f
2020-12-03 17:09:56 +09:00
sophiez c80a2b392e Automated NDK API coverage used by Mainline modules build integration.
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Forrest run: android-build.googleplex.com/builds/forrest/run/L97800000742565481

Change-Id: I39e5aa7c74eff46aba06808642b2bd67ae45ba1e
2020-12-03 05:45:20 +00:00
Kiyoung Kim 24dfc1fbb2 Generate system stub library list at build time
List of stub libraries from system (and bionic APEX) is currently
managed in static file. This change generates the list at build time and
adds to the system config so linkerconfig can use it.

Bug: 172889962
Test: Build cuttlefish and confirmed list is generated in
/system/etc/linker.config.pb

Change-Id: Ie0400e9d1098b0de19c6a7c25f261384aadde0a3
2020-12-01 14:09:03 +09:00
Jingwen Chen cda22c9bb9 Refactor 'in_make' to mean Kati is not skipped.
In Nougat and before, Make wrote a marker file to indicate that
soong_build was invoked from Make to change certain behaviors of Soong
at build time.
https://cs.android.com/android/platform/superproject/+/android-7.1.2_r36:build/core/soong.mk;l=70-73;drc=ae18638b0406ad107b0882a02a13cdd8b92f2a4e

Things have changed, and now soong_build is invoked from soong_ui, which
supports a --skip-make configuration flag:
https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/build.go;l=31-33;drc=680387bf1d3ce7cbc77f535be7c42cec411b1687

Thus, the various remnants of 'EmbeddedInMake' and 'inMake'
configuration are misleading, since soong_build is no longer invoked
from Make. This CL refactors all instances to actually mean that
Kati is enabled (not skipped with --skip-make), and will run after
soong_build finishes, so Kati-specific behavior like the AndroidMk
singleton should run.

Test: TH presubmit

Change-Id: I576ab8e54f99f5c8ddf9feaf9a828019b279e266
2020-11-23 00:29:18 -05:00