Commit Graph

17033 Commits

Author SHA1 Message Date
Dan Willemsen 1bdbdec5a0 Take into account RAM usage for multiproduct_kati
Apparently we have some instances of Soong taking 16GB of RAM. While
that's a problem to solve on it's own, make multiproduct_kati's
auto-parallelism detection smarter by taking into account the total RAM
on the machine instead of just the number of CPUs.

Bug: 146925549
Test: check soong.log for autodetected parallelism values
Change-Id: Ice34c2cf73ea4f8f235521cbefc8654a79a04eef
2019-12-27 09:54:11 -08:00
Dan Willemsen 2bb82d0011 Add TotalRAM detection
Colin originally wrote this for the highmem pool:

https://android-review.googlesource.com/c/platform/build/soong/+/1168271

But while that's a larger change, this is useful to just be in the logs,
and for use in multiproduct_kati to better limit the number of
concurrent jobs.

Test: check soong.log on linux and mac
Change-Id: I0518d303a220d775f8d78dba9f660b2954e68e3e
2019-12-27 09:43:49 -08:00
Kiyoung Kim fb1e5fbf13 Merge "Move compat symlinks from makefile to soong" 2019-12-27 00:08:06 +00:00
Jooyung Han 4791cb507b Move compat symlinks from makefile to soong
Two APEXes (i18n, art) requires compat symlinks and these symlinks are
created via ld.config.txt module's POST_INSTALL_CMDS.

  $ m installclean
  $ m ld.config.txt
  $ tree $OUT/system

.../system
├── bin
│   ├── dalvikvm -> /apex/com.android.art/bin/dalvikvm
│   └── dex2oat -> /apex/com.android.art/bin/dex2oat
├── etc
│   └── ld.config.txt
└── usr
    └── icu -> /apex/com.android.i18n/etc/icu

By the way, ld.config.txt is going to removed since linkerconfig
generates it on device.

So, we're moving symlink creation from ld.config.txt(rootdir/Android.mk)
to APEXes' POST_INSTALL_CMDS.

  $ m installclean
  $ m com.android.i18n
  $ tree $OUT/system

.../system
├── apex
│   └── com.android.i18n.apex
└── usr
    └── icu -> /apex/com.android.i18n/etc/icu

  $ m installclean
  $ m com.android.art
  $ tree $OUT/system

.../system
├── apex
│   └── com.android.art.debug.apex
├── bin
│   ├── dalvikvm -> /apex/com.android.art/bin/dalvikvm
│   └── dex2oat -> /apex/com.android.art/bin/dex2oat

Bug: 143192278
Test: m && boot (since these two apexes are bootstrap apexes)
Change-Id: Ib29ea9f9ac40e74f78cd530e7daef8c51292fd24
2019-12-24 14:00:47 +09:00
Treehugger Robot 29d1336d11 Merge "Run extractor for Go files in build/soong directory" 2019-12-23 20:55:25 +00:00
Sasha Smundak ca54088aa8 Run extractor for Go files in build/soong directory
Bug: 137798757
Test: run cd build/soong &&  ../../prebuilts/build-tools/linux-x86/bin/go_extractor --goroot=$PWD/../../prebuilts/go/linux-x86 --rules=vnames.go.json --canonicalize_package_corpus --output $OUT_DIR/soong/golang.kzip ./...
Change-Id: Ib1edde4bf36e6b0683f49da0986f243690c81d6b
2019-12-23 11:34:15 -08:00
Paul Duffin e2bc36f12c Merge "Generate .srcjar for prebuilt_stubs_sources" 2019-12-23 13:32:08 +00:00
Jooyung Han 97e7279cc6 Merge "apex: drop `java_import` support" 2019-12-23 09:14:32 +00:00
Jooyung Han 5a80d9f352 apex: drop `java_import` support
java_import doesn't support dex jar which is necessary to be packaged in
an APEX. Supporting this would require non-trivial work. By the way
java_import is not used and there is a workaround.(defining java_library
and static-linking with it)

We drop the support for `java_import` in APEX.

Bug: 139175488
Test: m (soong tests amended)
Change-Id: I924386571079090c701276d87f665ce7fbb6f074
2019-12-23 06:46:39 +00:00
Jiyong Park d061df9ef3 android_*_stubs_current modules are moved to frameworks/base
The modules were moved from developement/build to frameworks/base.
In addition, they no longer uses 'sdk_version:"none"'. Reflect that
change here.

Bug: N/A
Test: m

Exempt-From-Owner-Approval: cherry-pick from internal

Merged-In: I6f69f7cb64a6ed3c2032a116500436eabe562dca
(cherry picked from commit 12cc126760)
Change-Id: I6f69f7cb64a6ed3c2032a116500436eabe562dca
2019-12-22 17:20:27 +09:00
Treehugger Robot 7e143af6ee Merge "Add support for java_test_helper_app." 2019-12-22 05:21:37 +00:00
Dario Freni 6f3937cb29 Add support for java_test_helper_app.
Bug: 142712057
Bug: 138429615
Test: apex_test passes, tried building an apex with a test app.
Change-Id: Ib75bd48042d8b1fe102038bef08cd1be708e996d
2019-12-21 08:50:12 +00:00
Treehugger Robot 727173ecca Merge "Create product variant for product partition" 2019-12-21 00:56:45 +00:00
Jaewoong Jung dcee89cad6 Merge "Make apex.overrides overridable by override_apex." 2019-12-20 18:20:50 +00:00
Paul Duffin a0843f6a6f Differentiate between cc library and binary
The cc library and cc binary (and other cc module types) are all
instances of cc.Module. So, to differentiate between them and make
sure that only appropriate library instances can be added to
native_shared/static_libs this adds a special sdkMemberTypes field to
Module which if set specifies the SdkMemberTypes the module supports.
If it is not set then the module type cannot be used in the sdk at all.

Corrects an issue with one of the tests where a prebuilt cc
library was added to the sdk instead of a source cc library.

Adds a new test to ensure that cc_library_(shared|static)_host module
types work with the sdk as well and another test to ensure that
cc_library can be used as either.

Bug: 142918168
Test: m checkbuild
Change-Id: I359cdbdd15328ca571f276d2b6ce9a229ebb2c86
2019-12-20 15:45:17 +00:00
Paul Duffin 6c26dc7392 Dedup cc library module type registration
Test: m checkbuild
Bug: 146540677
Change-Id: I6685338550f587212e70f3eba8d4342b66f566e2
2019-12-20 15:22:52 +00:00
Martin Stjernholm d732beed90 Merge "Add script for building all target arch's needed in AML (Android Mainline) prebuilts." 2019-12-20 11:53:33 +00:00
Martin Stjernholm c1ecc43aec Add script for building all target arch's needed in AML (Android Mainline)
prebuilts.

This runs Soong in skip-make mode, using normal in-make mode only to query
platform versions.

The same ${OUT_DIR} cannot be used for both skip-make and in-make builds,
because Soong generates a smaller build.ninja file in in-make builds where
many build targets are expected to be provided by the mk files. Thus this
script avoids using ${OUT_DIR} if it's an in-make build, defaulting instead
to out-aml/.

The script is based on build-ndk-prebuilts.sh, but uses a separate Soong
variable Aml_abis to enable the appropriate target architectures for
Mainline modules. Aml_abis is very similar to Ndk_abis, except "armeabi-v7a"
is used instead of "armeabi", which is necessary to match prebuilt
dependencies, e.g. for LLVM.

Test: build/soong/scripts/build-aml-prebuilts.sh libart libdexfile_external
  (verify that libraries for arm, arm64, x86, x86_64 are built)
Test: build/soong/scripts/build-aml-prebuilts.sh \
  out-aml/soong/.intermediates/external/conscrypt/conscrypt-module-sdk/android_common/conscrypt-module-sdk-current.zip
  (verify that the zip file contains libconscrypt_jni.so's for all four arches)
Test: build/soong/scripts/build-aml-prebuilts.sh com.android.art.{release,debug,testing,host}
  (verify that the build completes)
Test: Two identical build/soong/scripts/build-aml-prebuilts.sh runs after each other
  (verify that the 2nd run completes both Soong and ninja steps quickly without any building)
Change-Id: I35712f9f8f0b1cbb77107314c5927c6720e6c3bf
2019-12-20 11:53:21 +00:00
Mathew Inwood 431b8a2fdd Merge "Update build rule for updated script." 2019-12-20 09:33:42 +00:00
Justin Yun 5f7f7e8a39 Create product variant for product partition
When PRODUCT_PRODUCT_VNDK_VERSION is set to 'current', product
modules are enforced to use only VNDK libs from the system partition
as BOARD_VNDK_VERSION does to vendor partition.

Modules with 'vendor_available: true' create product variant as well
as core and vendor variants. The product variant as an image variant
is used for the modules in /product or /system/product.

It must not affect the current build behavior without
PRODUCT_PRODUCT_VNDK_VERSION set.

Bug: 134099726
Bug: 138966004
Bug: 144534640
Test: build without PRODUCT_PRODUCT_VNDK_VERSION set
Change-Id: I4d3585c110d84493e45bf76d550dc240bb26137f
2019-12-20 17:56:08 +09:00
Paul Duffin 9b478b0831 Generate .srcjar for prebuilt_stubs_sources
Changes prebuilt_stubs_sources to generate a .srcjar from its input
instead of just exposing the srcs it is given. This ensures that it can
be used as a drop in replacement for a droidstubs module.

Updates the test for prebuilt_stubs_sources to be more representative
of the actual use made of it by sdk snapshot which outputs a directory
not a glob pattern. Added some documentation of the
prebuilts_stubs_sources srcs property to make it clear that it is
supposed to be a set of directories.

Extracts common code from sdk/testing.go for normalizing path/paths
for testing.

Bug: 143678475
Test: m conscrypt-module-sdk conscrypt-module-host-sdk conscrypt-module-test-sdk
      unzip those in place of external/conscrypt
	  build core-current-stubs-source which expects it to provide a .srcjar.
Change-Id: I8204a022557a9b0b45e19eac79ecba98ff95213d
2019-12-20 08:13:45 +00:00
Treehugger Robot 120d73fe4f Merge "Support java_sdk_library as java_libs of apex" 2019-12-20 05:16:31 +00:00
Treehugger Robot 076651765a Merge "Generate Compdb file in the output directory" 2019-12-20 04:01:00 +00:00
Jaewoong Jung 7abcf8ead8 Make apex.overrides overridable by override_apex.
Test: apex_test.go
Change-Id: Id47e5e5bec45ec1ada68f9d2d806585c5141a2f9
2019-12-19 17:32:06 -08:00
Treehugger Robot 4565aba98c Merge "Dedup cc build components registration" 2019-12-20 01:27:00 +00:00
Treehugger Robot de85fa9f5f Merge "Improve cc tests by adding fuzzer mutators" 2019-12-20 01:26:01 +00:00
Treehugger Robot 7d211f1793 Merge "Improve cc tests by adding sanitizer mutators" 2019-12-20 01:25:56 +00:00
Treehugger Robot 82f4f99780 Merge "Dedup prebuilt mutator registration" 2019-12-20 01:25:51 +00:00
Treehugger Robot 24de9ea201 Merge "Dedup registration for cc default test config" 2019-12-20 01:25:42 +00:00
Elliott Hughes 936a18d8da Merge "Switch to toybox realpath." 2019-12-20 00:27:10 +00:00
Changyeon Jo 96279252d7 Generate Compdb file in the output directory
This change modifies the location of generated compdb file from
$ANDROID_BUILD_TOP to where compdbOtuputProjectsDirectory points to,
under $OUT_DIR.  This is especially required for the build environment
that maps $ANDROID_BUILD_TOP as read-only, such as Gitc.

Test: m SOONG_GEN_COMPDB=1 nothing
      examine generated compile_commands.json
Signed-off-by: Changyeon Jo <changyeon@google.com>

Change-Id: I86700f89b613a443a33982ecd5501b0f68a62be7
2019-12-20 00:10:02 +00:00
Ulyana Trafimovich 3b56e37f19 Merge "Rollback to the legacy way of computing boot image locations." 2019-12-20 00:06:13 +00:00
Jooyung Han 58f26aba4e Support java_sdk_library as java_libs of apex
When a java_sdk_library module is added, both impl jar and permission
xml files are packaged together.

For example, when a java_sdk_library "foo" is listed, following two
entries will be in an APEX package.

/javalibs/foo.jar
/etc/permissions/foo.xml

Bug: 145474221
Test: m com.android.cronet
      deapexer list com.android.cronet.apex

Change-Id: If5883c02255e9309f20810b1532d3fbe73bf4e95
2019-12-20 08:50:34 +09:00
Treehugger Robot 6b295312f6 Merge "Remove unused testWritablePath and associated methods" 2019-12-19 23:14:55 +00:00
Treehugger Robot f5f1ecb746 Merge "Extend RegistrationContext to support pre/post deps mutators" 2019-12-19 23:06:32 +00:00
Treehugger Robot b237de1914 Merge changes I7e9440a0,I7f1a4b64,If5d6fdac
* changes:
  Dedup cc prebuilts module type registration
  Detect registration of duplicate module/singleton types
  Dedup prebuilt apis module type/mutator registration
2019-12-19 22:43:02 +00:00
Colin Cross f6b213f050 Merge "Add helper functions for getting output files from modules" 2019-12-19 22:36:04 +00:00
Colin Cross 35a261de1f Merge "Make sh_binary a HostToolProvider" 2019-12-19 22:35:48 +00:00
Treehugger Robot 4326e7375c Merge changes I982fcb8d,I17f24fe1,If5740476
* changes:
  Dedup system modules and sdk library module type registration
  Dedup java genrule module type registration
  Dedup droiddoc module type registration
2019-12-19 22:28:44 +00:00
Ulya Trafimovich 3380531d80 Rollback to the legacy way of computing boot image locations.
This fixes JIT-zygote builds (b/146462581). The fix is a workaround:
boot image location is computed from boot image filename instead of
using the pre-computed location from the module dexpreopt config.

Currently parts of the dexpreopt module config use boot image name
that is hard-coded in Makefiles, and other parts use boot image name
that is generated by Soong. This is exacerbated by the fact that
"default" boot image config in JIT-zygote builds consists partly of
the "apex" boot image, and partly of "boot" boot image.

Test: walleye_jitzygote-userdebug builds.

Bug: 146462581
Change-Id: I9a91fc48e54d7d43abec2cb2b5a11e3581db380b
2019-12-19 22:17:08 +00:00
Paul Duffin 3eb9dfe136 Remove unused testWritablePath and associated methods
Bug: 143678475
Test: m checkbuild
Change-Id: I4855f97499d3e4f3f5024fd67069b1bb2f9c349f
2019-12-19 20:38:53 +00:00
Hamzeh Zawawy 5c5cc67ee2 Merge "Adding capability to granular disable of fuzzer" 2019-12-19 19:54:07 +00:00
Paul Duffin 036e7001ae Dedup cc build components registration
Effectively adds the cc_defaults module type to the set that are
registered for tests that rely on cc default deps so needed to remove
a few references of that.

Test: m checkbuild
Bug: 146540677
Change-Id: I9df3d33a0cf3f6b22d270efed8366d0183eccaec
2019-12-19 19:19:40 +00:00
Paul Duffin 075c417837 Improve cc tests by adding fuzzer mutators
This fixes an issue that arises when deduping cc build component
registration code.

The fuzzer and fuzzer_deps post deps mutators were not previously
added when running cc tests. That meant the tests were not actually
testing the same behavior as at runtime.

Adding the mutators breaks the TestFuzzTarget test as the mutator
changes the variant of the fuzz_smoke_test that is created and so the
test fails to find it.

This fixes the test by correct the variant name.

Test: m checkbuild
Bug: 146540677
Change-Id: I9d2be0023affb7722f0399663fa18a2a5fb610d8
2019-12-19 19:19:40 +00:00
Paul Duffin d686791c16 Improve cc tests by adding sanitizer mutators
This fixes an issue that arises when deduping cc build component
registration code.

The sanitize_runtime_deps and sanitize_runtime post deps mutators were
not previously added when running cc tests. That meant the tests were
not actually testing the same behavior as at runtime.

Adding the mutators breaks the TestFuzzTarget test as the mutator adds
libclang_rt.ubsan_standalone-aarch64-android as a dependency of libc++
and the former is not available.

This fixes the test by adding the missing dependency as a cc prebuilt
shared library.

Test: m checkbuild
Bug: 146540677
Change-Id: Ie13c7e6fcefef7d9cb1cc5364be3dc563ce40de5
2019-12-19 19:19:40 +00:00
Paul Duffin 0c4979bbdf Dedup prebuilt mutator registration
Test: m checkbuild
Bug: 146540677
Change-Id: I9f8a11adf61fe6f352ec1eabc7c5bac332348ba2
2019-12-19 19:19:40 +00:00
Paul Duffin 77980a8bb9 Dedup registration for cc default test config
The cc.GatherRequiredDepsForTest() method returns some default module
definitions that are required when using cc module types like
cc_library. Previously, the registration of the module types and
mutators needed to process those default definitions was duplicated
in the test config initialization.

This change removes that duplicated code and replaces it with calls
to cc.RegisterRequiredBuildComponentsForTest(ctx) which registers all
the required build components.

Test: m checkbuild
Bug: 146540677
Change-Id: I80b6913c5691ff164ce9d308b9e1da24940f2d42
2019-12-19 19:19:40 +00:00
Paul Duffin 2ccaffd1d7 Extend RegistrationContext to support pre/post deps mutators
Test: m checkbuild
Bug: 146540677
Change-Id: If5b30f1124ce5671c26ec76ca1f7b029841e20b1
2019-12-19 19:19:40 +00:00
Paul Duffin 59986b23cc Dedup cc prebuilts module type registration
Test: m checkbuild
Bug: 146540677
Change-Id: I7e9440a075ef9c683729ed83e0033ab529fe4ac0
2019-12-19 19:19:40 +00:00
Paul Duffin 43dc1cc2bc Dedup system modules and sdk library module type registration
Test: m checkbuild
Bug: 146540677
Change-Id: I982fcb8d723e8e2f7679434051ddc427d4fbd7be
2019-12-19 19:19:40 +00:00