Commit Graph

1883 Commits

Author SHA1 Message Date
Paul Duffin 91b883d541 java_sdk_library - replace .docs suffix with .stubs.source
Previously, the .docs suffix was used for droidstubs internal modules
that generate the stubs source. The name was confusing so this change
replaces the suffix with .stubs.source to clarify the purpose of those
modules.

Bug: 148080325
Test: m droid
Change-Id: I9d7a2ab19e4a331b515408912ac8f7e8b0e1b35d
2020-02-11 13:11:28 +00:00
Paul Duffin 0bdcb27970 java_sdk_library_import - Allow it to replace java_sdk_library
Previously a java_sdk_library_import module did not replace the
corresponding java_sdk_library module, even when it was marked as
prefer=true. That is because the java_sdk_library_import had an empty
set of sources.	However, the stubs modules that are created by the
java_sdk_library_import did override the stubs modules created by the
corresponding java_sdk_library module.

That created inconsistent behavior between the cases where only the
prebuilt is available and both source and prebuilt are available and
the prebuilt is preferred. e.g. assume a java_sdk_library/import module
called SDKLIB.

When both prebuilt and source modules are available for SDKLIB then
even if the prebuilt is preferred then any dependencies on the
SDKLIB module would use the source module.

This change fixes that inconsisteny by making the array of sources
non-empty.

Bug: 148080325
Test: m droid && TARGET_BUILD_APPS=Camera2 m
Change-Id: I25395e020393921735ada20c5492f27f1260f6c5
2020-02-10 10:45:29 +00:00
Paul Duffin 3b531e3c85 Merge "java_sdk_library - pass patch_module through to stubs library" 2020-02-10 10:42:34 +00:00
Nicolas Geoffray d862f0bce7 Merge "Remove UseApexImage and GenerateApexImage." 2020-02-09 22:53:56 +00:00
Anton Hansson 26bf49b0b2 Fix sdk aidl include for droiddoc
Prior to this change droidstubs modules that set sdk_version
did not get framework.aidl added to its aidl includes.

Bug: 149138391
Test: patch CL in bug && m system_aidl_test-droidstubs
Change-Id: I92ab344c8a4311e10c1e5c8ebf525fa2dc704075
(cherry picked from commit f278ca60e06da86c67f6a3865c290f8451657ce9)
2020-02-08 20:37:07 +00:00
Paul Duffin ab8da5dd53 java_sdk_library - pass patch_module through to stubs library
When building java.lang classes it is necessary to compile them using
patch_module: "java.base". This change causes patch_module to be passed
through to the java_library created to compile the stubs to allow this
to be used to generate stubs for java.lang.

Test: m droid
Change-Id: I7c27953a5d782eeedd7f25e849ab444d28e28228
2020-02-08 10:38:52 +00:00
Treehugger Robot 568a82a36e Merge "Add java_system_modules to sdk/module_exports" 2020-02-07 17:57:08 +00:00
Nicolas Geoffray 73d5983af0 Remove UseApexImage and GenerateApexImage.
These options are not used anymore.

Bug: 119800099
Test: m
Change-Id: I4568ff23ed71a5c288ed87828aed6e4bc4f8bd4d
2020-02-07 14:19:28 +00:00
Vladimir Marko 396b950a1b Merge "Support dexpreopt against the primary boot image." 2020-02-07 14:12:52 +00:00
Paul Duffin 7b81f5e9d7 Add java_system_modules to sdk/module_exports
Adds an SdkMemberType implementation for java_system_modules. It
specifies that java_system_modules can be used with sdk as well as
module_exports, and also that the libs property should be included
as transitive members in the sdk.

It also adds support for treating appropriate tagged properties in
the snapshot prebuilts module as references to sdk members so that
they are correctly transformed when creating the versioned modules.

Bug: 142940300
Test: m nothing
Change-Id: Ic10b5a6d5b92b6018334fe876f06feaf79cc55e9
2020-02-07 14:03:03 +00:00
Treehugger Robot 9be2556d90 Merge "java_sdk_library - Allow it to be replaced by prebuilt" 2020-02-07 11:16:41 +00:00
Vladimir Marko 40139d6422 Support dexpreopt against the primary boot image.
Framework and other dex files are used without image.

Test: taimen-userdebug boots when built with
      DEXPREOPT_USE_ART_IMAGE=true
Test: Check logcat for checksum verification failures.
      (Build ART with extra logging in OatFileAssistant.)
Test: Check that bootclasspath-checksums from some prebuilt
      oat files (say input.odex) contain only one image
      checksum followed by dex file checksums with
          grep -az -A1 -E '^bootclasspath-checksums$' <oat-file> | \
          xargs -0 echo | gawk '{print $2}'
Bug: 119800099
Change-Id: I65c2f247656e41f2c37df1ecb9e06af7dabab76e
2020-02-07 10:32:50 +00:00
Treehugger Robot 7300095182 Merge "java_sdk_library - Use prebuilt/prefer for unbundled app builds" 2020-02-07 05:28:35 +00:00
Paul Duffin e74ac73bc9 java_sdk_library - Allow it to be replaced by prebuilt
Previously, a java_sdk_library called "SDKLIB" would create a
prebuilt_etc module called "SDKLIB.xml" which installs the generated
XML permission file to /etc/permissions/SDKLIB.xml. That module
depended on the java_sdk_library "SDKLIB" to generate the XML file
as one of its outputs by specifying srcs: [":SDKLIB{.xml}"].

If the java_sdk_library is replaced by a prebuilt then the SDKLIB.xml
module expects the prebuilt to provide the XML permissions file which
it doesn't because that is an implementation detail and so the build
breaks.

A couple of alternative approaches were looked at to fix this. One was
to have the logic that replaced the source module with the prebuilt to
inform the source module that it was being replaced so it could disable
its created module. That lead to a dependency cycle where
    SDKLIB -> SDKLIB.xml -> SDKLIB{.xml}

Another solution was to mark dependency tags in such a way that the
prebuilt could automatically identify and disable the SDKLIB.xml
module. Similar to how the visibility code will ignore dependencies
that are tagged with ExcludeFromVisibilityEnforcementTag. That became
very convoluted.

Instead the java_sdk_library was changed so that it was not responsible
for creating the XML permissions file. Instead it created a genrule
called "gen-SDKLIB.xml" to create it and then "SDKLIB.xml" depended on
that. The java_sdk_library also depended on the genrule to make the XML
permissions file available for APEX and testing.

Some refactoring of the APEX code and tests was necessary because they
had knowledge of the internal implementation of java_sdk_library. The
refactoring insulates them a little better from those details.

Bug: 148080325
Test: m droid && TARGET_BUILD_APPS=Camera2 m
Change-Id: I597bccbb177b6b6320c3a3edeff467243230d384
2020-02-06 15:58:05 +00:00
Jiyong Park 0f80c1848a apex_available tracks static dependencies
This change fixes a bug that apex_available is not enforced for static
dependencies. For example, a module with 'apex_available:
["//apex_available:platform"]' was able to be statically linked to any
APEX. This was happening because the check was done on the modules that
are actually installed to an APEX. Static dependencies of the modules
were not counted as they are not installed to the APEX as files.

Fixing this bug by doing the check by traversing the tree in the method
checkApexAvailability.

This change includes a few number of related changes:

1) DepIsInSameApex implementation for cc.Module was changed as well.
Previuosly, it returned false only when the dependency is actually a
stub variant of a lib. Now, it returns false when the dependency has one
or more stub variants. To understand why, we need to recall that when
there is a dependency to a lib having stubs, we actually create two
dependencies: to the non-stub variant and to the stub variant during the
DepsMutator phase. And later in the build action generation phase, we
choose one of them depending on the context. Also recall that an APEX
variant is created only when DepIsInSameApex returns true. Given these,
with the previous implementatin of DepIsInSameApex, we did create apex
variants of the non-stub variant of the dependency, while not creating
the apex variant for the stub variant. This is not right; we needlessly
created the apex variant. The extra apex variant has caused no harm so
far, but since the apex_available check became more correct, it actually
breaks the build. To fix the issue, we stop creating the APEX variant
both for non-stub and stub variants.

2) platform variant is created regardless of the apex_available value.
This is required for the case when a library X that provides stub is in
an APEX A and is configured to be available only for A. In that case,
libs in other APEX can't use the stub library since the stub library is
mutated only for apex A. By creating the platform variant for the stub
library, it can be used from outside as the default dependency variation
is set to the platform variant when creating the APEX variations.

3) The ApexAvailableWhitelist is added with the dependencies that were
revealed with this change.

Exempt-From-Owner-Approval: cherry-pick from internal

Bug: 147671264
Test: m

Merged-In: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
(cherry picked from commit fa89944c79)
Change-Id: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
2020-02-06 14:45:08 +09:00
Treehugger Robot c4930248c4 Merge "sdk_version: "module_current" is supported" 2020-02-06 03:50:43 +00:00
Jaewoong Jung 387ad5c576 Merge "Add rules to handle asset resources." 2020-02-05 22:24:39 +00:00
atrost 6e12625c0f Add support for compat config in APEX.
apex module accepts PlatformCompatConfigIntf as prebuilt,
and places it in the etc folder of the apex.

Test: m
Test: flash device with dummy config in mediaprovider APEX -
the config is present
Change-Id: Ifc62cd262f6c6571c1bf6c2943879aa20877ecad
2020-02-05 13:33:50 +00:00
Paul Duffin 50061511d4 java_sdk_library - Use prebuilt/prefer for unbundled app builds
Previously, the java_sdk_library had special support for disabling the
stubs library when the build was configured to use prebuilts for sdks,
e.g. in an unbundled app build. That caused the prebuilt version of the
stubs library to be used instead. The disabling was done irrespective
of whether a prebuilt was available which prevents java_sdk_library
from being used in situations when prebuilts are not available, e.g.
when they have not been created yet.

This change moves the logic into tha java_sdk_library_import and
leverages the existing prebuilt/prefer mechanism to ensure that the
prebuilt version is used when required.

* Adds a ForcePrefer() method to Prebuilt to allow a module to forcibly
  set the value of the prefer property.
* Sets prefer true on the java_sdk_library_import and the stubs modules
  it creates when the prebuilt version should be used.
* Refactors PrebuiltJars for use by both java_sdk_library and
  java_sdk_library_import as they both need to provide access to
  prebuilts for previously released versions of the library.
* Removes disabling logic from java_sdk_library.

This will probably require some additional java_sdk_library_import
modules to be added to prebuilts/sdk/current/Android.bp.

Bug: 148080325
Test: m droid && TARGET_BUILD_APPS=Camera2 m
Change-Id: I0b5f751e82a2179a967ae64ca03dc9b9e7665c16
2020-02-05 08:09:57 +00:00
Paul Duffin 50fea3b6a7 Merge "java_sdk_library use prebuilt jars for numeric sdk_version" 2020-02-05 08:06:52 +00:00
Anna Trostanetski 026ffecb9d Merge "Build rules for compat config docs generation." 2020-02-04 16:28:40 +00:00
Mathew Inwood abd49ab4df Build rules for compat config docs generation.
We add a compat config build rule to extract the merged config, and
then update the droiddoc build rule to consume that.

Test: m -j offline-sdk-docs
Bug: 144927670
Change-Id: Ib1e85f97895c89227882e665572bda9bfc2a8cba
Exempt-From-Owner-Approval: ag/10097965 approved internally, Colin requested to patch to aosp
2020-02-04 16:28:22 +00:00
Paul Duffin a2db18fb1e java_sdk_library use prebuilt jars for numeric sdk_version
Previously, when a library that used a numeric sdk version also
referenced a java_sdk_library it would use the current version of its
API. That was dangerous as there is an expectation that an app building
against a numbered version will also be targeted at that version and
so building against a later version of the API could hide runtime
incompatibilities.

This change will use prebuilt versions of the java_sdk_library's api
when being built for a numbered sdk version.

Bug: 148080325
Test: m droid
Change-Id: I3fd416553950785a443c1702e495a96debc33331
2020-02-04 14:47:04 +00:00
Paul Duffin 1b57531573 Merge changes from topic "expose-system-test"
* changes:
  Remove legacy properties from java_sdk_library_import
  java_sdk_library_import - expose system and test stubs
2020-02-04 08:26:30 +00:00
Jaewoong Jung 411a98a917 Merge "Export the cert path for runtime_resource_overlay." 2020-02-03 17:43:46 +00:00
Paul Duffin fcfd79166c Remove legacy properties from java_sdk_library_import
The legacy usages have all been updated so the legacy properties
can be safely removed. The Libs property is kept so it can be used to
specify properties common to all scopes.

Bug: 148080325
Test: m droid
      TARGET_BUILD_APPS=Camera2 m

Change-Id: I252ebbedbb463db3c7346e86d86b5880eea76fe9
2020-02-03 15:58:08 +00:00
Paul Duffin 56d4490d59 java_sdk_library_import - expose system and test stubs
Previously, the java_sdk_library_import only exposed the public stubs.
This change adds support for exposing system and test stubs too by adding
separate structures for public, system and test scopes. The existing
properties are kept for legacy reasons (and because libs can be common
across the differents scopes).

It extracts some code that is common to both sdk library and sdk
library import.

The legacy support will be removed in a future change once all existing
usages have been switched over.

Bug: 148080325
Test: m droid
      TARGET_BUILD_APPS=Camera2 m
Change-Id: I0b26cc8af9ee044437ff3b80c1eca611816b9386
2020-02-03 15:58:08 +00:00
Jaewoong Jung 78ec5d8d6c Export the cert path for runtime_resource_overlay.
Fixes: 119811120
Test: app_test.go
Test: Converted and built IconPackFilledSystemUIOverlay
Change-Id: I3f7eae3127996773c5abf9f9bdb9d57a198ddaa3
2020-01-31 10:39:40 -08:00
Jiyong Park 50146e9c8e sdk_version: "module_current" is supported
module_* is a new API surface for OS modules (e.g. APEXes). It has
slightly bigger API surface than the system_* SDK. Specifically, APIs
with @SystemApi(client=MODULE_LIBRARIES) are added there.

Bug: 146757305
Test: m

Change-Id: I8980e50c0e3a4cd843048e0de1f638e854384f46
2020-01-31 23:01:34 +09:00
Paul Duffin 726d23c26a Improve java_sdk_library handling of test_current
When building a library against sdk_version: system_current it uses the
system stubs of any java_sdk_library that it references. Previously,
when building against sdk_version: test_current this used the public
stubs of any java_sdk_library. This change causes it to use the
test stubs instead to be consistent with the handling of system and
public.

Bug: 148080325
Test: updated the test
      ran m nothing which failed as expected
	  fixed the code
	  ran m nothing again which succeeded
Change-Id: I58ec5bd243701c5a5c75664e2bb615ce7b2f2441
2020-01-31 13:42:41 +00:00
Paul Duffin d1b3a92f05 Parameterize java_sdk_library by api scope
The java_sdk_library duplicated a lot of code related to api scopes and
also did a lot of switching on apiScope. This change eliminates that
duplication and switching by converting apiScope from an enum to a
structure containing api scope specific information.

It associates dependencies with corresponding scope by using a
scopeDependencyTag that contains a reference to the associated
apiScope.

Tested by:
1) Running the command below without the changes.
2) 'mv out/dist/apistubs $ANDROID_BUILD_TOP'
3) Applying these changes.
4) Running the command again.
5) Verifying that the out/dist/apistubs was identical to before the
   change using 'meld out/dist/apistubs $ANDROID_BUILD_TOP/apistubs'

Bug: 148080325
Test: m TARGET_PRODUCT=sdk TARGET_BUILD_VARIANT=userdebug dist sdk
      m droid
Change-Id: I0de3268b12254122e94ca83d09309b06ca2f9dbe
2020-01-30 17:26:18 +00:00
Treehugger Robot 44fc5d201f Merge "Use merge_csv python_binary." 2020-01-30 15:18:58 +00:00
Treehugger Robot f9192060bd Merge "Set LOCAL_CERTIFICATE for flattened apex APKs" 2020-01-29 17:42:51 +00:00
Ulyana Trafimovich 28e36b1455 Merge "Store uncompressed and aligned DEX files for java libs in APEXes." 2020-01-29 10:25:33 +00:00
Colin Cross 503c1d06e8 Set LOCAL_CERTIFICATE for flattened apex APKs
Leaving out LOCAL_CERTIFICATE for flattened apex APKs causes
the apkcerts.txt to have empty keys for those APKs, which
confuses the signing tools.  Set LOCAL_CERTIFICATE for them.

Also refactor the Certificate support to avoid introducing
duplicated handling for presigned certificates.

Bug: 147765187
Test: m apkcerts-list
Change-Id: Ife07661761cd5a89c9f009b8ce041db4dff9ec54
2020-01-28 14:03:08 -08:00
Ramy Medhat 5e0dbe4e3e Merge "Add support for experimentally enabling RBE support on specific rules." 2020-01-28 19:49:05 +00:00
Ramy Medhat 8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Ulyana Trafimovich 911083e3e1 Merge "Fix --boot-image argument in dex2oat command for boot image extension." 2020-01-28 17:04:38 +00:00
Ulya Trafimovich b0a2d37dcb Fix --boot-image argument in dex2oat command for boot image extension.
The primary image name should be "boot.art" in case of
"boot-framework.art", and "apex.art" in case of "apex-framework.art".
Although "boot.art" and "apex.art" are identical binaries, dex2oat
has hard-coded logic based on the image name.

This has been broken since boot image extension has been enabled for
JIT-zygote config in CL I5493e575ebf90bad1d5ad2850004d54590bbc079.

Test: compare boot-framework.art and apex-framework.art,
  they were identical before this CL, and differ after this CL.

Test: JIT-zygote config boots, steps 1-2:

  1. Temporarily enable Jit zygote in the product device config (in
    this case device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. Build and flash:
    $ lunch aosp_walleye-userdebug && m \
    && adb reboot bootloader && fastboot flashall -w

Change-Id: I98de271852ecc33feb9fd4c9b0addf0feba01856
2020-01-28 14:42:41 +00:00
Treehugger Robot 057dd35669 Merge "Abstract sdk_version string using sdkSpec type" 2020-01-28 00:07:37 +00:00
Martin Stjernholm 699cf17f53 Merge changes from topic "revert-1211982-dex2oat-soong-dep-LLLKNULXYJ"
* changes:
  Revert "Separate dexpreopt.GlobalSoongConfig to allow independen..."
  Revert "Move the Once cache for dexpreopt.GlobalConfig into the ..."
  Revert "Get the dex2oat host tool path from module dependency on..."
2020-01-27 13:47:31 +00:00
Hans Boehm e4b5342d43 Revert "Separate dexpreopt.GlobalSoongConfig to allow independen..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: Ice3990225635a737e49e9aed7373f06516fccea3
2020-01-27 13:44:24 +00:00
Hans Boehm 453bf09853 Revert "Move the Once cache for dexpreopt.GlobalConfig into the ..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I6b656afb5feaad70d958b9d38b6c6eab7b03fba1
2020-01-27 13:44:03 +00:00
Hans Boehm 7b2e6f3ed7 Revert "Get the dex2oat host tool path from module dependency on..."
Revert submission 1211982-dex2oat-soong-dep

Reason for revert: Build failures. See b/148312086.

Reverted Changes:
Ibc427a9a8: Make dex2oat(d) visible for use as implicit dexpre...
I71df11c1e: Move the Once cache for dexpreopt.GlobalConfig int...
I38317f2d5: Get the dex2oat host tool path from module depende...
I440a09dba: Separate dexpreopt.GlobalSoongConfig to allow inde...

Bug: 148312086
Bug: 145934348
Exempt-From-Owner-Approval: Plain revert
Change-Id: I0cf6fef04434fd3c0b7957ee9abdcaabeb20b9d8
2020-01-27 13:01:16 +00:00
Ulya Trafimovich f491dde8ea Store uncompressed and aligned DEX files for java libs in APEXes.
Test: aosp_walleye-userdebug boots.

Test: In internal, cherry-picked this CL, walleye-userdebug boots.

Test: In internal, ensure that DEX file wifi-service.jar in the
  com.android.wifi APEX is now stored uncompressed:

    $ zipinfo out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Archive:  out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Zip file size: 3086879 bytes, number of entries: 3
    -rw----     2.0 fat  3086304 b- stor 08-Jan-01 00:00 classes.dex
    -rw----     2.0 fat      339 b- defN 80-Feb-01 00:00 META-INF/proguard/androidx-annotations.pro
    -rw----     2.0 fat       45 b- defN 08-Jan-01 00:00 META-INF/MANIFEST.MF
    3 files, 3086688 bytes uncompressed, 3086480 bytes compressed:  0.0%

  Before the change it was:

    $ zipinfo out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Archive:  out/target/product/walleye/apex/com.android.wifi/javalib/wifi-service.jar
    Zip file size: 1241938 bytes, number of entries: 3
    -rw----     2.0 fat  3086304 bl defN 08-Jan-01 00:00 classes.dex
    -rw----     2.0 fat      339 b- defN 80-Feb-01 00:00 META-INF/proguard/androidx-annotations.pro
    -rw----     2.0 fat       45 b- defN 08-Jan-01 00:00 META-INF/MANIFEST.MF
    3 files, 3086688 bytes uncompressed, 1241526 bytes compressed:  59.8%

Bug: 148099857
Bug: 146984232

Change-Id: Ia302967b280be85665032cc870c5d5d2419586fc
2020-01-27 10:19:10 +00:00
Jiyong Park 6a927c4e6a Abstract sdk_version string using sdkSpec type
The value format that sdk_version (and min_sdk_version, etc.) can have
has consistently evolved and is quite complicated. Furthermore, with the
Mainline module effort, we are expected to have more sdk_versions like
'module-app-current', 'module-lib-current', etc.

The goal of this change is to abstract the various sdk versions, which
are currently represented in string and is parsed in various places,
into a type called sdkSpec, so that adding new sdk veresions becomes
easier than before.

The sdk_version string is now parsed in only one place 'SdkSpecFrom', in
which it is converted into the sdkSpec struct. The struct type provides
several methods that again converts sdkSpec into context-specific
information such as the effective version number, etc.

Bug: 146757305
Bug: 147879031
Test: m
Change-Id: I252f3706544f00ea71c61c23460f07561dd28ab0
2020-01-25 21:56:43 +09:00
Baligh Uddin cf0bba7a30 Merge "Revert "Revert "Allow codename.fingerprint format for minSdkVersion""" 2020-01-25 04:12:30 +00:00
Treehugger Robot 6cc0515e82 Merge "Create a highmem pool and put metalava into it" 2020-01-25 01:09:43 +00:00
Baligh Uddin f62013736a Revert "Revert "Allow codename.fingerprint format for minSdkVersion""
This reverts commit 230e241f58.

Reason for revert: This is a revert of a revert.  Downstream problem has been fixed and have been validated locally and via Forrest build.

Change-Id: I89c51d25b3adb818ea44a983d0ac681a88790d8c
2020-01-24 23:15:44 +00:00
Martin Stjernholm 335d596384 Get the dex2oat host tool path from module dependency on the binary
module.

This uses the Once cache for GlobalSoongConfig to propagate the dex2oat
path from a module dependency to the singletons (both the one that
writes out dexpreopt_soong.config and the one that creates the
dexpreopted boot images). Unless dexpreopting is disabled altogether
through DisablePreopt in dexpreopt.config, that means:

- We must ensure at least one module registers a dex2oat tool
  dependency and resolves a GlobalSoongConfig using it, or else the
  singletons fail. That means we litter dex2oat dependencies in java
  modules even when they won't get dexpreopted and hence don't really
  need them.

- We still assume there's at least one java_library or android_app in
  the build.

Bug: 145934348
Test: m
  (check that out/soong/dexpreopt_soong.config points to dex2oatd64)
Test: env USE_DEX2OAT_DEBUG=false m
  (check that out/soong/dexpreopt_soong.config points to dex2oat)
Test: env OUT_DIR=out-tools prebuilts/build-tools/build-prebuilts.sh
  on the aosp-build-tools branch
Change-Id: I38317f2d5e5cf6822f87daae25745767c8a3656c
2020-01-24 22:18:42 +00:00
Martin Stjernholm dae8a80426 Move the Once cache for dexpreopt.GlobalConfig into the dexpreopt
package.

Preparation for a future CL that will need to get the make-written
GlobalConfig from within dexpreopt.

Also rename the Load*Config functions to Parse*Config, since they don't
actually load the config files anymore.

Bug: 145934348
Test: m
Change-Id: I71df11c1e042ca8135d273a7263e9539ea3cd68f
2020-01-24 22:18:42 +00:00
Martin Stjernholm be9d0d21d1 Separate dexpreopt.GlobalSoongConfig to allow independent caching of
it.

Introduce a Once cache for GlobalSoongConfig to allow it to get binary
tool paths from ordinary module dependencies (coming in a future CL)
that are then reused in singletons.

Bug: 145934348
Test: m
Change-Id: I440a09dba7d337965a196527566b0966a18e3653
2020-01-24 22:09:54 +00:00
Colin Cross 8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Jaewoong Jung f0f747c949 Do not remove RRO resources
Resource configs should not be deduped when building RROs since it
would be impossible to override some resource configs with the same
value as the default config. Also, aapt2 removes resources that do not
have default configurations. If an overlay attempts to overlay a
non-default configuration without overlaying the default, the resource
will be removed and the value will not be overlaid at all.

Bug: 146227008
Fixes: 119811120
Test: app_test.go
Change-Id: I834a58b18d1e74a0f6b3de3d0523009788787e42
2020-01-24 10:35:52 -08:00
Treehugger Robot 84f71af41c Merge "Add runtime_resource_overlay." 2020-01-24 00:23:42 +00:00
Jaewoong Jung 9befb0c326 Add runtime_resource_overlay.
Fixes: 119811120
Test: app_test.go
Test: Converted and built IconPackFilledSystemUIOverlay
Change-Id: I71841148c25f820ba829f751a201d2c771c8bd20
2020-01-23 14:24:41 -08:00
Ulya Trafimovich 7eebb4f4e8 For JIT-zygote config, add "apex" boot image files to the ART apex.
JIT-zygote boot image was split into primary and framework extension
in CL I5493e575ebf90bad1d5ad2850004d54590bbc079, but that CL forgot to
add JIT-zygote boot image files to the ART apex. Add them now.

Test: Steps 1-3:

  1. Temporarily enable Jit zygote in the product device config (in
    this case device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. Build and flash:
    $ lunch aosp_walleye-userdebug && m \
    && adb reboot bootloader && fastboot flashall -w

  3. Ensure that apex.art and apex-framework.art are mapped in the
    JIT-zygote address space:

    $ adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex.art
    6ffde000-70102000 rw-p 00000000 00:00 0                                  [anon:dalvik-/apex/com.android.art/javalib/apex.art]
    701e5000-70262000 rw-p 00000000 00:00 0                                  [anon:dalvik-/apex/com.android.art/javalib/apex.art]
    72715cf000-72715d4000 r--p 000ba000 07:98 93                             /apex/com.android.art/javalib/arm64/apex.art

    $ adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex-framework.art
    7084b000-70cc2000 rw-p 00000000 00:00 0                                  [anon:dalvik-/system/framework/apex-framework.art]
    70fd8000-71109000 rw-p 00000000 00:00 0                                  [anon:dalvik-/system/framework/apex-framework.art]
    7271517000-7271526000 r--p 00295000 fc:00 1268                           /system/framework/arm64/apex-framework.art

Change-Id: I985e15de9f38f345ba3e99451c6d9b8e2cb5d698
2020-01-22 15:27:08 +00:00
Treehugger Robot 0c9c0f045f Merge "Remove unused java_sdk_library_import properties" 2020-01-22 06:03:25 +00:00
Paul Duffin 29f5576357 Remove unused java_sdk_library_import properties
The properties are just passed through to the java_import module that
it creates to import the stubs so could be used if set. However, none
of the current usages of java_sdk_library_import actually set them and
it is not clear why the following would ever be set on
java_sdk_library_import:
* installable
* exclude_files
* exclude_dirs

The above properties look as though they were simply copied from the
ImportProperties struct.

The libs property is also unused but it seems as though it should be
used to improve compatibility with the source module. e.g.
the source module for android.test.runner has android.test.base and
android.test.mock in its libs. However, the prebuilt version of it
does not.

Bug: 148080325
Test: m nothing
Change-Id: I710e3962b2e34b2b0cc81685d07d4948cef4e6ad
2020-01-21 18:37:58 +00:00
Pete Gillin 230e241f58 Revert "Allow codename.fingerprint format for minSdkVersion"
This reverts commit 014a85712d.

Reason for revert: Caused vendor/google/build/build_mainline_modules.sh to fail with `Error: minSdkVersion (10000) is greater than maxSdkVersion (30)`.

Bug: 130541924
Change-Id: Ifa233bf40a674481d21b61ee816c5fdde8201080
2020-01-21 13:44:50 +00:00
Artur Satayev b01dd44266 Use merge_csv python_binary.
Instead of using .py file directly.

Bug: 145132366
Test: diff -w out/soong/hiddenapi/hiddenapi-greylist.csv
Change-Id: Ib79019234b4ae05486e831b30d27bd1be3bbba67
2020-01-20 17:59:02 +00:00
Baligh Uddin 014a85712d Allow codename.fingerprint format for minSdkVersion
Use codename.fingerprint format for minSdkVersion if it is unset
in the manifest and
UNBUNDLED_BUILD_TARGET_SDK_WITH_API_FINGERPRINT=true.

Using a utility function in sdk.go to check whether to apply
api.fingerprint.

BUG: 130541924
Change-Id: I748a25c419033bf54b63171d334644fcd0ecc78f
2020-01-20 07:46:12 -08:00
Jaewoong Jung a929cd52ce Merge "Overwrite test-file-name in test config." 2020-01-17 19:07:12 +00:00
Jaewoong Jung 3998234d8c Overwrite test-file-name in test config.
If the install apk name is different than the module name, use
test_config_fixer to update the test-file-name value in the config.

Test: app_test.go
Fixes: 147375216
Change-Id: I2141eeebbb3552995400b45634712306673fd812
2020-01-17 06:08:12 -08:00
Anton Hansson 91133e6e44 Add framework-sdkextensions to jars to instrument
Required to get code coverage.

Bug: 143937446
Test: verify LOCAL_SOONG_JACOCO_REPORT_CLASSES_JAR is set
      for framework-sdkextensions
Change-Id: I2cdb8ce60939a91f8473c2ddd2a8ff198cd7c92e
Merged-In: I2cdb8ce60939a91f8473c2ddd2a8ff198cd7c92e
(cherry picked from commit 9492b09dae)
2020-01-17 09:57:52 +00:00
Jaewoong Jung 6431ca7a3a Add rules to handle asset resources.
AAPT2 ignores assets in overlay resource inputs, so separate rules are
required to extract and merge assets from dependencies.

Test: app_test.go
Test: Added assets to Settings-core for testing
Test: Confirmed Settings.apk contains them with this change.
Fixes: 146655310
Change-Id: Iab8bc61b719541dae64f0e3502bc9cb45a353687
2020-01-16 12:19:49 -08:00
Treehugger Robot 1be2d48005 Merge "Save compilation units in protobuf format." 2020-01-16 18:55:26 +00:00
Treehugger Robot a35294889f Merge changes I0668ff5a,Iff2d7063,I6f8b1e2d,I126db49d,Ic6582260
* changes:
  Exclude source->prebuilt deps from visibility enforcement
  Dedup package build components registration
  Support registering hard coded pre arch mutators
  Added java_system_modules_import
  Document java_system_modules
2020-01-16 02:09:36 +00:00
Treehugger Robot 0b3a5781a9 Merge "Add basic test for java_system_modules" 2020-01-16 02:03:44 +00:00
Jaewoong Jung 275eb69d0a Merge "Keep signatures of presigned prebuilt test apps." 2020-01-15 23:17:02 +00:00
Sasha Smundak 6c2d4f91f8 Save compilation units in protobuf format.
Bug: 146224091
Test: manual
Change-Id: I7a856bab13f54c78efa0061421c3fcb9341bc6e7
2020-01-15 12:40:03 -08:00
Jaewoong Jung 7c5bd835d0 Keep signatures of presigned prebuilt test apps.
Skip the JNI uncompress step for presigned prebuilt test apps. They
don't need it, and they can invalidate the signature. Also, fix the
install partition of prebuilt test apps.

Fixes: 143472191
Test: app_test.go
Test: Converted CtsShimPrivUpgradePrebuilt to bp and checked the
final output apk signature.

Change-Id: Ied7d3576b5db4de40a9ca9e388336229d07212f5
2020-01-15 14:33:32 +00:00
Paul Duffin a7b9f42587 Add basic test for java_system_modules
Bug: 142940300
Test: m nothing
Change-Id: I8855df0832f93e40d31e457191590458c274bb5c
2020-01-15 11:17:57 +00:00
Paul Duffin 90169baf0e Added java_system_modules_import
A prebuilt version of java_system_modules. It does not import the
generated system module, it generates the system module from imported
java libraries in the same way that java_system_modules does. It just
acts as a prebuilt, i.e. can have the same base name as another module
type and the one to use is selected at runtime.

Bug: 142940300
Test: m nothing
Change-Id: I126db49d18294fcd6e2b7ad0237f83e9c2fdef7a
2020-01-15 11:17:57 +00:00
Paul Duffin cded5ecfc3 Document java_system_modules
Bug: 142940300
Test: m nothing
Change-Id: Ic658226088615dbeeef15f17dbb095a968e0742d
2020-01-15 11:17:57 +00:00
Treehugger Robot 814bebb8ea Merge "Reland^2 "m <apex_name>-deps-info prints the internal/external deps of the APEX"" 2020-01-15 06:12:45 +00:00
Inseob Kim a46c2b1994 Merge "Allow sdk and core modules to use sysprop_library" 2020-01-15 00:32:00 +00:00
Anton Hansson ee76fba905 Merge "Make android_test_helper_app support apex_available" 2020-01-14 12:59:17 +00:00
Jiyong Park 83dc74b770 Reland^2 "m <apex_name>-deps-info prints the internal/external deps of the APEX"
This reverts commit 7cb4d378e7.

Test: m
Test: ALLOW_MISSING_DEPENDENCIES=true DIST_DIR=out/dist ./art/tools/dist_linux_bionic.sh -j80 com.android.art.host
(in the master-art-host branch)

Change-Id: I9beca73aafdf42f03bfa19cf1634b2641dac417b
2020-01-14 18:40:56 +09:00
Jiyong Park a7503367cf Merge "Revert "Reland "m <apex_name>-deps-info prints the internal/external deps of the APEX"""" 2020-01-14 08:58:17 +00:00
Jiyong Park 7cb4d378e7 Revert "Reland "m <apex_name>-deps-info prints the internal/external deps of the APEX"""
This reverts commit 956305c61c.

Reason for revert: broke master-art-host branch
Exempt-From-Owner-Approval: reverting a bad change

Change-Id: Id7faed4ee85328c7c65847a3543ea9e67a3d50b3
2020-01-14 08:58:02 +00:00
Inseob Kim c52395135f Allow sdk and core modules to use sysprop_library
This allows javaSdk and javaCore modules to link against Public stub of
Platform owned sysprop_library.

Bug: 141246285
Test: m nothing
Change-Id: Ie11c43cc72ddd0ee13e4163955775c13eb9dbe92
2020-01-14 15:58:59 +09:00
Treehugger Robot 17c51a4ff6 Merge "Reland "m <apex_name>-deps-info prints the internal/external deps of the APEX""" 2020-01-14 02:40:42 +00:00
Ulyana Trafimovich 4d9c97bd77 Merge "Use boot image extension in the JIT-zygote experiment." 2020-01-13 10:38:01 +00:00
Ulyana Trafimovich 289bd28f17 Merge "Do not add jacocoagent to framework libraries in static coverage builds." 2020-01-13 10:25:16 +00:00
Yo Chiang 72d009e1ec Merge "Don't dexpreopt APEX variant modules" 2020-01-13 03:15:50 +00:00
Treehugger Robot ae7cddd5ad Merge "Sandbox soong_build by changing to root directory" 2020-01-11 06:00:21 +00:00
Colin Cross 988414c2cf Sandbox soong_build by changing to root directory
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.

Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
2020-01-11 01:11:46 +00:00
Colin Cross e734832d25 Merge "Revert "Sandbox soong_build by changing to root directory"" 2020-01-10 22:00:59 +00:00
Colin Cross 47e4f9e1e8 Revert "Sandbox soong_build by changing to root directory"
This reverts commit 05c25ccb4a.

Reason for revert: broke absolute OUT_DIR
Bug: 146437378

Change-Id: I523ed79d40e1c1ef040212ba794a7a084abea75d
2020-01-10 18:51:04 +00:00
Colin Cross e35e961aa9 Merge "Sandbox soong_build by changing to root directory" 2020-01-10 17:35:22 +00:00
Ulya Trafimovich 57547452cf Use boot image extension in the JIT-zygote experiment.
Test: temporarily enable JIT-zygote configuration, build, boot the
  device and ensure that the JIT-zygote specific boot image
  apex-framework.art is mapped in the zygote address space:

  1. enable Jit zygote in the product device config (in my case
    device/google/muskie/aosp_walleye.mk):

    +# System server should not contain compiled code.
    +PRODUCT_SYSTEM_SERVER_COMPILER_FILTER := verify
    +
    +# Use the apex image for preopting.
    +DEXPREOPT_USE_APEX_IMAGE := true
    +
    +# Have the runtime pick up the apex image.
    +PRODUCT_PROPERTY_OVERRIDES += \
    +    dalvik.vm.boot-image=/apex/com.android.art/javalib/apex.art:/system/framework/apex-framework.art

  2. lunch aosp_walleye-userdebug \
    && m \
    && adb reboot bootloader \
    && fastboot flashall -w

  3. adb shell cat /proc/`adb shell ps | grep zygote64 | awk {'print $2'}`/maps | grep apex-framework.art
  6fe44000-7025c000 rw-p 00000000 fc:02 1179718                            /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art
  70571000-70696000 rw-p 0072d000 fc:02 1179718                            /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art
  75a339f000-75a33ac000 r--p 00852000 fc:02 1179718                        /data/dalvik-cache/arm64/apex@com.android.art@javalib@apex-framework.art

Change-Id: I5493e575ebf90bad1d5ad2850004d54590bbc079
2020-01-10 15:29:25 +00:00
Anton Hansson 3d2b6b452a Make android_test_helper_app support apex_available
Needed in order to enforce apex_available is set for
everything that gets added to apexes.

Bug: 147364041
Test: apex_test.go
Change-Id: I083ec35283c4c7b5101d00139416d0734b97bab4
2020-01-10 15:06:01 +00:00
Colin Cross 05c25ccb4a Sandbox soong_build by changing to root directory
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
2020-01-09 14:19:46 -08:00
Ulyana Trafimovich a773c39bfc Merge "Revert^2 "Use boot image extension for framework libraries."" 2020-01-09 13:20:36 +00:00
Yo Chiang dbdf8f9c2e Don't dexpreopt APEX variant modules
Java modules that are APEX variant won't be dexpreopted.

After applying this CL,
- $(ANDROID_PRODUCT_OUT)/apex/com.android.appsearch/javalib/service-appsearch.jar
- $(ANDROID_PRODUCT_OUT)/apex/com.android.cronet/javalib/org.chromium.net.cronet.jar
- $(ANDROID_PRODUCT_OUT)/apex/com.android.os.statsd/javalib/service-statsd.jar
- $(ANDROID_PRODUCT_OUT)/apex/com.android.telephony/app/StkLib/StkLib.apk

above four files are not preopted, thus the dex files aren't uncompressed.

FIle size of JAR/APK is (before / after):
- service-appsearch.jar:        697594 / 273354
- org.chromium.net.cronet.jar:  607206 / 239990
- service-statsd.jar:           90806 / 41857
- StkLib.apk:                   6424332 / 2942719
File size of APEX is (before / after):
- com.android.appsearch.apex:   854851 / 432963
- com.android.cronet.apex:      4982739 / 4618195
- com.android.os.statsd.apex:   322319 / 322319
- com.android.telephony.apex:   6596717 / 3111021

Bug: 146984232
Test: lunch aosp_cf_x86_phone-userdebug; m
Test: Build aosp_cf_x86_phone with and w/o this CL and compare artifacts.
      After applying this CL, javalibs and apps embedded inside APEX
      were not preopted, thus dex file are not uncompressed.

Change-Id: I296173bc29d0df0c4d5cc59e4b070e5c4d3b50d5
2020-01-09 20:08:15 +08:00
Ulya Trafimovich 38dfa0fa18 Do not add jacocoagent to framework libraries in static coverage builds.
Framework libraries need special handling in static coverage builds:
they should not have static dependency on jacoco, otherwise there
would be multiple conflicting definitions of the same jacoco classes
coming from different bootclasspath jars.

This CL does two things:

- Move the code that enables instrumentation of framework libraries
  from AndroidGenerateBuildActions phase to the earlier DepsMutator
  phase. This is necessary because DepsMutator phase already does some
  things that depend on the instrumentation flag.

- Explicitely exclude framework libraries from those libraries
  which have static dependency on jacoco.

This CL does not fix any apparent build problems: prior to it the
framework libraries were not excluded properly, but this was masked by
wrong order of checking / setting instrumentation flag.

Note that static coverage builds without framework coverage fail to
boot, namely this build command:

  $ build/soong/soong_ui.bash --make-mode \
    SKIP_ABI_CHECKS=true \
    TARGET_PRODUCT=aosp_walleye TARGET_BUILD_VARIANT=userdebug droid \
    EMMA_INSTRUMENT=true \
    EMMA_INSTRUMENT_STATIC=true \
    NATIVE_COVERAGE=true

..causes the following boot-time errors in logcat:

    01-08 12:31:48.670  1252  1252 E System  : java.lang.StackOverflowError: stack size 8192KB
    01-08 12:31:48.670  1252  1252 E System  :      at org.jacoco.agent.rt.internal.Offline.$jacocoInit(Unknown Source:13)
    01-08 12:31:48.670  1252  1252 E System  :      at org.jacoco.agent.rt.internal.Offline.getProbes(Unknown Source:0)

Also note that static coverage with framework coverage failed to build
prior to CL Iaa198b8505aaff36e6685559642ff721637ce55f (dex2oat failed
to create boot image due to missing classes).

Test: non-static coverage without framework coverage boots:
  $ build/soong/soong_ui.bash --make-mode \
    SKIP_ABI_CHECKS=true \
    TARGET_PRODUCT=aosp_walleye TARGET_BUILD_VARIANT=userdebug droid \
    EMMA_INSTRUMENT=true \
    NATIVE_COVERAGE=true

Test: non-static coverage with framework coverage boots:
  $ build/soong/soong_ui.bash --make-mode \
    SKIP_ABI_CHECKS=true \
    TARGET_PRODUCT=aosp_walleye TARGET_BUILD_VARIANT=userdebug droid \
    EMMA_INSTRUMENT=true \
    EMMA_INSTRUMENT_FRAMEWORK=true \
    NATIVE_COVERAGE=true

Test: static coverage with framework coverage boots:
  $ build/soong/soong_ui.bash --make-mode \
    SKIP_ABI_CHECKS=true \
    TARGET_PRODUCT=aosp_walleye TARGET_BUILD_VARIANT=userdebug droid \
    EMMA_INSTRUMENT=true \
    EMMA_INSTRUMENT_FRAMEWORK=true \
    EMMA_INSTRUMENT_STATIC=true \
    NATIVE_COVERAGE=true

Change-Id: I700f979a5d638ce632f5e8b920b9d0adb3c80248
2020-01-09 10:39:13 +00:00
Jiyong Park 956305c61c Reland "m <apex_name>-deps-info prints the internal/external deps of the APEX""
This reverts commit 4513f703f9.

Bug: 1190898
Test: m
Test: EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true NATIVE_COVERAGE=true  m dist

Change-Id: I50fdccabb43e4751694db83ee451d388657257e0
2020-01-09 18:42:27 +09:00
Jiyong Park 8ea6bc4a05 Merge "Revert "m <apex_name>-deps-info prints the internal/external deps of the APEX"" 2020-01-09 02:20:42 +00:00
Jiyong Park 4513f703f9 Revert "m <apex_name>-deps-info prints the internal/external deps of the APEX"
This reverts commit 114ff53f5e.

Reason for revert: broke cf_x86_phone-userdebug_coverage 

Change-Id: I7221fd0b0baee5407a7322240c170aef8e549e4d
2020-01-09 02:05:18 +00:00