Commit Graph

178 Commits

Author SHA1 Message Date
Nikita Ioffe da6dc31389 Rename test_only_no_hashtree to generate_hashtree
In order not to introduce any changes in the system image default value
of generate_hashtree is currently set to false. Once all the APEXes that
shouldn't have hashtree enable (i.e. non-updatable ones) explicitly set
`generate_hashtree: false` default value will be switched to true.

Test: checked that are no diffs between ${OUT}/system/apex without this
  change and with this change.
Test: m
Bug: 190621617
Change-Id: I5a3144f8f6ea7dee25678af43e2b07a8c85daa9e
2021-06-09 19:43:46 +01:00
Jaewoong Jung 6a328ff8ac Merge "Make apex.key and certificate overridable." 2021-04-22 15:37:22 +00:00
Jaewoong Jung 4cfdf7de26 Make apex.key and certificate overridable.
Test: apex_test.go
Fixes: 185477325
Change-Id: I8992ac5606fa92f53f07e6870a034f2d3352e618
2021-04-20 16:21:24 -07:00
Nikita Ioffe b6ea6c2d3c Don't compress apexes when building in unbundled mode
Test: TARGET_BUILD_APPS=com.android.neuralnetworks m apps_only
Bug: 185708645
Change-Id: I3fdf96f2b8decdff1422549bca413e7660ffb706
2021-04-20 10:20:15 +01:00
Nikita Ioffe bc03588bec Generate hashtree for compressed apexes
Bug: 184615428
Test: flashed build on device, checked compressed apex has hashtree
Change-Id: I83828f581e84504c47047e7692817da5c44a6ec1
2021-04-14 21:35:24 +01:00
Nikita Ioffe 05b999ecb2 Don't compress apexes produced by apex_test module
AFAIU, apex_test is used by modules to create a test version of their
apex for e2e testing of the update flow. Since it's a regular pattern
for test_apex to use the same apex_defaults as the "production" apex,
and due to the fact that we are only going to compress pre-installed
version of the apex, it makes sense to explicitly not compress
test apexes.

Bug: 185082717
Test: build test_com.android.media verified it's not compressed
Test: atest sdkextensions_e2e_tests
Change-Id: I1179235efe068fefc582369a4218ba967e7efbf3
2021-04-14 21:08:13 +01:00
Jiyong Park 9231537fe2 SdkSpec is fully using ApiLevel
Previously, SdkSpec was constructed only from the user string. It didn't
make use of the Config struct where information about the latest stable
SDK version, etc. is recorded. As a result, the build system couldn't
check if the sdk version "current" is referring to the in-development
(i.e.  not-yet-frozen) SDK version or the latest stable version.
"current" was always assumed to be in-development (IsPreview() returns
true) even when Platform_sdk_final == true.

As the first step for fixing that, this change requires
android.EarlyModuleContext to be passed when constructing SdkSpec from
the user string.

In the following changes, "current" will be mapped to either
FutureApiLevel (10000) or one of the FinalApiLevels() depending on
whether the platform SDK was finalized or not.

Bug: 175678607
Test: m
Change-Id: Ifea12ebf147ecccf12e7266dd382819806571543
2021-04-08 11:27:24 +09:00
Jiyong Park f1691d2a2c Move java.sdkSpec to the android package
... in preparation for making the handling of sdk versions consistent
across java and cc modules.

Bug: 175678607
Test: m
Change-Id: I598f0454bce9b7320621022115412fbe97403945
2021-04-03 08:25:12 +09:00
Artur Satayev 533b98cde3 Don't track modules that are only available to APEXes.
Modules that are not available for platform are developed with
updatability in mind, and do not require manual approvals.

Bug: 181223240
Test: checkbuild
Change-Id: I10b91053b3ef5a9ff5400d9d7a68fae3144a671c
2021-03-11 18:13:18 +00:00
Colin Cross 69f0a24762 Pass list of module libraries to gen_ndk_backed_by_apex.sh
Running find on the entire module directory can lead to flaky failures
when a directory is modified by another rule while find is traversing
it.  Pass a list of libraries in the module into the script instead
of finding them on disk.

Fixes: 178509901
Test: TARGET_BUILD_APPS=com.android.runtime dist
Change-Id: Ie4bf288f0e3d99cbbed5ca8770687a9bbe5377e4
2021-02-08 16:52:07 -08:00
Nikita Ioffe 82aab58aea Turn apex compression from opt-out into opt-in
Compression is useful only for big enough apexes. We can manually opt-in
them, instead of trying to find list of apexes that should be opted-out.

Test: m
Bug: 178713634
Change-Id: Ice86fabb7c1db4a1e21e4c983d595cfd3a1c0b7a
2021-01-28 20:25:23 +00:00
Jooyung Han ed124c308f apex: checks min_sdk_version for preview/current
If we don't check "current", it won't be checked even in the finalized
branch.

If we don't check "preview", it should be done during the SDK
finalization. It'd be better done before the SDK finalization regarding
that setting min_sdk_version is to get approval from deps library owners.

Bug: 177833148
Test: m (soong tests)
Change-Id: I712b61cfe5a134fbb69c73956d26fb3a1e5c011e
2021-01-26 12:09:07 +09:00
Sophie Zheng 9b6596bb60 Merge "Revert "Revert "Automated NDK API coverage used by Mainline modules build integration.""" 2021-01-26 00:13:46 +00:00
sophiez 6bde0b5f9d Revert "Revert "Automated NDK API coverage used by Mainline modules build integration.""
This reverts commit 556b2ad77e.

Original change-id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15

Reason for revert: Add excutable permission to the gen_ndk_backedby_apex.sh.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Change-Id: Ib587ba200cd7f2f61d478452b43c329d72b2de06
2021-01-25 19:52:10 +00:00
Riddle Hsu f17b07fc52 Merge "Revert "Automated NDK API coverage used by Mainline modules build integration."" 2021-01-25 06:15:29 +00:00
Riddle Hsu 556b2ad77e Revert "Automated NDK API coverage used by Mainline modules build integration."
This reverts commit 87ff51db84.

Reason for revert: build breakage b/178327796

Change-Id: I51c56938540794097eda579ebe47fa5c0a4d8c12
2021-01-25 01:44:34 +00:00
Sophie Zheng 0bf77198b5 Merge "Automated NDK API coverage used by Mainline modules build integration." 2021-01-22 22:15:21 +00:00
sophiez 87ff51db84 Automated NDK API coverage used by Mainline modules build integration.
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Change-Id: Ib1b2f0dd2f9ae85b1545c6cc5bb4c5bbdfac1c15
2021-01-21 18:08:50 +00:00
Mohammad Samiul Islam 9ac0e32979 Fix capex build rule when building using RBE
Presubmit builds were failing cause they were being built using RBE.

Bug: 172911362
Test: presubmit-builds are passing now
Change-Id: I65ffd912ccf2f4fb8cb9824e7a9579f66908d7b3
2021-01-19 11:32:29 +00:00
Mohammad Islam fdb9e89642 Merge "Add property to apex soong modules to forcefully compress an APEX" 2021-01-12 13:51:50 +00:00
Cindy Zhou 18417cbd72 Skip apex dep check when sanitizer diag is enabled
To avoid adding ubsan to the apex allowed_dep list, this commit adds a
check on depedency tags to see if apex check should be skipped.
The check is only used on sharedLib dependencies when diag mode are enabled
for sanitizers.

Bug: 158010610

Test: make build for aosp-sargo and aosp_cf_x86_phone-userdebug

Change-Id: I3d7dbb70d8c80ffae1854819cf8cf9e6b0b15c00
2020-12-29 14:48:37 +00:00
Jaewoong Jung 18aefc1977 Remove unnecessary snake case variables.
Test: m nothing + TreeHugger
Change-Id: I99f7162944daa6c57c6ae4763261e108bb5cb6b1
2020-12-22 12:38:35 -08:00
Mohammad Samiul Islam a8008f9595 Add property to apex soong modules to forcefully compress an APEX
This makes creating compressed APEX artifacts for testing simpler.

Bug: 172911820
Test: atest apex_compression_platform_tests
Change-Id: I77b8bea4662c273420e7b37d5fde181b86c9cb78
2020-12-22 10:47:50 +00:00
Mohammad Samiul Islam 3cd005d347 Enable soong build tool to handle APEX compression
1. Soong can now detect PRODUCT_COMPRESSED_APEX flag
     We don't want APEX to be compressed on all devices. Only those that
     have explicitely set PRODUCT_COMPRESSED_APEX flag.
2. Handle "compressible" field in soong build rule
     On devices that supports APEX compression, all APEX will be
     compressed by default. If any apex does not want to be compressed,
     they will need to state that by setting "compressible" field to
     false
3. Can use apex_compression_tool to compress APEX
     Note we compress the APEX after it has been signed. That way, when
     we decompress we will get a signed APEX.
4. Place the compressed APEX in system with .capex extension
     This makes it easy to identify. We still preserve the original
     extension so that when we decompress, we can just rename by cuttif
     off the .capex extension.

Note: with this change, we can create a system image with compressed
APEX, but we cannot boot with it since platform doesn't know how to
handle .capex files. Platform support will be added on follow up CLs.

Bug: 172911362
Test: OVERRIDE_PRODUCT_COMPRESSED_APEX=true m (apex_test.go)
Test: observed $OUT/system/apex has .capex files
Change-Id: I20ac4c4ceb521924c751a6017f979b2d808fdded
2020-12-08 13:06:25 +00:00
Sophie Zheng 8baa8b50a8 Merge "Automated NDK API coverage used by Mainline modules build integration." 2020-12-03 23:30:21 +00:00
sophiez c80a2b392e Automated NDK API coverage used by Mainline modules build integration.
Add gen_ndk_usedby_apex.sh script to generate NDK API list used by Mainlain modules when modules get build.

Test: TARGET_BUILD_APPS=com.android.adbd m dist apps_only

Forrest run: android-build.googleplex.com/builds/forrest/run/L97800000742565481

Change-Id: I39e5aa7c74eff46aba06808642b2bd67ae45ba1e
2020-12-03 05:45:20 +00:00
Colin Cross f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Jiyong Park b81b99032d Documenting apex/builer.go
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reordering functions, reordering statements
in logical order, etc.

Bug: 173472337
Test: m
Change-Id: Iefdc8e463bcda60187b98e8c90661e220e8cdf40
2020-11-25 09:53:46 +09:00
Jiyong Park c0ec6f99d7 Documenting apex/apex.go
Mostly documentation changes, but includes a few refactorings like
changing the variable names, reording functions, reordering statements
in logical order, etc.

Bug: 173472337
Test: m

Change-Id: Ie1799c0972d63da823ad375f008018de782529d1
2020-11-24 09:28:50 +09:00
Colin Cross cf371cc1f7 Replace android.WriteFile rule with android.WriteFileRule
The android.WriteFile rule takes careful escaping to produce the
right contents.  Wrap it in an android.WriteFileRule that handles
the escaping.

Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
2020-11-14 16:26:00 -08:00
Nikita Ioffe 5335bc471a Propagate min_sdk_version to apexer
This way it will be correctly set in the auto-generated
AndroidManifest.xml.

Bug: 171096530
Bug: 157078772
Test: m
Change-Id: I268f08ca5dd17fa9d6763ad70cc9a3f83891a8d6
Merged-In: I268f08ca5dd17fa9d6763ad70cc9a3f83891a8d6
(cherry picked from commit 97fd49235a)
2020-10-21 14:26:44 +01:00
Jooyung Han 51ce6b0acc Merge "apex: emit file_contexts for flattened apex" 2020-10-06 03:26:14 +00:00
Jooyung Han 7f146c0da1 apex: emit file_contexts for flattened apex
Flattened APEXes are installed in /<partition>/apex/<name>, so its
file_contexts file should be amended accordingly.
For example, file_contexts of an apex "com.android.foo" with should be
modified by prepending actual install path to the path regex.

/bin/foo <sepolicy label>
=> /system/apex/com\.android\.foo/bin/foo <sepolicy label>

File_contexts file is emitted to "make" via LOCAL_FILE_CONTEXTS variable
for apex_manifest.pb module.

Make collects these files and use them to build file_contexts.bin.

Also removed global state which is used to keep the list of all
flattened apexes.

Bug: 166518492
Test: m file_contexts.bin
      check intermediate file_contexts.modules.tmp file
      in $OUT/obj/ETC/file_contexts.bin_intermediates/
Change-Id: I780370b6e576964a5c09f9477f3361f341dd576b
2020-09-23 15:43:39 +00:00
Dan Albert 4f378d75aa Convert more versions in config to ApiLevel.
The test case I removed is invalid. The codename has had its int
assigned, but the config claims it is not final.

If this ever does need to be supported it's just a matter of making
sure the Q -> 29 mapping (or whatever) in the finalized codenames map
in android/api_levels.go.

Test: treehugger
Bug: http://b/154667674
Change-Id: I4f42ec2fd4a37750519ee3937938a1c65b6bb1e8
2020-09-22 16:01:56 -07:00
Dan Albert c8060536e8 Replace ApiStrToNum uses with ApiLevel.
Test: treehugger
Bug: http://b/154667674
Change-Id: I2954bb21c1cfdeb305f25cfb6c8711c930f6ed50
2020-09-22 15:04:48 -07:00
Xin Li 80dc4f137a Merge "Merge Android R" 2020-09-10 17:22:09 +00:00
Kousik Kumar 3e0b9c031c Merge "Check UseRBE is set before replacing any template with the RE version." 2020-09-10 09:24:17 +00:00
Xin Li 429c23c1fa Merge Android R
Bug: 168057903
Merged-In: I9d5d0da0f409bd6b131f7e0f6363be061d3045c1
Change-Id: Ie7feaf6a3d0787c750de17540969b876a4306b0a
2020-09-09 20:21:02 -07:00
Jiyong Park 41f637d653 Notice files for prebuilt_etc are included in APEX
This change fixes a bug that notice files for some module type (e.g.
prebuilt_etc) were not included in APEX. This happened because we relied
on WalkPayloadDeps which actually doesn't traverse module types that
don't implement ApexModule interface. prebuilt_etc is one such module
type. Fixing the problem by also iterating the filesInfo array which
has info about all modules that are included in the APEX.

Bug: 166575301
Test: m com.android.tzdata and inspect the built artifact.
NOTICE.html.gz is there.

Change-Id: Iceb055b60184aef2a3e65e44680304853eb79a53
2020-09-09 13:19:56 +09:00
Ramy Medhat 16f23a4038 Check UseRBE is set before replacing any template with the RE version.
Test: presubmit
Change-Id: I6df58b4e700f0d231367af2710672d731d20a736
2020-09-03 01:29:49 -04:00
Xin Li 65cb5c69ea Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ifca7b65f4e27bf14cdc30f72f790b0de90130bae
Change-Id: I3a39be5f0b8736de4822c6a14072c78d4e4ad89d
2020-08-29 01:17:45 -07:00
Nikita Ioffe a0254b5289 Merge "Add F2FS support for APEX build system" 2020-08-28 21:55:24 +00:00
Theotime Combes 4ba38c1a3f Add F2FS support for APEX build system
Add the new property payload_fs_type in apex module type. Either 'f2fs'
or 'ext4'. Default 'ext4'.

Test: m

Bug: 158453869
Change-Id: I36f373251bd597e11acb62af75437a87e2a531ec
Merged-In: I36f373251bd597e11acb62af75437a87e2a531ec
2020-08-28 12:03:00 +00:00
Jooyung Han b9007609f6 apex: dedup jni_libs when generating apex_manifest
When we package both 32/62, there could be two instances of the same
library. We need to dedup the list of jni_libs.

Bug: n/a
Test: m com.android.cronet (which uses jni_libs)
      check apex_manifest.pb
Change-Id: I9a9f42f0f5219ab7bc50d945b54113b5d823b7f3
2020-08-28 16:33:16 +09:00
Colin Cross 08dca38eb0 Build transitive lint reports for apex modules
Build and export transitive lint report zips for apex modules.

Bug: 153485543
Test: m TARGET_BUILD_APPS=com.google.android.wifi lint-check dist
Change-Id: I5a1805440452301a7e2c4ca91482b989638b54fb
2020-07-22 20:16:14 -07:00
Liz Kammer 0a51aa2c98 Use RelativeInstallPath for DataPaths in apex
The RelativeInstallPath is currently unused for apexes; however, as it
stands, relative install paths would be handled differently for
flattened and unflattened apexes.

Test: go java test
Test m nothing

Change-Id: Iad9350642cad27c4f745cea6e5fc9e11e7b06ee4
2020-07-21 13:29:37 -07:00
Chris Parsons 216e10a0f6 Install data_libs using relative_install_path property
Test: Added to data_libs tests
Test: Manually verified with bionic-unit-tests
Test: Treehugger
Change-Id: I28a8e08e3409f1e7c7bb72f4351310b57f35f513
2020-07-20 13:59:24 -04:00
Colin Cross 56b330f8e7 Merge "Use unzip -DD" am: b7cea24fe3 am: cae5903677
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1361337

Change-Id: I68b97b33bd19206dfd96aca36bf4575a0bca5459
2020-07-13 06:21:07 +00:00
Colin Cross d783bbbace Use unzip -DD
We put reproducible timestamps in zip files so that the artifacts
are consistent, but that leads to old timestamps in the output
directory if they are unzipped as part of the build.  Use
unzip -DD when unzipping to update the timestamps.

Bug: 161015009
Test: touch -d 2020-01-01 ref; find $OUT/system -not -newer ref
Change-Id: I70407a627cb070e24be510faa6a774e3d9eae3a8
2020-07-12 05:37:46 +00:00
Treehugger Robot ab22ca31e9 Merge "apex: add / in file_contexts as system_file" am: fae9e0cf9a am: 062e9afe7e
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1351465

Change-Id: I9b4e3f4340b937df93be56867e557e53bb86fa30
2020-07-02 04:41:34 +00:00