Commit Graph

50640 Commits

Author SHA1 Message Date
Anton Hansson 528fdc021f Fix non-existing modules check
This variable was accidentally changed to refer to the
shorthand PRODUCT_XYZ instead of PRODUCTS.<mk>.PRODUCT_XYZ
in a previous mass-migration, but that shorthand didn't
exist for these variables. Add it in.

Also whitelist the offender introduced since it was broken.

Test: lunch generic; m nothing
Change-Id: I29dd73c29796743e770afa0c2ceb4f8583dc6e1e
2019-06-25 21:52:58 +01:00
Tao Bao c931547560 Stop passing `--block` and MKBOOTIMG to ota_from_target_files.
- `--block` has been a no-op since Oreo [1] -- ota_from_target_files
  will always build block-based OTAs or A/B OTAs.
- MKBOOTIMG is no longer needed when calling ota_from_target_files, at
  least for the callsite in Makefile. Because target_files.zip has been
  including boot/recovery images since Lollipop [2].

This CL also switches to long options to improve readability.

[1] commit 457cbf6a8a
[2] commit 3c84f56948

Test: TreeHugger
Change-Id: I801bd0525d2529156bbf783698661d90ba24017a
2019-06-25 13:04:06 -07:00
Dan Willemsen b88f792d25 Install shared library dependencies for 64-bit windows
Test: m host_cross_libandroidicu_64; see libs in out/host/windows-x86/lib64
Change-Id: Ibf816aeb96e70b70cfcb9b58df9605c3af87db39
2019-06-25 12:56:41 -07:00
Chih-hung Hsieh 7b3a8fb7ec Merge "Fix pylint warnings and now prefer python3" 2019-06-25 18:32:26 +00:00
Tao Bao 86b529a23e releasetools: Make apex_utils Python 3 compatible.
In order to get a Python 2 and 3 compatible re-raise behavior, this CL
removes the stack traceback for the lines within apex_util module (i.e.
sys.exc_info()[2]). It's not a big loss in practice, since we only have
one line within the try-except block (`common.RunAndCheckOutput()`)
that's no longer reported in the traceback.

Using `six` module could better solve this, but only after building
releasetools as python_binary_host modules where we can properly handle
the module dependency.

Bug: 131631303
Test: TreeHugger
Test: `python -m unittest test_apex_utils`
Test: `python3 -m unittest test_apex_utils`
Change-Id: I0c5a72ec9fad5ff9d8c9c94d29e813e433ec2921
2019-06-25 11:10:55 -07:00
Tao Bao ec53f441fa Merge "Clean up the dependencies for user image creation." 2019-06-25 17:57:43 +00:00
Isaac Chen 17a3aa4819 Merge "Add lpunpack to otatools for rebuilding super.img" 2019-06-25 17:19:46 +00:00
Tao Bao ec8e2b4e02 Merge changes Idc7776be,I409fe30a
* changes:
  releasetools: Make verity_utils Python 3 compatible.
  releasetools: Make common Python 3 compatible.
2019-06-25 16:35:35 +00:00
Elliott Hughes b82ef911f9 Don't use tar's -p with -rf, where it makes no sense.
-p only applies to tar invocations that cause extraction, but -rf is
just appending to a tar file.

toybox tar doesn't have -p, and it doesn't seem worth implementing if
this is our only "user".

Test: treehugger
Change-Id: I025c496b04cb57cb31ba81fd516fd40b158bc06b
2019-06-25 09:07:41 -07:00
Tao Bao 9e893c3cfd releasetools: Make verity_utils Python 3 compatible.
Bug: 131631303
Test: TreeHugger
Test: `python -m unittest test_verity_utils`
Test: `python3 -m unittest test_verity_utils`
Change-Id: Idc7776be2cf78fbdb0b3ba87231d05013da8d3ac
2019-06-24 21:07:19 -07:00
Tao Bao da30cfae96 releasetools: Make common Python 3 compatible.
Bug: 131631303
Test: TreeHugger
Test: `python -m unittest test_common`
Test: `python3 -m unittest test_common`
Change-Id: I409fe30a5d71975c1d7b66e5e749843de530f1f9
2019-06-24 21:02:12 -07:00
Isaac Chen eec4a7cba4 Add lpunpack to otatools for rebuilding super.img
lpunpack is needed to repackage super.img for mixed-image virtual devices.
For Cuttlefish, it's included in host package. We need to include it in
otatools.zip so Goldfish or other devices can use it to mix their super images.
Necessary SOs are the same as lpmake and have been included in otatools.zip.

Bug: 134461288
Test: $ lunch aosp_x86-userdebug
      $ m otatools-package
      $ unzip -l $OUT/ota* | grep lpunpack
        467824  2008-01-01 00:00   bin/lpunpack

Change-Id: Ie2092f18afb88721b29b1738847a0b781dbcd431
2019-06-25 11:50:58 +08:00
Tao Bao dba635a3b7 Clean up the dependencies for user image creation.
The dependency on MAKE_EXT4FS (mke2fs) has been covered by the rule for
MKEXTUSERIMG (mkuserimg_mke2fs):
https://android.googlesource.com/platform/system/extras/+/refs/heads/master/ext4_utils/Android.bp

The dependency on mksquashfs and img2simg comes from mksquashfsimage.sh,
which has been moved into mksquashfsimage's building rule.

This CL also fixes the ordering of INTERNAL_USERIMAGES_BINARY_PATHS,
which should come after adding all the binaries to
INTERNAL_USERIMAGES_DEPS.

Test: TreeHugger
Change-Id: I8b30bf87860d9711e88eecca4b76db72f5cd8241
2019-06-24 15:47:47 -07:00
Tao Bao 9392fb3905 Merge "Fix an issue in setting up PATH." 2019-06-24 22:34:53 +00:00
Chih-Hung Hsieh 9018ea49d8 Fix pylint warnings and now prefer python3
* Read/write warning messages as utf-8 strings.
* No more embedded raw utf-8 code in file.

Test: ./warn.py --csvpath warnings.csv build.log > warnings.html
Change-Id: Ie51700721a32bede1d3b250f4a42fd8facd6af75
2019-06-24 15:24:39 -07:00
Tao Bao cf821fc963 Fix an issue in setting up PATH.
`PATH=$(foreach p,$(INTERNAL_USERIMAGES_BINARY_PATHS),$(p):)$$PATH` will
produce a space separated string due to `foreach`, if
$(INTERNAL_USERIMAGES_BINARY_PATHS) has more than one string. We didn't
hit the issue in the past because $(INTERNAL_USERIMAGES_BINARY_PATHS)
had contained only one path in practice.

Test: TreeHugger
Change-Id: I74ef4356668af63d871a81f6bfd4c9324aa1d956
2019-06-24 14:08:36 -07:00
Treehugger Robot f26a4e459e Merge "Revert "Revert "Deprecate bash m* commands and use the soong's modules-in* build commands instead.""" 2019-06-24 20:31:29 +00:00
Treehugger Robot 4188cc5c5c Merge "Allow passing --legacy in LOCAL_AAPT_FLAGS" 2019-06-24 17:13:44 +00:00
Tao Bao eb7cb468e4 Merge "releasetools: Remove some legacy paths for prop rewrites." 2019-06-24 15:47:33 +00:00
Kiyoung Kim 62e2231b82 Set INSTALLED_BOOTIMAGE_TARGET in case of TARGET_NO_KERNEL is true and
BOARD_PREBUILT_BOOTIMAGE is set.

Based on current Makefile INSTALLED_BOOTIMAGE_TARGET always kept as
empty if TARGET_NO_KERNEL is true, so build fails with empty out path.
This change is to set up INSTALLED_BOOTIMAGE_TARGET with
$(PRODUCT_OUT)/boot.img in case of TARGET_NO_KERNEL is true and
BOARD_PREBUILT_BOOTIMAGE is defined.

Bug: 135648566
Test: m -j passed
Change-Id: I9658f215c2e2cc5d87040d5085521ef44722c4ef
2019-06-24 09:45:28 +09:00
Colin Cross fd5fa34775 Don't pass --legacy by default to aapt2 compile
Don't pass --legacy by default, it is rarely necessary and converts
some errors into warnings that crash at runtime.  The modules that
need it have had --legacy added to LOCAL_AAPT_FLAGS.

Bug: 135597368
Test: m java
Change-Id: I0cb213599d6612746d988b8966cbd529b5328db0
2019-06-22 13:04:51 -07:00
Colin Cross 18f28c7781 Allow passing --legacy in LOCAL_AAPT_FLAGS
--legacy will soon no longer be passed by default to all aapt2
compiles.  Allow it to be specified in LOCAL_AAPT_FLAGS by passing
it to aapt2 compile when it is present and filtering it out from
the flags passed to aapt2 link.

Bug: 135597368
Test: m java
Change-Id: I92792cf6a0d4c6ecf5e0971523bd0ab76c81345f
2019-06-22 13:03:09 -07:00
Tao Bao 338c1b7e90 releasetools: Remove some legacy paths for prop rewrites.
sign_target_files_apks.py only needs to take care of the current
release. The legacy paths of ODM/build.prop, VENDOR/odm/build.prop, and
BOOT/RAMDISK/default.prop no longer exist in the target_files.zip from
master.

The other two cases of ROOT/default.prop and
RECOVERY/RAMDISK/default.prop are still kept in the code, as they will
still exist (as symlink or conditionally).

Test: Run sign_target_files_apks.py against
      aosp_taimen-target_files.zip. Check the rewritten prop files.
Test: `python -m unittest test_sign_target_files_apks`
Change-Id: I5e70bc2ccc0f3dcf0eace0718c59a3b0f89a9ff4
2019-06-21 10:26:15 -07:00
Tao Bao 4daa755fb2 Merge "releasetools: Make blockimgdiff Python 3 compatible." 2019-06-21 14:23:09 +00:00
Treehugger Robot 4ba3984b4c Merge "Remove experimental pass manager for fuzzer builds." 2019-06-21 00:51:26 +00:00
Patrice Arruda fa7204b0e1 Revert "Revert "Deprecate bash m* commands and use the soong's modules-in* build commands instead.""
This reverts commit c4dae4d7ab.

Reason for revert: Fix available: go/aog/987391

Change-Id: I0a6cd679699488da716c30b95c62af89b6aa4f1a
2019-06-20 23:40:33 +00:00
Dan Willemsen 27c05b2405 Merge "Revert "Deprecate bash m* commands and use the soong's modules-in* build commands instead."" 2019-06-20 22:35:06 +00:00
Patrice Arruda c4dae4d7ab Revert "Deprecate bash m* commands and use the soong's modules-in* build commands instead."
This reverts commit 610b6302cf.

Reason for revert: m mlibc from root dir works. Create a tmp dir and then cd in to tmp. Run the same command and it fails.

Change-Id: I0f0c056d7e668da621ae99d347a055ecdf2dec07
2019-06-20 22:32:46 +00:00
Tao Bao b813120af1 releasetools: Make blockimgdiff Python 3 compatible.
Bug: 131631303
Test: `python -m unittest test_blockimgdiff`
Test: `python3 -m unittest test_blockimgdiff`
Change-Id: I8cf072ca4af6a525bc675cba0be998ff7be7f4f1
2019-06-20 14:30:30 -07:00
Dan Shi 3771cff834 Merge "Create individual build artifact for test suite build output" 2019-06-20 20:30:34 +00:00
Dan Shi 51b789e422 Create individual build artifact for test suite build output
This change creates following build artifacts that are currently part of
device-tests.zip and general-tests.zip. The smaller build artifacts are
used to reduce the download size required to run TradeFed suite.
Detailed design can be found in the bug.

From device-tests
device-tests_configs: include all test config files in device-tests, zip
  file is less than 1MB.

From general-tests
general-tests_configs: include all test config files in general-tests,
  zip file is less than 1MB.
general-tests_host-shared-libs: include tools and shared libraries
  required to run host side test, zip file is about 60MB

Bug: 73786521
Test: make device-tests and general-tests, verify output.
Change-Id: I34c9aa3c1c672d5d58eab6de92198a7b8e4a681c
2019-06-20 10:02:36 -07:00
Tao Bao 1f1aafa30d Merge "releasetools: Make rangelib Python 3 compatibile." 2019-06-20 14:51:21 +00:00
Treehugger Robot fde688ebc8 Merge "Fixes a typo in a comment." 2019-06-20 04:20:36 +00:00
Treehugger Robot 345ed53d98 Merge "Enforce property split in GSI" 2019-06-20 01:53:15 +00:00
Treehugger Robot 9bcf0376ab Merge "Deprecate bash m* commands and use the soong's modules-in* build commands instead." 2019-06-20 00:20:23 +00:00
Patrice Arruda 610b6302cf Deprecate bash m* commands and use the soong's modules-in* build commands instead.
The functionality of m* envsetup.sh commands has been implemented in
soong_ui. The bash functionality of m* commands can be replaced by
soong's modules-in* build commands.

Fixes: b/130049705
Test: * Loaded the envsetup.sh on a new environment
      * Executed each m* command.
      * Verified with "type <build command>" to make sure
        the function generation is correct.
      * Ran on Darwin environment.

Change-Id: I8792e4f1f1727aba2ca6ee4bdfe657f6ac42302a
2019-06-19 14:16:38 -07:00
Tao Bao 9521c802ec Merge "releasetools: Make common.ZipWriteStr Python 3 compatible." 2019-06-19 21:09:01 +00:00
Tao Bao d660c8d8f3 releasetools: Make rangelib Python 3 compatibile.
Bug: 131631303
Test: `python -m unittest test_rangelib`
Test: `python3 -m unittest test_rangelib`
Change-Id: Ia8e26623d5967f2eea003252ee56b861350d626b
2019-06-19 10:30:34 -07:00
Treehugger Robot 435dcbb845 Merge "Disable apex update for GSI" 2019-06-19 10:38:42 +00:00
Treehugger Robot 9871d36024 Merge "Add more OWNERS" 2019-06-19 09:34:14 +00:00
Tao Bao c1a1ec30df releasetools: Make common.ZipWriteStr Python 3 compatible.
Python 2 and 3 behave differently when calling ZipFile.writestr() with
zinfo.external_attr being 0. Python 3 uses `0o600 << 16` as the value
for such a case (since
18ee29d0b8),
which seems to make more sense. Otherwise the entry will end up with
0o000 as the permission bits. This CL updates common.ZipWriteStr to
follow the logic in Python 3, in order to get consistent behavior
between using the two versions.

Bug: 131631303
Test: `python -m unittest test_common.CommonZipTest`
Test: `python3 -m unittest test_common.CommonZipTest`
Change-Id: If8429855d922ef1ad76591f703215a0ce5089f0f
2019-06-18 23:53:53 -07:00
Treehugger Robot e346d8c25e Merge changes from topics "rbe-deps-dexpreopt-images", "rbe-deps-system-modules"
* changes:
  Use dexpreopt image deps from Soong
  Depend on all the files from system modules
2019-06-18 22:48:22 +00:00
Tao Bao aadc9b9159 Merge "releasetools: Update partitions in deterministic order." 2019-06-18 22:20:26 +00:00
Treehugger Robot 0405f780fc Merge "Update fs_config documentation" 2019-06-18 22:08:46 +00:00
Tao Bao f1113e97ae releasetools: Update partitions in deterministic order.
Previously it was using regular dict.

Test: python -m unittest test_common.DynamicPartitionsDifferenceTest
Change-Id: If108a4512aeaf9d3c8775c030cad6e44342b9d3d
2019-06-18 12:14:32 -07:00
Neil Fuller c4924c616c Merge "Remove a host simulation of the runtime module" 2019-06-18 15:33:11 +00:00
SzuWei Lin 79e0d25c33 Enforce property split in GSI
Pure GSI build targets has no vendor partition, such as
aosp_$arch_ab and gsi_$arch. The system properties defined by
PRODUCT_PROPERTY_OVERRIDES will be in /system/build.prop.

The patch defined a fake BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE to
let these system properties flow to vendor and won't pollute the
system.img.

The bug also move some properties to /product/build.prop.

Bug: 135508595
Bug: 131162245
Bug: 134781120
Test: check the /system/build.prop do not have "ro.carrier=unknown"
Test: adb remount on GSI Q on P
Change-Id: Ib200d66cf98fea572c26338e058bce29eb5e0cd7
2019-06-18 18:23:18 +08:00
Isaac Chen fae280264e Disable apex update for GSI
Currently, GSI does not include apex support and TARGET_FLATTEN_APEX is true.
This will cause issues for vendors with apex support (e.g. Cuttlefish).
This change set ro.apex.updatable to false in product to override that sets
in vendor.

Bug: 135411972
Bug: 134673003
Test: $ lunch aosp_x86-userdebug; m -j; emulator
      $ lunch aosp_cf_x86_phone-userdebug
      # Replace system.img in super.img with GSI
      # The resulted CF could boot and browse the web successfully.

Change-Id: I08fd7a1b254aac276926329e064c35b714764936
2019-06-18 15:57:51 +08:00
Dan Willemsen cb4c357661 Use dexpreopt image deps from Soong
Instead of just depending on the single image file, depend on the entire
image that gets loaded -- the .oat / .vdex files, and the modules that
are part of the image that have their own files.

This is necessary for RBE, where the rule only gets the files that it
depends upon.

Test: treehugger
Test: build a system image with RBE
Change-Id: I93a580b42cf4ef1824ee1397e551c065f083348a
2019-06-18 01:19:04 +00:00
Dan Willemsen 70b934ee64 Depend on all the files from system modules
Test: treehugger
Change-Id: Ie84dda3d05a84658ad9386a3034d861138f25b2c
2019-06-18 01:19:04 +00:00