Commit Graph

7256 Commits

Author SHA1 Message Date
Nikita Ioffe 6068e8dd52 Refactor apex signing logic in preparation for compressed apexes
Logic for signing compressed apex will be added in the follow-up cl.

Test: python -m unittest test_sign_apex
Bug: 172912232
Change-Id: I8dc1d334c17e11f9eed8fe0a575b6dfcf337ab5d
2021-01-12 22:29:58 +00:00
Tianjie 2e0b835d28 Fix errors from validate_target_files
We should skip lines that don't contain '=', e.g. empty lines.
Also, pop a warning instead of an error if a prop is defined
multiple times with the same value

Bug: 177240467
Test: unittest, run validate_target_files
Change-Id: Ifc9eadb91e5dda7170a19d875016e5a47e8fc592
2021-01-12 14:04:58 -08:00
Yifan Hong 14a8712c4d Add missing dependencies to releasetools_common and binary defaults.
When GetBootImageBuildProp is moved to common, its dependencies
(toybox, lz4 and unpack_bootimg) aren't moved accordingly.
Copy the dependencies over.

Test: pass
Change-Id: Iaa8f41ae0109e8eb1c058ecd7dd854bb7de9391a
2021-01-12 14:04:19 -08:00
Yifan Hong 7dc5117ffd GetBootImageBuildProp returns None if file does not exist
If system/etc/ramdisk/build.prop does not exist in the ramdisk,
GetBootImageBuildProp should return None instead of a path
to a non-existant file.

Test: pass
Bug: 177171706
Change-Id: I34e447bc315d203e136e8c0d59d00961f81301e3
2021-01-12 11:27:55 -08:00
Nikita Ioffe c3fdfed11b Fix test_sign_apex
Test: python -m unittest test_sign_apex
Change-Id: I2a27a394b3de32fa29e487bea0b7721502a8bd98
2021-01-11 23:50:31 +00:00
Yifan Hong cb09afc04a Merge changes from topic "common.GetBootImageTimestamp"
* changes:
  Load boot build props to info_dict properly.
  Add GetBootImageBuildProp.
  Split PARTITIONS_WITH_BUILD_PROP from *_WITH_CARE_MAP
  Move GetBootImageTimestamp to common.
2021-01-08 22:18:51 +00:00
Colin Cross 383e482c22 zipalign: log more I/O errors
b/177069043 is a flake that happens in zipalign during I/O, but
the error doesn't contain enough information to determine what the
problem is.  Add more logging on I/O errors.

Test: atest --host zipalign_tests
Bug: 177069043
Change-Id: I89573e5b3867abe05d855c837bb2aa5b5c630aa7
2021-01-08 11:11:40 -08:00
Yifan Hong 10482a2a56 Load boot build props to info_dict properly.
Add boot to PARTITION_WITH_BUILD_PROP. Let PartitionBuildProps handle
boot partitions by extracting build.prop file.

Test: generate OTA and call
      `payload_info.py payload.bin`, check that boot image timestamp is
      set
Bug: 162623577

Change-Id: I71955e4ee494ffbb5d111c3d02a129c0f879b6d1
2021-01-07 17:20:22 -08:00
Yifan Hong 85ac5012aa Add GetBootImageBuildProp.
Add a function that extracts build.prop from boot image.

Test: TH
Bug: 162623577
Change-Id: I673ccd8e7c4846ca4c41a86349a388b2e800724e
2021-01-07 17:20:22 -08:00
Yifan Hong 5057b95572 Split PARTITIONS_WITH_BUILD_PROP from *_WITH_CARE_MAP
Right now they are the same content. In the future, boot will be added
to PARTITIONS_WITH_BUILD_PROP, but it is not added to
PARTITIONS_WITH_CARE_MAP. Boot partition has a cpio filesystem in the
ramdisk, so it contains a build.prop file, but it doesn't make sense to
create care map from it.

Test: TH
Bug: 162623577

Change-Id: I9b5a20fe2d774b52cf7d5eae9deecbc75122a3dd
2021-01-07 17:20:22 -08:00
Yifan Hong c65a0545dd Move GetBootImageTimestamp to common.
Test: TH
Bug: 162623577

Change-Id: Icbd8b07dce91006a1abca1f0dc40726de8a10219
2021-01-07 17:20:22 -08:00
Bob Badour 7a95820907 Revert^2 "Rough-in license metadata support to make."
Changed shell script to /bin/sh syntax for portability to mac_os etc.

54066744f0

Change-Id: Ibd472fc1bedfdc228a704108f0280b13ab1c9333
2021-01-06 20:52:43 -08:00
Jerome Gaillard 54066744f0 Revert "Rough-in license metadata support to make."
Revert "Export soong license data to make."

Revert "Add ability to declare licenses in soong."

Revert submission 1377717-metalics

Reason for revert: This has broken renderscript_mac target for aosp-master, see b/176909442

Reverted Changes:
I26ac54ca9:Define the standard license_kind rules.
I656486070:Export soong license data to make.
If9d661dfc:Export soong license data to make.
I97943de53:Add ability to declare licenses in soong.
Icaff40171:Rough-in license metadata support to make.
Ib8e538bd0:Add variables for notice deps, license kinds etc.

Change-Id: Iaf2243cba16f6191b022371a14f81ef8194a47d1
2021-01-06 19:00:05 +00:00
Bob Badour 20fc1b3719 Rough-in license metadata support to make.
Change-Id: Icaff40171b94538219a8caa697b182dbdcc8229a
2021-01-05 08:42:48 -08:00
SzuWei Lin baf5c8114b Make ro.product.cpu.abilist.* to be fetched dynamically
The patch rename the original ro.product.cpu.abilist* to
ro.system.product.cpu.abilist*. The property service will
define ro.product.cpu.abilist* by the intersect ABIs of
ro.{partition}.product.cpu.abilist*.

Bug: 176520383
Test: build aosp_arm64-userdebug and check the system/build.prop
Change-Id: I26e404632b28e9f6abd462920e0084ad2022a763
2020-12-31 16:59:27 +08:00
Treehugger Robot fdb2f2c5db Merge "Swich TEST_MAPPING config to unit_tests:true" 2020-12-22 21:32:31 +00:00
Julien Desprez dc0663f66f Merge "Remove the zipalign_test mapping file" 2020-12-22 00:25:05 +00:00
Julien Desprez 6d9e78fd5d Swich TEST_MAPPING config to unit_tests:true
All unit_tests:true are run in presubmit which avoids
the explicit definition of TEST_MAPPING.

Test: presubmit runs all those tests
Bug: 175408655
Change-Id: Ibfad3fe2dd8d2fa1ebc6f6bcbd8ab34c2a0069dc
2020-12-21 13:56:05 -08:00
Julien Desprez 2bbc159e97 Remove the zipalign_test mapping file
Not needed as the host-unit-tests suite runs in
presbumit now and cover this

Test: None
Bug: 175408655
Change-Id: Iea3e3cd7f696f43b6c686ef9cbee1194f514e589
2020-12-21 09:58:17 -08:00
Yo Chiang d21e7dc833 Add support for packing multiple vendor ramdisks in vendor_boot
Add support for partitioning the vendor_boot kernel modules into
multiple vendor ramdisk fragments. The partition granularity is kernel
module directory. This mechanism builds upon the existing
BOARD_KERNEL_MODULE_DIRS mechanism. For example, say we have three
kernel module directories:

BOARD_KERNEL_MODULE_DIRS := foo bar baz

We can then define a vendor ramdisk fragment:

BOARD_MKBOOTIMG_ARGS += --header_version 4
BOARD_VENDOR_RAMDISK_FRAGMENTS := dlkm_foobar

And let said ramdisk to contain the DLKM directories "foo" and "bar":

BOARD_VENDOR_RAMDISK_FRAGMENT.dlkm_foobar.KERNEL_MODULE_DIRS := foo bar
BOARD_VENDOR_RAMDISK_FRAGMENT.dlkm_foobar.MKBOOTIMG_ARGS := <mkbootimg args>

The built vendor_boot image would contain two ramdisks.
The first one being the "default" ramdisk, which contains DLKM directory
"baz" and the rest of the files that get's installed to
$(TARGET_VENDOR_RAMDISK_OUT).
The second one is the "dlkm_foobar" ramdisk, which contains the two DLKM
directories.

Design doc: go/vendor-boot-v4

Bug: 162864255
Test: Modify BoardConfig.mk to have a product build v4 vendor_boot
Test: Use unpack_bootimg to verify the vendor_boot image
Test: Teach a bootloader how to handle v4 boot image, flash boot &
  vendor_boot and boot device
Change-Id: Ibb1bbd7ebe36430c55ec6c4818c1d3888a319089
2020-12-17 11:07:04 +08:00
Julien Desprez 31839fe1cd Merge "mark zipalign_test as unit tests" 2020-12-14 16:11:41 +00:00
Elliott Hughes 27de935d83 Update OWNERS.
Test: treehugger
Change-Id: I5f9ddd03d5cd34fb398b842c933dc6a365be4b39
2020-12-11 12:56:27 -08:00
Julien Desprez 74b8e09094 mark zipalign_test as unit tests
Test: make
Bug: 175408655
Change-Id: I03a7a0bb19ea408830acbf1ebe12ffb25012af91
2020-12-11 11:52:45 -08:00
Jaegeuk Kim 74e57f6482 Merge "Enable system partition compression" 2020-12-10 00:41:04 +00:00
Treehugger Robot 70c61bc6e9 Merge changes I1deda85d,I43d7b2c6
* changes:
  calculation partition size use compressed image size
  add erofs to non partition spanning fs
2020-12-09 06:50:56 +00:00
Robin Hsu 3e51f4234c Enable system partition compression
* Prerequisites: external/f2fs-tools: sload compression support
* Must work with corresponding changes in system/extras repository
* If Board config does not change, it falls back to old behavior,
  i.e. no compression for the system partition
* Kernel f2fs compression support is a prerequisite if the Board config
  enables the compression (see below)
* Necessary board config change
 (e.g. device/<vendor>/<product>/BoardConfig-common.mk)

BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE := f2fs
BOARD_SYSTEMIMAGE_PARTITION_RESERVED_SIZE := 200000000
BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS := true
BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS := <sload compress sub-options>

Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_COMPRESS to true enables both the
compression support when the initial empty file system be made (mkfs.f2fs)
and the compression flag (-c) when the system image files be side-loaded
by sload.

Sload compress sub-options (i.e. options other than -c) will be provided
by BOARD_SYSTEMIMAGE_F2FS_SLOAD_COMPRESS_FLAGS.   If it is not given, or
is empty, the default sub-options will be used

Please refer to the sload.f2fs manual page.

Setting BOARD_SYSTEMIMAGE_FILE_SYSTEM_TYPE to f2fs is trivially necessary.

* File system table (fstab), notably the file 'fstab.hardware', should
  also changed:
  - The file type must be changed to f2fs
  - Perhaps also other f2fs-specific options

Bug: 170918499
Test: Pixel4a userdebug build (from build id 6918751)
Signed-off-by: Robin Hsu <robinhsu@google.com>
Change-Id: Id9d67b5cb35dc806e06ff1320e89114abc996a28
2020-12-09 07:24:55 +08:00
Rupert Shuttleworth 729427460b Ensure file permissions are writable before attempting to sign.
Test: Manually using USE_BAZEL=1 use_rbe m dist (fixes a build error).
Change-Id: I1aac185f12ac430514c975103547ce211ea6c99e
2020-12-08 10:03:34 +00:00
Huang Jianan 35f015ea5e calculation partition size use compressed image size
If dynamic partitioning is enabled and the partition size is not set,
du will be used to calculate the partition size.

For compressed file systems, it's better to use the compressed size
to avoid wasting space.

Bug:174816929
Test: erofs image size is smaller than the original file
Change-Id: I1deda85d312c19620680531223fffcfb815e5fd4
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
2020-12-07 07:09:28 +00:00
Huang Jianan 62d926e2fe add erofs to non partition spanning fs
Bug:174816929
Test: sparse image size will be fixed after rebuilding erofs image
Change-Id: I43d7b2c61c57301935981f6f1282afb685cce331
Signed-off-by: Huang Jianan <huangjianan@oppo.com>
2020-12-04 09:08:00 +00:00
Colin Cross bbc94b0d83 Use jni_libs instead of required for signapk conscrypt JNI
Using jni_libs tells Soong to install the JNI library into
a genrule sandbox alongside the java binary.

Bug: 124313442
Test: m checkbuild
Change-Id: Ia550102c7aa483497f95ac2ea8bb01bce89ae9c1
2020-12-02 11:37:21 -08:00
Colin Cross 40c9513f7f Follow argument changes to RuleBuilder
Pass pctx and ctx to NewRuleBuilder instead of RuleBuilder.Build,
and don't pass ctx to RuleBuilderCommand.BuiltTool.  Follows the
changes in I63e6597e19167393876dc2259d6f521363b7dabc.

Test: m checkbuild
Change-Id: I5d11e07ae1a24a967e379c151f4f5d7d188c6cc0
2020-12-01 16:21:51 -08:00
Daniel Norman e9fdfb9fea Merge "Runs host_init_verifier on merged target files packages." 2020-11-25 18:13:30 +00:00
Chih-hung Hsieh 5364ffafa9 Merge "Dump a new CSV with warning messages" 2020-11-25 17:38:21 +00:00
Daniel Norman 21c34f78e8 Runs host_init_verifier on merged target files packages.
This verifies the init rc files in the merged result.

Bug: 163089173
Test: test_common.py
Test: Run merge_target_files.py to merge two target-files packages where
      one has init_rc errors. Observe script failure.
Test: Run merge_target_files.py on two good target-files packages,
      observe no failure.
Change-Id: I86c8e5a2bc07c2c1896ac40afd32bc1d055447ee
2020-11-24 17:26:27 +00:00
Saeid Farivar Asanjan 75dc8d2dab Dump a new CSV with warning messages
Bug: 171068870
Test: manual
Change-Id: Ibb528940daf9a6cd99ece8e5a63d67aacdfc131c
2020-11-24 05:46:33 +00:00
Treehugger Robot 84e9bf1a94 Merge "Try to find host tools in the same directory as the caller" 2020-11-24 02:05:39 +00:00
Treehugger Robot bda655cead Merge "build: make fs_config_generator.py python2 explicitly" 2020-11-23 23:08:22 +00:00
Kelvin Zhang 788055c00b Merge "Add --disable_verity_computation option to ota script" 2020-11-23 19:43:24 +00:00
Kelvin Zhang caf7bbc39e Add --disable_verity_computation option to ota script
Test: treehugger, generate an OTA with --disable_verity_computation, serve it to
VABC device. Make sure it works

Change-Id: I0a5f3c018727cb3eed3edd004d60d09d13e34039
2020-11-23 13:24:34 -05:00
Jiyong Park c8c94ac38e Try to find host tools in the same directory as the caller
common.py is used by many python-based host tools. When a host tool
invokes another host tool, it's usually done with the name of the tool,
not the absolute path of the tool. For example, ["cmd", "arg"] instead
of ["out/soong/host/linux-x86/bin/cmd", "arg"].

Previously, the caller of the tool has to teach all the locations of the
sub-tools that the tool will internally use. But this isn't ideal;

1) It's against the abstraction. The existence of the sub tools and their
names are implementation detail of the containing tool. It's also
subject to change.

2) This isn't scalable. Sometimes a host tool invokes a large number of
host tools. Furthermore, the sub tools might invoke other sub-sub tools.
Then the location of the sub-sub-tools had to be known to the top-level
tool.

The idea here is to make use of the fact that

a) dependencies to the sub (and sub-sub) tools are already described in
the Android.bp and

b) the dependencies are guaranteed to be up-to-date and installed to the
host tool directory (out/soong/host/linux-x86/bin) by Soong.

Then by the time a host tool is invoked, all of its sub tools should
be found in the host tool directory. So, when "cmd" is about to be
invoked, common.py first tries to find it from the user-given paths. If
not found there, it falls back to search the directory where the current
tool is located at. Then finally falls back to the original name "cmd"
and expects it to be found under PATH.

Bug: 172414391
Test: m
Change-Id: Id7b44f9021be3bbf0631ddafe382ea3990f7ea74
2020-11-20 10:40:54 +09:00
Daniel Norman 38888d390f Turns on building vbmeta.img when merging if avb_enable=true.
Needed after commit 8218225794
which skips regenerating vbmeta.img if avb_building_vbmeta_image!=true.
Partial builds that disable vbmeta generation do not set this field.

Bug: 169892879
Test: Run merge_target_files.py on builds that have avb_enable=true but
      PRODUCT_BUILD_VBMETA_IMAGE=false. Observe vbmeta.img generation.
Change-Id: Id498dbd28c909e19cd35178b95a0b5dc22a1ac70
2020-11-19 14:51:15 -08:00
Yifan Hong 187e975927 Merge changes from topic "gki_vintf_level"
* changes:
  Pass kernel release file to check_vintf.
  Pass kernel version file to check_vintf.
2020-11-16 23:48:47 +00:00
Treehugger Robot 59dd38cdb0 Merge "Include complete boot image on builds containing gki apex" 2020-11-16 18:04:46 +00:00
Treehugger Robot fc65231af1 Merge "Support PRODUCT_BUILD_VBMETA_IMAGE for target files" 2020-11-16 13:59:41 +00:00
Kelvin Zhang 072efc92e2 Merge "Allow downgradeable secondary payload" 2020-11-14 03:09:49 +00:00
Yifan Hong 72e78f21b2 Pass kernel version file to check_vintf.
check_vintf now accept a path to a file that contains the kernel
version string. Use it.

Test: m check-vintf-all and manual inspect
Bug: 161317193
Change-Id: I4812c93f352686d73c5832d59e43a297d93e67a2
2020-11-13 18:45:03 -08:00
Fabien Sanglard 3d38ff9f38 Merge "Improve Zipalign test" 2020-11-13 20:43:11 +00:00
Fabien Sanglard e5fd58a62d Merge "Fix zipalign alignment error" 2020-11-13 20:42:50 +00:00
Bowgo Tsai 8218225794 Support PRODUCT_BUILD_VBMETA_IMAGE for target files
Currently we support skip building vbmeta.img when
PRODUCT_BUILD_VBMETA_IMAGE is set to false, which is used
when generating a system-only or vendor-only build artifacts.

However, when using `make dist` to build target files,
vbmeta.img is still generated. This change fixes this by
passing the related info to misc_info.txt for the release
tool to skip building vbmeta.img when necessary.

To make this happen, a device config should:
 1. Set PRODUCT_BUILD_VBMETA_IMAGE := false
 2. Remove 'vbmeta' from AB_OTA_PARTITIONS:
     AB_OTA_PARTITIONS += \
	 boot \
	 system \
    -    vbmeta \
	 dtbo \
	 product \
	 system_ext \
         vbmeta_system

Bug: 161659370
Test: `make dist` and checks the target file does not include vbmeta.img
Change-Id: Iddca9fe0b171a50a80e5a88aa462e118f18a7e6f
2020-11-13 11:28:17 +08:00
Fabien Sanglard a720635be2 Fix zipalign alignment error
Problem: Zipalign operates over several false assumptions. First it
assumes that zip entries are in the same order in the body and in
the Central Direcotry. Second, it assumes there are not space
between entries. This makes alignment incorrect when these asserts
are not true.

Solution: Don't align entries by tracking bias based on input zip
entry location. Calculate the expected alignment based on the out-
put zip file and correct with extra padding.

Fixes: 162117652
Test: Units Tests
Change-Id: Ia179338f658cab18a377cba2c7c8e629089a2785
2020-11-11 17:00:22 -08:00
Fabien Sanglard 4b4b495d88 Improve Zipalign test
Problem: Zipalign test invokes process but not verify.
Solution: Invoke both during test.

Test: Unit Test
Bug: NA
Change-Id: If603df301048f4b508e3429c4a8c77339ab351ae
2020-11-11 16:50:18 -08:00
Kelvin Zhang 6cdf39168f Allow downgradeable secondary payload
Test: treehugger
Bug: 172811557
Change-Id: I58f3123b9bc69342427f95250a5ee57ede4ac926
2020-11-11 14:23:00 -05:00
Kelvin Zhang 83ea783146 Filter out ab_partitions in source_info as well
When generating a incremental partial OTA, we need to filter out
partitions in ab_partitions which are not in partial list

Test: Generate a partial incremental OTA
Change-Id: Iff7748ce6181a9a231557de0539004211587232c
2020-11-11 14:23:00 -05:00
Kelvin Zhang 65029a2801 Include complete boot image on builds containing gki apex
Test: treehugger
Bug: 171906589
Change-Id: I231dac425ab14602492346006e331e4666173614
2020-11-11 09:16:32 -05:00
Kelvin Zhang 6f83343e7e Merge "Allow downgradable partial OTA" 2020-11-11 14:05:47 +00:00
Kelvin Zhang 80195724e7 Allow downgradable partial OTA
Cuttlefish doesn't support updatin vendor partition. So in testing we
need to exclude vendor and generate a downgrade partial package.

Test: generate a partial downgrade OTA
Bug: 171999375
Change-Id: I97e450d4063320cf55003a7a5c5ef913c321e067
2020-11-10 09:11:20 -05:00
Treehugger Robot b65e0b7648 Merge "Add the janitors and someone from Studio." 2020-11-10 08:55:31 +00:00
Elliott Hughes 3fc740b7cb Merge "Run zipalign tests in presubmit." 2020-11-10 00:19:31 +00:00
Elliott Hughes 3d8fb39b0e Add the janitors and someone from Studio.
Studio uses this too, not just the platform build.

Test: treehugger
Change-Id: I65c8a8f150e4d10ca31f67a816c9882d46f08ebf
2020-11-09 14:06:02 -08:00
Treehugger Robot 7bd7db619a Merge "Fix zipalign tests with GetExecutableDirectory" 2020-11-07 23:08:55 +00:00
Fabien Sanglard 6dfc6fb4c4 Fix zipalign tests with GetExecutableDirectory
Test: Self-tested
Bug: NA
Change-Id: Idc15c0e028b74a2b38c26af0e7d4a51f32210145
2020-11-06 17:04:59 -08:00
Treehugger Robot ba1811bf83 Merge "Checks for APK sharedUserIds that cross partition group boundaries." 2020-11-06 22:29:46 +00:00
Elliott Hughes c2039151ba Run zipalign tests in presubmit.
Test: treehugger
Change-Id: I2779c04313976d29c29c50920c0b356309c08439
2020-11-05 16:37:40 -08:00
Daniel Norman d33515623a Checks for APK sharedUserIds that cross partition group boundaries.
This check is used when merging target files to ensure that a merged
build does not contain any APKs that share UID across builds.

Bug: 171431774
Test: test_common
Test: Use merge_target_files.py to merge two partial builds,
      observe no failures for inputs without colliding APKs.
Test: Use merge_target_files.py to merge two partial builds,
      observe failure for inputs that have an APK that shares a
      UID across input partition groups.
Change-Id: I9dc57216882741ae46a99cfd7847f34702c75582
2020-11-05 15:17:05 -08:00
Treehugger Robot 6be3371c23 Merge "Refactor zipalign to allow unit tests" 2020-11-05 19:10:22 +00:00
Treehugger Robot 50c0f97fe7 Merge "Finds APK shared UID violations when merging target files." 2020-11-04 21:44:40 +00:00
Kelvin Zhang 31233e5922 Disable FEC computation for VABC
Test: treehugger, generate a package and make sure --disable_fec is
passed to delta_generator

Change-Id: If256af90b11a7fa03299c1272466d35ef3798a24
2020-11-03 15:45:30 -05:00
Fabien Sanglard 0f29f54695 Refactor zipalign to allow unit tests
Test: Self-tested
Bug: NA
Change-Id: I322b78c5e18082f7abf7427cdc62dbabcb18b4a0
2020-11-03 10:03:30 -08:00
Treehugger Robot 0806152b83 Merge "Fix partition timestamps for secondary payload" 2020-11-02 21:22:17 +00:00
Kelvin Zhang a59bb27754 Fix partition timestamps for secondary payload
When generating secondary payload, we exclude many partitions. These
partitions should be excluded from partition timestamps as well.

Bug: 172022126
Test: ota_from_target_files --include_secondary
Change-Id: Ia3121919154181dc882c4694ad2591dcb81e719f
2020-10-30 14:54:40 -04:00
Alistair Delva 38d49ca61a Merge "Allow flashing erofs system imgs" 2020-10-30 06:41:25 +00:00
Daniel Norman b8d52a2fdc Finds APK shared UID violations when merging target files.
This involved moving the find-shareduid-violation.py script to
releasetools to simplify the cross-tool usage. This new location aligns
this script with other similar python host tools.

In a future change this violation file will be used to check for
shared UID violations across the input build partition boundary.

Bug: 171431774
Test: test_merge_target_files
Test: Use merge_target_files.py to merge two partial builds,
      observe shared UID violations file contents in the result.
Test: m dist out/dist/shareduid_violation_modules.json
      (Checking that existing behavior in core/tasks is presereved)
Change-Id: I7deecbe019379c71bfdbedce56edac55e7b27b41
2020-10-29 09:33:26 -07:00
Chema Gonzalez f8b158b312 build: make fs_config_generator.py python2 explicitly
Test: run script in Fedora 31 with (default) python3 interpreter.

```
$ env python --version
Python 3.7.9
```

Change-Id: Ib4e68c5df54b092380e9fca473cd0dbbc0c18db7
2020-10-28 10:52:14 -07:00
Baligh Uddin 865b6605ca Merge "Move {libbacktrace,libunwindstack} from system/core -> system/unwinding/" 2020-10-28 05:16:46 +00:00
Baligh Uddin 4dc00c60da Move {libbacktrace,libunwindstack} from system/core -> system/unwinding/
BUG: 170465278
Test: TH
Change-Id: I7f0d7b1254e7a28a55599aa4ee6bb6f2ac789b60
2020-10-27 19:44:34 +00:00
Gao Xiang 961041ac80 Allow flashing erofs system imgs
Bug: 158339527
Test: build & boot
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
Change-Id: Ie74db8ebcd7214ffbec49d42b305b2f3ac2e8c9d
2020-10-28 01:24:39 +08:00
Felka Chang b511e7f33a Let genrule have the entrance depend on signapk
Currently, it may be a good idea to let genrule module depend on
java_library_host by changing soong. To change signapk module type from
java_library_host to java_binary_host makes the genrule can depend on the
signapk module directly and generate the signapk.jar at the same time.

Test: for i in `seq 1 10` ; \
        do\
            make clean; \
            make -j64 apexer_test_host_tools; \
        done
Bug: 160632228
Bug: 157878202
Change-Id: I63ae661b9b419ddc6e69ccfc9a29fd28b116c0b5
Merged-In: I63ae661b9b419ddc6e69ccfc9a29fd28b116c0b5
(cherry picked from commit d8442b4c88)
2020-10-27 14:50:39 +00:00
Joanne Chung 2c892f8700 Merge "Revert "Refactor zipalign to allow unit tests"" 2020-10-27 04:32:43 +00:00
Treehugger Robot 89dccb3cf9 Merge "Cleanup references to system/core/liblog -> system/logging/liblog" 2020-10-27 03:33:37 +00:00
Joanne Chung 2736c6f046 Revert "Refactor zipalign to allow unit tests"
This reverts commit d4f71a97bd.

Reason for revert: Looks to have broken apsp master b/171754371

Change-Id: I17ec7ab20834f6b0dbdf49668c41d3e76c0c1e41
2020-10-27 03:22:02 +00:00
Treehugger Robot 454e099c65 Merge "Refactor zipalign to allow unit tests" 2020-10-26 23:42:01 +00:00
Fabien Sanglard d4f71a97bd Refactor zipalign to allow unit tests
Test: Self-tested
Bug: NA
Change-Id: If5e42735853e1508fbeb9fb420a9a9f129c2218f
2020-10-26 12:31:03 -07:00
Baligh Uddin 84280dfa4c Cleanup references to system/core/liblog -> system/logging/liblog
BUG: 170387172
Test: TH
Change-Id: I206a8c43ba914381771826518f369bad328cc257
2020-10-26 14:34:45 +00:00
Baligh Uddin fabc1f6757 Merge "Cleanup references to system/core/base -> system/libbase" 2020-10-24 01:40:48 +00:00
Treehugger Robot 13e76a3e8f Merge "Formatting and general cleanup of merge_target_files." 2020-10-23 21:04:20 +00:00
Baligh Uddin 71fab5c75a Cleanup references to system/core/base -> system/libbase
BUG: 157656545
Test: TH
Change-Id: I0dc00f5f90cfdf439e3dc5d63e5aff178b1e9206
2020-10-23 14:40:36 +00:00
Daniel Norman dbbf5a3912 Formatting and general cleanup of merge_target_files.
Also adds a new validation that IMAGES/<partition>.img must come from
the same build that provides <PARTITION>/*.

Bug: 171431774
Test: python3 -m unittest test_merge_target_files
Test: Use merge_target_files to merge a build.
Change-Id: Ia1f3f2e65a0ca90962216fb0c5cdd24c4c150cae
2020-10-22 16:37:13 -07:00
Treehugger Robot 0b0d8d5b53 Merge "OTA: Support A/B devices custom images update." 2020-10-22 04:59:08 +00:00
Hongguang Chen 49ab1b90df OTA: Support A/B devices custom images update.
Add a new custom_image option to configure which custom images to use to
update custom partitions in A/B update.

This change also moves oem_settings to common option as A/B update will
use it to set oem properties too.

BUG: 171225290
Test: unittest pass, generate OTAs, flash to devices and check results
Change-Id: I279477d6b2954fb3705d7efede0a8bcd330c108b
2020-10-21 20:20:52 +00:00
Justin Yun 75f7cf9a2d Add fs_config_(dirs|files) for product and system_ext
These files were added to devices with the fs_config_files_nonsystem
module in base_vendor.mk. As they are partition-specific, move them
to each base_<partition>.mk file.

To add the fs_config_* files to the base_<partition>.mk files, it is
required to define the fs_config_* regardless of the existance of
partitions.

Bug: 170282998
Test: build and check if they are installed.
Change-Id: Ib8a2c75e2e0e93bb7030da981494e880f8465e5a
2020-10-21 03:39:22 +00:00
Tianjie Xu ec952237e2 Merge "sign_target_files_apks: Fix password encrypted keys handle" 2020-10-20 21:45:54 +00:00
Daniel Norman 29fda6ce51 Merge "Make merge_target_files more lenient, needed for cross-release merging." 2020-10-20 20:31:17 +00:00
Oleh Cherpak e555ab1848 sign_target_files_apks: Fix password encrypted keys handle
This patch restores the possibility of using password encrypted
keys for build signing.

Bug: 171221825
Test: 1. Generate password encrypted keys (write non empty passwords):
        $ subject='/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'
        $ mkdir ~/.android-certs
        $ for x in releasekey platform shared media; do \
            ./development/tools/make_key ~/.android-certs/$x \
            "$subject"; \
          done
      2. Create a file with passwords:
        Example of passwd file:
            [[[ 12345678 ]]] /home/user/.android-certs/releasekey
            [[[ 12345678 ]]] /home/user/.android-certs/platform
            [[[ 12345678 ]]] /home/user/.android-certs/shared
            [[[ 12345678 ]]] /home/user/.android-certs/media
            [[[ 12345678 ]]] /home/user/.android-certs/networkstack
      3. Tell system where to find passwords:
        $ export ANDROID_PW_FILE=/path/to/file/with/passwords
      4. Generate a release image:
        $ make dist
        $ sign_target_files_apks \
        -o --default_key_mappings ~/.android-certs \
        out/dist/*-target_files-*.zip \
        signed-target_files.zip

Signed-off-by: Oleh Cherpak <oleh.cherpak@globallogic.com>
Change-Id: I3e9d5318f69a2c3ac6eec64b36163b6544d49c90
2020-10-20 18:29:35 +00:00
Tianjie Xu d53d6f7169 Merge "Add ota script support to generate partial updates" 2020-10-19 18:47:12 +00:00
Treehugger Robot 6a2135b6e6 Merge changes from topic "split_ramdisk"
* changes:
  Exclude board cmdline/dtb/pagesize/base in generic boot image
  Add BOARD_MOVE_RECOVERY_RESOURCES_TO_VENDOR_BOOT.
  Add BOARD_EXCLUDE_KERNEL_FROM_RECOVERY_IMAGE.
  Add GKI global variable.
2020-10-17 00:40:54 +00:00
Tianjie Xu 11e5f33e2c Merge "Update ota script to match init's logic" 2020-10-16 23:10:37 +00:00
Tianjie b37c5be8cc Update ota script to match init's logic
In aosp/1459042, init has updated it's logic to use
ro.build.version.release_or_codename for fingerprint calculation.
The ota script needs the same update for ota targeting to work
correctly.

As a best effort to support pre-S dev stage builds, use
ro.build.version.release for sdk version < 30.

Bug: 170968068
Bug: 158483506
Test: unit tests, generate an OTA for S build
Change-Id: I01ca8a3b7b8b58f94b10f90ed0d27e688a72b866
2020-10-16 11:46:03 -07:00
Jaegeuk Kim ed754fb339 Add build property to enable fscompression
Bug: 162253911
Bug: 110272315
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Change-Id: I3cb7d0a055a7f83c9d0321d2b16f11f40a74f7aa
Merged-In: I3cb7d0a055a7f83c9d0321d2b16f11f40a74f7aa
2020-10-15 17:10:25 -07:00
Yifan Hong 38ab4d81ed Add ota script support to generate partial updates
Allow the ota generation script to take a list of partitions;
and thus generating a partial ota update package. This allow the
testing of partial update functionality, e.g. system-only updates.

Bug: 170921953
Test: unit tests, generate and apply a partial update on coral with system, system_ext, vbmeta_system
Change-Id: I0d8f93806dfbf7b781ea911117b02fc5c0971434
2020-10-15 16:30:13 -07:00
Yifan Hong 63c5ca19ab Add BOARD_EXCLUDE_KERNEL_FROM_RECOVERY_IMAGE.
BOARD_EXCLUDE_KERNEL_FROM_RECOVERY_IMAGE removes kernel from the
recovery image.

Test: build recovery image and unpack_bootimg
Test: build target files and unpack_bootimg IMAGES/recovery.img
      on device with GKI and dedicated recovery partition.

Bug: 156098440

Change-Id: I5f37d74ed954b26fe3dd778708b6ab9cfdc51f1e
2020-10-15 14:58:34 -07:00
Yifan Hong 125d0b63b9 Add function to extract timestamp from boot image
Also add toybox to otatools list.

Bug: 169169031
Test: build OTA
Change-Id: Ib14dbf46a8385ccf2a9c5a9c6f4e7fa9399cf0ba
2020-10-15 19:32:39 +00:00
Daniel Norman b0c75911b8 Make merge_target_files more lenient, needed for cross-release merging.
These changes were necessary to begin merging a new cross-release target
whose vendor half is frozen.

- MergeDynamicPartitionInfoDicts
  - Filters combined fields to remove duplicates
  - Merges `super_block_devices` as well as other keys that were not
    previously used by other targets consuming this tool.
- Introduces --allow-duplicate-apkapex-keys. This gives a warning rather
  than fatal error on duplicate apk/apex keys. This flag is needed for
  targets that cannot update a frozen half.
- (Formats merge_target_files.py)

Bug: 170683837
Test: Use merge_target_files to merge an S+R build, and boot.
Change-Id: Id5f787e730de8f8ef697e1f2f29ac6514221e58d
2020-10-15 10:16:31 -07:00
Treehugger Robot 5b7e8e0406 Merge "Allow zip64 support when opening zip files" 2020-10-08 23:35:56 +00:00
Yo Chiang d311799206 Merge changes from topic "exclude-system-shared-libs-from-elfcheck-suggestions"
* changes:
  Pass the value of system_shared_libs from Android.bp to check_elf_file.py
  Exclude system shared libs from fix suggestion
2020-09-28 03:54:27 +00:00
Chris Gross 69c6598b21 Merge "Misc fixes to release tools." 2020-09-26 18:13:24 +00:00
Stephen Hines 7f5d326ce2 Fix ordering of checks based on llvm-readobj output.
llvm-readobj has reordered some of its output, causing issues when
checking for DT_SONAME and other fields.

Bug: http://b/169195129
Test: m for a failing target
Change-Id: Ifbc120b703ec47b52e77c2baf0e4d080960e9ef9
2020-09-22 20:54:21 -07:00
Kelvin Zhang 928c2341a6 Allow zip64 support when opening zip files
When opening an zip file through zipfile.ZipFile(), python2 by default
disables zip64 support. To support update files >4GB, we manually add
allowZip64 to override the setting.

Test: generate && serve an OTA

Change-Id: I9645e963ced830cc2d3a4b72bc63b9369a1cefe8
2020-09-22 16:53:39 -04:00
Logan Chien 751a987bcc Exclude system shared libs from fix suggestion
This commit removes system shared libs (e.g. libc, libdl, or libm) from
the prebuilt ELF check fix suggestion.

Bug: 141925662
Test: Write a bad Android.mk module and check fix suggestions
Change-Id: I4a827d07b24a976c1910b814126790abbeccc793
2020-09-22 18:55:11 +08:00
Chih-hung Hsieh d890f596ce Merge "Add new Java/C++/Yacc warning patterns" 2020-09-22 07:11:54 +00:00
Chris Gross 435b8fe7ba Misc fixes to release tools.
- Remove vendor_dlkm and odm_dlkm from the default list of items that
need to be merged
- Allow a prebuilt system.img to be used when adding images to the
target files archive

Test: Built Pixel 3a and inspected the output
Change-Id: If5291fff559ed9517133f6b476d8dee2500c5b4c
2020-09-21 13:37:21 -07:00
Tom Cherry 787ef71f29 Merge "Use libcutils_headers for android_filesystem_config.h" 2020-09-21 17:26:10 +00:00
Chih-Hung Hsieh e8f4a716ad Add new Java/C++/Yacc warning patterns
Test: warn.py --url=http://cs/android --separator='?l=' build.log > warnings.html
Test: warn.py --gencsv build.log > warnings.csv
Change-Id: I49eafa322902f0587738b583b6262117595318ee
2020-09-18 21:51:06 -07:00
Tom Cherry 54752bf0ae Use libcutils_headers for android_filesystem_config.h
android_filesystem_config.h is found since system/core/include is on
the include path for all projects and contains a symlink to the real
android_filesystem_config.h.  This is fragile and the below bug seeks
to remove this symlink and have users correctly depend on
libcutils_headers.

Bug: 165825252
Test: build
Change-Id: Ie921c4fd96d6e4e42d127bfbfe7a892756ad9495
2020-09-18 22:49:10 +00:00
Treehugger Robot 281affb8a8 Merge "fs_config: Update docs" 2020-09-17 17:15:56 +00:00
Tom Cherry 329545d912 fs_config: Update docs
1. Convert to README.md
2. Sync come content with the updates on source.android.com
3. fs_config uses bionic/libc/kernel/uapi/linux/capability.h, not
system/core/include/private/android_filesystem_capability.h as the
capability header, so update the documentation accordingly.

Test: n/a
Change-Id: I24a084d7a804d3f5d2259cfcea85b8ff4e79d290
2020-09-16 16:21:30 -07:00
Yifan Hong 4e501b58b8 Merge "Revert "Add modules partition."" 2020-09-16 22:45:54 +00:00
Yifan Hong c08cbf0f4a Revert "Add modules partition."
Revert submission 1413808-modules_partition

Reason for revert: modules partition no longer needed
Reverted Changes:
Iceafebd85:Add modules partition
I2fa96199a:rootdir: Add modules directory
Ie397b9ec6:Add modules partition.
I4200d0cf5:fastboot: add modules partition

Bug: 163543381

Change-Id: Iec1d9421bbfeb114fb705f85d910c6def1f211b6
2020-09-15 19:09:26 +00:00
Luca Stefani e0fc4caea9 Sync modules required by releasetools
Test: m otapackage
Change-Id: Id9e43909f3600224af5649b2615dffd7d59723eb
2020-09-12 06:10:13 +00:00
Joe Onorato 55c090ebe0 Merge changes I4a43604b,I7200f5f9,If3402a0a
* changes:
  Add CSV output of remaining makefiles for bp converstion
  Refactor the mongo main() into a class in mk2bp_catalog.py
  Add per-partition summaries and "easy" transitions to mk2bp_catalog
2020-09-11 05:46:48 +00:00
Yifan Hong b1e66600c0 Merge "Add modules partition." 2020-09-11 00:25:24 +00:00
Joe Onorato 3198607cfd Add CSV output of remaining makefiles for bp converstion
Test: m out/target/product/$(get_build_var TARGET_DEVICE)/mk2bp_remaining.html out/target/product/$(get_build_var TARGET_DEVICE)/mk2bp_remaining.csv
Change-Id: I4a43604b2d536387c787b02a4627a0ad9e2a6ff9
2020-09-10 14:09:29 -07:00
Joe Onorato 8ade9b2e00 Refactor the mongo main() into a class in mk2bp_catalog.py
Test: m out/target/product/$(get_build_var TARGET_DEVICE)/mk2bp_remaining.html
Change-Id: I7200f5f9a6735bb0da3928aa5dc74723d69752d3
2020-09-10 14:09:28 -07:00
Joe Onorato 934bd8dcfc Add per-partition summaries and "easy" transitions to mk2bp_catalog
"Easy" means makefiles that are both clean themselves and
only depend on clean targets themselves too.

Test: m out/target/product/$(get_build_var TARGET_DEVICE)/mk2bp_remaining.html
Change-Id: If3402a0ada5d9db5fb04c617d197ab28d695b03c
2020-09-10 14:09:28 -07:00
Yifan Hong c0f187f5f9 Add modules partition.
Modules partition is a dynamic read-write partition.
- AVB is not enabled on the partition
- OTA is file-based; see follow up CL for details
- No build prop files; in particular, no build fingerprint
- No fs_config
- No notice files; notice files are included in individual APEXes

Test: build on CF
Bug: 163543381

Change-Id: Ie397b9ec61dfd1c158450d050196024604854d4d
2020-09-09 15:00:02 -07:00
Kelvin Zhang 7a1364a36a Merge "Generate partition timestamps in ota_from_target_files" am: 49fe034137
Original change: https://android-review.googlesource.com/c/platform/build/+/1402687

Change-Id: I23b48a511fded51330d7620ec748d36225175e2f
2020-09-04 12:37:45 +00:00
Kelvin Zhang 39aea44a96 Generate partition timestamps in ota_from_target_files
Test: make an OTA package
Bug: 162553432
Change-Id: I17b9f1b24307255b1e5115de12fa516126b32365
2020-09-03 14:04:21 -04:00
Treehugger Robot 5040d41e20 Merge "Add ota_metadata_pb2 to otatools.zip" am: 58c793ab0b
Original change: https://android-review.googlesource.com/c/platform/build/+/1416089

Change-Id: I77eeb78be1bcdfdae0a7f7b942af0a0307e1cc75
2020-09-02 12:56:13 +00:00
Kelvin Zhang 4d50ea4dd7 Add ota_metadata_pb2 to otatools.zip
Sometimes tests are being run by executing python -m unittest *.py. In
that case, invocation will fail due to missing ota_metadata_pb2, which
is auto generated from ota_metadata.proto(added in recent change).

Bug: 166718341
Test: unzip otatools.zip and python releasetools/test_utils.py
Change-Id: I597c7ef6dbd8e46ccde020b3122cc38a8df28dc9
2020-09-01 16:58:08 -04:00
Xin Li 37a9f38115 Merge "Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)" into stage-aosp-master 2020-09-01 20:03:45 +00:00
Tianjie Xu f55227e25f Merge "Add per-partition build info into ota metadata" 2020-09-01 18:43:30 +00:00
Tianjie 2bb14864e0 Add per-partition build info into ota metadata
As part of the ota metadata work to support partial update, we should
add the per-partition device name, fingerprint, and version to the ota
metadata.

Bug: 151088567
Bug: 166154730
Test: unit tests pass, generate a package
Change-Id: I7a71a4860aad651daf9e62df39c60d72727c14ff
2020-08-31 14:17:10 -07:00
Tianjie d5563371d4 Add more enums to the ota type
We'd better have a default type for enums. It's safe to change the
proto definition now since no one is parsing the new meta.

Also add the brick type for manually generated brick pkgs.

Bug: 166154730
Test: unit tests pass
Change-Id: Ie6d4e695c298313276b559601509c9a7a7ee91b1
2020-08-31 14:06:31 -07:00
Xin Li c2fd58fe67 Merge Android R (rvc-dev-plus-aosp-without-vendor@6692709)
Bug: 166295507
Merged-In: Ie00dbea32e074838466eeed00f316250c2b4f3c2
Change-Id: I9ee3441fd4d063b9dd489cb1b01d3dbc11332457
2020-08-29 01:19:44 -07:00
Treehugger Robot 002863d698 Merge "Switch to aapt2" 2020-08-29 00:30:28 +00:00
Kelvin Zhang f5f3eaf01c Switch to aapt2
We already moved to aapt2 in Android.bp, no aapt binary in otatools.zip.
So tests should be updated accordingly.

Test: unittest
Change-Id: I29776d75e108e16bcceda49b8be47a338ee8bc1c
2020-08-28 17:40:29 -04:00
Ram Muthiah 2ad69ab090 Merge "Revert "Revert "Add prebuilt bootloader var and add bootloader t..."" 2020-08-28 04:04:24 +00:00
Ram Muthiah 0c4a352409 Revert "Revert "Add prebuilt bootloader var and add bootloader t..."
Revert submission 1411608-revert-1402265-cf-x86-bootloader-XGKLNEQAPF

Reason for revert: Breakage originally caused by change was fixed. Refer to tracking bug for details.
Reverted Changes:
I89e95d4a6:Revert "Add prebuilt bootloader var and add bootlo...
I8fe199351:Revert "Added bootloader binary to cf x86 based ta...

Bug: 166380489
Bug: 164917252
Test: local build and Treehugger build
Change-Id: Ia895aedd77f2ec825aefecaf74c6dfc1ce631c93
2020-08-27 13:18:24 -07:00
Tianjie a2076137a1 Define the protobuf for OTA metadata
Background in http://go/android-partial-updates. For partial update
(e.g. system-only) or devices with mixed build, the current
fingerprint & device name no longer suffice as the precondition to
install the package.

Therefore, we need to additionally include the per-partition build
props into the ota metadata. We also define a protobuf for the metadata
so it can be extended later. The metadata of the legacy format is also
kept for backward compatibility.

Bug: 151088567
Test: unittest pass, generate an OTA and check the result
Change-Id: I716f7da54a393cd340280dbddc3c92b3460f8ef8
2020-08-26 16:17:35 -07:00
Kelvin Zhang ff2c7454fe Merge "Fix lint errors on ota_utils.py" 2020-08-26 13:34:19 +00:00
Ankit Goyal 3c94b2710a Merge "Revert "Add prebuilt bootloader var and add bootloader to img zip"" 2020-08-26 06:54:34 +00:00
Ankit Goyal 6805873751 Revert "Add prebuilt bootloader var and add bootloader to img zip"
Revert submission 1402265-cf-x86-bootloader

Reason for revert: Droidcop-triggered revert due to breakage https://android-build.googleplex.com/builds/quarterdeck?branch=git_master&target=adt3-userdebug&lkgb=6791187&lkbb=6791249&fkbb=6791216, bug 166380489
Reverted Changes:
Id67b7bddd:Add prebuilt bootloader var and add bootloader to ...
Id713eaf20:Added bootloader binary to cf x86 based targets

Change-Id: I89e95d4a6a0534c8217785be9b90632fa110bc41
2020-08-26 06:40:59 +00:00
Treehugger Robot ff6d873873 Merge "build_image relies on mkuserimg_mke2fs." 2020-08-26 04:46:56 +00:00
Ram Muthiah 861f05fb83 Merge "Add prebuilt bootloader var and add bootloader to img zip" 2020-08-26 02:59:22 +00:00
Yifan Hong 770234520d build_image relies on mkuserimg_mke2fs.
Add missing dependency.

Test: TH
Bug: 162921175
Change-Id: Ide1fe0b823e5e7a06c6c0252a5bef7bfaf6f7c28
2020-08-25 16:15:30 -07:00
Ram Muthiah 41a03d3aee Add prebuilt bootloader var and add bootloader to img zip
Needed to enable a prebuilt bootloader to make it to the dist and img
directories.

Added a flag to ensure only devices that request will have the update
package populated with a bootloader

Bug: 164917252
Test: local build and Treehugger build
Change-Id: Id67b7bddda14b51a523f794976591c6de04d71ca
2020-08-25 22:12:27 +00:00
Kelvin Zhang 2e41738566 Fix lint errors on ota_utils.py
Pylint reports lot of error on ota_utils.py, because some members of
OPTIONS object aren't defined. This CL moves some definition from
ota_from_target_files.py to ota_utils.py to fix these lint errors.

Test: Make an OTA package
Change-Id: I1e9f255d5919712b13329046c72650dfac184701
2020-08-25 15:58:24 -04:00
Kelvin Zhang dd833dcfb2 Searches for debugfs_static in search path
When signing APks on build server, there might not be an android repo.
Usually deapexer.py will be run from otatools.zip after exctration. So
we should look for debugfs_static in search path

Bug: 165347497
Test: Use sign_apk_from_target_files without android repo
Change-Id: Ifaf91764ca0dc08d010f43a24bfe975a07a71e7f
2020-08-21 14:13:13 -04:00
Treehugger Robot 78439f4668 Merge "Searches for debugfs_static in search path" 2020-08-19 23:27:08 +00:00
Kelvin Zhang d6b799a627 Searches for debugfs_static in search path
When signing APks on build server, there might not be an android repo.
Usually deapexer.py will be run from otatools.zip after exctration. So
we should look for debugfs_static in search path

Bug: 165347497
Test: Sign using self-service
Change-Id: I608870b0184bd8f5f07afa53355b39de47a97e1c
2020-08-19 16:12:23 -04:00
Tianjie e3c31eac61 Fix the syntax in install-recovery.sh
We recently modify the logic in install-recovery.sh to support
non-A/B updates on VAB launched devices. And the position of the
$(getprop ro.boot.slot_suffix) is misplaced for the recovery image.
For example:
"if ! applypatch --check EMMC$(getprop ro.boot.slot_suffix):/dev/block/recovery:SHA1"
should be:
"if ! applypatch --check EMMC:/dev/block/recovery$(getprop ro.boot.slot_suffix):SHA1"

Bug: 156979431
Test: rerun add_image_to_target_file, then validate_target_files
Change-Id: Id2d45e281151a46d66b37ea8fdb7b5cb4ae72e81
(cherry picked from commit e55f62c971)
2020-08-17 18:38:13 -07:00
Yo Chiang 76279b102f Merge "Revert^2 "releasetools: Disable host module on darwin"" 2020-08-11 04:56:55 +00:00
Yo Chiang 096c515976 Revert^2 "releasetools: Disable host module on darwin"
0ac3666667

Exempt-From-Owner-Approval: Original change was approved

Change-Id: I4030dbc072a90f9b8704926c0956787c75bc5583
2020-08-11 04:56:31 +00:00
Treehugger Robot b83c9d0461 Merge "extract_kernel.py: add support to output compiler information" 2020-08-11 02:32:23 +00:00
Kelvin Zhang 2f6571884d Merge "Move non-AB OTA generation code to a separate file" 2020-08-11 01:35:23 +00:00
Kelvin Zhang cff4d7606d Move non-AB OTA generation code to a separate file
Test: Generate a non-AB OTA, apply it
Change-Id: I2f1afbe70d17356fcbf4d59901d201a76a3d6c4f
2020-08-10 16:22:22 -04:00
Yongqin Liu 2c8b8eca5e extract_kernel.py: add support to output compiler information
Test: tested with android-mainline gki kernel from
    https://android.googlesource.com/kernel/prebuilts/mainline/arm64/

Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
Change-Id: Ibf3eeb3679b233cd0d54501d385f442e0c0c34ec
2020-08-08 17:15:19 +08:00
Treehugger Robot e9bceb662b Merge "Add more details to divergent vndk lib error." 2020-08-06 21:27:09 +00:00
Steven Moreland 9d27901e4a Add more details to divergent vndk lib error.
Bash better broadcast better build-backstop baggage because baffled
beings' bewilderement begets badness.

Fixes: 162379658
Test: view error by removing "libbinder" from
  VndkMustUseVendorVariantList.
Change-Id: Ib425e30bc7b5388b78af4bfab84dd24072d550a6
2020-08-06 18:02:31 +00:00
Insaf Latypov 0ac3666667 Revert "releasetools: Disable host module on darwin"
Revert "conscrypt: Disable host module on darwin"

Revert submission 1385032-mac-required-fix

Reason for revert: breaks the build on aosp-master
Reverted Changes:
I13335299c:releasetools: Disable host module on darwin
I728a4677a:conscrypt: Disable host module on darwin
I8f3435ff1🎨 Remove dependencies on platform unavailable m...
Ibb9248ef4🎨 Disable host module on darwin
I7db68a629:libcore: Disable host module on darwin

Bug: 163030797
Change-Id: I3b3991b91652c426d8f22d60c59c44d640059619
2020-08-06 15:48:12 +00:00
Yo Chiang 79d10eddbf Merge "releasetools: Disable host module on darwin" 2020-08-06 04:02:52 +00:00
Treehugger Robot 090c0c9c6e Merge "Delete unused subdirectory." 2020-08-05 04:50:59 +00:00
Yo Chiang d65e8127f1 releasetools: Disable host module on darwin
Disable releasetools components that depend on disabled
modules on darwin host.

Bug: 162102724
Bug: 7456955
Test: TH build mac host target
Change-Id: I13335299ca0872cfa956ed756650a966f269080e
2020-08-04 07:12:57 +00:00
Yifan Hong 3a9bd34ec2 Merge "Add SetHostToolLocation." 2020-07-31 22:43:31 +00:00
Tianjie Xu 8001114aee Merge "Update language to comply with Android's inclusive language guidance" 2020-07-31 18:15:12 +00:00
Treehugger Robot c21f30ca8a Merge "Update language to meet Android's inclusive language guidance" 2020-07-31 14:49:05 +00:00
Treehugger Robot bb62634fbd Merge "Let extract_kernel return full kernel release." 2020-07-31 03:00:29 +00:00
Yifan Hong 351b6b811d Let extract_kernel return full kernel release.
Also add relevant soong rules for genrules to use it.

Test: use it in GKI APEX
Bug: 161563386
Change-Id: I04f80ba91748ee5783088d4e20e6438418863f62
2020-07-30 23:53:25 +00:00
Joel Galenson 154ac1d870 Update language to meet Android's inclusive language guidance
See https://source.android.com/setup/contribute/respectful-code for reference.

Bug: 161896447
Test: Build
Change-Id: I8d33207c84f424e89f367b1c917661347402ae77
2020-07-30 15:05:55 -07:00
Yifan Hong 8e332ff3a7 Add SetHostToolLocation.
Add common.SetHostToolLocation so that, when calling
various common.Run* functions, replace the first
argument with previously set paths. This is needed
when the script is executed in a Soong sandbox.

Bug: 161563386
Test: pass
Change-Id: If3b40b518fc7fe068677d39d604e3f6578a12ea3
2020-07-29 19:44:27 -07:00
Tianjie a85bdf0bf0 Update language to comply with Android's inclusive language guidance
More details in: https://source.android.com/setup/contribute/respectful-code

Bug: 161896447
Test: build, run unittests
Change-Id: I35732333da0dd0c871f3e37c4af1b2211647d1e3
2020-07-29 12:06:00 -07:00
Ivan Lozano ad970e69f3 Merge "Update language to comply with Android’s inclusive language guidance" 2020-07-29 13:24:06 +00:00
Treehugger Robot bc02bb355c Merge "Update language to comply with Android's inclusive language guidance" 2020-07-29 05:45:10 +00:00
Ivan Lozano b021b2aeb3 Update language to comply with Android’s inclusive language guidance
See https://source.android.com/setup/contribute/respectful-code for reference

 #inclusivefixit

Bug: 161896447
Test: N/A
Change-Id: I9fd970726d739da658f6419a1ccd3e4952b17c78
2020-07-28 15:43:15 -04:00
Kelvin Zhang b1cb5b82a6 Merge "Fix some wording to comply with respectful-code" 2020-07-28 17:58:35 +00:00
Yo Chiang 1237c1f577 Update language to comply with Android's inclusive language guidance
See https://source.android.com/setup/contribute/respectful-code for
reference

Bug: 161896447
Test: TH
Change-Id: I2a93e520120fa52a190a445b4d4c417c299b9727
2020-07-29 01:20:01 +08:00
Dan Albert 57600a4735 Delete unused subdirectory.
Directory was flagged for not being compliant with inclusive language
guidance, and there are no references to this directory anywhere that
I can find, so just delete it.

Test: None
Bug: 161896447
Change-Id: I28892cfd32b54987380d472d032d741eb78f4156
2020-07-24 14:07:35 -07:00
Kelvin Zhang c693d95f86 Fix some wording to comply with respectful-code
https: //source.android.com/setup/contribute/respectful-code
Test: Run unit tests
Bug: 161896447
Change-Id: I9a3676b6f7bb6be43756fdf18b1d8b9ec41fb4cf
2020-07-23 10:04:12 -04:00
Bob Badour 5e9e1fbe83 Add support for combining the notices from 2 roots.
Apparently needed for cts tests due to apks putting intermediate files under
TARGET_OUT even when going to HOST_OUT.

Test: m with and without change verified no differences in notice files

Change-Id: Iea50b6bd9af9452a2600c97e2f7cb89218b9be7f
2020-07-20 17:32:14 +00:00
Yifan Hong 76a337f65e Add odm_dlkm partition. am: f496f1b94f am: 4f2ea25d5f
Original change: https://android-review.googlesource.com/c/platform/build/+/1363859

Change-Id: I4aaa3967e889923f0958afaa3227474176e36ea9
2020-07-17 22:52:14 +00:00
Yifan Hong 4f2ea25d5f Add odm_dlkm partition. am: f496f1b94f
Original change: https://android-review.googlesource.com/c/platform/build/+/1363859

Change-Id: I8c7af808f72de5611cb52f34c0138bc97ceb6254
2020-07-17 22:27:15 +00:00
Yifan Hong 997d227648 Merge changes from topic "odm_dlkm"
* changes:
  Create $OUT/{vendor,odm}/lib before symlink modules
  Add odm_dlkm/etc/build.prop
  Install ODM dlkm to appropriate place and symlink
  Add notice files for odm_dlkm
  Add odm_dlkm partition.
2020-07-17 22:05:26 +00:00
Treehugger Robot 29e347e4be Merge changes Iacfe74ae,I7c380b60 am: ce13163827 am: 5771787ccb
Original change: https://android-review.googlesource.com/c/platform/build/+/1365306

Change-Id: I81debee224374fd8f60d1cad12a4e7ac6c0591ab
2020-07-17 04:59:48 +00:00
Joe Onorato 02fb89a4d7 Add mk2bp_catalog.py that outputs more data about makefiles to be converted to soong.
- Adds makefile and which files are installed to the SOONG_CONV CSV file
- Updates soong_to_convert.py to be able to parse that
- Adds new script that is more detailed.
- Outputs that file as part of the droidcore build to
  $(OUT_DIR)/target/product/$(TARGET_DEVICE)/mk2bp_remaining.html

Test: m out/target/product/$(get_build_var TARGET_DEVICE)/mk2bp_remaining.html
Change-Id: I7c380b6070754f4329bf3965595751e4dac794a0
2020-07-16 18:19:15 -07:00
Yifan Hong f496f1b94f Add odm_dlkm partition.
Test: define odm_dlkm in CF.
Bug: 156020364
Change-Id: I5593d09a1cdf13e651a13e951336555a7475865e
2020-07-16 12:34:29 -07:00
Tianjie fb27b477d8 Use apksigner in check_target_files_signatures
Some apk files, e.g. Chrome.apk has switched to use v2+ scheme.
And the apk file no longer has a META-INF/CERT.RSA. So, the
signature parsing script should use apksigner to dump the cert
info.

Leave the parsing of META-INF/CERT.RSA as a fallback, as some apks
fail the `apksigner verify`. The script also switches to store the
digest of the cert instead of the raw bytes.

Bug: 157735036
Test: run check_target_files_signatures on aosp|signed target-files
(cherry picked from commit a9a50cf6e3)

Change-Id: I910cd1aa91d1e446bef7097871af4171c285685d
2020-07-14 21:48:48 -07:00
Tianjie Xu 25d2fb7a35 Merge "Use apksigner in check_target_files_signatures" am: 8542706908 am: e5eda8950f
Original change: https://android-review.googlesource.com/c/platform/build/+/1362881

Change-Id: I3c90e26190b8d07f1e57894f50b99a4919fa71fc
2020-07-15 03:57:33 +00:00
Tianjie Xu 8542706908 Merge "Use apksigner in check_target_files_signatures" 2020-07-15 03:25:25 +00:00
Tianjie a9a50cf6e3 Use apksigner in check_target_files_signatures
Some apk files, e.g. Chrome.apk has switched to use v2+ scheme.
And the apk file no longer has a META-INF/CERT.RSA. So, the
signature parsing script should use apksigner to dump the cert
info.

Leave the parsing of META-INF/CERT.RSA as a fallback, as some apks
fail the `apksigner verify`. The script also switches to store the
digest of the cert instead of the raw bytes.

Bug: 157735036
Test: run check_target_files_signatures on aosp|signed target-files

Change-Id: I910cd1aa91d1e446bef7097871af4171c285685d
2020-07-14 17:23:07 -07:00
Yifan Hong 9fce02b625 Add vendor_dlkm partition. am: cfb917a1c7 am: 58af66905e
Original change: https://android-review.googlesource.com/c/platform/build/+/1316438

Change-Id: I84a4ffc60242ba92ade09dba87ba53a15394fe9f
2020-07-14 19:47:16 +00:00
Yifan Hong cfb917a1c7 Add vendor_dlkm partition.
Test: define vendor_dlkm for CF.
Bug: 156020364

Change-Id: I459059e1c20df7a285a377b8b3bbc8decc60b124
2020-07-13 14:20:56 -07:00
Kelvin Zhang 922c4bd519 Merge "Fix validate_target_files for target files modified by" am: 757fd2768e am: dd940c163d
Original change: https://android-review.googlesource.com/c/platform/build/+/1350724

Change-Id: I038ec5cb51a174e174d3b68565424de6feaee95e
2020-07-09 00:57:17 +00:00
Kelvin Zhang 757fd2768e Merge "Fix validate_target_files for target files modified by" 2020-07-09 00:32:52 +00:00
Kelvin Zhang 951799f56e Merge "Check for duplicate entries in build.prop in validation script" am: 46a6ab8c05 am: 11a1e1b3bd
Original change: https://android-review.googlesource.com/c/platform/build/+/1352903

Change-Id: Ia764ae32e9e39effb73348d68fb9067be669c3d4
2020-07-06 21:59:49 +00:00
Kelvin Zhang 46a6ab8c05 Merge "Check for duplicate entries in build.prop in validation script" 2020-07-06 21:13:01 +00:00
Kelvin Zhang f2e846fb91 Check for duplicate entries in build.prop in validation script
Test: Added Unit test
Bug: 143013566
Change-Id: Ida14507a4a6034990bc81d5263ef1cea34f589c1
2020-07-06 15:13:11 -04:00
Treehugger Robot 92346502ba Merge "Fix up _CheckSecondTokenNotSlotSuffixed function." 2020-07-01 08:39:42 +00:00
Yifan Hong c8afa54838 Merge "Fix up _CheckSecondTokenNotSlotSuffixed function." into rvc-dev am: 2206bb29fd
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/+/12051901

Change-Id: Ib30dd8adc20af705060d2e5633641d759f363d3b
2020-07-01 01:48:17 +00:00
Yifan Hong 2206bb29fd Merge "Fix up _CheckSecondTokenNotSlotSuffixed function." into rvc-dev 2020-07-01 01:38:10 +00:00
P.Adarsh Reddy 1ed2d66259 Fix up _CheckSecondTokenNotSlotSuffixed function.
Incremental OTA generation was failing since the
function _CheckSecondTokenNotSlotSuffixed was in
broken state, this change fixes the same.

Bug: 160215626
Test: Incremental OTA now gets generated and applied
      successfully a non-ab device.

Change-Id: Ifb3886396fe65dcdaeba16c568419b9fa084193d
Merged-In: Ifb3886396fe65dcdaeba16c568419b9fa084193d
2020-07-01 01:37:56 +00:00
P.Adarsh Reddy 6924eb7d40 Fix up _CheckSecondTokenNotSlotSuffixed function.
Incremental OTA generation was failing since the
function _CheckSecondTokenNotSlotSuffixed was in
broken state, this change fixes the same.

Bug: 160215626
Test: Incremental OTA now gets generated and applied
      successfully a non-ab device.

Change-Id: Ifb3886396fe65dcdaeba16c568419b9fa084193d
(cherry picked from commit 52defe48d52a770c326f489357ab8877b211541e)
Merged-In: Ifb3886396fe65dcdaeba16c568419b9fa084193d
2020-06-30 14:49:04 -07:00