* changes:
fs_config: add unit tests
fs_config: drop fs_config_files/dirs PRODUCT_PACKAGES requirement
fs_config: add group to build
fs_config: introduce group generator
fs_config: add passwd to build
fs_config: introduce passwd generator
fs_config: generate friendly in AID class
fs_config: limit characters for AID_<name> sections
fs_config: generate oem AID header file
fs_config: android_id header generator
fs_config: support parsing android_filesystem_config.h
fs_config: modularize fs_config_generator
Add support for specifying number of inodes when creating
system, vendor, oem partitions. These are all read-only
and have no use for extra inodes. Removing extra inodes
saves a lot of space.
Bug: 32246383
Change-Id: I13f1d4614b64a4abc752c42a1c65d3d151481c21
(cherry picked from commit b59eca3586)
In two-step OTAs, we write recovery image to /boot as the first step so
that we can reboot from there and install a new recovery image to
/recovery. However, bootloader will show "Your device is corrupt"
message when booting /boot with the recovery image. Because the recovery
image encodes the path of "/recovery" as part of the signature metadata,
which fails the verified boot.
This CL generates a special "recovery-two-step.img" in addition to the
regular recovery.img. This image encodes "/boot" when being signed,
which will be flashed to /boot at stage 1/3 in a two-step OTA.
Here are the desired changes:
- 'IMAGES/recovery-two-step.img' exists in target_files.zip for non-A/B
targets (e.g. bullhead). The image should not exist for targets that
don't have a recovery partition (e.g. A/B devices like sailfish).
- <device>-img.zip should not contain 'recovery-two-step.img'.
- Nothing should change when building non-two-step OTAs. For two-step
OTAs, 'recovery-two-step.img' should be included in the OTA package;
'updater-script' should flash this image to /boot at stage 1/3.
- When building a two-step OTA with an input TF.zip that doesn't have
IMAGES/recovery-two-step.img, it should use the existing
IMAGES/recovery.img instead.
Bug: 32986477
Test: Tested the steps above on bullhead and sailfish.
Change-Id: I34e6c599bcf2011d4cd5c926999418b3975d6d0f
(cherry picked from commit d42e97ebb4)
In two-step OTAs, we write recovery image to /boot as the first step so
that we can reboot from there and install a new recovery image to
/recovery. However, bootloader will show "Your device is corrupt"
message when booting /boot with the recovery image. Because the recovery
image encodes the path of "/recovery" as part of the signature metadata,
which fails the verified boot.
This CL generates a special "recovery-two-step.img" in addition to the
regular recovery.img. This image encodes "/boot" when being signed,
which will be flashed to /boot at stage 1/3 in a two-step OTA.
Here are the desired changes:
- 'IMAGES/recovery-two-step.img' exists in target_files.zip for non-A/B
targets (e.g. bullhead). The image should not exist for targets that
don't have a recovery partition (e.g. A/B devices like sailfish).
- <device>-img.zip should not contain 'recovery-two-step.img'.
- Nothing should change when building non-two-step OTAs. For two-step
OTAs, 'recovery-two-step.img' should be included in the OTA package;
'updater-script' should flash this image to /boot at stage 1/3.
- When building a two-step OTA with an input TF.zip that doesn't have
IMAGES/recovery-two-step.img, it should use the existing
IMAGES/recovery.img instead.
Bug: 32986477
Test: Tested the steps above on bullhead and sailfish.
Change-Id: I34e6c599bcf2011d4cd5c926999418b3975d6d0f
The 'system_img_path' parameter was introduced in commit
d995f4b04d, but became obsolete since
commit 2ce63edab7.
Test: m dist
Change-Id: Iffd496d929db5cc3dfc955a48bfc1b1317bd012f
Add some initial unit tests.
The unit tests themselves are inlined into the tool and
can be executed by running the "test" commandlet.
Example:
$ python -m unittest test_fs_config_generator.Tests
.............
----------------------------------------------------------------------
Ran 13 tests in 0.004s
OK
Test: run the test commandlet and observe for failures.
Change-Id: I1bada385fa841fd50fa958997d440f1198e15198
Signed-off-by: William Roberts <william.c.roberts@intel.com>
When configuring fs_config_files or fs_config_dirs for file_system
capabilities, drop the requirement that OEMs must add the target
to PRODUCT_PACKAGES. This limits the configuration requirement
to only needing to set the new and preferred TARGET_FS_CONFIG_GEN
or the older TARGET_ANDROID_FILESYSTEM_CONFIG_H method.
Test: That only setting TARGET_FS_CONFIG_GEN results in passwd and
group in the build image.
Change-Id: I818854fa1b3e94edaff59a32bd7cf23cf9b504aa
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Introduce a generator that outputs group files per man(5) group.
Succinctly, the output is a colon delimited string containing the following
fields:
* group name
* encrypted password (optional)
* gid (int)
* userlist (str,...)
Multiple colon delimited lines may exist, but will not be separated
across lines.
Sample generator output:
foo::2900:
foo_bar::2901:
custom_oem1::2902:
Test: That make group produces the group file.
Change-Id: Idd3fe925a09a227c6e894e1b5d2b3873b01531c6
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Introduce a generator that outputs passwd files per man(5) passwd.
Succinctly, the output is a colon delimited string containing the following
fields:
* login name
* encrypted password (optional)
* uid (int)
* gid (int)
* User name or comment field
* home directory
* interpreter (optional)
Multiple colon delimited lines may exist, but will not be separated
across lines.
When run, produces:
foo::2900:2900::/:/system/bin/sh
foo_bar::2901:2901::/:/system/bin/sh
custom_oem1::2902:2902::/:/system/bin/sh
Note that this generator allows for 0 or more config.fs files. This allows for:
* Unconditional inclusion of /system/etc/passwd in the generated image
* A blank passwd file if no config.fs files are specified.
This ensures that when OEMs add config.fs files, there is no additional steps
for proper functionality (simpler for OEMs).
The one draw back is the additional inode consumption on system for a possible
blank file.
Test: That it produces a valid passwd file.
Change-Id: I19691c8260f02147ed861f8a319aeab3f5b1738e
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Generate the friendly name in one location. This prepares
the tool for generatting passwd and group files.
Also support mapping friendly names to identifiers.
Test: That output files stay the same as before.
Change-Id: I12198611126613eae81ca61614ed269c2439b72b
Signed-off-by: William Roberts <william.c.roberts@intel.com>
When generating files restrict the characters allowed in
AID_<name> to upercase, numbers, and underscores.
This detects errors ahead of time for generated C files as
well as handles seperation characters for passwd/group files.
This also lends itself to automatic detection of collisions
on friendly names, since freindly names are the lowercase
version of <name>.
Test: That invalid aid values result in a build failure.
Test: That the output files are consistent with ones before
this change (hash and diff checks)
Change-Id: Ie8ec44c1157ba9c22100e9169d9187f615e71280
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Generate an OEM AID_<name> header file seperate from fs_config
header file and provide details on how to export this interface
into native code.
Test: That ls, ps, chown and services function for built in
services as before.
Change-Id: Ie8ce6585e0721b52633ee50d62dcfe796e178f65
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Implement an android_id generator that takes the data
acquired from parsing private/android_filesystem_config.h
and generates the android_id friendly name to uid mapping
for consumption in Bionic.
Test: That ls, ps, mkdir, chown, chgrp and services for built
in names work.
Change-Id: I1e55a401be0fca0ad162f8dc1e072e6afde7b927
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Rather than hardcode the OEM ranges, parse and extract
AID values from android_filesystem_config.h.
An AID is defined to the tool as:
* #define AID_<name>
An OEM Range is defined to the the tool as:
* AID_OEM_RESERVED_START
* AID_OEM_RESERVED_END
or
* AID_OEM_RESERVED_N_START
* AID_OEM_RESERVED_N_END
Where N is a number.
While parsing, perform sanity checks such as:
1. AIDs defined in the header cannot be within OEM range
2. OEM Ranges must be valid:
* Cannot overlap one another.
* Range START must be less than range END
3. Like the C preproccessor, multiple matching AID_<name> throws
en error.
The parser introduced here, prepares the tool to output android_ids
consumable for bionic.
Note that some AID_* friendly names were not consistent, thus a small
fixup map had to be placed inside the tool.
Test: tested parsing and dumping the data from android_filesystem_config.h
file.
Change-Id: Ifa4d1c9565d061b60542296fe33c8eba31649e62
Signed-off-by: William Roberts <william.c.roberts@intel.com>
This internally structures fs_config_generator.py to be able
to plug in generators to produce different outputs. This
prepares this tool for group and pwd file outputs.
Test: Checked diff and hash of before and after files.
Change-Id: Ie558518ac227dd946d70ab48027698b72a9bc94a
Signed-off-by: William Roberts <william.c.roberts@intel.com>
We are investigating replacing make_ext4fs with the upstream tool mke2fs.
To mitigate the trouble that may arise if the new tool behave differently
compared to the old one, there will be a transition period.
Devices that want to use the new way of creating ext4 images can set the
variable "TARGET_USES_MKE2FS" to true in their BoardConfig.mk
By default, the build system will choose the old tool 'make_ext4fs'.
Test: m otapackage with TARGET_USES_MKE2FS={,false,true}
Change-Id: I282bcb9efe335a86c53986283090ca947d65c7f8
Test: Checked diff and hash of before and after output files, same.
Test: Tested build for basic boot and functionality.
Change-Id: If7806427e3a2a9ddb7a2c9aa14e1e4f9bf696acf
Signed-off-by: William Roberts <william.c.roberts@intel.com>
Testing on earlier versions of Python indicated an error in the
README. Python's ConfigParser prior to version 3.2 allows for
sections within the same file to override later sections.
Correct the README.
Test: Checked diff and hash of output file from before and after.
Change-Id: Ifd3f78414b4803e4b74decb1fdf6816d4902c4a2
Signed-off-by: William Roberts <william.c.roberts@intel.com>
For some partition sizes, we currently build an image that's 1-2
blocks smaller than the actual partition, which causes fs_mgr to
not find metadata. This change adds padding to FEC metadata that
correctly positions the metadata header at the end.
Bug: 28865197
Bug: 32789520
Change-Id: Ie0e044715a9c5ae8ba395e7d2ff9fbd7cffc0b4c
(cherrypicked from commit 433905f50e)
* Allow other callers of this module to:
* pass any input stream to parse_input_file,
* pass any warning_lines to parallel_classify_warnings,
* call dump_csv or dump_html to get output.
* No output change.
* Capture and ignore signal.SIGTERM at the end,
to avoid bad warning/error messages from the exit clean-up process.
Test: run warn.py build.log.
Change-Id: I1414797a536c0ee622e2a34c226578621be1ddab
Prior to this CL, it was calling the hard-coded "java" although it was
accepting a "--java_path" option.
Also switch OPTIONS.java_args from string to list. Otherwise it won't
work when providing multiple args.
Bug: 32737832
Test: Specify "--java_path=" and "--java_args" when invoking
sign_target_files_apks.py with "-v". Check the commands being
called.
Change-Id: Id7ef98e778646d532027434de7fba9b7a104dbd0
(cherry picked from commit e95540e060)
Prior to this CL, it was calling the hard-coded "java" although it was
accepting a "--java_path" option.
Also switch OPTIONS.java_args from string to list. Otherwise it won't
work when providing multiple args.
Bug: 32737832
Test: Specify "--java_path=" and "--java_args" when invoking
sign_target_files_apks.py with "-v". Check the commands being
called.
Change-Id: Id7ef98e778646d532027434de7fba9b7a104dbd0
set() doesn't keep elements according to the order of insertion. So
Transfers managed with set() in intermediate steps may not appear in the
same order across runs. This leads to slightly different output packages
when generating the same incremental OTA.
This CL fixes the issue by replacing set() with OrderedDict() in
blockimgdiff.GenerateDigraph() and blockimgdiff.FindVertexSequence().
It also adds a testcase that ensures blockimgdiff.GenerateDigraph()
preserves the insertion order for Transfer.goes_after set.
Bug: 32220816
Test: ota_from_target_files.py gives identical package when running
multiple times.
Change-Id: I56d551e5ca926993ab46896e33c80e0ce42e506a
(cherry picked from commit 3348228423)
Keep a set of all visited inodes, and prevent recursing into a symlink
to an already visited inode.
Test: m -j
Test: compare `build/tools/findleaves.py --prune=.repo --prune=.git --mindepth=2 --dir=. Android.mk` before and after
Change-Id: Ied14c40d3066ef9f8e8a2b1535f56f7bbbbd0ab6
Purpose of this change is not to save some execution time,
but to avoid forking another process. In other applications
that use a wrapper to call this python script, it is difficult
to get overwritten file I/O functions work in a subprocess.
So the wrapper will call warn.py with --processes=1.
Test: run "warn.py --processes=1 build.log"
Change-Id: I5998d5c70d81a456c86eb4002f444a4a60135477
set() doesn't keep elements according to the order of insertion. So
Transfers managed with set() in intermediate steps may not appear in the
same order across runs. This leads to slightly different output packages
when generating the same incremental OTA.
This CL fixes the issue by replacing set() with OrderedDict() in
blockimgdiff.GenerateDigraph() and blockimgdiff.FindVertexSequence().
It also adds a testcase that ensures blockimgdiff.GenerateDigraph()
preserves the insertion order for Transfer.goes_after set.
Bug: 32220816
Test: ota_from_target_files.py gives identical package when running
multiple times.
Change-Id: I56d551e5ca926993ab46896e33c80e0ce42e506a
system/extras/verity/build_verity_metadata.py now accepts
"--signer_args" to specify verity signer args.
Also remove the duplicate "--verity_signer_args" in
add_img_to_target_files.py, as we already have that in common.py.
Bug: 31500665
Test: Building and signing work w/ and w/o --signer_args.
Change-Id: I02f59c50a1ebf15c5505e9fffd5b9bbbbaa785be
(cherry picked from commit 458104266f)
Add support for specifying number of inodes when creating
system, vendor, oem partitions. These are all read-only
and have no use for extra inodes. Removing extra inodes
saves a lot of space.
Bug: 32246383
Change-Id: I13f1d4614b64a4abc752c42a1c65d3d151481c21
Adds installed-files.json in addition to installed-files.txt
Further sorts the file list to be ordered within the same size bucket.
Test: manual, cross-checked checksums with sha256sum utility
checked build outputs.
Bug: 19988819
Merged-in: Ifb632eb4df65ec48645c8f93e36bae44ccc52ba8
Change-Id: Ifb632eb4df65ec48645c8f93e36bae44ccc52ba8
Signed-off-by: Sasha Levitskiy <sanek@google.com>
The original CL in AOSP is for 'system' partition.
Howerver, after automerger merges the CL from aosp to goog gerrit, it
patches part of the properties to 'system_other' partition. Correct this by
moving the system_avb* properties to 'system'.
Bug: 31264226
Test: build aosp_angler-userdebug successfully
Change-Id: I1a7128d29c1152315e9b50cfe5bbe13b47b131a7
Currently, whether contains patch or verbatim, compute with file size
and patch size.
But ota file must be compressed with zip, so it should be better with
compressed size than uncompressed.
Test: aosp_shamu-user build without proprietary blobs between MOB30P and NRD90S
$ du -k ota_shamu_old.zip ota_shamu_new.zip
217252 ota_shamu_old.zip
216520 ota_shamu_new.zip
Change-Id: If68cb1fbe2f7815067451915a0dcfe93ea5ba8d6
Signed-off-by: YOUNG HO CHA <ganadist@gmail.com>
* Add a --processes flag to specify number of parallel processes,
with default multiprocessing.cpu_count().
* Wrap long line to suppress pylint warning.
Test: run warn.py with a large build.log file.
Change-Id: I9a93a9324bc531c1bce741367013051ce40a67fa
system/extras/verity/build_verity_metadata.py now accepts
"--signer_args" to specify verity signer args.
Also remove the duplicate "--verity_signer_args" in
add_img_to_target_files.py, as we already have that in common.py.
Bug: 31500665
Test: Building and signing work w/ and w/o --signer_args.
Change-Id: I02f59c50a1ebf15c5505e9fffd5b9bbbbaa785be
These projects are all used by RenderScript, but it would be good to
differentiate warnings for each.
Change-Id: I983279d590630175a36075b01e8eec9100c07d26
Test: Ran warn.py
Adds installed-files.json in addition to installed-files.txt
Further sorts the file list to be ordered within the same size bucket.
Test: manual, cross-checked checksums with sha256sum utility
checked build outputs.
Bug: 19988819
Merged-in: Ifb632eb4df65ec48645c8f93e36bae44ccc52ba8
Change-Id: Ifb632eb4df65ec48645c8f93e36bae44ccc52ba8
Signed-off-by: Sasha Levitskiy <sanek@google.com>
Adds installed-files.json in addition to installed-files.txt
Further sorts the file list to be ordered within the same size bucket.
Test: manual, cross-checked checksums with sha256sum utility
checked build outputs.
Bug: 19988819
Change-Id: Ifb632eb4df65ec48645c8f93e36bae44ccc52ba8
Signed-off-by: Sasha Levitskiy <sanek@google.com>
* changes:
fat16copy: Fix allocation logic when extending directories.
fat16copy: Sort new directory entries.
Make fat16copy.py add . and .. entries to directories
Default.prop can interfere with UsbDeviceManager's management of usb state.
Adb is now enabled for userdebug and eng builds directly from UsbDeviceManager.
Bug: 31814300
Test: Manually verified that logcat is not kicked
Change-Id: I69c998c0dd184f9440cc236d82cc2f25f36e8734
Since build/ was moved to build/make/, the droiddoc templates now need
another ../ to reach the android root.
Test: m online-sac-docs
Change-Id: I2393f6c791df9b67ccafb4eb047fd8ccc52481cb
Since build/ was moved to build/make/, the droiddoc templates now need
another ../ to reach the android root.
Test: m online-sac-docs
Change-Id: I2393f6c791df9b67ccafb4eb047fd8ccc52481cb
* Warning messages must start with source file path.
This will reject lines containing only 'warning:'
but not a source file path prefix.
* Escape both backslash and quotation marks in the
dump of strings to JavaScript string literals.
Bug: 32060052
Test: run through build.log files
Change-Id: Ib064768b1ba2954f974604ea054f7a6d2ad15ae7
Remove Android root directory path from long
absolute source file path.
Test: run warn.py with build.log
Bug: 32059187
Change-Id: Ib630cebd8c56a9fef443139ca502d97e59c7b42e
Warning from the static analyzer:
build/tools/zipalign/ZipFile.cpp:503:5: warning: Function call argument
is an uninitialized value
pEntry->setDataInfo(uncompressedLen, endPosn - startPosn, crc,
Specifically, it's referencing `crc`, which would be uninitialized if we
hit either of these two error cases, since we'd return `NO_ERROR`.
Note that the warning is still there, but that's only because the static
analyzer can't see the asserts. If we #undef NDEBUG in the file, then
the warning disappears.
Bug: none
Test: With NDEBUG undefined, the warning is gone.
Change-Id: Iaed66127746c38add2c842ab027f2e1982d0e2fd
* Separate clang-tidy's clang-analyzer-* warnings from
other clang-tidy warnings.
Bug: 31559947
Test: build with WITH_TIDY=1 and DEFAULT_GLOBAL_TIDY_CHECKS=*,-readability-*,-google-readability-*
Change-Id: If03c421c9319741ef6588bc0d803b6da0cc4433a
This silences a warning null pointer passed as an argument to a
'nonnull' parameter
BUG=None
Test: The warning is gone.
Change-Id: I10a17d4674bfb373d7fe8666863abd5f10a4d11f
While the system.img images currently built with AVB support verify
correctly, mounting the filesystem content fails. This is because
'avbtool add_hashtree_footer' used to claim some of the unused /
DONT_CARE space for stashing the verity tables and this resulting in the
mapped device ending up being smaller causing the mount failure.
Fix this by leaving enough room for AVB hashtree and metadata before
building the image. This is achieved by moving the AVB hashtree support
into build_image.py and using a just added '--calc_max_image_size'
option to 'avbtool add_hashtree_footer' to figure out how much space to
leave out.
This depends on https://android-review.googlesource.com/#/c/281821/
Bug: 31264226
Test: Mounting dm-verity set up from system.img now works.
Merged-In: I4c5de1004c1059f8c582e76b3b8517d427aa1a87
Change-Id: I945a5f1f6782791736cd319f216cfa6b448fb04d
openJdk based android javadoc contains <a name=... tags
that are non-empty. Currently they are being hidden,
causing badly formatted pages for some java.* classes
(missing text and random whitespace blocks).
This change makes the existing a[name] hiding rule
apply only to empty tags.
Test: make docs
Bug: 31700998
Change-Id: Ifa75bbabe308d7aed0cce6165c1113e5a382a121
(cherry picked from commit a7edd92c45)
Set up a signal handler in makeparallel that will forward SIGHUP,
SIGINT, SIGQUIT, and SIGTERM to the child process.
Bug: 31907490
Test: m -j & killall make; pgrep -a ninja
Test: make makeparallel_test
Change-Id: I306e5335ed1b2c7056804d5da377a2f283877f30
While the system.img images currently built with AVB support verify
correctly, mounting the filesystem content fails. This is because
'avbtool add_hashtree_footer' used to claim some of the unused /
DONT_CARE space for stashing the verity tables and this resulting in the
mapped device ending up being smaller causing the mount failure.
Fix this by leaving enough room for AVB hashtree and metadata before
building the image. This is achieved by moving the AVB hashtree support
into build_image.py and using a just added '--calc_max_image_size'
option to 'avbtool add_hashtree_footer' to figure out how much space to
leave out.
This depends on https://android-review.googlesource.com/#/c/281821/
Bug: 31264226
Test: aosp_x86_64-eng 'm' and 'm dist' successful.
Change-Id: I4c5de1004c1059f8c582e76b3b8517d427aa1a87
Remove the obsolete workaround for API 24. Also make it pylint clean.
Test: 1. Sign a target_files.zip and get identical results.
2. `pylint --rcfile=pylintrc sign_target_files_apks.py` gives 10.00/10.
Change-Id: I21736f959c5182486fd8ccebea9bbc594edef9fb
sign_target_files_apks.py calls common.GetBootableImage() but without
calling 'OPTIONS = common.OPTIONS' first. In common.GetBootableImage(),
we should use the local info_dict parameter instead of OPTIONS.info_dict.
Test: sign_target_files_apks.py generates signed-TF.zip successfully.
Change-Id: Ia3d32b88691c26e5fb98feea709e3e3c3eb70fdb
This updates the build system for the new Android Verified Boot
codebase. As this is based on Brillo Verified Boot, this change replaces
the existing BVB support.
Android Verified Boot is enabled by the BOARD_AVB_ENABLE variable
BOARD_AVB_ENABLE := true
This will make the build system create vbmeta.img which will contain a
hash descriptor for boot.img, a hashtree descriptor for system.img, a
kernel-cmdline descriptor for setting up dm-verity for system.img and
append a hash-tree to system.img.
Additionally, the descriptors are left in boot.img and system.img so a
third party can create their own vbmeta.img file linking - using the
option --chain_partition - to these images. If this is not needed
footers can be erased using the 'avbtool erase_footer' command. It's
also harmless to just leave them in the images.
By default, the algorithm SHA256_RSA4096 is used with a test key from
the AVB source directory. This can be overriden by the
BOARD_AVB_ALGORITHM and BOARD_AVB_KEY_PATH variables to use e.g. a
4096-bit RSA key and SHA-512:
BOARD_AVB_ALGORITHM := SHA512_RSA4096
BOARD_AVB_KEY_PATH := /path/to/rsa_key_4096bits.pem
To prevent rollback attacks, the rollback index should be increased on a
regular basis. The rollback index can be set with the
BOARD_AVB_ROLLBACK_INDEX variable:
BOARD_AVB_ROLLBACK_INDEX := 5
If this is not set, the rollback index defaults to 0.
The variable BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS can be used to specify
additional options passed to 'avbtool make_vbmeta_image'. Typical
options to be used here include '--prop', '--prop_from_file', and
'--chain_partition'.
The variable BOARD_AVBTOOL_BOOT_ADD_HASH_FOOTER_ARGS can be used to
specify additional options passed to 'avbtool add_hash_footer' for
boot.img. Typical options to be used here include '--hash_algorithm' and
'--salt'.
The variable BOARD_AVBTOOL_SYSTEM_ADD_HASHTREE_FOOTER_ARGS can be used
to specify additional options passed to 'avbtool add_hashtree_footer'
for systems.img. Typical options to be used here include
'--hash_algorithm', '--salt', and '--block_size'.
BUG=31264226
TEST=Manually tested on edison-eng by inspecting {boot, system,
vbmeta}.img in out/ directory as well as their counterparts in
the IMAGES/ directory of edision-target_files-eng.zeuthen.zip
Merged-In: Ic9a61cfc65c148b12996e57f04da5432eef6b982
Change-Id: I97042655bca15e7eac899f12c5bada2f6184d307
This updates the build system for the new Android Verified Boot
codebase. As this is based on Brillo Verified Boot, this change replaces
the existing BVB support.
Android Verified Boot is enabled by the BOARD_AVB_ENABLE variable
BOARD_AVB_ENABLE := true
This will make the build system create vbmeta.img which will contain a
hash descriptor for boot.img, a hashtree descriptor for system.img, a
kernel-cmdline descriptor for setting up dm-verity for system.img and
append a hash-tree to system.img.
Additionally, the descriptors are left in boot.img and system.img so a
third party can create their own vbmeta.img file linking - using the
option --chain_partition - to these images. If this is not needed
footers can be erased using the 'avbtool erase_footer' command. It's
also harmless to just leave them in the images.
By default, the algorithm SHA256_RSA4096 is used with a test key from
the AVB source directory. This can be overriden by the
BOARD_AVB_ALGORITHM and BOARD_AVB_KEY_PATH variables to use e.g. a
4096-bit RSA key and SHA-512:
BOARD_AVB_ALGORITHM := SHA512_RSA4096
BOARD_AVB_KEY_PATH := /path/to/rsa_key_4096bits.pem
To prevent rollback attacks, the rollback index should be increased on a
regular basis. The rollback index can be set with the
BOARD_AVB_ROLLBACK_INDEX variable:
BOARD_AVB_ROLLBACK_INDEX := 5
If this is not set, the rollback index defaults to 0.
The variable BOARD_AVB_MAKE_VBMETA_IMAGE_ARGS can be used to specify
additional options passed to 'avbtool make_vbmeta_image'. Typical
options to be used here include '--prop', '--prop_from_file', and
'--chain_partition'.
The variable BOARD_AVBTOOL_BOOT_ADD_HASH_FOOTER_ARGS can be used to
specify additional options passed to 'avbtool add_hash_footer' for
boot.img. Typical options to be used here include '--hash_algorithm' and
'--salt'.
The variable BOARD_AVBTOOL_SYSTEM_ADD_HASHTREE_FOOTER_ARGS can be used
to specify additional options passed to 'avbtool add_hashtree_footer'
for systems.img. Typical options to be used here include
'--hash_algorithm', '--salt', and '--block_size'.
Bug: 31264226
Test: aosp_x86_64-eng 'm' and 'm dist' successful.
Change-Id: Ic9a61cfc65c148b12996e57f04da5432eef6b982
In BBOTA, we generate patches based on _all_ the blocks of a pair of
input files (src and tgt). For security incremental OTAs, one common
pattern is that only a few blocks are changed in odex files (e.g.
headers). We don't really need to stash/patch the unchanged blocks.
This CL analyzes the unchanged blocks in odex files and computes the
diff for the changed blocks only. It reduces the OTA install time by
about 25% to 40% in our experiments, by paying an increase of 5% to 30%
OTA generation time cost.
Bug: 31570716
Test: Generate an incremental and apply on device.
Change-Id: If842c1afeff6894a3d27eb60b7e8f65a179b7977
(cherry picked from commit 08c8583aea)
* Rename 'severity' to 'Severity'.
* Fix bad line indentations and space.
Test: run warn.py --byproject build.log
Change-Id: I33d701cca408b5ca4715be8d44c890942b7d132a
* Add more project patterns.
* Add more top level comments for global variables and functions.
* Resequence severity numbers to match the dump order.
* Emit warning messages and tables to static HTML JavaScript arrays.
* Replace old static HTML table dumper functions with
new dynamic HTML JavaScript to generate sections of warnings.
* Warning messages are grouped into sections by severity or projects.
* Better descriptions for SKIP warning patterns.
* Replace output function with print.
Bug: 31377083
Test: run warn.py --byproject build.log
Change-Id: I7b44ef6223d5b2f1aa31655a5a47d854f9a1dedc
* Rename variables and functions to follow Google coding style.
* Fix line indentations to follow Google coding style.
* Add module and function docstrings.
* Change shebang line to avoid pylint warning.
* Suppress some pylint warnings to avoid too many changes at once.
* Fix typos.
Test: run warn.py --byproject build.log
Change-Id: I569961981b562e35cb5609a96cd5ebb40a80829c
openJdk based android javadoc contains <a name=... tags
that are non-empty. Currently they are being hidden,
causing badly formatted pages for some java.* classes
(missing text and random whitespace blocks).
This change makes the existing a[name] hiding rule
apply only to empty tags.
Test: make docs
Bug: 31700998
Change-Id: Ifa75bbabe308d7aed0cce6165c1113e5a382a121
When extending a directory, empty directory entries were left behind
signaling the end of a directory.
Bug: 31570546
Test: make dist; mdir shows all the files.
(cherry picked from commit d4516300b010c8155861e310dac871e8c10d3382)
Change-Id: Ie3d82cfaad51be73911293a54d18746e8adf62a2
Sort the entries returned by os.listdir to give a consistent ordering
across build.
Bug: None
Test: `make dist`
(cherry picked from commit ef34e5dca216e61fd81020e818f247f6ce47d26e)
Change-Id: I2f80889684f108208f07e13d725acaa7bc0ab099
In BBOTA, we generate patches based on _all_ the blocks of a pair of
input files (src and tgt). For security incremental OTAs, one common
pattern is that only a few blocks are changed in odex files (e.g.
headers). We don't really need to stash/patch the unchanged blocks.
This CL analyzes the unchanged blocks in odex files and computes the
diff for the changed blocks only. It reduces the OTA install time by
about 25% to 40% in our experiments, by paying an increase of 5% to 30%
OTA generation time cost.
Bug: 31570716
Test: Generate an incremental and apply on device.
Change-Id: If842c1afeff6894a3d27eb60b7e8f65a179b7977
* Use a loop to initialize empty 'members' and 'option' of warnpatterns.
* Create severity.{color,columnheader,header} from an attributes list.
* Compute totalbyproject and totalbyseverity from a constructor.
* Skip all-zero rows and columns in the warning count table.
* Remove redundant spaces and newlines in tablerow output.
Test: run through build.log files
Change-Id: I4d3fa4ecd92e1afab91d85b7535d03f8696e83d8
tempfile.TemporaryFile() complains when 'None' is passed as the
prefix/suffix. It uses prefix='tmp' and suffix='' as the default values
and we should do the same.
Test: Call check_ota_package_signature.py and ota_from_target_files.py
and they still work.
Change-Id: I7fb023a3fd0b1a57c009631d0c57a7bb8e4cb5a3
Currently it supports verifying packages signed with RSA algorithms
(v1-v4 as in bootable/recovery/verifier.cpp). No support for ECDSA (v5)
signed packages yet.
$ ./build/tools/releasetools/check_ota_package_signature.py \
bootable/recovery/tests/testdata/testkey_v1.x509.pem \
bootable/recovery/tests/testdata/otasigned_v1.zip
Package: bootable/recovery/tests/testdata/otasigned_v1.zip
Certificate: bootable/recovery/tests/testdata/testkey_v1.x509.pem
Comment length: 1738
Signed data length: 2269
Use SHA-256: False
Digest: 115e688ec3b77743070b743453e2fc6ce8754484
VERIFIED
Bug: 31523193
Test: Used the tool to verify existing packages (like above).
Change-Id: I71d3569e858c729cb64825c5c7688ededc397aa8
This was printing "error:", but not actually triggering an error.
Instead of trying to write a single line bash script to handle this,
move the actual check into python. This allows us to print all of the
errors for a single module before triggering the failure.
Also updates the warning format and the warn.py script to properly parse
these warning. Many of the java:sdk -> java:platform warnings are false
positives due to the lack of LOCAL_SDK_VERSION markings on prebuilts.
Individual tags can be marked as warnings now, which lets us check for
system libraries linking against vendor libraries (which won't work on
AOSP). I'm not sure this is a completely valid check, which one reason
that it's just a warning.
Test: m all_link_types (with some missing libs commented out)
Change-Id: I333e418c9a4511b7c7e826891ae481da08fbf6f9
* Use 't1' CSS class for main warning tables,
and default table style for new warning count table.
* Count total skipped warnings.
* Append count of warnings in each severity header.
* Fix some skipped warning patterns and count/dump skipped warnings.
* Add dumpskippedanchors function to fix expand_collapse
error when --byproject flag is used and severity.SKIP
is not emitted.
* Minor coding style changes:
* Loop through severity.kinds.
* Remove unused import.
* Add spaces around assignment operators.
* Adjust indentation.
Bug: 31377083
Test: run warn.py with build.log
Change-Id: I68949edc4601b10ea2f7ac84e48e6c3da4ab1e4f
This workaround to clean package intermediates is no longer necessary,
as ninja reruns commands whose command line changes.
Test: switch between product overlays that use add-resource, verify
R.java is regenerated.
Change-Id: Icb1f76360e39882e4401180414a031956777b020
The switch to apksig changed the Created-By header value in .SF file
from "1.0 (Android SignApk)" to "1.0 (Android apksigner)". This commit
reverts the value back to "1.0 (Android SignApk)".
Change-Id: I2fc462cade40a5b31bb6191996fd6f18fabbf08f
fat16copy.py can be used to copy files or directories in to the root of
a fat16 image.
Usage: fat16copy <image> <file> [<file> ...]
Test: Used to compose a Raspberry Pi 3 boot image
Bug: 28912590
Change-Id: I29c9eec3786e2c5cc94f9a160360bac850809a93
For some partition sizes, we currently build an image that's 1-2
blocks smaller than the actual partition, which causes fs_mgr to
not find metadata. This change adds padding to FEC metadata that
correctly positions the metadata header at the end.
Bug: 28865197
Change-Id: Ie0e044715a9c5ae8ba395e7d2ff9fbd7cffc0b4c
I actually just deleted templates-sdk/ then made a copy of templates-sdk-dev/
and then renamed it to templates-sdk/. So this is a clean copy.
Once this goes through cleanly, I'll delete templates-sdk-dev/ and update
all doc targets to use templates-sdk/.
Change-Id: I276695969e02fcf718370564e84b077a9d43725e
William Roberts