This is useful for devices with low disk space with different
build variants.
Bug: 37469715
Test: Regular image builds successfully, errors occur when
the headroom size is greater than available partition space.
Change-Id: I526cdd0f84981bbd16e3afcfe1cd7fc43dce98ef
Xml files will be generated instead of html files, and then Settings
will generate html file on runtime from xml files of partitions.
Test: build succeeded and tested on sailfish
Bug: 37099941
Merged-In: Id7899381a537b1947cbb6164ed2b4a98492334af
Change-Id: Id7899381a537b1947cbb6164ed2b4a98492334af
(cherry picked from commit 3b7560bf88)
find_android_root works only when warn.py is found
so it works for build bot, but not when warn.py is
run against a stand alone build.log file.
Now get the TOP directory path from build.log
if it is found.
Bug: 37725427
Test: warn.py --gencsv build.log
Change-Id: I7b776c2e71a8d8d769f0600b08abe37acbd6ce7c
Misspelled fs_config_generate_extra_partition_list when used to
calculate LOCAL_REQUIRED_MODULES dependencies. This resulted in
fs_config_dirs and fs_config_files failing to generate the vendor,
oem or odm variants (ie: fs_config_files_vendor)
Test: build generates correct fs_config override files.
Bug: 37721445
Bug: 36071012
Change-Id: I053dd5cdb357c7cea05a5ff63b0686bd0bc99df6
This reverts commit b1e59b8a73.
This CL enables generating compatibility zip entry in an A/B OTA
package. We're not enabling the support for non-A/B OTAs for now until
they have the necessary support (e.g. /proc/config.gz exists).
Bug: 36810913
Test: Generate full and incremental packages for marlin and applly under
normal boot and sideload respectively.
Change-Id: I69f0a3feb7a0efc7f727f92f31c9e63f47ee6530
(cherry picked from commit b63c952cc7)
Add build targets to generate the vendor, oem and odm slices of the
fs_config_dirs and fs_config_files targets for each of the respective
filesystems, and automatically trigger the individual and enabled
partition build objects based as dependents of fs_config_dirs and
fs_config_files.
Sensitive to the following board definition pairs:
BOARD_VENDORIMAGE_FILE_SYSTEM_TYPE BOARD_USES_VENDORIMAGE
BOARD_OEMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_OEMIMAGE
BOARD_ODMIMAGE_FILE_SYSTEM_TYPE BOARD_USES_ODMIMAGE
Which when defined and set means the images are generated for vendor,
oem or odm respectively.
Test: manually build w/ m 'fs_config_(dirs|files)_(vendor|oem|odm)'
and check the expected products, or fails if not pertinent.
Test: build for product that produces split to system and vendor and
subsequently boots with vendor products correctly configured.
Bug: 36071012
Change-Id: I44289a3dd50b7657367c826ad813c671a3de6afc
Add a -P option to fs_config_generate that allows us to filter on
a comma separated partition whitelist or minus prefixed -blacklist.
A partition is defined as accessible as <partition>/ or
system/<partition>/.
No change for legacy, where no -P flag is specified.
For system.img, but not vendor, oem or odm file references:
fs_config_generate -P -vendor,-oem,-odm
For vendor.img file references:
fs_config_generate -P vendor
For oem.img file references:
fs_config_generate -P oem
For odm.img file references:
fs_config_generate -P odm
Add and fortify tests for each of these cases.
SideEffects: none
Test: gTest host fs_config-unit-test
Bug: 36071012
Change-Id: I0537453bb289d41ef98ad76e69f78ae49358a1b3
Bug: None
Test: Ran warn.py on a build log. Code links in resultant HTML open in
new tabs when clicked.
Change-Id: Ia3def5ea14fe216700cb2ffabe4a39b98e69d55a
Uses avbtool to sign vendor.img if BOARD_AVB_ENABLE is set.
It also allows appending additional arguments to avbtool via
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS.
e.g.,
BOARD_AVB_ENABLE := true
BOARD_AVB_VENDOR_ADD_HASHTREE_FOOTER_ARGS := --generate_fec
Bug: 35415839
Test: "make" with the above variables and use avbtool to check vbmeta is
appended to vendor.img
Test: "make dist" with the above variables
Change-Id: I8ada38dff3def6d34613e77c67944def8a49f464
While we are generating the html file also expose a way for a user to
save off a csv file. This prevents having to run the tool twice.
* Add a new flag called csvpath that takes a file path for the csv dump.
* Use csv.writer to simply print formats.
* Modify the dump_csv and count_severity to take a csv writer.
Test: run with a build.log file, compare before and after html/csv dumps.
Change-Id: I27ed730625a352fdc3fb2928433eb5e8b2ea229c
This patch updates the logic to use the PRODUCT_IOT variable instead of
BRILLO.
Bug: 36702887
Test: `make`; Image doesn't have the dev key.
Change-Id: I1751e97d8cdfeba83c7e4720a017a5f4dcfd49da
This CL is to generate every static RRO package for its target package
automatically at build-time.
BOARD_ENFORCE_RRO build variable is added to specify whether enforcing
RRO is required or not.
BOARD_ENFORCE_RRO_EXEMPT_SOURCES build variable is added to specify
the module list of which item should be exempt from enforcing RRO.
Test: tested on bullhead and sailfish
Bug: 34097942
Change-Id: I455b2ce34e66c57a540c299b5e378b7c4e78d5b8
(cherry picked from commit 3070610b72)
The test broke due to api change in constructor of Transfer.
Bug: 36456381
Test: blockimgdiff test passes
Change-Id: Ic73b66f8fb6141ff6f50c113b0d8cc55d34e0b5d
fileslist.go is still disabled by default. To enable, use:
USE_FILESLIST_GO=true m -j
Bug: 36274890
Test: Manual, with m -j
Change-Id: I722e17fa7fb6ba91e2b2c44cd7524d974cbe4a56
We've just created the target files package and still have the extracted
directory, so pass that to ota_from_target_files so that it doesn't have
to re-extract what we just compressed.
This saves a little bit of time -- for bullhead-userdebug on internal
master, this brings the time to build an OTA from ~340s -> ~310s. Much
of the time is still spent generating and signing the OTA.
Test: lunch bullhead-userdebug; m -j otapackage
Test: bullhead-ota-*.zip is identical before/after
Change-Id: Ib51cf6b02123e9c49280e9957c6590645470a52c
system_root_image has been dead since commit
2ce63edab7.
Test: pylint --rcfile=pylintrc add_img_to_target_files.py
Change-Id: Id791747b7313923b82279a21d264a998455a92d4
When bsdiff/imgdiff fails, dump the output along with the src/tgt
filenames and block ranges. Also, exit the script gracefully instead
of raising an exception about iterating over 'NoneType'.
Bug: 31381530
Test: Package generates successfully for angler, and error outputs
correctly with error injection.
Change-Id: I06a2dfe545fbdff7043de05fee34b378453a9291
The line was unintentionally removed in commit 7f804ba.
Test: ota_from_target_files.py generates a full OTA. Check the package
metadata.
Change-Id: Icae88e2a9bb2bfc450a3d0d7ab524d6a6eac9df5
BBOTA v1 and v2 (introduced in L and L MR1 respectively) don't support
resumable OTA. We shouldn't generate packages using v1/v2 at the risk of
bricking devices.
BBOTA v3 (since M) and v4 (since N) both support resumable OTAs. BBOTA
v4 additionally supports using FEC to possibly recover a corrupted
image.
Bug: 33694730
Test: Generate full and incremental OTAs w/ and w/o the CL. They should
give identical packages (in v4).
Change-Id: Ib89d9cd63ba08e8e9aa4131bed18876b89d244c0
Remove the following functions that are needed for file-based OTAs only:
- SetPermissions()
- SetPermissionsRecursive()
- MakeSymlinks()
- DeleteFiles()
- DeleteFilesIfNotMatching()
- RenameFiles()
- SkipNextActionIfTargetExists()
Bug: 35853185
Test: Verified there's no reference to these functions.
Change-Id: Iff24a9d705476211effaef28eed2a147fa5fcbce
In addition to the current behavior of add_img_to_target_files working
on an existing zip file, allow passing in a directory where the target
files have already been extracted. When in this mode, it writes the
images out to that directory instead of the zip file.
This allows us to call add_img_to_target_files on the temp directory
used during the build to create the target files package, saving the
time and space of unzipping what we just zipped. This also allows us to
use the parallel soong_zip, which compresses the images much faster.
Test: aosp_marlin target_files zip is the same before/after this change
Test: marlin target_files zip is the same before/after this change
Test: bullhead target_files zip is the same before/after this change
Change-Id: I155654cdc7ad7754ba4ef76ec69c31f504a58198
We have stopped shipping devices with file-based OTAs, and are not
actively maintaining the support. Devices using file-based OTAs
should be moved to block-based, if not A/B OTAs.
We will also need to clean up EdifyGenerator class, which will be
handled in follow-up CLs.
Bug: 35853185
Test: Generate full and incremental OTAs w/ and w/o the CL, and they
give identical packages.
Test: Not specifying --block also generates block-base OTAs.
Change-Id: I3b0fc8ce5600e109f3251fe41f655534aaa298c7
We must have created the images (system.img, system.map etc) prior to
calling ota_from_target_files.py (added by commit
2e0d8fcf08, into Lollipop).
Remove the obsolete suppport for handling "old" target_files zips that
don't have such images. This simplies the logic for BuildSystem() and
BuildVendor(), which now would only be called by
add_img_to_target_files.py itself.
Test: Generating full and incremental OTAs give the same results, w/ and
w/o this CL.
Change-Id: I0ea243d09d3378249d7982701ae4ec339b04b7b8
The first one in WriteVerifyPackage() is mismatching function parameters
that can be trivially fixed.
The other one is in WriteABOTAPackageWithBrilloScript(), where we don't
have edify script instance.
Test: `pylint --rcfile=pylintrc ota_from_target_files.py`.
Change-Id: Ie238ef5b296dfe9e725b61376992447b662d2376
The major issue with the existing implementation is unnecessarily
holding too much data in memory, such as HashBlocks() which first reads
in *all* the data to a list before hashing. We can leverage generator
functions to stream such operations.
This CL makes the following changes to reduce the peak memory use.
- Adding RangeSha1() and WriteRangeDataToFd() to Image classes. These
functions perform the operations on-the-fly.
- Caching the computed SHA-1 values for a Transfer instance.
As a result, this CL reduces the peak memory use by ~80% (e.g. reducing
from 5.85GB to 1.16GB for the same incremental, as shown by "Maximum
resident set size" from `/usr/bin/time -v`). It also effectively
improves the (package generation) performance by ~30%.
Bug: 35768998
Bug: 32312123
Test: Generating the same incremental w/ and w/o the CL give identical
output packages.
Change-Id: Ia5c6314b41da73dd6fe1dbe2ca81bbd89b517cec
This reverts commit a7316ce094.
This CL differs from the original CL by not unzipping RADIO/*. This is
because: a) AOSP targets don't have RADIO/ entries in the TF.zip; b)
we're not using the unzipped RADIO files (but reading them from the zip
files directly) - checked all the device-specific releasetools for
angler, bullhead, ryu, shamu, volantis, fugu, marlin and sailfish.
Test: `m dist` with AOSP targets (tested fugu and bullhead).
Change-Id: I4d0c67214ddd6202fc27c71bb79f52b5f4d40c64
This reverts commit aa3a04f19d.
Reason for revert: Some AOSP targets don't include RADIO/ in the TF.zip. We may possibly skip unzipping RADIO/, or by always creating a dummy RADIO folder in TF.zip. Revert this CL for now.
Change-Id: I8e90d322706a4fe82129bdfab5ffa1eab989c648
Julius D'souza