* Use 't1' CSS class for main warning tables,
and default table style for new warning count table.
* Count total skipped warnings.
* Append count of warnings in each severity header.
* Fix some skipped warning patterns and count/dump skipped warnings.
* Add dumpskippedanchors function to fix expand_collapse
error when --byproject flag is used and severity.SKIP
is not emitted.
* Minor coding style changes:
* Loop through severity.kinds.
* Remove unused import.
* Add spaces around assignment operators.
* Adjust indentation.
Bug: 31377083
Test: run warn.py with build.log
Change-Id: I68949edc4601b10ea2f7ac84e48e6c3da4ab1e4f
This workaround to clean package intermediates is no longer necessary,
as ninja reruns commands whose command line changes.
Test: switch between product overlays that use add-resource, verify
R.java is regenerated.
Change-Id: Icb1f76360e39882e4401180414a031956777b020
The switch to apksig changed the Created-By header value in .SF file
from "1.0 (Android SignApk)" to "1.0 (Android apksigner)". This commit
reverts the value back to "1.0 (Android SignApk)".
Change-Id: I2fc462cade40a5b31bb6191996fd6f18fabbf08f
fat16copy.py can be used to copy files or directories in to the root of
a fat16 image.
Usage: fat16copy <image> <file> [<file> ...]
Test: Used to compose a Raspberry Pi 3 boot image
Bug: 28912590
Change-Id: I29c9eec3786e2c5cc94f9a160360bac850809a93
For some partition sizes, we currently build an image that's 1-2
blocks smaller than the actual partition, which causes fs_mgr to
not find metadata. This change adds padding to FEC metadata that
correctly positions the metadata header at the end.
Bug: 28865197
Change-Id: Ie0e044715a9c5ae8ba395e7d2ff9fbd7cffc0b4c
I actually just deleted templates-sdk/ then made a copy of templates-sdk-dev/
and then renamed it to templates-sdk/. So this is a clean copy.
Once this goes through cleanly, I'll delete templates-sdk-dev/ and update
all doc targets to use templates-sdk/.
Change-Id: I276695969e02fcf718370564e84b077a9d43725e
The userdata.img and cache.img entries are not useful in signed builds;
because fastboot doesn't look at these two entries in the *img.zip when
flashing a device. And they aren't used elsewhere. Therefore, skip
building the image files for them when signing the target files with
sign_target_files_apks. Also, add an option "--is_signing" to avoid
adding these two images when we call add_img_to_target_files.
Change-Id: I39ba91a86d9a856d7d01771f6d1403dbf21f2011
Test: Run sign_target_files_apks on a target file and userdata/cache.img doesn't not generate.
Bug: 30642470
(cherry picked from commit b48589af57)
The output will be in the `m dist` results as soong_to_convert.txt, or
can be built using:
$ m $OUT/soong_to_convert.txt
The output is a list of modules that are probably ready to convert to
Soong:
# Blocked on Module (potential problems)
283 libEGL (srcs_dotarm)
246 libicuuc (dotdot_incs dotdot_srcs)
221 libspeexresampler
215 libcamera_metadata
...
0 zram-perf (dotdot_incs)
The number at the beginning of the line shows how many native modules
depend on that module.
All of their dependencies have been satisfied, and any potential
problems that Make can detect are listed in parenthesis after the
module:
dotdot_srcs: LOCAL_SRC_FILES contains paths outside $(LOCAL_PATH)
dotdot_incs: LOCAL_C_INCLUDES contains paths include '..'
srcs_dotarm: LOCAL_SRC_FILES contains source files like <...>.c.arm
aidl: LOCAL_SRC_FILES contains .aidl sources
dbus: LOCAL_SRC_FILES contains .dbus-xml sources
objc: LOCAL_SRC_FILES contains Objective-C sources
proto: LOCAL_SRC_FILES contains .proto sources
rs: LOCAL_SRC_FILES contains renderscript sources
vts: LOCAL_SRC_FILES contains .vts sources
Not all problems can be discovered, but this is a starting point.
Change-Id: I45674fe93fd267d4d1fb0bc3bc9aa025e20c5ac6
* Generalize regular expression to match new warning messages.
* Remove old unused pattern.
Change-Id: I54f2e5bdb255b290cb3751ba56558f5fe8d7245e
Test: Run warn.py with build.log
The userdata.img and cache.img entries are not useful in signed builds;
because fastboot doesn't look at these two entries in the *img.zip when
flashing a device. And they aren't used elsewhere. Therefore, skip
building the image files for them when signing the target files with
sign_target_files_apks. Also, add an option "--is_signing" to avoid
adding these two images when we call add_img_to_target_files.
Change-Id: I39ba91a86d9a856d7d01771f6d1403dbf21f2011
Test: Run sign_target_files_apks on a target file and userdata/cache.img doesn't not generate.
Bug: 30642470
add_img_to_target_files.py fails when the target_files.zip is over 4GiB
when adding IMAGES/ folder. Specify the flag to allow creating
target_files.zip with ZIP64 extension.
Other zip artifacts (-img.zip, -ota.zip etc) remain in non-ZIP64 format.
zip2zip is not affected, which still creates non-ZIP64 zips even when
copying from target_files in ZIP64.
Bug: 30961841
Test: "make dist" with large system image and check the artifacts.
Change-Id: I0568745f01ef8f0239081f783eac92288d4fdd84
Do not copy the "META/care_map.txt" from the source zipfile when
signing the target files with sign_target_files_apks. Because we'll
generate a new care_map after rebuilding the system/vendor images;
and we'll write the new "META/care_map.txt" to the signed-target-file.
Change-Id: I6919cfdf8314a4084b5f612a9c89469f391486a4
Test: Run sign_target_files_apks locally, and the entry is updated.
Bug: 30812253
(cherry picked from commit 4f09900e67)
Do not copy the "META/care_map.txt" from the source zipfile when
signing the target files with sign_target_files_apks. Because we'll
generate a new care_map after rebuilding the system/vendor images;
and we'll write the new "META/care_map.txt" to the signed-target-file.
Change-Id: I6919cfdf8314a4084b5f612a9c89469f391486a4
Test: Run sign_target_files_apks locally, and the entry is updated.
Bug: 30812253
On A/B devices (i.e. system_root_image="true"), /default.prop is
packaged at ROOT/default.prop (as opposed to BOOT/RAMDISK/default.prop
for non-A/B devices). Update the path so that we handle properties like
ro.bootimage.build.fingerprint properly.
The one for recovery is not affected, which stays at
BOOT/RAMDISK/default.prop for A/B devices and gets updated correctly.
Bug: 30811237
Test: Verify the property in the generated signed-TF.zip.
Change-Id: Id201a042d7ea988a64f89c6d04f43326a9851e27
(cherry picked from commit 28e2fa1726)
On A/B devices (i.e. system_root_image="true"), /default.prop is
packaged at ROOT/default.prop (as opposed to BOOT/RAMDISK/default.prop
for non-A/B devices). Update the path so that we handle properties like
ro.bootimage.build.fingerprint properly.
The one for recovery is not affected, which stays at
BOOT/RAMDISK/default.prop for A/B devices and gets updated correctly.
Bug: 30811237
Test: Verify the property in the generated signed-TF.zip.
Change-Id: Id201a042d7ea988a64f89c6d04f43326a9851e27
The update-payload-key is used by update_engine_sideload from recovery
to verify an update payload.
Bug: 27178350
(cherry picked from commit b3e8ce6d1d)
Change-Id: Iac239732251e550e9966bf284b68bc9d578f22ff
The update-payload-key is used by update_engine_sideload from recovery
to verify an update payload.
Bug: 27178350
Change-Id: I7a0a307ae565e5e9cbf2c9b58fbcc055e87771ce
* LOCAL_CLANG_EXCEPTION_PROJECTS is the list of project prefixes.
* New warning message is detected by warn.py as high severity.
Bug: 30575506
Change-Id: I87a38b2ba6bab1d2e5e7a9d5ec9557f0e609727c
Test: build and find new warning in build.log
We were using the package name as the key to index APKs. APKs from the
same package got messed up and gave wrong signature summary. Switch to
using the package filename as the key, which is identical in a given build.
Also fix the trailing space when printing the signature summary.
Bug: 30418268
Test: Run with a target_files.zip that has multiple APKs from the same package.
Change-Id: I6317e8c05e987c5690915e05c294153d10e2f0ab
(cherry picked from commit 6a54299fbd)
We were using the package name as the key to index APKs. APKs from the
same package got messed up and gave wrong signature summary. Switch to
using the package filename as the key, which is identical in a given build.
Also fix the trailing space when printing the signature summary.
Bug: 30418268
Test: Run with a target_files.zip that has multiple APKs from the same package.
Change-Id: I6317e8c05e987c5690915e05c294153d10e2f0ab
* Use const reference parameter type to avoid unnecessary copy.
Bug: 30407689
Change-Id: Iefab05c077367f272abf545036b853e8a295c8cd
Test: build with WITH_TIDY=1
* Use --byproject flag to dump warnings in HTML file by projects.
* Add external/google and vendor/google projects, vs other
external/non-google and vendor/non-google projects.
* Refactor code in dumpcategory to dump warnings by projects
when --byproject is specified.
* Recognize new compiler -Wvarargs warning pattern.
* No change to default HTML output format.
Change-Id: I2406ad392ab18683357ff0ee0cf6c41f87cf399c
Test: run through current build.log files.
* This enables 5 performance related checks:
performance-faster-string-find
performance-for-range-copy
performance-implicit-cast-in-loop
performance-unnecessary-copy-initialization
performance-unnecessary-value-param
* Recognize these new warnings in warn.py.
Bug: http://b/27779618
Change-Id: I7af3fdc262dd44db90d3b62eb37c5eac5a2eaa01
Test: build with WITH_TIDY=1 and run warn.py with build.log
* No change to html output, except removing unused "PageTop" anchor.
* Add --gencsv flag to output a CSV file instead of html file.
CSV files are combined to a table for dashboard charts.
* Count warnings from selected source project directories.
* CSV output lines are:
num_of_warnings,project_name,warning_description
* project_name is empty for sum of warnings from all projects.
* warning_description could be:
(1) "<severity>: <warning type description>"
for one warning type, or
(2) "<severity> warnings"
for all warning types of the severity.
Change-Id: I1cf976ed0cf142091226b22665b7f28d383608fd
Test: run through current build.log files.
* Count new clang-tidy warnings:
google-runtime-references
google-runtime-operator
* Count new clang warnings:
-Wconst-conversion similar to -Wliteral-conversion
-Wundefined-var-template
* Modified static analyzer warning message about
"Null passed to a callee that requires a non-null ..."
* Disable google-runtime-references by default and
disable misc-macro-parentheses in vendor projects;
too many old C/C++ code have this warning.
Change-Id: Ib35e0af85875aada17c92b0237b76cd5ae3bfcc8
Test: build with clang-tidy.
Used to skip only immediately followed identical warning lines.
Now classifywarning is called only once for each unique warning line.
Change-Id: Ie3b77ded70b41dafad91c042dbe15ad2be119e62
Test: run through build.log file
We should disable using imgdiff if *any* of the source and target
partitions uses squashfs.
Bug: 30004734
Test: Create an incremental with two builds with one of them uses squashfs.
Change-Id: I826cd13d7b852c548e4b45e61f5ae00f6407cac3
(cherry picked from commit f8acad1480)
We use imgdiff to handle files in zip format (e.g. jar/zip/apk) for
higher compression ratio.
For system/vendor in squashfs, a) all files are compressed in LZ4
format; b) we use 4096-byte block size in their sparse images, but the
files in squashfs may not be laid out as 4K-aligned. So the blocks for
a given file as listed in block map may not form a valid zip file, which
may fail the patch generation with imgdiff.
Disable using imgdiff for squashfs images, and use bsdiff instead.
Bug: 22322817
Change-Id: Ie76aa4cece5c9d38cb1d1a34c505a4a8f37512d3
(cherry picked from commit 293fd135c7)
Generate a new file containing care_data of system (and vendor)
partition, and add it under META/ of target file package. For
A/B update, copy this file to OTA package for later use by
update_verifier.
Bug: 27175949
Change-Id: I90bb972703afaeb94bc3efe718fd81b1cfbcabcc
update_engine expects the extracted public key instead of the
certificate.
Bug: 28701652
Change-Id: I292d39da9e039f96d01a4214226aeb46f8cb881d
(cherry picked from commit afaf295cb8)
The first line returned by awk is empty. In consequence, when
fs_config computes its mode it will consider this line as a file (no
trailing slash) and affect the default android_files mode which is
0644. The mode for the root directory should be the default
android_dirs mode 0755.
Add a special case in fs_config to consider empty line as a directory.
Change-Id: I9f33f6fcf4be05c31914db898e65c92b0a611518
Signed-off-by: Gaelle Nassiet <gaellex.nassiet@intel.com>
This moves build/tools/apksigner/core to its own project tools/apksig.
The move also renames the moved Java packages from
com.android.apksigner.core.* to com.android.apksig.* to reflect the
new name of the library.
Bug: 27461702
Change-Id: Iab812ae2b8f0a741014f842460c78e35bc249d43
We should disable using imgdiff if *any* of the source and target
partitions uses squashfs.
Bug: 30004734
Test: Create an incremental with two builds with one of them uses squashfs.
Change-Id: I826cd13d7b852c548e4b45e61f5ae00f6407cac3
add_img_to_target_files.py has an option of "-a" to add missing
images only. Under this option, the script should skip copying
the radio images for A/B devices when given image exists already
under "IMAGES/".
Test: Run the command on an A/B device, the existing radio images under "IMAGES/" don't get overwritten; and missing images are added correctly.
Bug: 29608905
Change-Id: Ie034b85a5d777d53e367f99470cea4d19cb1aaaf
This adds an APK signer primitive which preserves as much of the input
APK as possible. For example, it preserves the order of APK entries
and preserves their contents, including compressed form and alignment
of data.
Bug: 27461702
Change-Id: I51d07c530480182a66379e70a00f680544ff6214
For AB devices, support flashing two system partitions for factory use.
The normal system image on one partition, but without dex preopt. And a
system_other image that just contains the odex files. The dex files will
not be stripped out of the system image, in case the second system
partition is wiped.
Setting BOARD_USES_SYSTEM_OTHER_ODEX := true in the BoardConfig.mk
enables this behavior.
One can control which directories are placed in system_other by the
SYSTEM_OTHER_ODEX_FILTER configuration variable. Currently we default
to only copying only app and priv-app odexs.
Bug: 29278988
Change-Id: I7f4e87da919e7dc6a89fd8c668193cd4e98631bc
system_root_image expects the key at ROOT/verity_key as opposed to
BOOT/verity_key. Also refactor the verity key replacement lines.
Bug: 29397395
Test: 'sign_target_files_apks.py --replace_verity_private_key newkey --replace_verity_public_key newkey.pub target_files.zip signed-target_files.zip' and verify the replaced key in boot.img.
Change-Id: I58a5defff4be008ad55d4b5a5b7148569c3b8d66
(cherry picked from commit e0ee794fa1)
For A/B OTAs, by default it calls 'openssl pkeyutl' to sign the payload
and metadata with the package private key. If the private key cannot be
accessed directly, a payload signer that knows how to do that should be
supplied via "--payload_signer <signer>".
The signer will be called with "-inkey <path_to_private_key>",
"-in <input_file>" and "-out <output_file>" parameters.
Test: Use a dummy signer, call 'ota_from_target_files.py --payload_signer <signer> <target_files.zip> <ota.zip>' and verify the signatures in the generated package.
Bug: 28701652
Change-Id: I26cfdd3fdba6fc90799221741b75426988e46fd3
(cherry picked from commit dea0f8bfed)
Replace verity keyid with the keyid extracted from cert
passed through --replace_verity_keyid. The veritykeyid in the
BOOT/cmdline of input target files is replaced with keyid
extracted from --replace_verity_keyid and written to the
output target files.
BUG: 28384658
Change-Id: Ic683f36f543c4fcd94b6f95e40f01200fbf45ee1
(cherry picked from commit b58d23fe00)
It replaces the package verification key (change of path due to
system_root_image flag), as well as the payload verification key.
Bug: 29397395
Change-Id: I10435072aaf4356f2d8b5e1b6e82eb9cead7ad62
(cherry picked from commit 24a7206430)
Limit the number of blocks in each 'new' command to 1024. This should
decrease the chance of fsync error during an OTA update, similiar to
what we have done in b/29535618.
Bug: 29607757
Change-Id: I89f0a845d71eb810766e39860ab667c61b61a417
system_root_image expects the key at ROOT/verity_key as opposed to
BOOT/verity_key. Also refactor the verity key replacement lines.
Bug: 29397395
Test: 'sign_target_files_apks.py --replace_verity_private_key newkey --replace_verity_public_key newkey.pub target_files.zip signed-target_files.zip' and verify the replaced key in boot.img.
Change-Id: I58a5defff4be008ad55d4b5a5b7148569c3b8d66
* Add expand/collapse-all buttons to expand/collapse all warnings.
* Use HTML styles to reduce output file size.
Change-Id: Ica188cc4f123ce0ab8547f88315325c3e0560a39
Test: Checked output html file with Chrome browser.
This should make it easier to add parameters/options without breaking
existing clients.
Bug: 27461702
Change-Id: Ia4577f78d703a6b91828dd08492c78d5e9afb110
Limit the number of blocks in command zero to 1024 blocks. This
prevents the target size of one command from being too large and
might help to avoid fsync errors during the OTA update.
Bug: 29535618
Change-Id: Ic630cea2599138516162bd0029e2e4b2af75bf4f
(cherry picked from commit bb848c54a7)
Limit the number of blocks in command zero to 1024 blocks. This
prevents the target size of one command from being too large and
might help to avoid fsync errors during the OTA update.
Bug: 29535618
Change-Id: Ic630cea2599138516162bd0029e2e4b2af75bf4f
For A/B OTAs, by default it calls 'openssl pkeyutl' to sign the payload
and metadata with the package private key. If the private key cannot be
accessed directly, a payload signer that knows how to do that should be
supplied via "--payload_signer <signer>".
The signer will be called with "-inkey <path_to_private_key>",
"-in <input_file>" and "-out <output_file>" parameters.
Test: Use a dummy signer, call 'ota_from_target_files.py --payload_signer <signer> <target_files.zip> <ota.zip>' and verify the signatures in the generated package.
Bug: 28701652
Change-Id: I26cfdd3fdba6fc90799221741b75426988e46fd3
Replace verity keyid with the keyid extracted from cert
passed through --replace_verity_keyid. The veritykeyid in the
BOOT/cmdline of input target files is replaced with keyid
extracted from --replace_verity_keyid and written to the
output target files.
BUG: 28384658
Change-Id: Ic683f36f543c4fcd94b6f95e40f01200fbf45ee1
It replaces the package verification key (change of path due to
system_root_image flag), as well as the payload verification key.
Bug: 29397395
Change-Id: I10435072aaf4356f2d8b5e1b6e82eb9cead7ad62
This surfaces relevant NoSuchAlgorithmExceptions to the caller instead
of rethrowing as other exception types. Some setups need to be able to
distringuish issues due to their own misconfiguration
(required crypto algorithm mising -- NoSuchAlgorithmException) from
issues with the APK being signed or verified.
Bug: 27461702
Change-Id: I993f73edb29b2cd4cc485734a89a924ec357ef19
This enables verification of APKs which are served to a specific
range of Android platform versions, or to replicate behavior of
particular platform versions.
Bug: 27461702
Change-Id: I44ab4c99419eb97d72c4ccd109137fe1efda577d
Chih-hung Hsieh