* Use --byproject flag to dump warnings in HTML file by projects.
* Add external/google and vendor/google projects, vs other
external/non-google and vendor/non-google projects.
* Refactor code in dumpcategory to dump warnings by projects
when --byproject is specified.
* Recognize new compiler -Wvarargs warning pattern.
* No change to default HTML output format.
Change-Id: I2406ad392ab18683357ff0ee0cf6c41f87cf399c
Test: run through current build.log files.
* This enables 5 performance related checks:
performance-faster-string-find
performance-for-range-copy
performance-implicit-cast-in-loop
performance-unnecessary-copy-initialization
performance-unnecessary-value-param
* Recognize these new warnings in warn.py.
Bug: http://b/27779618
Change-Id: I7af3fdc262dd44db90d3b62eb37c5eac5a2eaa01
Test: build with WITH_TIDY=1 and run warn.py with build.log
* No change to html output, except removing unused "PageTop" anchor.
* Add --gencsv flag to output a CSV file instead of html file.
CSV files are combined to a table for dashboard charts.
* Count warnings from selected source project directories.
* CSV output lines are:
num_of_warnings,project_name,warning_description
* project_name is empty for sum of warnings from all projects.
* warning_description could be:
(1) "<severity>: <warning type description>"
for one warning type, or
(2) "<severity> warnings"
for all warning types of the severity.
Change-Id: I1cf976ed0cf142091226b22665b7f28d383608fd
Test: run through current build.log files.
* Count new clang-tidy warnings:
google-runtime-references
google-runtime-operator
* Count new clang warnings:
-Wconst-conversion similar to -Wliteral-conversion
-Wundefined-var-template
* Modified static analyzer warning message about
"Null passed to a callee that requires a non-null ..."
* Disable google-runtime-references by default and
disable misc-macro-parentheses in vendor projects;
too many old C/C++ code have this warning.
Change-Id: Ib35e0af85875aada17c92b0237b76cd5ae3bfcc8
Test: build with clang-tidy.
Used to skip only immediately followed identical warning lines.
Now classifywarning is called only once for each unique warning line.
Change-Id: Ie3b77ded70b41dafad91c042dbe15ad2be119e62
Test: run through build.log file
We should disable using imgdiff if *any* of the source and target
partitions uses squashfs.
Bug: 30004734
Test: Create an incremental with two builds with one of them uses squashfs.
Change-Id: I826cd13d7b852c548e4b45e61f5ae00f6407cac3
(cherry picked from commit f8acad1480)
We use imgdiff to handle files in zip format (e.g. jar/zip/apk) for
higher compression ratio.
For system/vendor in squashfs, a) all files are compressed in LZ4
format; b) we use 4096-byte block size in their sparse images, but the
files in squashfs may not be laid out as 4K-aligned. So the blocks for
a given file as listed in block map may not form a valid zip file, which
may fail the patch generation with imgdiff.
Disable using imgdiff for squashfs images, and use bsdiff instead.
Bug: 22322817
Change-Id: Ie76aa4cece5c9d38cb1d1a34c505a4a8f37512d3
(cherry picked from commit 293fd135c7)
update_engine expects the extracted public key instead of the
certificate.
Bug: 28701652
Change-Id: I292d39da9e039f96d01a4214226aeb46f8cb881d
(cherry picked from commit afaf295cb8)
The first line returned by awk is empty. In consequence, when
fs_config computes its mode it will consider this line as a file (no
trailing slash) and affect the default android_files mode which is
0644. The mode for the root directory should be the default
android_dirs mode 0755.
Add a special case in fs_config to consider empty line as a directory.
Change-Id: I9f33f6fcf4be05c31914db898e65c92b0a611518
Signed-off-by: Gaelle Nassiet <gaellex.nassiet@intel.com>
This moves build/tools/apksigner/core to its own project tools/apksig.
The move also renames the moved Java packages from
com.android.apksigner.core.* to com.android.apksig.* to reflect the
new name of the library.
Bug: 27461702
Change-Id: Iab812ae2b8f0a741014f842460c78e35bc249d43
add_img_to_target_files.py has an option of "-a" to add missing
images only. Under this option, the script should skip copying
the radio images for A/B devices when given image exists already
under "IMAGES/".
Test: Run the command on an A/B device, the existing radio images under "IMAGES/" don't get overwritten; and missing images are added correctly.
Bug: 29608905
Change-Id: Ie034b85a5d777d53e367f99470cea4d19cb1aaaf
This adds an APK signer primitive which preserves as much of the input
APK as possible. For example, it preserves the order of APK entries
and preserves their contents, including compressed form and alignment
of data.
Bug: 27461702
Change-Id: I51d07c530480182a66379e70a00f680544ff6214
system_root_image expects the key at ROOT/verity_key as opposed to
BOOT/verity_key. Also refactor the verity key replacement lines.
Bug: 29397395
Test: 'sign_target_files_apks.py --replace_verity_private_key newkey --replace_verity_public_key newkey.pub target_files.zip signed-target_files.zip' and verify the replaced key in boot.img.
Change-Id: I58a5defff4be008ad55d4b5a5b7148569c3b8d66
(cherry picked from commit e0ee794fa1)
For A/B OTAs, by default it calls 'openssl pkeyutl' to sign the payload
and metadata with the package private key. If the private key cannot be
accessed directly, a payload signer that knows how to do that should be
supplied via "--payload_signer <signer>".
The signer will be called with "-inkey <path_to_private_key>",
"-in <input_file>" and "-out <output_file>" parameters.
Test: Use a dummy signer, call 'ota_from_target_files.py --payload_signer <signer> <target_files.zip> <ota.zip>' and verify the signatures in the generated package.
Bug: 28701652
Change-Id: I26cfdd3fdba6fc90799221741b75426988e46fd3
(cherry picked from commit dea0f8bfed)
Replace verity keyid with the keyid extracted from cert
passed through --replace_verity_keyid. The veritykeyid in the
BOOT/cmdline of input target files is replaced with keyid
extracted from --replace_verity_keyid and written to the
output target files.
BUG: 28384658
Change-Id: Ic683f36f543c4fcd94b6f95e40f01200fbf45ee1
(cherry picked from commit b58d23fe00)
It replaces the package verification key (change of path due to
system_root_image flag), as well as the payload verification key.
Bug: 29397395
Change-Id: I10435072aaf4356f2d8b5e1b6e82eb9cead7ad62
(cherry picked from commit 24a7206430)
* Add expand/collapse-all buttons to expand/collapse all warnings.
* Use HTML styles to reduce output file size.
Change-Id: Ica188cc4f123ce0ab8547f88315325c3e0560a39
Test: Checked output html file with Chrome browser.
This should make it easier to add parameters/options without breaking
existing clients.
Bug: 27461702
Change-Id: Ia4577f78d703a6b91828dd08492c78d5e9afb110
Limit the number of blocks in command zero to 1024 blocks. This
prevents the target size of one command from being too large and
might help to avoid fsync errors during the OTA update.
Bug: 29535618
Change-Id: Ic630cea2599138516162bd0029e2e4b2af75bf4f
(cherry picked from commit bb848c54a7)
This surfaces relevant NoSuchAlgorithmExceptions to the caller instead
of rethrowing as other exception types. Some setups need to be able to
distringuish issues due to their own misconfiguration
(required crypto algorithm mising -- NoSuchAlgorithmException) from
issues with the APK being signed or verified.
Bug: 27461702
Change-Id: I993f73edb29b2cd4cc485734a89a924ec357ef19
This enables verification of APKs which are served to a specific
range of Android platform versions, or to replicate behavior of
particular platform versions.
Bug: 27461702
Change-Id: I44ab4c99419eb97d72c4ccd109137fe1efda577d
There's only a single library, libhost, and it's only used by acp and
atree in build/tools, move it there.
Bug: 28001743
Change-Id: Ie404d2793710de4e265a6fa95d462c32d4042623
This addresses the TODO to mimic the behavior of Android when
verifying APK JAR signatures. Unfortunately, the behavior of Android
kept changing in interesting ways between different platform versions.
This is hard-coded as a big lookup.
Bug: 27461702
Change-Id: I49bc181ee05f774ef8ee041af870385b35212c23
update_engine now accepts POWERWASH=1 to schedule a factory reset in
the post-install phase. Hook up with the --wipe_user_data flag in the
OTA script.
Bug: 28700985
Change-Id: Ie73876a61db90d124d2af588d674757376e9aabc
(cherry picked from commit 38ca0be399)
Instead of specifying character encoding by name, the faster, cleaner,
and safer way is to use StandardCharsets.UTF_8.
Bug: 27461702
Change-Id: I897284d3ceeb44a21cc74de09a9b25f6aec8c205
For acp, we've been using an old prebuilt in prebuilts/sdk, but it's not
part of the SDK. Instead, we'll use a prebuilt in the build-tools
repository.
For ijar, we've been using the host libstdc++ to workaround the lack of
libc++ on some unbundled branches. Instead, use a prebuilt that can use
libc++.
For ziptime, we've been disabling it on unbundled branches, due to the
lack of libc++. Instead, use a prebuilt version of ziptime that can use
the prebuilt libc++.
Change-Id: If80f845ea06f76e3fe6765964e77c864eaf303d0
This switches PKCS#7 SignedData generation code from Bouncy Castle to
OpenJDK's proprietary internal API. This is to avoid depending on a
huge library that's not really needed. In the longer term, it's best
to add our own implementation of PKCS#7 SignedData building, parsing,
and verification. This will give the code more power to mimic what the
Android platform does.
Bug: 27461702
Change-Id: I29f1ceea1293e35424fde69c0f2969d551345100
This replaces the less general DataSink which outputs into a
ByteArrayOutputStream with a more general DataSink which outputs into
an OutputStream.
Bug: 27461702
Change-Id: I9467f38c41f586b71f35edb3602fd6e57153184f
We use imgdiff to handle files in zip format (e.g. jar/zip/apk) for
higher compression ratio.
For system/vendor in squashfs, a) all files are compressed in LZ4
format; b) we use 4096-byte block size in their sparse images, but the
files in squashfs may not be laid out as 4K-aligned. So the blocks for
a given file as listed in block map may not form a valid zip file, which
may fail the patch generation with imgdiff.
Disable using imgdiff for squashfs images, and use bsdiff instead.
Bug: 22322817
Change-Id: Ie76aa4cece5c9d38cb1d1a34c505a4a8f37512d3
Needed to generate image.map and to optimize the OTA size
BUG: 22322817
Change-Id: I7802f4dbb9116a94ea99a00b68c3a7ff180ef08a
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
26f00cda4b introduced a bug where an
APK entry's extra field is padded for alignment purposes when no
padding is necessary because the entry is aligned without any padding
bytes.
Bug: 27461702
Change-Id: Icb164dbaa26d9686412e2920318a9f40c5ce9751
This patch uses subprocess.communicate instead of subprocess.wait to
prevent deadlock if any of the child processes outputs too much data,
and redirects the subprocess output to stdout when running in verbose
mode.
With this patch `ota_from_target_files -v` prints the delta_generator
output in stdout, and no output if '-v' is not passed.
Bug: None
TEST=ota_from_target_files -v ...
Change-Id: Id66e4f3360a6f91d61a3ce96d53afbccdaa19da5
Add the build property "build.version.incremental" of the source (if
present) and target files to the metadata of the ota update package.
Example of metadata:
....
post-build-incremental=2951741
post-timestamp=1465345123
pre-build-incremental=2943039
pre-device=bullhead
...
Bug: 28658632
Change-Id: I889e8ccf39633b1b35590751001a42d1b05d5514
This adds the ApkVerifier class which verifies APKs using APK
Signature Scheme v2 only. In a follow-up commit this class will be
extended to verify APKs using JAR signature scheme when necessary.
The APK verifier is designed to not just verify an APK, but also
report errors, warnings, and information about signers in a structured
way, to enable tools to surface this information to users in various
ways.
Bug: 27461702
Change-Id: I10c6ba436021d86b6dbf6d3cf44494652adacb66
This modifies JAR signing code to produce DSA signatures which are
accepted by all Android platforms rather than only API Level 9 and
higher.
The issue is that by default Bouncy Castle uses OID 1.2.840.10040.4.3
(dsaWithSha1) in PKCS #7 CMS SignerInfo whereas Android accepts that
only since API Level 9. However, OID 1.2.840.10040.4.1 (dsa) is
accepted by all Android platforms.
Bug: 27461702
Change-Id: I24256a255bcdc2108bdb447557af7568a2c096e3
In preparation for including them in prebuilts/build-tools.
acp: We use a prebuilt in prebuilts/sdk, but it's not part of the sdk.
ijar: We use the host libstdc++ to workaround the lack of libc++ on
some unbundled branches.
ziptime: We disable this on unbundled builds, due to the lack of libc++.
Change-Id: Ib9766b1dbddd151c38ff27c529865200ab37fce1
am: 732b29a5c3
* commit '732b29a5c38a3a33bdffea662875e6a8dab894b3':
Improve efficiency of using the DataSource abstraction.
Change-Id: I4b2e05ffc80d59ef19a1b78c3560f0d21c5b04b2
This adds getByteBuffer and copyTo methods to the DataSource
abstraction. These methods enable the client to avoid unnecessary
copying of the data source's data.
Bug: 27461702
Change-Id: If4e9f902ea75c1ca5c7be0e20c0e7218faf9c504
findleaves.py needs to be able to search multiple directories for
multiple filenames (Android.bp and Android.mk). Add support for a new
--dir= option to specify one or more directories to search, and then
treat any remaining options as filenames. If no directories are
specified, fall back to treating only the last option as a filename and
the rest as directories.
Change-Id: Ie879facd49192c6b4fafae070ad00eb21b1faa32
This switches signapk's APK signing from its own signing logic to that
offered by apksigner-core library. OTA update package signing logic
remains inside signapk codebase.
Bug: 27461702
Change-Id: Ibf8435c555fe3f2b621d5189e7ae44f79082c810
am: 4ea527597e
* commit '4ea527597e72ddff825a32fc24c6e4970d50da8f':
Store entry alignment information in APK.
Change-Id: I3ab22f58eba48743a6533265217f09ab007840f6
1. This adds way to obtain a DataSource which represents a region of
data contained in the DataSource.
2. This fixes a design bug in "feed" method where the size parameter
was an int instead of long.
3. This fixes a bug in ByteBufferDataSource where its mSize field was
a long instead of an int (ByteBuffer's length cannot be more than
2^31).
Bug: 27461702
Change-Id: Ib0812784beb581f19d2412e667b8bd018f0a3c78
am: a5bce98621
* commit 'a5bce98621ef4aa7632ead4de267118f4832d6c2':
Fix improper date reference in blog reader.
Change-Id: Iedec1fc31a923e12a9733427a7ee2666c761b3e5
am: 53cd365b2d
* commit '53cd365b2d2c2b074d628147a8a382275075bd1d':
Fix improper date reference in blog reader.
Change-Id: I300892c3c4439354917872839c18e4c029160df1
Chih-Hung Hsieh