Commit Graph

3637 Commits

Author SHA1 Message Date
Alex Klyubin a85e0aa883 Merge "Fix inefficiency in APK entry data alignment." 2016-06-13 18:14:18 +00:00
Alex Klyubin ab2a3b0061 Fix inefficiency in APK entry data alignment.
26f00cda4b introduced a bug where an
APK entry's extra field is padded for alignment purposes when no
padding is necessary because the entry is aligned without any padding
bytes.

Bug: 27461702
Change-Id: Icb164dbaa26d9686412e2920318a9f40c5ce9751
2016-06-11 17:47:20 -07:00
Alex Deymo d8d96ecdae Redirect subprocess stderr to stdout in verbose mode.
This patch uses subprocess.communicate instead of subprocess.wait to
prevent deadlock if any of the child processes outputs too much data,
and redirects the subprocess output to stdout when running in verbose
mode.

With this patch `ota_from_target_files -v` prints the delta_generator
output in stdout, and no output if '-v' is not passed.

Bug: None
TEST=ota_from_target_files -v ...

Change-Id: Id66e4f3360a6f91d61a3ce96d53afbccdaa19da5
2016-06-10 16:38:31 -07:00
Alex Klyubin 8740e9df19 APK Signature Scheme v2 APK verifier.
This adds the ApkVerifier class which verifies APKs using APK
Signature Scheme v2 only. In a follow-up commit this class will be
extended to verify APKs using JAR signature scheme when necessary.

The APK verifier is designed to not just verify an APK, but also
report errors, warnings, and information about signers in a structured
way, to enable tools to surface this information to users in various
ways.

Bug: 27461702
Change-Id: I10c6ba436021d86b6dbf6d3cf44494652adacb66
2016-06-08 08:23:30 -07:00
Alex Klyubin ebe68ec237 Merge "Fix DSA APK signatures for API Level 8 and lower." 2016-06-08 15:17:42 +00:00
Treehugger Robot c52d09e694 Merge "Switch some build tools to Soong" 2016-06-06 22:51:57 +00:00
Alex Klyubin 819b5354e0 Fix DSA APK signatures for API Level 8 and lower.
This modifies JAR signing code to produce DSA signatures which are
accepted by all Android platforms rather than only API Level 9 and
higher.

The issue is that by default Bouncy Castle uses OID 1.2.840.10040.4.3
(dsaWithSha1) in PKCS #7 CMS SignerInfo whereas Android accepts that
only since API Level 9. However, OID 1.2.840.10040.4.1 (dsa) is
accepted by all Android platforms.

Bug: 27461702
Change-Id: I24256a255bcdc2108bdb447557af7568a2c096e3
2016-06-06 15:05:08 -07:00
Dan Willemsen 56eb2f9f54 Switch some build tools to Soong
In preparation for including them in prebuilts/build-tools.

acp: We use a prebuilt in prebuilts/sdk, but it's not part of the sdk.

ijar: We use the host libstdc++ to workaround the lack of libc++ on
some unbundled branches.

ziptime: We disable this on unbundled builds, due to the lack of libc++.

Change-Id: Ib9766b1dbddd151c38ff27c529865200ab37fce1
2016-06-06 14:51:52 -07:00
Treehugger Robot a571e83095 Merge "Remove dead rgb2565" 2016-06-06 21:35:47 +00:00
Alex Klyubin 7353eabd14 Merge "Improve efficiency of using the DataSource abstraction." 2016-06-06 21:07:52 +00:00
Dan Willemsen 08b12dc2c8 Remove dead rgb2565
Change-Id: I2397dc97768f24a7c416bff96dd983fc796cbb53
2016-06-06 13:32:11 -07:00
Alex Klyubin 1a39e8eb34 Merge "Switch signapk to apksigner-core." 2016-06-06 19:40:23 +00:00
Alex Klyubin b02eefacbf Improve efficiency of using the DataSource abstraction.
This adds getByteBuffer and copyTo methods to the DataSource
abstraction. These methods enable the client to avoid unnecessary
copying of the data source's data.

Bug: 27461702
Change-Id: If4e9f902ea75c1ca5c7be0e20c0e7218faf9c504
2016-06-06 11:17:03 -07:00
Treehugger Robot 40623e2a99 Merge "Add --dir option to findleaves.py" 2016-05-28 00:53:47 +00:00
Colin Cross 9808645cd9 Add --dir option to findleaves.py
findleaves.py needs to be able to search multiple directories for
multiple filenames (Android.bp and Android.mk).  Add support for a new
--dir= option to specify one or more directories to search, and then
treat any remaining options as filenames.  If no directories are
specified, fall back to treating only the last option as a filename and
the rest as directories.

Change-Id: Ie879facd49192c6b4fafae070ad00eb21b1faa32
2016-05-27 16:05:10 -07:00
Alex Klyubin fa1da6c311 Switch signapk to apksigner-core.
This switches signapk's APK signing from its own signing logic to that
offered by apksigner-core library. OTA update package signing logic
remains inside signapk codebase.

Bug: 27461702
Change-Id: Ibf8435c555fe3f2b621d5189e7ae44f79082c810
2016-05-27 14:45:51 -07:00
Alex Klyubin ce4c9d7108 Merge "Improve DataSource interface." 2016-05-27 20:00:59 +00:00
Alex Klyubin 7b977ea0e9 Improve DataSource interface.
1. This adds way to obtain a DataSource which represents a region of
   data contained in the DataSource.
2. This fixes a design bug in "feed" method where the size parameter
   was an int instead of long.
3. This fixes a bug in ByteBufferDataSource where its mSize field was
   a long instead of an int (ByteBuffer's length cannot be more than
   2^31).

Bug: 27461702
Change-Id: Ib0812784beb581f19d2412e667b8bd018f0a3c78
2016-05-27 10:28:52 -07:00
Alex Klyubin 26f00cda4b Store entry alignment information in APK.
Data of uncompressed APK entries is often aligned to a multiple of 4
or 4096 in the APK to make it easier to mmap the data. Unfortunately,
the current method for achieving alignment suffers from two issues:
(1) the way it uses the Local File Header extra field is not compliant
with ZIP format (for example, this prevents older versions of Python's
zipfile from reading APKs: https://bugs.python.org/issue14315), and
(2) it does not store information about the alignment multiple in the
APK, making it harder/impossible to preserve the intended alignment
when rearranging entries in the APK.

This change solves these issues by switching to a different method for
aligning data of uncompressed APK entries. Same as before, alignment
is achieved using Local File Header entry field. What's different is
that alignment is achieved by placing a well-formed extensible data
field/block into the extra field. The new field/block contains the
alignment multiple (e.g., 4 or 4096) as well as the necessary padding
(if any). Compared to the original alignment method, the new method
uses 6 more bytes for each uncompressed entry.

Bug: 27461702
Change-Id: I8cffbecc50bf634b28fca5bc39eb23f671961cf9
2016-05-23 17:46:18 -07:00
Alex Klyubin ac68c28831 Merge "Improve concurrency for JAR entry inspection requests." 2016-05-23 19:26:14 +00:00
Alex Klyubin 094caf99aa Merge "No need to JAR-sign OTA update packages." 2016-05-16 17:11:04 +00:00
Alex Klyubin 2af45bacb4 Improve concurrency for JAR entry inspection requests.
This moves the creation of potentially expensive objects from the
thread which creates JAR entry inspection requests, to the thread
which fulfills these requests, increasig concurrency opportunities.

Bug: 27461702
Change-Id: If753b2de7fb04ee5e2e4bbcb27d42269d7fa5def
2016-05-13 18:51:18 -07:00
Ian Rogers 86c3b72d9c Merge "Make Error Prone warnings in sync with Error Prone documenation." 2016-05-13 18:19:18 +00:00
Ian Rogers 6e52003b54 Make Error Prone warnings in sync with Error Prone documenation.
Use summary information from annotation within Error Prone to auto-generate
the warnings list.

Bug: 28681096
Change-Id: Ib3a5f734bd229726fcd7332eaa9e0e1bd1200c25
2016-05-13 10:18:18 -07:00
Mohamad Ayyash 465acf8631 Use fs_config files for mksquashfs
BUG: 27467028
Change-Id: I7648030ad4b31d70a5d439e9552fd2cbfe288b74
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
BUG: 28704419
2016-05-12 18:44:48 -07:00
Alex Klyubin 6975f03fd8 Merge "Offer an ApkSignerEngine implementation." 2016-05-12 18:43:08 +00:00
Alex Klyubin 0caa16a6d1 No need to JAR-sign OTA update packages.
This removes the logic for JAR signing from -w (whole-file signing)
mode. This mode is designed specifically for OTA update packages. When
such packages are verified, their JAR signatures are ignored. Thus,
there is no need to JAR-sign in -w mode.

For context, OTA update packages are protected by a special signature
residing in the ZIP End of Central Directory record (at the very end
of the file). This is the signature verified when update packages are
being applied to Android.

Change-Id: Ia852a11ed6774ce746087cdd7f028b191ef6bc8b
2016-05-12 11:32:43 -07:00
Alex Klyubin e305f45f00 Offer an ApkSignerEngine implementation.
This adds an implementation of ApkSignerEngine to the apksigner-core
library.

Bug: 27461702
Change-Id: I5f977b98555ca507a0dfcd3e92eecb9758aa8370
2016-05-12 10:56:06 -07:00
Treehugger Robot 648ea82b04 Merge "Include BRILLO_VENDOR_PARTITIONS in target zip generation" 2016-05-11 21:49:56 +00:00
Chih-hung Hsieh bc0ab6d133 Merge "Fix misc-macro-parentheses warnings." 2016-05-11 20:30:47 +00:00
Wei Wang 2e735ca34e Include BRILLO_VENDOR_PARTITIONS in target zip generation
This change will enable the target zip to include partition defined in
BRILLO_VENDOR_PARTITIONS. This is necessary for OTA to support
partitions other than boot and system in ab_partitions.txt.

The target zip will include images in BRILLO_VENDOR_PARTITIONS if
defined, under VENDOR_IMAGES/ with path kept. Also any vendor partitions
defined in AB_OTA_PARTITIONS will be copied to IMAGES/ in the target zip.

BUG: 28623063
Change-Id: Ic479048dfb8ac506acf827865e784fcb0432a1d5
2016-05-11 12:26:04 -07:00
Ian Rogers adf2141b66 Merge "Add code search links to warnings." 2016-05-11 19:07:36 +00:00
Chih-Hung Hsieh f536e7ba34 Fix misc-macro-parentheses warnings.
Add parentheses around macro arguments used beside binary operators.

Bug: 28705665
Change-Id: I8594b9463e2389a46d6e0235757b93de065cd007
2016-05-11 11:55:33 -07:00
Tao Bao 7ff7c2d870 Merge "Revert "Include vendor partitions in target zip generation"" 2016-05-11 15:11:57 +00:00
Wei Wang 05e6f679e6 Revert "Include vendor partitions in target zip generation"
This reverts commit 421fe44ae8.

Change-Id: Id45781de73735783bc7dfb6fc2838957578d00d3
2016-05-11 14:49:50 +00:00
Ian Rogers f382973250 Add code search links to warnings.
Add argument parsing using the argparse library.
Make HTML easier to read by adding a \n after each table row.

Change-Id: I6b745fc60afb96fb51e5f69df7d607b5fb8da0e5
2016-05-10 22:36:38 -07:00
Treehugger Robot bb05fd768c Merge "Include vendor partitions in target zip generation" 2016-05-11 02:18:58 +00:00
Ian Rogers 32bb9bd03a Add error prone patterns to warn.py
Bug: 28681096

(cherry picked from commit df79adacab)
Change-Id: I06ac0ed4593b03cea0a8163ca85e393ff1358855
2016-05-10 13:54:36 -07:00
Treehugger Robot afd3d55261 Merge "ApkSignerEngine abstraction for apksigner-core library." 2016-05-10 19:13:41 +00:00
Wei Wang 421fe44ae8 Include vendor partitions in target zip generation
This change will enable the target zip to include partition defined in
BRILLO_VENDOR_PARTITIONS. This is necessary for OTA to support
partitions other than boot and system in ab_partitions.txt.

BUG: 28623063
Change-Id: I6e0969c31c9ad2e8285c6f560825c160aa8c4d55
2016-05-10 11:00:09 -07:00
Ian Rogers 2f4ce82d13 Sort warnings with most frequent first.
Change-Id: Id608c20d065bec83dcb2993658cc743d5e9dcb46
2016-05-10 09:34:29 -07:00
Alex Klyubin 5f987967da ApkSignerEngine abstraction for apksigner-core library.
This adds the ApkSignerEngine abstraction to the apksigner-core
library. This abstraction enables the separation of
parsing/building/writing of an APK from signing the APK.

Users of ApkSignerEngine invoke it during specific I/O events (e.g.,
JAR entry encountered in input APK, all output JAR entries ready).
The engine responds by occasionally providing instructions about what
to do next. For example, it may provide JAR entries to be added to
the output APK, thus making the output APK signed.

The default implementation of ApkSignerEngine will be provided in  a
follow-up commit.

Bug: 26516150
Bug: 27461702
Change-Id: Ia85fd09497a6e2ff3103ba10c53a1848ba731f7b
2016-05-09 13:20:03 -07:00
Alex Klyubin 265db0826e APK Signature Scheme v2 signing logic for apksigner-core.
apksigner-code library will offer a high-level primitive (future
commit) for signing APKs. This is meant to be used by
build/tools/signapk and Android Studio's APK builder/signer.

This commit adds a lower-level APK Signature Scheme v2 (aka v2
signing) code which will be used by the future APK signing abstraction
exposed by this library.

All classes (except DataSource and DataSources) added by this commit
are internal (i.e., implementation details of this library). Clients
of this library should not be using these classes.

Bug: 26516150
Change-Id: I98d4da0666cf122667c67565108ea4fb28ac51e6
2016-05-09 12:16:33 -07:00
Treehugger Robot b60b340597 Merge "JAR signing logic for the apksigner-core library." 2016-05-05 22:42:43 +00:00
Alex Klyubin 85b8a0146e JAR signing logic for the apksigner-core library.
apksigner-code library will offer a high-level primitive (future
commit) for signing APKs. This is meant to be used by
build/tools/signapk and Android Studio's APK builder/signer.

This commit adds a lower-level JAR signing (aka v1 signing) code which
will be used by the future APK signing abstraction exposed by this
library.

All classes added by this commit are internal (i.e., implementation
details of this library). Clients of this library should not be using
these classes.

Bug: 26516150
Change-Id: I5cecd435e63aab206d63868be5e0d0e289e7c423
2016-05-05 10:53:56 -07:00
Alex Klyubin b141ded82b SHA-256 with ECDSA supported only on API Level 21 and higher.
Turns out APK signatures using SHA-256 with ECDSA are accepted only by
platforms with API Level 21 and higher, not 18 and higher.

Bug: 28296599

Change-Id: I3fab5be17bf3a9bdbf4d84d90d51448027c7e761
2016-05-03 18:15:39 -07:00
Alex Klyubin cc3440cc1d Merge "Separate logic for alg selection for OTA and APK signing." 2016-05-03 20:50:54 +00:00
Shinichiro Hamaji a6da20e2d2 Merge "Initialize an uninitialized field in ijar" 2016-05-02 19:00:43 +00:00
Shinichiro Hamaji d6c694b302 Initialize an uninitialized field in ijar
The upstream patch was merged:
https://github.com/bazelbuild/bazel/commit/a5004430338bc72817c7

Bug: 28347190
Change-Id: I9b8b9bc830bd16299b94e8c92c22dba8aa36affb
2016-05-03 03:19:23 +09:00
Chih-hung Hsieh 169cb4b68c Merge "Fix google-explicit-constructor warnings." 2016-05-02 18:08:16 +00:00