Commit Graph

15 Commits

Author SHA1 Message Date
Dan Willemsen 263dde779e Fix par file zip offsets
The zip file format does support scripts/programs prefixed to the
archive, though many of the offsets are supposed to start from the
beginning of the file. Some tools (python and zipinfo) are able to
read zip files with arbitrary prefixes, but others (libziparchive and
zipdetails) fail.

So pass the file to prefix onto the zip file into merge_zips so that we
can set the offsets from the real start of the file.

Test: m sepolicy_tests  (runs the embedded python interpreter)
Test: zipinfo out/host/linux-x86/bin/sepolicy_tests
Test: zipdetails out/host/linux-x86/bin/sepolicy_tests
Change-Id: If73d4c2465581f7de5aa47959284ecf2059df091
2018-12-03 15:55:15 -08:00
Colin Cross dc1e829b59 Always allow duplicates with identical CRC32 and size
Don't warn on duplicate files in merge_zips if they have identical
CRC32 and size values.

Test: m checkbuild
Test: merge_zips_test.go
Change-Id: I61336ca4d4d3b7402c24a7abd337bd350fe10930
2018-10-17 15:49:53 -07:00
Colin Cross 05518bc13b soong_zip: Add tests
Add test that cover basic command line usage of soong_zip.  -D
is not covered yet as the implementation will be replaced with
one that is also more easily testable in the next patch.

Bug: 116751500
Test: zip_test.go
Change-Id: I5a1bcee74ebc9cb3cf332c36f89bc12c0e807ad2
2018-09-28 13:56:06 -07:00
Colin Cross 4c03f68763 Make merge_zips -stripFile use blueprint style globs
merge_zips -stripFile was only considering the name of the file and
ignoring the path.  Make it more useful by supporting blueprint style
globs.  The previous behavior can be recreated by prefixing with **/.

Bug: 111389216
Test: m checkbuild
Change-Id: I25760fe3f1f77704dd9da9d107d9a38a415d681f
2018-07-16 15:24:43 -07:00
Colin Cross 2486065c43 Add tests for merge_zips
Test: merge_zip_test.go
Change-Id: I8fc577946b40cad193864aa2ebda1bef3a0e050e
2018-07-16 15:24:22 -07:00
Nan Zhang 1db85407aa Remove timestamp based filelist file for tracking Python dependencies
Each Python module will generate a zip file containing source & data
files. The Python binary will collect all its dependencies and use
merge_zips to merge each zip file to create a final .par file.

Test: m -j checkbuild && real examples:
Bug: b/70568913

Change-Id: I9ff232d461d33e1c06026e7dcb5b124bf02c3ce5
2018-03-09 14:12:48 -08:00
Nan Zhang 5925b0fda4 Add -p, -e options in merge_zips for supporting par format.
Move __init__.py files population to merge_zips, and add options: -p, and
-e. Since par needs entry_point.txt to search the program entry point, so
add -e option to support it.

Test: ./merge_zips -p -e entry_point.txt out in1.zip in2.zip
Bug: 70568913
Change-Id: Ida10faf125e1b94dffc98b30240db3b90fd75b25
2017-12-20 12:57:10 -08:00
Colin Cross e909e1e079 Add support for --ignore_duplicates to merge_zips
Add a --ignore_duplicates option to cause merge_zips to
silently take the first entry in the case of duplicate entries
in the zips to be merged.  This will be used for soong Jacoco
support to combine the instrumented jar with the original jar
to pick up any classes that were excluded.

Bug: 69629238
Test: m EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true SKIP_BOOT_JARS_CHECK=true WITH_DEXPREOPT=false
Change-Id: I49ede3f07ce0ed7701b4db7058da2e4f11b8043e
2017-11-22 16:08:35 -08:00
Colin Cross 5c6ecc179f Don't panic in turbine with no classes
ALLOW_MISSING_DEPENDENCIES=true can cause missing dependencies to be
hidden from GenerateAndroidBuildActions, which is triggering the
len(jars) == 0 check in compileJavaHeader.  For the
ALLOW_MISSING_DEPENDENCIES=true case the check is unnecessary, as the
build rules will be replaced with runtime errors.  On the off chance
the check would have been hit in a real build, allow merge_zips to
create an empty zip file if it is asked to.

Test: m ALLOW_MISSING_DEPENDENCIES=true
Change-Id: I3171f921d51229ddf38a1a647d32566658c673fa
2017-10-24 16:01:44 -07:00
Colin Cross 0cf45cdc1e Add -stripFile argument to merge_zips
Putting resources in dex jars is going to need to pull files from
classpath jars that do not match *.class.

Test: m -j checkbuild
Change-Id: Ia37b8b9387e5f5636769afc6937b9aa3c9ec1ced
2017-10-04 17:19:43 -07:00
Colin Cross 635acc9446 Rearrange manifest file handling in merge_zips and soong_zip
Jar always puts default MANIFEST.MF files in if none was specified.
Copying that behavior in soong_zip causes problems with merge_zips,
because it ends up taking the default manifest from the classes.jar
instead of the user's manifest from res.jar.  We don't want the
user's manifest in the classes.jar, otherwise a change to the
manifest will cause all the class files to rebuild.  Instead,
move the manifest insertion to the final merge_zips stage.

Test: m -j checkbuild
Change-Id: Id6376961dbaf743c2fb92843f9bdf2e44b963be0
2017-09-20 13:20:45 -07:00
Nan Zhang 13f4cf5c9e Add -stripDir and -zipToNotStrip option in merge_zips.
In some cases, we need delete META-INF dir to void mis-using some
classes files in the Java runtime.

For now, when we delete META-INF, we need keep Manifest file in the jar
since merge_zips doesn't have ability to add a default one.

Bug: b/65455145
Test: m clean && m -j

Change-Id: Iea44b1a98d348cd8df1fa7374907733b1add6935
2017-09-19 18:44:43 -07:00
Nan Zhang d5998cce7d Don't add data_descripters when merging uncompress zip entries for merge_zips.
Also filter out META-INF/TRANSITIVE dir, and report warnings when
merge_zips see duplicates entries with different CRC hash.

Bug: b/65455145
Test: m clean && m -j java (locally)

Change-Id: I47172ffa27df71f3280f35f6b540a7b5a0c14550
2017-09-13 23:01:49 -07:00
Colin Cross 34540315a0 Allow some duplicates in merged jars
Only take the first MANIFEST.MF or module-info.class file.

Test: m -j checkbuild
Change-Id: Ifbf9fe272437ef2c2bd51ab4849ac8d7ef37b6fc
2017-09-06 15:01:07 -07:00
Jeff Gaston 8bab5f2015 Implement merge_zips
For merging .zip and/or .jar files

Bug: 64691570
Test: merge_zips merged.zip in1.zip in2.zip
Change-Id: I9a080d25b92a506c9b5451670c4786895b2722ab
2017-09-01 13:52:13 -07:00