Commit Graph

155 Commits

Author SHA1 Message Date
Narayan Kamath ecc0b6c605 Merge "zip_archive_test: compare error codes directly." am: d1e9e7bc06
am: 9be5dbed91

Change-Id: I42d449eba683e2f2b158992573cacbbde2b28e27
2017-06-20 10:58:30 +00:00
Narayan Kamath d1e9e7bc06 Merge "zip_archive_test: compare error codes directly." 2017-06-20 10:50:27 +00:00
Narayan Kamath 1ef9d2dfbe zip_archive_test: compare error codes directly.
Make them available to tests so that they can compare values
directly instead of comparing error code strings.

Test: zip_archive_test
Change-Id: I1c78c24651ff7549b5125a49a4e67bbd8295c617
2017-06-19 13:03:41 +01:00
Elliott Hughes a431e2961a Merge "Add libziparchive-based unzip." am: 3e67944d2e
am: 3cd4306ea2

Change-Id: I091fe18bb661a190b06ac6e884f285ebc9063dd6
2017-06-15 21:47:38 +00:00
Elliott Hughes 3e67944d2e Merge "Add libziparchive-based unzip." 2017-06-15 21:30:56 +00:00
Narayan Kamath 08ffb1b94a Merge "zip_archive: validate data descriptor contents." am: b001cc5b1c
am: ec95719844

Change-Id: Ie1d5c733c478100b4e7b09be4ef4f59f9bb6d688
2017-06-15 11:00:52 +00:00
Narayan Kamath 162b705801 zip_archive: validate data descriptor contents.
- Ensure that the compressed and uncompressed lengths in the data
  descriptor match those in the central directory.

- Calculate CRCs for compressed entries and ensure that those CRCs
  match the information in the central directory / data descriptor.
  This is currently guarded by a flag and will be turned on at a
  later stage.

- Finally, always trust the LFH when it comes to indicating the
  existence of a data descriptor. We no longer require this info to
  match the central directory to bring this in line with other zip
  processing tools. (see NOTE in the included change).

Bug: 62047801
Test: zip_archive_test
Change-Id: Icceea66a8b1937c08a4af9aee97bc77ad7218c0f
2017-06-14 14:29:53 +01:00
Adam Lesinski 1d105aa597 libziparchive: verify that gpb flags match
The Central File Header and Local File Header should
have the same general purpose bit flags set for
consistency.

Bug: 36686974
Test: existing tests pass (ziparchive-tests)

(cherry picked from commit 73b3aa541f)

Change-Id: Ia672a28732154a564ea2f2e3520238bb77924a56
2017-06-14 14:25:55 +01:00
Elliott Hughes 55fd293dfe Add libziparchive-based unzip.
Bug: N/A
Test: new toybox unzip.test
Change-Id: I00be388578be9a0a0390a9e2ecfac664c6ab39ca
2017-06-04 11:53:48 -07:00
Sebastian Pop e8fa0313ef Merge "add a performance benchmark for libziparchive" am: f8e96ea2d1 am: 48a6049eca
am: 8b44403ad6

Change-Id: Ib389678a5692faefff82077fb1af2835a9660661
2017-05-30 20:01:24 +00:00
Sebastian Pop 8b44403ad6 Merge "add a performance benchmark for libziparchive" am: f8e96ea2d1
am: 48a6049eca

Change-Id: I6667e494ac8cbca9663294e656344638b92f8c64
2017-05-30 19:57:53 +00:00
Treehugger Robot f8e96ea2d1 Merge "add a performance benchmark for libziparchive" 2017-05-30 19:44:52 +00:00
Sebastian Pop 0a94c56463 add a performance benchmark for libziparchive
This patch adds two benchmarks that measure the performance of some
operations of libziparchive.

Both benchmarks are creating a temporary zip file containing file
names of uniformly distributed lengths.  The creation of the zip
file is not timed in the benchmarks.

- FindEntry_no_match tries to find an inexisting entry in the files
  of the zip archive, in order to force the code to examine all the
  files in the archive.

- Iterate_all_files uses the iterate function to list all the files
  in the archive.

Bug: N/A
Test: adb shell /data/ziparchive-benchmarks
Change-Id: Ibdb524ba1c5ae55caddf0416ebbc09f8b6df0021
2017-05-30 10:50:14 -07:00
Jayant Chowdhary 34daec25b2 Merge "Mark libziparchive vendor_available." am: e0ee6d31fd am: 55a59f9906
am: 6d6673314c

Change-Id: I5f81256ca1c6c957322dcef8777f69cbfa1d61b7
2017-05-25 17:28:33 +00:00
Jayant Chowdhary 6d6673314c Merge "Mark libziparchive vendor_available." am: e0ee6d31fd
am: 55a59f9906

Change-Id: Ie9439e0165583696d63f3453e352a52dcfd02cc2
2017-05-25 17:23:10 +00:00
Jayant Chowdhary 2c87bccf46 Mark libziparchive vendor_available.
libziparchive is a library which belongs to vndk-cap. Mark it
vendor_available to enable header-abi-dumper's invocation to identify it
as a part of the vndk.

Details: https://android-review.googlesource.com/368372

Test: mm -j64

Bug: 38244611

Change-Id: Ibe490cc6c2cfca0f8d58df45317bb3a491d530f0
2017-05-24 12:33:12 -07:00
Tianjie Xu e170e3876b Fix out of bound read in libziparchive am: d9fd1863f4 am: 6908996434 am: 25c0d68d29 am: ed33065bfc am: bc8c8e4b56 am: afddaba0cc am: 0b1ad23070 am: d4b4f0a1e1
am: 2aea56db64

Change-Id: Ib2eebcb64ffb95737ef4a739dddf181dcdaa9fae
2017-04-11 20:11:24 +00:00
Tianjie Xu f4bb9db0a8 Fix out of bound read in libziparchive am: d9fd1863f4 am: 6908996434 am: 25c0d68d29 am: ed33065bfc am: bc8c8e4b56 am: afddaba0cc am: 0b1ad23070 am: d4b4f0a1e1
am: 2aea56db64

Change-Id: Icecc6630e82f43360d55b925a5946ffd0d608f58
2017-04-11 20:11:00 +00:00
Tianjie Xu bc8c8e4b56 Fix out of bound read in libziparchive am: d9fd1863f4 am: 6908996434 am: 25c0d68d29
am: ed33065bfc

Change-Id: Ia2b0e05ae0e248d692d9ad4ad62e53613203f08e
2017-04-11 19:45:58 +00:00
Adam Lesinski d987c9dd9e ZipWriter: Keep LFH and CFH in sync
We change the GPB in the LocalFileHeader if the entry can not
have a trailing DataDescriptor. Make sure to patch the
CentralFileHeader to have the same bits set.

Modify ZipArchive to check that the data descriptor bit is consistent
between Central and Local file headers.

(cherry-pick of commit e0eca55fe6)

Test: make ziparchive-tests
Bug: 36686974
Change-Id: Ied167570abcf6426b1c678cd40123e5ad65909db
2017-04-11 01:45:25 +00:00
Adam Lesinski 73b3aa541f libziparchive: verify that gpb flags match
The Central File Header and Local File Header should
have the same general purpose bit flags set for
consistency.

Bug: 36686974
Test: existing tests pass (ziparchive-tests)
Change-Id: I4602336fa96359219b8a3fd69fd318eed680c43d
2017-04-10 12:10:53 -07:00
Adam Lesinski e0eca55fe6 ZipWriter: Keep LFH and CFH in sync
We change the GPB in the LocalFileHeader if the entry can not
have a trailing DataDescriptor. Make sure to patch the
CentralFileHeader to have the same bits set.

Modify ZipArchive to check that the data descriptor bit is consistent
between Central and Local file headers.

Test: make ziparchive-tests
Bug: 36686974
Change-Id: Ied167570abcf6426b1c678cd40123e5ad65909db
2017-04-10 12:05:43 -07:00
Adam Lesinski 47f5581927 Merge "ZipWriter: Do not write DataDescriptor for STORED files" 2017-04-07 00:33:20 +00:00
Adam Lesinski e2fa70bcb0 ZipWriter: Do not write DataDescriptor for STORED files
Older implementations of ZIP (Java's ZipInputStream) don't
like a Data Descriptor trailing after an uncompressed entry.

If the FILE is seekable, and the entry is uncompressed, seek
back to the header and write out the crc and sizes.

(cherry-pick of commit 639814d946)

Bug: 36686974
Test: make ziparchive_tests
Change-Id: I61664515e5afa3e2ba814874eeac847a2aaac319
2017-04-06 17:03:32 -07:00
Tianjie Xu d9fd1863f4 Fix out of bound read in libziparchive
We should check the boundary of central directory before checking its
signature. Swap the order of these two checks.

Bug: 36392138
Test: libziparchive doesn't read the signature after boundary check fails.
Change-Id: Ie89f709bb2d1ccb647116fb7ccb1e23c943e5ab8
(cherry picked from commit 74464a1361)
2017-04-06 18:07:02 +00:00
Adam Lesinski 639814d946 ZipWriter: Do not write DataDescriptor for STORED files
Older implementations of ZIP (Java's ZipInputStream) don't
like a Data Descriptor trailing after an uncompressed entry.

If the FILE is seekable, and the entry is uncompressed, seek
back to the header and write out the crc and sizes.

Bug: 36686974
Test: make ziparchive_tests
Change-Id: I61664515e5afa3e2ba814874eeac847a2aaac319
2017-04-05 10:54:15 -07:00
Adam Lesinski aa1d43230c Merge "libziparchive: fix mac os breakage" am: 390f3b364c am: 7cfe1d69b0
am: cdbadafb4a

Change-Id: Ie327879eaea542911603640e36f0d867e05afd61
2017-03-23 21:12:58 +00:00
Adam Lesinski b02d690336 libziparchive: fix mac os breakage
Add ftruncate64 to utils/Compat.h definitions for mac.

Change-Id: I82cb46927be911e867b606f4f4429a5e1b1987f7
Test: builds on mac
2017-03-23 12:02:09 -07:00
Adam Lesinski e80f0cc5fb Merge "libziparchive: Add ability to backup in ZipWriter" am: 0e19795a62 am: a41597ba0e
am: 797852c435

Change-Id: I10f76ce5bec477f1e27ef7456e20cdedd9e3833f
2017-03-23 18:15:01 +00:00
Adam Lesinski 537713bace libziparchive: Add ability to backup in ZipWriter
Based on the compressed size of a file entry,
the decision needs to be made to instead store the file
uncompressed. This adds support to ZipWriter to backup
its last file entry.

The file is now always truncated when the EOCD is written out,
to account for the case where a file entry is backed-up and the
resulting file written is much smaller, leaving garbage data at
the end of the file.

This change also includes a rename of FileInfo -> FileEntry.
This struct was private (now public), so it shouldn't affect any
clients.

Bug: 35461578
Test: make ziparchive-tests
Change-Id: I23dc584406274ab7b8ce62b3fbc3562ca4c2603e
2017-03-22 16:46:42 -07:00
Mark Salyzyn 271a1a7cda resolve merge conflicts of a45b1d6f3 to master
Test: compile
Bug: 30465923
Change-Id: I648855539df3cfa176c6ecac19b6a562ba6feaf7
2017-01-11 11:41:38 -08:00
Mark Salyzyn 30f991f251 liblog: use log/log.h when utilizing ALOG macros
Test: compile
Bug: 30465923
Change-Id: Id6d76510819ebd88c3f5003d00d73a0dbe85e943
2017-01-11 09:31:15 -08:00
Jaekyun Seok 1faf78e32f Merge "Use shared lib of libutils, libz and libbase." am: 4bf9c82857 am: 70f6d5878e am: 3b00f1cc8f
am: da100377b0

Change-Id: I947f7edc8e3ef3c56502387e2b764a3826244025
2017-01-05 23:39:16 +00:00
Treehugger Robot 4bf9c82857 Merge "Use shared lib of libutils, libz and libbase." 2017-01-05 23:24:21 +00:00
Elliott Hughes cee26e196b Merge "libziparchive: use _FILE_OFFSET_BITS=64" am: 10a7b9bb8b am: d9bd95e1c1 am: 45c7500b9a
am: 3d42d99996

Change-Id: I47c15f203c8dbc5e356e54984b139be9f1d68192
2016-12-30 20:50:41 +00:00
Jaekyun Seok 86e80b9be0 Use shared lib of libutils, libz and libbase.
libutils, libz and libbase are being used as shared lib by many other
modules.
So using their shared lib will reduce total image size.

Size diffs on angler build image are as follows.

libziparchive.so  : 103844 -> 41680 (-62164)
libnativeloader.so:  50824 -> 25104 (-25720)
total             : (-87884)

Test: building succeeded, and the image was tested on angler.
Bug: 33056637
Change-Id: I015afe5b8f4d87d495b706e2e78d60f44a910e87
2016-12-22 07:03:26 +09:00
Christian Poetzsch 3081c5ebfb libziparchive: use _FILE_OFFSET_BITS=64
On 32 bit system those calls may fail if one tries to unpack files which
are bigger than 2GB.

Use large file system support to fix this problem.

Test: unpack a file bigger than 2GB on 32 bit Android systems

Change-Id: Ibd9bd5fc4a2f8dc7df98bd595f4fd1638a4f0d4a
Signed-off-by: Christian Poetzsch <christian.potzsch@imgtec.com>
2016-12-21 12:05:10 +00:00
Dan Willemsen 00ae90ff80 Enable libbase/libutils/libziparchive on host bionic am: ab34b47b35 am: 69b0123549 am: 5bc670c841
am: 55d4de90a6

Change-Id: I11d8cb68123cedddd1450d05bef473037d41d550
2016-11-30 06:22:30 +00:00
Dan Willemsen 69b0123549 Enable libbase/libutils/libziparchive on host bionic
am: ab34b47b35

Change-Id: If8e762eace58a494efede9ef5cb09be99b74e5b8
2016-11-30 06:05:59 +00:00
Dan Willemsen ab34b47b35 Enable libbase/libutils/libziparchive on host bionic
Bug: 31559095
Test: Test linux_bionic compile
Change-Id: Ib42c063532c1cbeb3e1418ab16a1fb54d5902aab
2016-11-29 13:38:53 -08:00
Tao Bao 26a5c41255 Merge "ziparchive: Allow ExtractEntryToFile() to work with block device." am: 787482ecd9 am: af2a8858c4 am: 9b1b0627b9
am: 87cbd3aaf9

Change-Id: I490b2e9098288c98b5397c72d9aad451ad4a8793
2016-11-15 22:34:37 +00:00
Tao Bao af2a8858c4 Merge "ziparchive: Allow ExtractEntryToFile() to work with block device."
am: 787482ecd9

Change-Id: Icdce4207cb64e084e200b6f7f8ae194831d5c4f0
2016-11-15 22:11:21 +00:00
Tao Bao a456c21348 ziparchive: Allow ExtractEntryToFile() to work with block device.
FileWriter::Create() calls ftruncate(2) to truncate the destination
file, which doesn't work with FD that references a block device. It
leads to kIoError when calling ExtractEntryToFile() to extract an entry
to block device FD.

As a result, it fails the package_extract_file() command in OTA updates
(e.g. 'package_extract_file("boot.img",
"/dev/block/platform/soc.0/f9824900.sdhci/by-name/boot")').

This CL skips the call to ftruncate(2) if FD references a block device.

Bug: 32903624
Test: ziparchive-tests works.
Test: Build an OTA updater (m updater) and call package_extract_file().

Change-Id: Ia81116f1a8d7cab802396bdc32c6096b4cb56a3c
2016-11-15 10:19:04 -08:00
Mark Salyzyn b4c9ab960e Merge "system/core: preparation to pull back interfaces from android/log.h" am: 27d2d49f48 am: 8a7297a09f
am: c561af07ae

Change-Id: I7bfb6dd925739345624a1ed16c0c3cb2106ad098
2016-10-20 18:21:18 +00:00
Mark Salyzyn 8a7297a09f Merge "system/core: preparation to pull back interfaces from android/log.h"
am: 27d2d49f48

Change-Id: I604bb1d4cf62636663fa92e3d14a55887dbcae23
2016-10-20 18:03:57 +00:00
Mark Salyzyn cfd5b080af system/core: preparation to pull back interfaces from android/log.h
Point to log/log.h where necessary, define LOG_TAG where necessary.
Accept that private/android_logger.h is suitable replacement for
log/logger.h and android/log.h.

Correct liblog/README

Effectively a cleanup and controlled select revert of
'system/core: drop or replace log/logger.h' and
'system/core: Replace log/log.h with android/log.h'.

Test: compile
Bug: 30465923
Change-Id: Ic2ad157bad6f5efe2c6af293a73bb753300b17a2
2016-10-20 08:11:39 -07:00
Tianjie Xu 769c601b91 resolve merge conflicts of 34486c5 to master
Change-Id: I12fc8eb4760bd6c4ae5c97583eb6c659122c9fa9
2016-10-18 15:46:56 -07:00
Tianjie Xu b2ca85184b Merge "Add functions in recovery/minzip to libziparchive"
am: 4bc51d1ea4

Change-Id: Ica94e8050286bed1c5bc23a33305050ab8da8ec5
2016-10-18 21:57:30 +00:00
Tianjie Xu 18c25920c2 Add functions in recovery/minzip to libziparchive
Add two functions libziparchive that libminzip has. And create
corresponding unit tests.
1. Open a zip archive from a memory mapped region.
2. A new writer that takes a call back function pointer.
(Used by the OTA updater to stream the data.)

Test: Unit tests passed
Bug: 19472796
Change-Id: I2b2daec71174afe221030357e39bff5faea51e72
2016-10-15 23:16:34 -07:00
Tianjie Xu bcc4431f24 Check filename memory bound when parsing ziparchive
Add a check to ensure the filename boundary doesn't exceed the mapped
memory region. Also add the corresponding unit test.

Bug: 28802225
Test: New unit test passes.
Change-Id: Ibf543a7da3d7898952e9eb332c84cdfc67cf5aa4
2016-10-10 14:26:15 -07:00