Commit Graph

48 Commits

Author SHA1 Message Date
Alex Deymo b0c395955d Remove "_host" and "_static" suffix from libsparse definition.
This now combines all the "libsparse" libraries into the same soong
target. A minor side-effect of this change is that the libsparse
static library depends on the libz shared library instead of the libz
static library. This minor change has no effect since targets using
the static libsparse library need to explicitly include either the
static libz or the shared one.

Bug: 34220783
Change-Id: I8f41586cf4c3336791cfa57ab4f5ae59a76d7ffa
2017-01-11 19:15:11 -08:00
Adrien Schildknecht a26a6bd6f3 libsparse: add a function to retrieve the data blocks
Test: m libsparse

Change-Id: I04bd3912bb4364e591b064ec2aab782cf02f6bd7
2016-11-30 19:16:28 -08:00
Jeremy Compostella fca594c2de libsparse: output_file.c, fix large data chunk issue
CHUNK_HEADER structure type allows DATA chunk to be up to MAX_UINT
large.

The write() callback LEN parameter should be unsigned int.

Also:
- write() callback should continue to write data if less than expected
  data have been written.
- gzerror() returns LEN on success, 0 otherwise

Test: manual
Change-Id: Id46d664b84c1a506f419524fe28055f784c2ae7a
Signed-off-by: Jeremy Compostella <jeremy.compostella@intel.com>
2016-10-06 09:53:42 -07:00
Elliott Hughes 34a4f0b8c7 Test for _WIN32 in instead of USE_MINGW.
Bug: http://b/23392815
Test: builds
Change-Id: I7fce8de1f0e2314170cce456cd49346dcee9d134
2016-10-05 09:37:44 -07:00
Dan Willemsen 45f05240c2 Convert more Android.mk files to Android.bp
These modules have their dependencies satisfied, and aren't doing
anything strange.

Change-Id: I72039a15256cbd5e5eee0d79a15d66d74a6c087d
2016-07-13 17:41:45 -07:00
Chih-Hung Hsieh eabd5108b3 Fix misc-macro-parentheses warnings in libion, libsparse, libmem*
Bug: 28705665
Change-Id: I3dd5c086787f5e48ab100a71a42109ea0e417bf9
2016-05-18 15:41:16 -07:00
Colin Cross 1eb743ba8b Fix static analyzer issues caught by clang
system/core/libsparse/sparse_read.c:260:51: warning: Dereference of null pointer (loaded from variable 'crc_ptr')
                        ret = process_crc32_chunk(fd, chunk_data_size, *crc_ptr);
system/core/libsparse/sparse_read.c:404:9: warning: Potential leak of memory pointed to by 'buf'
        return 0;

Fixes leak in sparse_file_read_normal, and null pointer dereference
crash if an image with an obsolete CRC chunk was read with CRC checking
disabled.

Bug: 26904425

Change-Id: Ibc72cd37602929ae2c248bea1cdd1d22ea03baaf
2016-02-01 11:19:26 -08:00
Colin Cross b5619d9630 Turn off CRC checking for append2simg
make_ext4fs doesn't write out a CRC chunk, and append2simg doesn't
either, but append2simg was still performing a CRC on every input
block.  Cuts append2simg time in half.

Change-Id: I678f807abbb741042461ed68a0f61b406d3665fb
2015-12-17 17:28:44 -08:00
caozhiyuan f21f0f16e7 Fix parameter to lseek
Change-Id: I9c497a3ebc24ca1db92d0b9d3d1e9ac49dc40f75
2015-10-20 14:15:31 -07:00
Daniel Micay af090a6ea8 fix misuse of formatting specifiers
The Clang/GCC formatting warning triggers for usage of %lld with off_t
on 64-bit because it's defined as a long int, not a long long int. It
isn't important, but it's technically undefined.

This fix is in anticipation of adding __attribute__((format(...))) to
many functions where it is currently missing.

Change-Id: I2bf33e6563a2892d2f54d7c582cbdeadf867e84f
2015-10-13 16:22:33 -04:00
Dan Willemsen 87a419c8b1 Remove USE_MINGW/CYGWIN; Whitelist windows modules
CYGWIN is not supported, USE_MINGW and HOST_OS==windows are being
replaced with LOCAL_..._windows variables.

Bug: 23566667
Change-Id: I3e4a1e4097dc994cf5abdce6939e83a91758fd75
2015-09-02 17:10:35 -07:00
lei wang wang c227a1d855 libsparse: use strcmp and validate last_used pointer
This patch is used to fix two Bugs in backed_block.c

First, fix wrong comparing string way: we should use strcmp rather than
just compare their address. Second, fix using illegal memory risk in
bbl->last_used pointer. When entering queue_bb function,
bbl->last_used = new_bb, but in the following code if
queue_bb(xx, bb, new_bb) return ok, the space of new_bb is released. So
next time, if you use bbl->last_used pointer, may cause segment fault !

Change-Id: I6abb505f9b903b697448639fc64fb7518df5cca1
2015-08-21 14:23:31 +00:00
Tao Bao a29c0127fa libsparse: Wrap sparse_crc32.h to be C++ compatible.
Change-Id: Ic6b1721859937984619a262c459f1f734d81c149
2015-07-11 12:48:42 -07:00
Eric Miao 4cc3978276 libsparse: fix crc chunk parsing in simg_dump.py
Change-Id: Ia5b7cedd88809b0e421a056704d25afa190470b8
2015-04-30 16:36:17 +00:00
Jeremy Compostella cfd3a03d3d libsparse: move_chunks_up_to_len() does not account skip chunks
I caught the fastboot host command sending more data than the fastboot
device can accept.  Fastboot host command was sending 36 surplus bytes
because of 3 skip chunks that were not taken into account in
move_chunks_up_to_len() algorithm.

Change-Id: I39a4a033c9b15893bd70e553f17116735ee4a48e
2015-04-08 10:10:17 +02:00
Mohamad Ayyash 80cc1f6864 Add verbose param to sparse_file_import_auto
Change-Id: I0c5607f7aa5e964abc2031bbe71ff5c6e6ef56cc
Signed-off-by: Mohamad Ayyash <mkayyash@google.com>
2015-03-31 12:09:37 -07:00
Jeremy Compostella 9f0d6bdd66 fastboot/sparse: propagate error code
sparse_file_write_block calls functions that might failed.  This patch
makes sparse_file_write_block catch the error code and propagate it.

Without this patch, fastboot crashes on a segmentation fault if
usb_write() fail during a sparse file image flashing.

Change-Id: If9c0e8dfac8fa380628a2504e13abe8cf7d62726
Signed-off-by: Jeremy Compostella <jeremy.compostella@intel.com>
2015-03-01 11:54:02 +01:00
Trevor Drake a840300bab Cleanup libsparse Android.mk
Removed unnecessary external/zlib reference from LOCAL_C_INCLUDES

Change-Id: Ib61da00dc1e0eac89a68f244881b12e2a4e92e64
2014-12-12 04:19:27 +00:00
Mark Salyzyn 51067914d1 am d4d09637: am db32e788: am a2b03c4a: Merge "libsparse: turn on -Werror"
* commit 'd4d096370154492433f02df56a7508eb99f564e2':
  libsparse: turn on -Werror
2014-05-08 21:43:07 +00:00
Mark Salyzyn 8116c8c37a libsparse: turn on -Werror
- Deal with some -Wunused issues

Change-Id: I9667d74c0dfbe55c625e65b9d44d7c6f861c455d
2014-05-07 16:56:22 -07:00
Colin Cross 0e3f47e482 append2simg: write to temporary file
append2simg causes libsparse to write mmapped data from a file
back to that same file.  On btrfs, this sometimes causes a page
of zeroes to be written instead of the file data.  Work around
the issue by writing the output to a temporary file and then
renaming it over the original file.

Change-Id: Ia194b6ba0ddb8548747b63292b523756f544706a
2014-04-25 15:19:18 -07:00
Colin Cross b0cb721fb3 am 37d39eba: am 0cb82da2: am 958235bd: Merge "libsparse: allow including from C++"
* commit '37d39eba0b3da124be747dd6a014332817135264':
  libsparse: allow including from C++
2014-04-18 21:33:47 +00:00
Colin Cross 099824cce3 libsparse: allow including from C++
Change-Id: I3788fd07e2b52430a410f85fb79dc886c6a07fea
2014-04-18 14:22:22 -07:00
Mark Salyzyn 470185211b am d9f1702a: am 2b6791d0: am 0577a067: Merge "libsparse: deal with build warning messages"
* commit 'd9f1702a10541b59cfc9582762360d3347712e09':
  libsparse: deal with build warning messages
2014-02-28 17:10:02 +00:00
Mark Salyzyn 031a748b2a libsparse: deal with build warning messages
- mostly unused parameters, added defs.h to hold __unused macro
- read_all prototype not included, added output_file.h to acquire

Change-Id: Ib7ee00614acab53a6bd6a64f7e26a8c13dbe6297
2014-02-28 08:08:01 -08:00
Elliott Hughes b70375abc2 resolved conflicts for merge of ec9e5891 to master
Change-Id: I704e3159d529e436e060cb66c1739ef8e7b612d4
2014-01-16 14:42:35 -08:00
Elliott Hughes ccecf14254 system/core 64-bit cleanup.
This cleans up most of the size-related problems in system/core.
There are still a few changes needed for a clean 64-bit build,
but they look like they might require changes to things like the
fastboot protocol.

Change-Id: I1560425a289fa158e13e2e3173cc3e71976f92c0
2014-01-16 12:54:18 -08:00
Elliott Hughes 288ebfda3b am 9259663c: am f17f5591: am 36d44740: Merge "Fix a bunch of small system/core bugs."
* commit '9259663cdb8f15226844e3ecfc6e3aa53614ad03':
  Fix a bunch of small system/core bugs.
2013-10-29 18:36:16 -07:00
Elliott Hughes 14e28d39f7 Fix a bunch of small system/core bugs.
Missing frees in:
  adb/file_sync_client.c
  fastboot/fastboot.c
  libsparse/output_file.c

Missing closedirs in:
  adb/file_sync_service.c
  cpio/mkbootfs.c
  libcutils/dir_hash.c

Potential buffer overrun in:
  gpttool/gpttool.c

Incorrect NULL check in:
  libsparse/backed_block.c

Bug: https://code.google.com/p/android/issues/detail?id=61564
Change-Id: If97838a9e73a77aef7f416c31c237ce1fca4ce21
2013-10-29 14:12:46 -07:00
Elliott Hughes 60549075e8 am 4d5d633b: am 20eb7ac2: am 3c8bdef0: Merge "libsparse: Use BIONIC implementation of mmap64()"
* commit '4d5d633bf862dbb0a960019e80fadc40c93177f0':
  libsparse: Use BIONIC implementation of mmap64()
2013-09-20 09:58:54 -07:00
Daniel Leung fccfa414c9 libsparse: Use BIONIC implementation of mmap64()
Change-Id: I63692629d3a62b7aee21202b67241904b1cc9789
Signed-off-by: Daniel Leung <daniel.leung@intel.com>
2013-09-18 14:42:28 -07:00
Geremy Condra de807f2ed8 Add append2simg.
Append2simg is a small utility designed to glue data to the end
of a sparse image. Beware that it doesn't do anything clever with
that data.

Change-Id: I33af330cb4f77e0842375797a35aebbb6af60163
2013-08-06 22:16:07 -07:00
Hong-Mei Li 83a6d36d42 libsparse: Fix null pointer issue
Fix the issue that "out" pointer is dereferenced without checking.

Change-Id: I76a57ad7309841218231346f4807ef2be8628737
Signed-off-by: Hong-Mei Li <a21834@motorola.com>
2013-04-01 11:22:50 +08:00
Joe Onorato 5069b01e21 Support for multiple modules with the same name is going away.
Change-Id: I5006b467978071f9107babe532efa71d05061580
2012-08-15 19:52:52 -07:00
Colin Cross f1ec8ac84a libsparse: fix 32 bit overflow when calculating last chunk
last_block * s->block_size can overflow when writing large filesystems,
cast to 64 bits before multiplying.

Change-Id: I3e54097852ce7d0fd271eab53d65e666284898e4
2012-07-23 17:11:29 -07:00
Colin Cross b43828b247 libsparse: rename symbols that conflict with libext4_utils
Until ext4_utils switches to using libsparse, libext4_utils defines some
of the same symbols as libsparse.  Fastboot links statically against
both of them, and there is no easy way to make the symbols hidden, so
just rename them in libsparse.

Change-Id: Idc2cfe20efe3c3a7fb8233f453a89bbbeb0dcc8b
2012-07-09 22:09:37 -07:00
Colin Cross 317a09e2d4 libsparse: add sparse_file_len
Add sparse_file_len, which will compute the size of data that would
be produced if sparse_file_write was called.  Useful combined with
sparse_file_callback.

Change-Id: I1a156d1071760f5559483954a5c62ffc20298703
2012-07-09 22:09:37 -07:00
Colin Cross bdc6d39ed6 libsparse: add function to resparse a file and a utility to use it
Add sparse_file_repsarse, which splits chunks in an existing sparse
file such that the maximum size of a chunk, plus a header and footer,
is smaller than the given size.  This will allow multiple smaller
sparse files to result in the same data as a large sparse file.

Change-Id: I177abdb958a23d5afd394ff265c5b0c6a3ff22fa
2012-07-09 22:09:37 -07:00
Colin Cross 1e17b313a6 libsparse: add callback output file type
Add a new output file subclass that will call a callback for
each block as it is written.  Will be used to measure the space
used by each sparse block to allow resparsing files.

Also add sparse_file_callback, which will write out a sparse
file by calling the provided write function.

Change-Id: I18707bd9c357b68da319cc07982e93d1c2b2bee2
2012-07-09 22:09:37 -07:00
Colin Cross b4cd267db3 libsparse: pseudo-subclass output_file for normal and gz files
Create two subclasses of output_file that can handle normal
and gzipped files, and refactor open_output_fd.  Will allow
adding support for an output_file type that is not file
backed.

Change-Id: I26744c74d13f205cf17df1ea9caac1eea9c57357
2012-07-09 22:09:37 -07:00
Colin Cross 0c4c47f88d libsparse: add sparse_file read and convert tools to use it
Abstract the logic from simg2img into libsparse, and add logic
for reading a regular image into libsparse.  simg2img then
becomes a simple wrapper around libsparse.

img2simg was not actually making the file sparse, it was using
sparse files to create multiple files that could be pieced back
together.  Replace it with a simple wrapper around libsparse.
Its functionality will be replaced by an simg2simg that can
resparse a file into smaller chunks.

Change-Id: I266f70e1c750454183ce46c71a7bb66bbb033a26
2012-07-09 22:09:37 -07:00
Colin Cross 13a5606593 libsparse: fix windows image writing
Fix write_fd_chunk on windows.  Uses malloc and read instead of mmap.

Change-Id: I75f10db2e04f19e7f3a6ff46b6978d143cb5254e
2012-07-09 22:09:37 -07:00
Colin Cross be8ddcb35a libsparse: merge adjacent blocks of the same type
When a block is added that is adjacent to another block and of the same
type, merge it.  This will be useful for converting regular images to
sparse images, allowing the reader to add a single block at a time and
letting libsparse optimize into larger blocks as it goes.

Does not support merge two blocks that are backed by a data pointer,
only blocks that are backed by a file for now.

Change-Id: I95aa231714cbe01ac194e868c21385806c0bdb97
2012-07-09 22:09:37 -07:00
Colin Cross a21930b6b0 libsparse: add error reporting functions
Change-Id: I2f21355b6c5339d1d724b4c121ea30d575b2d366
2012-07-09 22:09:37 -07:00
Colin Cross 9e1f17e926 libsparse: add support for including fds
Add sparse_file_add_fd to include all or part of the contents
of an fd in the output file.  Will be useful for re-sparsing files
where fd will point to the input sparse file.

Change-Id: I5d4ab07fb37231e8e9c1912f62a2968c8b0a00ef
2012-07-09 22:09:37 -07:00
Colin Cross b55dceea98 libsparse: cleanups
Move block loops into sparse.c with iterator helpers in backed_block.c.
Simplify chunk writing by moving skip chunk calls from output_file.c to
sparse.c.
Rename variables to be consistent with new naming.
Remove use of u8, u32, u64.

Change-Id: Ic138ad58bef9f96239266ccee12ee83ea285e7eb
2012-07-09 22:09:37 -07:00
Colin Cross 411619e921 libsparse: remove static variables
Removes static variables in backed_block.c to allow multiple
sparse files to be open at the same time.

Change-Id: I012d8a424c6e21a7352408416adb7c72ee8add21
2012-07-09 22:09:36 -07:00
Colin Cross 28fa5bc347 system/core: move libsparse into system/core
This moves an exact copy of libsparse from
system/extras/ext4_utils/libsparse to system/core/libsparse in
preparation for linking tools in system/core against it.

Change-Id: If664e4fcfd6612844ac745589beb1517e7f9fe58
2012-07-09 22:09:36 -07:00