Commit Graph

1699 Commits

Author SHA1 Message Date
Daniel Rosenberg be27975fff Merge "Support casefolded encryption" 2020-01-24 02:09:55 +00:00
Treehugger Robot f3700b9e42 Merge "libsnapshot: fix null pointer deref" 2020-01-24 01:12:33 +00:00
Treehugger Robot 071c4b37e8 Merge "When encountered an unexpected status, mention it" 2020-01-23 21:30:31 +00:00
Yifan Hong 39de68cb6d libsnapshot: fix null pointer deref
Bug: 148129196
Test: TH
Change-Id: If6a628fe1bcb3006c7a21021162488a807a7f9f7
2020-01-23 12:58:25 -08:00
Daniel Rosenberg 8775ce017d Support casefolded encryption
On ext4, enable casefolding if it is requested, but not currently
enabled.

Test: Enable casefolding on device. Check fs configuration on /data
Bug: 138322712
Change-Id: I3d54ab8bf15f28cf52c5b4344aa3fa254af83d60
2020-01-22 20:33:53 -08:00
Yifan Hong e99ec483b1 Merge "SnapshotManager::WaitForMerge gives more info" 2020-01-22 03:13:16 +00:00
Woody Chow 8db23af93b Merge "Fix unsigned integer overflow of i, which gives SIGABRT" 2020-01-22 01:14:18 +00:00
Treehugger Robot 3ddd4304f5 Merge "Snapshot merge progress percentage" 2020-01-22 00:25:50 +00:00
Yifan Hong b98a2e36ae SnapshotManager::WaitForMerge gives more info
It now returns Ok() if successful, NeedsReboot() if merge
it should be checked again after reboot, and Error() for
other errors.

This wraps UpdateState to help clients interpret the UpdateState value.

Also separate SnapshotManager::Return from FiemapStatus since they are
for different libraries and have (potentially) different set of error codes.

Test: libsnapshot_test
Bug: 138808328
Change-Id: I8c95417c2b0b7b2a362beb12585f861453a79278
2020-01-21 13:34:55 -08:00
Woody Chow 7fb93c168e Fix unsigned integer overflow of i, which gives SIGABRT
Bug: 147640670
Test: None

Change-Id: I157aacbf9ddaa6a91eccd940573cf187b58e8d5e
2020-01-21 02:00:54 +00:00
Yifan Hong b37311d4dd SnapshotManager::Return -> Return
Now that the class becomes more complicated, move it out
of SnapshotManager.

Also, make it independent of FiemapStatus. libsnapshot and
libfiemap has different return classes.

Test: libsnapshot_test
Change-Id: If340959ba60bcd51c36e2b48f4d11c149ead907a
2020-01-17 14:43:27 -08:00
Alessio Balsini fb58f0d45a When encountered an unexpected status, mention it
Instead of just mentioning that something wrong has been found when
parsing UpdateState, also log the value of the erroneous status.

Bug: none
Test: m
Change-Id: I487a115a9347199059a684ef682bec79fa5f26a9
Signed-off-by: Alessio Balsini <balsini@google.com>
2020-01-17 16:56:05 +00:00
Howard Chen 62f35129d4 Merge "Refine the first stage mount for DSU" 2020-01-16 05:33:09 +00:00
Tom Cherry f633b62b63 Merge "Require liblog for libbase in all configurations" 2020-01-15 23:20:07 +00:00
Alessio Balsini 30c681dd37 Snapshot merge progress percentage
Compute and print the progress percentage of the dm-snapshot devices
merging after an update is marked as successful.
All the snapshot devices are queried for their update states and all
their allocated, metadata and total sectors are summed as if the system
consists of a single big snapshot device, as well as the initial
allocated sectors to keep trace of the absolute progress. Then the merge
percentage is computed on the numbers for this fake snapshot device.

Bug: 139088917
Test: manual OTA
Change-Id: Ia9a51742105c90b7fd043f3cbf4249877def7f08
Signed-off-by: Alessio Balsini <balsini@google.com>
2020-01-15 16:32:37 +00:00
Treehugger Robot 5c63dc9322 Merge "Refactor update status management as protobuf" 2020-01-15 14:04:25 +00:00
Nikita Ioffe 07892ea610 Merge "Reduce log spam when unmounting /data" 2020-01-14 10:05:33 +00:00
Yifan Hong b5c0ad7b21 Merge "Merge VtsFiemapWriterTest into fiemap_writer_test" 2020-01-14 00:11:14 +00:00
Yifan Hong d90c32fbe2 Merge VtsFiemapWriterTest into fiemap_writer_test
Delete AndroidTest.xml for VtsFiemapWriterTest and use auto-generated
test config for fiemap_writer_test.

Also add fiemap_writer_test to vts-core and device-tests.

Also require_root for fiemap_writer_test.

Test: atest fiemap_writer_test
Fixes: 144300653
Change-Id: Ifc3808ea56331809b58f0c02de2caac250d78352
2020-01-13 13:53:17 -08:00
Nikita Ioffe b03e0cffd4 Reduce log spam when unmounting /data
We are only interested in unmount that failed.

Test: /system/bin/vdc checkpoint startCheckpoint 1
Test: adb reboot userspace
Bug: 135984674
Bug: 147584366
Bug: 143970043
Change-Id: I9e0b174785fbaf76a95b13457a80765bfa856518
2020-01-13 16:05:07 +00:00
Alessio Balsini 824b77d24b Refactor update status management as protobuf
Convert UpdateState and introduce SnapshotUpdateStatus to protobuf to
simplify its access.
In addition, SnapshotUpdateStatus also stores the sum of all the
snapshot sector information. This additional data is used to improve the
merge progress progress.

Bug: 139088917
Test: manual OTA
Change-Id: Ic777d50244c1afa1cdd75fe9b2ffc6dd9ba19ade
Signed-off-by: Alessio Balsini <balsini@google.com>
2020-01-12 18:22:44 +00:00
Nikita Ioffe cb0c92e66d Add more logging to fs_mgr_remount_userdata_into_checkpoint
Sometimes unmounting userdata fails, this should help in debugging.
Ideally this information should be persisted somewhere so that we can
add it to the bugreport, but so far I don't have solution better than
writing it to a file in /metadata partition.

Test: adb reboot userspace
Bug: 135984674
Bug: 143970043
Change-Id: I92489600bf31f7f47b8b87de7cd4a882be21910e
2020-01-10 17:41:54 +00:00
Yifan Hong 84ef7f65e8 Merge "Add balsini@ as libsnapshot owner" 2020-01-10 10:28:42 +00:00
Tom Cherry 992163021d Require liblog for libbase in all configurations
liblog will soon be required for all of libbase's logging.  This
change proactively requires liblog in all configurations instead of
just Android.

Bug: 119867234
Test: build
Change-Id: I696162fbebc78d4ef23c6032412101ac51d397a4
2020-01-09 10:43:26 -08:00
Alessio Balsini b0b9c714d3 Add balsini@ as libsnapshot owner
Add balsini@ as libsnapshot owner to cover EMEA timezone.

Bug: n/a
Test: n/a
Change-Id: I2d5c900c62f202db2432df87f7847b0133872443
Signed-off-by: Alessio Balsini <balsini@google.com>
2020-01-09 10:11:45 +00:00
Howard Chen a841bd48f8 Refine the first stage mount for DSU
* Open the ImageManager according to the saved DSU slot.
* Load the saved install_dir when it presents. This is required
  when installing to an external storage and the current code
  seems to break it.
* Remove an unnecessary C++ using.

Test: gsi_tool install
Bug: 144247097

Change-Id: I5b7be25787f235518a25abacefa612c33861f6ef
2020-01-09 07:47:35 +00:00
Yifan Hong cd232f9734 SnapshotManager expose no space error
When there is not enough space on /userdata, CreateUpdateSnapshot
returns SnapshotManager::Return with ErrorCode::NO_SPACE.

Test: libsnapshot_test
Bug: 138808058

Change-Id: If2effe63f6a4324eff8d05d4db4ce98be8190262
2020-01-07 14:20:44 -08:00
Yifan Hong e4b44fc501 ImageManager returns FiemapStatus.
IImageManager::CreateBackingImage and ZeroFillNewImage now returns
FiemapStatus, which is an error code and provides more information
about the reason of error.

In particular, the error code is NO_SPACE if disk space is full during
allocation / writing.

Bug: 138808058
Test: libsnapshot_test

Change-Id: I500a3d9f55a2d7e60438b5b4ae70f8b5fed162fa
2020-01-07 13:01:58 -08:00
Yifan Hong 6170c7a02f libfiemap: Create/Open returns FiemapStatus
Add variants to SplitFiemap::Create and FiemapWriter::Open that
returns a FiemapStatus and take a unique_ptr<T>* out parameter to
get more information on the reason of the failure.

Test: fiemap_writer_test
Test: fiemap_image_test
Bug: 138808058
Change-Id: Iea77ac9810f98c2202e038b7aede15dd2d137b2b
2020-01-07 13:01:58 -08:00
Treehugger Robot 7cd37ba4dc Merge "Helper function to mount snapshot devices in recovery" 2020-01-07 20:26:43 +00:00
Bookatz 685e19dc97 CtsFsMgrTestCases secondary_user
Marks this CTS test module as supporting secondary_user, meaning that
the tests are eligible to be run from a regular Android user other
than the system user (i.e. other than user 0).

Note that 'user' here refers to the concept of users in Android
Multiuser; it is NOT the same as a uid.

Bug: 141773058
Test: module passed with secondary_user params
 (--enable-optional-parameterization --module-parameter secondary_user)

Change-Id: I1829a4297a49cd38b4b8a2558c92941acab8ab9a
2020-01-06 09:49:17 -08:00
David Anderson 81fbcbdbd4 Merge "overlayfs: Refactor how the scratch device is deduced, again." 2019-12-31 00:24:35 +00:00
Yifan Hong 0ede9b43e5 libfiemap: delete unuseful check
Just let fallocate fails. It also doesn't check for the delta between
the old file and the new file.

Test: unit tests
Change-Id: I05e12b097a973d9fe7fe696cc472bd7ec2d180c7
2019-12-27 15:36:41 -08:00
David Anderson 4e5b505751 Merge "libfiemap: Add a MapAllImages() helper." 2019-12-27 21:13:51 +00:00
David Anderson 31a35038a0 Merge "libfiemap: Add a way to get the block device path of a mapped image." 2019-12-27 21:13:33 +00:00
David Anderson 8edafa2393 Merge "libfiemap: Add helpers to remove images from recovery." 2019-12-27 20:28:48 +00:00
David Anderson 69def12f08 overlayfs: Refactor how the scratch device is deduced, again.
The previous refactoring did not uncover the full breadth of issues that
arise when trying to use /data for adb remount. In fact, there are a few
distinct use cases for the scratch device, and one function cannot
sensibly provide them all.

(1) First-stage init needs to know if there are dependent devices. This
    would be userdata, super_<other>, or system_<other>. This knowledge
    is dependent on the state in /metadata, fstab, and the kernel
    command-line.

(2) First-stage init and fastbootd need to know where the scratch
    partition is. If it's not in super_<other> or system_<other>, and
    there is no indicator on /metadata, then it might be in a dynamic
    partition.

(3) "adb remount" needs to find a place to put scratch, which
    effectively amounts to the first writable space it can find.
    However, for Virtual A/B, devices, scratch wants to be stored in
    /data, which requires more complex checks and binder calls.

Trying to encapsulate all of this into one function is too difficult, so
instead, this patch breaks GetScratchStrategy into separate functions:
one to return a physical location if a candidate exists, and another to
deduce the "boot" scratch device.

"adb remount" no longer calls GetScratchDevice, since it only needs to
know whether or not a physical candidate was possible.

fs_mgr_overlayfs_teardown calls GetBootScratchDevice, but now only
attempts to make a dynamic "scratch" partition if one definitely exists.
This makes the functionality clearer and reduces fastbootd uart spam.

Bug: 134949511
Test: adb_remount_test.sh on coral
      adb_remount_test.sh on crosshatch
      adb_remount_test.sh on walleye
Change-Id: I5f6a3677bc6adcaaf54c8ab3594643e4f285c04e
2019-12-25 22:32:31 -08:00
David Anderson d5745cce70 libfiemap: Add a MapAllImages() helper.
This adds a helper for first-stage init to easily map partitions backed
by /data. This can be used for the scratch partition as well as DSU
partitions.

Bug: 134949511
Test: fiemap_image_test
Change-Id: I46246b41ce19442d1476b9959e34df0e1bff58c3
2019-12-25 22:32:31 -08:00
David Anderson e1c879c816 Merge "fs_mgr: Move libfiemap into libfs_mgr, and introduce libfs_mgr_binder." 2019-12-24 04:37:05 +00:00
Treehugger Robot 5821125694 Merge "fs_mgr: Fix return in error case" 2019-12-21 06:59:02 +00:00
Greg Kaiser d8fdf60367 fs_mgr: Fix return in error case
We were doing "return -1" in an error case from a function that
returns bool.  Since it's non-zero, this gets implicitly cast to
'true', which almost certainly isn't what we wanted here.

Test: TreeHugger
Change-Id: I76c188c22d907fc425cf4d5b0b1575f640a78ee3
2019-12-20 19:39:16 -08:00
Treehugger Robot 1451e0f14c Merge "libsnapshot: no underscores in test names" 2019-12-21 00:54:25 +00:00
David Anderson 734047a231 libfiemap: Add a way to get the block device path of a mapped image.
This is needed for adb remount, to find the scratch device if it is
already mapped. Note that on devices without metadata encryption, this
cannot be done by querying device-mapper, since scratch will be a loop
device.

Bug: 134949511
Test: fiemap_image_test
Change-Id: Ia25d279c6f8a4838be32a8c01aefc67b5ec1e002
2019-12-20 16:41:35 -08:00
David Anderson f41c7bbb96 libfiemap: Add helpers to remove images from recovery.
ImageManager can map images in recovery, but not delete them, because
/data is not mounted. libsnapshot handles this by storing extra state
files, but this is complex to manage and inconvenient for
fs_mgr_overlayfs.

Instead, this patch introduces two new calls:

 - DisableImage(), which indicates the image should not be used. This is
   implemented by adding a new DISABLED attribute to
   LpPartitionMetadata. CreateLogicalPartitions ignores this flag, and
   thus recovery/fastbootd can disable the scratch partition and
   communicate that it can be deleted. This cannot be called from binder
   since it is intended for recovery/first-stage init only.

 - RemoveDisabledImages(), which walks the images for a given folder on
   /metadata and deletes any that are disabled. This can be called from
   binder.

Note that there is no metadata version bump for this flag. It's
considered to be included in the flag list for minor version 1, and
currently is not used for the actual super partition.

Bug: 134949511
Test: adb remount, fastboot flash system
Test: fiemap_image_test
Change-Id: Iaeca2d1eddb5637dd9a20202cafd11ae60b4d0e3
2019-12-20 16:41:35 -08:00
David Anderson 43482de3f9 fs_mgr: Move libfiemap into libfs_mgr, and introduce libfs_mgr_binder.
This eliminates libfiemap as a standalone library and instead bundles it
into libfs_mgr. All consumers of libfiemap need libfs_mgr, and having
them tightly coupled reduces linkage complexity.

When to use libfs_mgr: In recovery, first-stage-init, or when libfiemap
or overlayfs is not used.

When to use libfs_mgr_binder: When not in recovery/first-stage init, and
overlayfs or libfiemap is needed. In this case, IImageManager will proxy
over binder to gsid.

Bug: 134949511
Test: builds, fiemap_writer/image_test passes.
Change-Id: I4c1a5698afdbf9b3bf87accec98254c1fb6f4e0e
2019-12-20 16:41:34 -08:00
Nikita Ioffe 3d6a5fcb87 Now for real unmount bind-mounts on top of /data
Approach in
https://android-review.googlesource.com/c/platform/system/core/+/1191457
depended on ordering of entries. Instead fs_mgr now tries to unmount
everything with the same block device as /data until timeout is reached.
This is very similar to what is done in UmountPartitions in reboot.cpp,
but I'm a little bit reluctant to merge them together, since the later
unmount all read-write partitions, while here we are only interested in
/data.

Test: on both coral and blueline:
Test: adb shell /system/bin/vdc checkpoint startCheckpoint 1
Test: adb shell setprop sys.init.userdata_remount.force_umount_f2fs 1
Test: adb reboot userspace
Test: verified that userdata was remounted
Bug: 135984674
Change-Id: I929a058fa2f9639f0a2b73e9ea269cf1e07db889
2019-12-19 21:18:42 +00:00
Yifan Hong be230af10b libsnapshot: no underscores in test names
This is required for gtest.
Test: run it

Change-Id: I9b0d7ee651c283c00035104d4257d8e8addf4db0
2019-12-19 10:45:54 -08:00
David Anderson 43ad6362da Merge "liblp: Add a VIRTUAL_AB_DEVICE flag for LpMetadataHeader." 2019-12-18 15:42:44 +00:00
David Anderson 71b287bfd7 libfiemap: Fix fallocate() usage.
This fixes two bugs in how we use fallocate(). First, using ZERO_RANGE
is a mechanism to allocate holes, not blocks. Instead we should be
passing no flags.

The reason this code worked is because of our explicit writes to the
file, which is intended to defeat lazy block allocation. This is not
necessary for F2FS with proper file pinning support, so we now skip this
when possible.

Together, these fixes improve the speed of "adb remount" when backed
by libfiemap.

Bug: 134949511
Test: adb remount when backed by /data
Change-Id: I746053d183e0c83d56979d9c3fa6d96b40b5e478
2019-12-17 15:04:08 -08:00
Yifan Hong 18c6248ffe Merge changes from topic "wait_for_merge"
* changes:
  libsnapshot: add WaitForMerge
  libsnapshot: SnapshotUpdateTest::AddOperation
  libsnapshot: add GetCurrentSlot
2019-12-17 23:01:59 +00:00