All cases where we want to append are done via label, so we can remove
the far more complicated open mode for append. Since this only leaves
open, we can remove the mode entirely.
Bug: 172026020
Test: cow_api_test
Change-Id: Ibe305e56b434da25952a129764398f606ca58aff
This will be used by payload_generator to estimate the COW size without
using extra storage space.
Bug: 168554689
Test: cow_api_test, ota_from_target_files
Change-Id: I095c809e7d81eff5321b86f4c6bdfb6e9467e84e
* changes:
libsnapshot: Implement MapAllSnapshots and UnmapAllSnapshots.
Start snapuserd if needed as soon as possible during second-stage init.
libsnapshot: Add support for first-to-second-stage transitions of snapuserd.
This changes labels to belong at the end of the set of ops that they
refer to. We only sync after writing a label, or the footer, saving the
cost of syncing after ever op.
Change-Id: Iee9dd69132b8e3321eccfe1e43fa0c072a94d3bd
Bug: 172026020
Test: cow_api_test
A few values in the footer were being set inconsistently. We weren't
verifying them, so it didn't matter. This adds verification and sets
them properly.
Change-Id: I669a03a6e1e87ace31775aba5b67dde9b6e5ecf5
Bug: 168829493
Test: cow_api_test
A few minor issues snuck through code review.
Removed a debug line in cow_api_test to persist file.
Removed unused function declaration for cow writer
Switched PLOG to LOG where no errno exists.
Change-Id: Ibb774d6de518fc2a8746e4b9eefc3655c9973c30
Bug: 168554689
Test: cow_api_test
When Virtual A/B Compression is enabled, the manifest contains the
predicted COW size. Use this instead of the algorithm based on the
kernel COW format.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I545679b4834957ff80a930d91cb44afbadebb66c
This patch introduces the fundamentals needed to support booting off
dm-user. First, a method has been added to start snapuserd in
first-stage init. It simply forks and execs, creates a specially named
first-stage socket, then waits for requests.
Next, a new method has been added to SnapshotManager to perform a
second-stage handoff. This works by first launching a second copy of
snapuserd using init's normal service management functionality. The new
snapuserd runs alongside the original, but has correct privileges and a
correct selinux context. Next, we inspect each COW device, and if its
table uses dm-user, we replace the table with a renamed control
device. The new control device is bound to the new snapuserd.
device-mapper guarantees that such a table swap is safe. It flushes I/O
to the old table and then replaces it with the new table. Once the new
table is in place, the old dm-user control devices are automatically
destroyed. Thus, once all dm-user devices has been transitioned, the
first-stage daemon is idle and can gracefully exit.
This patch does not modify init. A few changes will be needed on top of
this patch:
(1) CreateLogicalAndSnapshotPartitions will need further changes to
start the first-stage daemon and track its pid. Additionally, it will
need to ensure the named socket file is deleted, so there is no further
IPC allowed after partitions are completed.
(2) init will need to propagate the pid to second-stage init so the
process can be killed (or signalled).
(3) first-stage snapuserd will need to gracefully exit once it has no
active handler threads.
(4) second-stage init will need to invoke the transition helper on
SnapshotMaanager, ideally as soon as feasible.
Bug: 168259959
Test: manual test
Change-Id: I54dec2edf85ed95f11ab4518eb3d7dbaf0bdcbfd
When the userdata is mounted, its result will be stored and return.
But the result is not stored when the userdata is mounted with
metadata encryption. Store the result of metadata encryption mount.
Bug: 172180818
Test: run cts-on-gsi -m CtsUserspaceRebootHostSideTestCases
Change-Id: I88b1b4f6a2b1ed81773e18243cb6c46244dc1ba5
Right now fs_mgr_overlayfs_teardown() does slightly different things
when called from the userspace or recovery. This is accomplished by many
runtime checks, runtime assumptions and conditional execution of code.
This makes the control flow of the function very hard to follow, and
assumptions becomes more fragile as the function becomes more complex.
This CL forks fs_mgr_overlayfs_teardown() and removes "recovery" bits
from it. A new entry point TeardownAllOverlayForMountPoint() is added
for recovery.
Recovery (fastbootd) should call TeardownAllOverlayForMountPoint() to
teardown overlays of all sources (cache dir, scratch device or DSU).
While fs_mgr_overlayfs_teardown() should only be called from the
userspace.
Also apply some linter suggestions.
Bug: 165925766
Test: adb-remount-test.sh
Change-Id: I7ff7e3409c910782e1ec207fcd02b967a9762bc1
OverlayFS on Device Mapper on Dynamic Partition / Split Fiemap doesn't
seem to report stable dev_t values of the underlying "wrapped" device.
For example, when not using overlayFS, the dev_t of a file in /system
may be something like 253:8, a dm-verity wrapped device (253 is device
mapper's major id).
When overlayFS is enabled, the /system mount point is mounted with a
overlayFS device wrapping a dm-linear device wrapping the actual storage
device. In this case, stat-ing a file in /system shows the dev_t to be
something like 0:23. Not only is the major id not correct, the minor id
is not stable across reboots, too.
For now, suppress the error and make it a warning so it don't report
false negatives. We may want to turn this check back on if the overlayfs
driver is fixed of this issue.
Bug: 165925766
Test: adb-remount-test.sh
Change-Id: I035caec3e00d179f227850cbc9835fb3aedf88e5
* changes:
libsnapshot: Remove the timeout on client recv().
libsnapshot: Integrate with snapuserd.
snapuserd: Add an API call to wait for device deletion.
This simple tool will dump the COW header and included ops to stdout.
Bug: N/A
Test: mm inspect_cow && inspect_cow <file>
Change-Id: I369c4a21a84c95ffc10670bd9eeb2ceccb2a56d6
This is so update engine can resume from the correct label.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: Ib04e80e8219f954f105d5a85f86efa7bb9097579
Bug: 168554689
Test: vts_libsnapshot_test
Test: full OTA with update_device.py
Test: incremental OTA with update_device.py
Change-Id: I3878abfd767d2e47cf8486bc2c06233da2f1ef08
Enable overlayFS (adb remount) within DSU only if the DM device
scratch_gsi exists. Under DSU mode the backing scratch device of
overlayFS must be scratch_gsi. The scratch_gsi partition must be
created by gsid on the host system and initialized during
first-stage-init. fs_mgr_overlayfs mustn't create any scratch device
for DSU, instead it should just check the existence of the dm node
and initialize the scratch partition (if any).
Bug: 165925766
Test: (In host)
adb shell gsi_tool create-partition --partition system \
--size $(du -b system.img | cut -f1) <system.img
adb shell gsi_tool create-partition --readwirte --partition userdata \
--size $((8 * 1024 * 1024 * 1024))
adb shell gsi_tool create-partition --readwirte --partition scratch \
--size $((200 * 1024 * 1024))
adb reboot
Test: (In DSU guest)
# Ensure next reboot is still DSU
adb shell gsi_tool enable
adb remount -R
# Check the output of "adb shell mount"; "/system", "/vendor" ...
# should be remounted as RW.
Test: adb-remount-test.sh in DSU system
Test: adb-remount-test.sh in normal system
Change-Id: I3267f551313e6b4d4ee63a4f1021040076126e6b
The value of entry.mount_point for data partition is "/data"
Fixes: 5ba5b90cd6 ("fs_mgr: try tune2fs for casefolding on /data only")
Test: got "Can't mount with encoding and encryption" problem reported
by the db845c build with the default 5.4.38 prebuilt kernel
Signed-off-by: Yongqin Liu <yongqin.liu@linaro.org>
Change-Id: I226a2275f5f2ee18503c5a3863ef5a1d2c2ed7be
Two seconds is a bit aggressive - considering this is analagous to a
synchronous binder call, let's drop the timeout entirely.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I2b3f5b33f79575d72b15ed314dbcc0ad20ebd9a8
This integrates libsnapshot with dm-user and snapuserd. Tests progress
significantly further now. Tests involving merging still fail as
snapuserd doesn't support this yet.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I464b683b464fe29a646f0f2823b7f4434a878614
This adds a new message to the daemon protocol, which waits for a device
to be deleted. The caller must ensure that the corresponding control
device is actually going away (eg, the device containing the dm-user
table entry has been deleted). Otherwise, this will hang.
This will allow libsnapshot to safely delete the cow since any
outstanding references will be closed.
This also refactors DmUserHandler so that it's freed (and removed from
the handler list) if its corresponding thread exits of its own accord.
Bug: 168554689
Test: vts_libsnapshot_test
Change-Id: I8e97c543eec84874c88795a493470e992dc476fc
This refactors SnapuserdClient so it retains a connection for its
lifetime. This allows SnapshotManager to ensure the daemon is running
and hold a connection open across all of its operations.
The main impetus of this change is to remove the ambiguity between first
and second-stage sockets. SnapshotManager should only ever connect to
the first-stage socket during first-stage init, or, to initiate the
"transition" step during second-stage init.
The transition steps are roughly:
(1) Start second-stage daemon.
(2) Load new device-mapper tables.
(3) Connect second-stage daemon to new dm-user devices.
(4) Activate the new tables, flushing IO to the first-stage daemon.
(5) Send a signal to the first-stage daemon to exit.
This patch makes it easier to hold these two separate connections.
Bug: 168554689
Test: manual test
Change-Id: I51cb9adecffb19143ed685e0c33456177ec3d81f
This is in preparation for moving to a traditional client/server model
where clients stay connected and the server multiplexes multiple
connections.
Client has been renamed to DmUserClient to differentiate it from local
socket clients.
poll() responsibilities have been moved into SnapuserdServer. In
addition, the server now tracks all open clients and polls them
together with the listen socket.
SnapuserDaemon is now only responsible for signal masking. These two
classes can probably be merged together - I didn't do that here because
the patch was already large.
Bug: 168554689
Test: manual test
Change-Id: Ibc06f6287d49e832a8e25dd936ec07747a1b0555
GetLastLabel returns the last Label that a reader is confident about.
InitializeAppend starts a writer up to append data after the last given
label, assuming all later labels are not relevant data.
Change-Id: I3339d5527bae833d9293cbbc63126136b94bd976
Bug: 168829493
Test: cow_api_test
This switches up the format to alternate ops with data, followed by a
footer containing additional meta information. This allows the file to
be resumed at arbitrary points if writing gets interrupted by power
loss.
Also adds a label op, which allows labeling future ops as connected.
If the footer is missing, Append will treat the last label as possibly
corrupt, and ignore it.
Change-Id: I126e15837d710776f9396e7afc9b0cd595e26b59
Bug: 168829493
Test: cow_api_test
As we change to a more resumable format, flush mostly writes the final
parts of the file that are needed, which would write extra data that is
not needed to continue writing, and would immediately be overwritten.
Additionally, in the next patch we will fsync the file after adding an
op, making the flush built in, and the Finalize name more appropriate.
Bug: 168829493
Test: builds
Change-Id: Iccc6580ac72ff066cfeeb32e3cdaf69c5ba615fc
Don't try to find a scratch partition in the fstab as it is a special
devices initialized by fs_mgr_overlayfs during "adb remount".
Don't hardcode the string "userdata_gsi", instead use the constant
kDsuUserdata defined in libgsi.h.
Bug: 165925766
Test: Build pass.
Test: Install and boot a DSU.
Change-Id: I4eaff9461e87e49704c976ddf2a096642957cf56
Daniel Rosenberg