Commit Graph

1528 Commits

Author SHA1 Message Date
Yifan Hong 1d99674e5d libsnapshot: Skip initializing snapshot if not created.
If no opeartions is to be executed on a partition, it is
possible that no snapshot is created for it. In that case,
don't crash when initalizing snapshots.

Test: libsnapshot_test
Change-Id: I7bb0b5596aa6e4eca319e9c88f6552d1907a6944
2019-10-01 19:43:33 -07:00
Yifan Hong 55fb297a0c libdm: Fix DmTable::num_sectors
Test: libdm_test
Change-Id: If147df2d5c4f3b3df7ca57200f16a9ad59ae8327
2019-10-01 19:43:33 -07:00
Yifan Hong 9c7903124c Fix fds libdm_test
Failed because of double free of fds.
Test: run it

Change-Id: I25d7d590ca52d57fb14a5483ff8751127f6a48a6
2019-10-01 19:43:33 -07:00
Yifan Hong 20ec0cd6ce Merge "libsnapshot: Remove GetSnapshotSize" 2019-10-01 18:37:48 +00:00
Yifan Hong 0a14d17665 libsnapshot: Remove GetSnapshotSize
A single number is not enough to represent the ranges
that needs to be snapshotted in the target partition. For
prototyping now, just use device_size instead.

Add test in PartitionCowCreatorTest to ensure that this
doesn't regress. Also fix some trivial tests.

In follow-up CLs, snapshot ranges should be represented
with a more complicated structure. See b/141889746.

Bug: 141889746
Test: libsnapshot_test

Change-Id: I1a508c2464abce216ad4049cc2533ffdaa8cd14f
2019-09-30 18:11:23 -07:00
Treehugger Robot a18907da54 Merge "[vts-core] Remove extra folder for vts_core_liblp_test and vts_libdm_test" 2019-09-30 05:34:15 +00:00
Tom Cherry aef269f11b Merge changes from topic "logwrapper-api-update"
* changes:
  Update init/fs_mgr for new logwrapper function
  logwrap: convert to C++, rename function logwrap_fork_execvp().
2019-09-27 15:31:04 +00:00
nelsonli 3beec54bf1 [vts-core] Remove extra folder for vts_core_liblp_test and vts_libdm_test
Remove the extra folder of vts_core for vts_core_liblp_test and
vts_libdm_test since we have a better auto-gen option of auto_gen_config.

Bug: 141684102
Test: 1. m -j vts_core_liblp_test
      2. m -j vts_libdm_test
      3. check out/.../vts_core_liblp_test.config
         and
         out/.../vts_libdm_test.config

Change-Id: Ibcdf96104205b50b3da522364199330ad0190c70
2019-09-27 15:00:12 +08:00
Yifan Hong 3edff134ef Merge "libsnapshot: refactor CreateUpdateSnapshots" 2019-09-27 01:11:22 +00:00
Tom Cherry 3a803eb2d4 Update init/fs_mgr for new logwrapper function
Test: cuttlefish boots, fsck logs seen
Change-Id: Icbc01161b44e097be813f4af648fa671739078b9
2019-09-26 10:29:57 -07:00
Colin Cross 2a62d42704 Merge "Use libcrypto_static instead of libcrypto" 2019-09-26 14:32:33 +00:00
Yifan Hong 98aa26fe69 libsnapshot: refactor CreateUpdateSnapshots
This function is too big. Refactor it a bit.

- Reorder checks / variable initializations a bit.
  They are independent of each other, so they can be
  safely reordered.
- Put snapshot creation and initialization to their own
  functions so that CreateUpdateSnapshots looks shorter.

Test: libsnapshot_test
Change-Id: I135b415d8e046dd91d31326fe7962ae44be4ccf8
2019-09-25 15:20:50 -07:00
Yifan Hong ab09a1764b Merge changes Iab867ded,Id37e34b4
* changes:
  libsnapshot: CreateUpdateSnapshots reuse COW space
  libsnapshot: fix tests
2019-09-25 22:19:50 +00:00
Yifan Hong bc71949dd7 Merge "libsnapshot: Fix mocked property fetcher" 2019-09-25 22:19:35 +00:00
Treehugger Robot 9e8f01101f Merge "liblp: convert more functions to use string_view (#5)" 2019-09-25 20:49:36 +00:00
Yifan Hong 30bdee9910 libsnapshot: CreateUpdateSnapshots reuse COW space
At the second update for Virtual A/B device,
PartitionCowCreator thinks the COW partitions in
the existing super metadata as occupied. In fact, these
partitions aren't used after the merge.

Now, unmap these partitions from the device mapper. If
something bad happens and the previous update has not been
merged yet, the unmap will fail. Then, delete these old
COW partitions from the device.

Test: Virtual A/B update twice
Bug: 135752105
Change-Id: Iab867ded19755089e6e0800e553a10fbcddbb931
2019-09-25 12:13:17 -07:00
Yifan Hong 3396e9327a libsnapshot: fix tests
- Add TestDeviceInfo(string, string) to avoid holding
the 'rebooted' TestDeviceInfo variable after ownership is
transferred

- OTA client calls BeginUpdate after unmapping partitions.

Test: run libsnapshot_test
Change-Id: Id37e34b421d728eb320c0e0906abef3b61897e7c
2019-09-25 12:13:17 -07:00
Yifan Hong 30adb65103 libsnapshot: Fix mocked property fetcher
Add a new SnapshotTestPropertyFetcher that sets up
good defaults for all test cases in libsnapshot_test.

Test: run libsnapshot_test
Change-Id: I6e0b31872d604bbd5f15b64ab7a8ea56d7c67405
2019-09-25 12:05:53 -07:00
Treehugger Robot b4a65cf1ec Merge "[vts-core] Add vts_core_liblp_test to vts-core" 2019-09-25 05:54:31 +00:00
Treehugger Robot 5781a185e1 Merge "[vts-core] Add libdm_test in vts-core" 2019-09-25 03:00:50 +00:00
Yifan Hong a1c1eb604d liblp: convert more functions to use string_view (#5)
Test: builds

Change-Id: I59ef370489b36410e3eaf90dc2623efaeefd613b
2019-09-24 19:34:54 -07:00
Treehugger Robot 8c055fbe3e Merge "snapshotctl: merge when boot" 2019-09-24 20:13:52 +00:00
Yifan Hong 71667636fb Merge "libsnapshot: add snapshotctl" 2019-09-24 18:06:53 +00:00
nelsonli 5642e23949 [vts-core] Add libdm_test in vts-core
Add libdm_test to vts-core suite and skip the test if the
first-api-level < 29.

Bug: 132702215
Test: atest vts_libdm_test
Change-Id: Ic805a4c08e5bf9f6c69019d4ea01825ae888e1fe
2019-09-24 11:34:59 +08:00
Yifan Hong 8a679e673f snapshotctl: merge when boot
Test: reboot after OTA on virtual A/B device, see merge
Bug: 135752105

Change-Id: I92eb8d0961635008366c336ead40e3038173257f
2019-09-23 18:50:12 -07:00
Yifan Hong 2feb47a29b libsnapshot: add snapshotctl
... which handles merge when boot completed. It also
dumps debug information when requested.

Bug: 135752105
Test: adb shell su 0 snapshotctl dump
Test: call snapshotctl merge at different stage of OTA:
  - before OTA (exit normally)
  - during OTA is applied (exit with error)
  - after OTA is applied but before reboot (exit with error)
  - after reboot (finish merge and exit normally)
  - Manually stop its execution during merge and run again
      (finish merge and exit normally)

Change-Id: Idcc6aea8d7bbeb9a1a288c966b8f5e14b3f6a3e7
2019-09-23 18:50:12 -07:00
Howard Chen e6d71689bb Merge "Mount multiple DSU partitions when present." 2019-09-21 02:48:37 +00:00
Yifan Hong e71efc3dc4 Merge changes from topic "libsnapshot_api_update_engine"
* changes:
  libsnapshot: Cancel/merge existing update before begin
  libsnapshot: operator<< for UpdateState
2019-09-20 20:56:50 +00:00
nelsonli 2e9edcb042 [vts-core] Add vts_core_liblp_test to vts-core
Add vts_core_liblp_test to vts-core suite and skip the test if the
first-api-level < 29.

Bug: 132702215
Test: atest vts_core_liblp_test
Change-Id: I09ea8ff923b5300bae513146f6d07dc029d36661
2019-09-20 16:06:18 +08:00
Howard Chen 1b094939e5 Mount multiple DSU partitions when present.
There might be partitions like product or system_ext in addition
to the system partition. Those partitions can contain dependencies
required by the system so we need to make the init to recognize
these partitions and mount them accordingly.

Bug: 140092208
Test: gsi_tool install & reboot
Change-Id: Ie5ef063292c3656e79cce9f887f23a8faa1ba2be
2019-09-20 14:10:52 +08:00
Treehugger Robot da6525c2db Merge "Construct the super_vbmeta image" 2019-09-19 10:21:53 +00:00
Kaiwen Szu 6dd098cb1e Construct the super_vbmeta image
This commit constructs the super_vbmeta image to eliminate the
need of adding /vbmeta_system and/or /vbmeta_vendor when AVB
chain partition is used with Android Dynamic Partition.

See BOARD_AVB_VBMETA_SYSTEM under the link:
https://android.googlesource.com/platform/external/avb/#build-system-integration

The structure of super_vbmeta :

|     VBMeta Table    | (fixed-length 2KiB)
| Backup VBMeta Table | (fixed-length 2KiB)
|     VBMeta Images   | (fixed-length 64KiB each)

The structure of VBMeta Table :

| Super VBMeta Header | (fixed-length 128B)
|  VBMeta Descriptors | (variable-length)

The VBMeta Table records the slot number of each
vbmeta image within the /super_vbmeta partition.

Bug: 137054296
Test: m libvbmeta_test
Test: ./out/host/linux-x86/nativetest/libvbmeta_test/libvbmeta_test
Change-Id: I01aeadd850750ae87d9125484c1b1f570bb84756
2019-09-19 12:05:09 +08:00
Colin Cross f117f34aed Use libcrypto_static instead of libcrypto
Replace libcrypto with libcrypto_static, which can be protected through
visibility to ensure only modules that don't affect FIPS certification
can use it.

Bug: 141248879
Test: m checkbuild
Change-Id: I5f0b9acfb57f68570f6f58f2395f2bb1bc015365
2019-09-18 11:04:35 -07:00
Yifan Hong 6aa77dd271 libsnapshot: Cancel/merge existing update before begin
Attempt to cancel or merge an existing update before starting
a new one.

Test: libsnapshot_test

Change-Id: Ic534db846f1fe97bd7ca1f176a9494fbc6a0f8d5
2019-09-17 16:58:34 -07:00
Yifan Hong 0b32b199ac libsnapshot: operator<< for UpdateState
Test: pass
Change-Id: Ifc88666d0b7c76f3ea9b7ec662398084d631c4e6
2019-09-17 16:58:34 -07:00
Yifan Hong 184bb8a77f libsnapshot: CreateUpdateSnapshot accepts DeltaArchiveManifest
DeltaArchiveManifest includes full update package metadata.
Now, CreateUpdateSnapshot reads the source
metadata, updates it, and write to target metadata slot as well.

Test: libsnapshot_test
Bug: 140868231

Change-Id: Ia885b336145d02111ecff1aad421cb9b1efd18c2
2019-09-17 12:55:23 -07:00
Yifan Hong d22459dff3 libsnapshot: Add snapshot metadata updater.
This class knows how to update super metadata for virtual A/B devices.

Test: libsnapshot_test
Bug: 138816109

Change-Id: I9e375c76814e0dcbb47fc2ea9e4903ba69ccf7f8
2019-09-17 12:55:23 -07:00
Yifan Hong 116d2ff503 Merge "liblp: convert more functions to use string_view (#4)" 2019-09-16 21:56:09 +00:00
Yifan Hong c2ff970b17 liblp: convert more functions to use string_view (#4)
Test: builds
Change-Id: Iad041762beb9eeab8b2174ab48a50677754248b5
2019-09-16 14:50:40 -07:00
Treehugger Robot 17438477cb Merge "libsnapshot: Fix typo in nullptr check" 2019-09-16 04:52:31 +00:00
Greg Kaiser 99ab97836b libsnapshot: Fix typo in nullptr check
We already know 'existing_extent' is non-null (or else we would
have crashed on the previous line).  We fix this to check
'existing_linear_extent' against nullptr.

Test: TreeHugger
Change-Id: I03338bca87d1c2bd0cbea0e1ec31244ba9570a40
2019-09-13 06:58:03 -07:00
Yifan Hong f6fccc466a Move digital_stroage.h to its own library
...so that it can be used by other tests.

Also, clean up the code before exposing it.

Test: libsnapshot_test
Test: liblp_test

Change-Id: I627326f696ea55b7113ff26b313f7dd04e341dc1
2019-09-12 15:50:51 -07:00
Yifan Hong 26d7d95391 libsnapshot: tests for public APIs.
Test: libsnapshot_test

Change-Id: I411ae32e77914845ed4037d7e67620598f8218cf
2019-09-11 18:32:57 -07:00
Yifan Hong 0e13bbade3 fastbootd: skip COW group
Skip importing COW group because they are dynamically
created by OTA clients and will never show up in built
images.
Test: flash

Change-Id: I44e7693cfb4c15e64455b56212a02f9abde88d61
2019-09-11 18:32:57 -07:00
Yifan Hong 65d94596f2 libsnapshot: no overlayfs during virtual a/b ota.
With overlayfs enabled, the scratch partition takes up
super partition, causing retrofit devices to create COW
images.
Test: OTA on retrofit device, no COW images created.

Change-Id: I35a1a2d02c8acf9efc33965c7e015f960bf45296
2019-09-11 18:32:57 -07:00
Yifan Hong 20a23bb41c libsnapshot: Also use empty space in super for COW
The super partition usually has some empty space even after
the target partitions are created, especially for retrofit
Virtual A/B devices. Use that empty space for COW before taking
up userdata space.

- PartitionCowCreator computes free regions in super partition metadata
  and use that space until it is used up. It returns a pair of numbers
  (cow_partition_size, cow_file_size) and let SnapshotManager to create
  the partition / images with proper sizes.
  - A region is considered free iff it is used by NEITHER target NOR
    source partitions
  - The list is in PartitionCowCreator's return value so that
    SnapshotManager can use it as a guide when creating partitions.
  - These partitions are created under the group named "cow".
    - Avoid mapping COW partitions directly during init first stage
      mount. Init only maps them when they are needed by the top-level
      device.
- CreateCowImage no longer zero-fills the first 4 bytes of the image.
  (See below)
- CreateUpdatePartitions: after creating the snapshot, also maps the COW
  devices (via MapCowDevices()) and zero-fills the first 4 bytes of the
  top-level COW device.
- Add a new SnapshotManager::MapCowDevices() function, which maps both
  the COW partition (in super) and the COW image (through
  IImageManager) (if necessary). Then, create a dm-linear target that
  concatenates them (if necessary).
- Add a new SnapshotManager::UnmapCowDevices() functions that does the
  reverse MapCowDevices().
- DeleteSnapshot also unmaps the top-level COW device and COW partition
  before unmapping the COW images (before deleting them).

Test: libsnapshot_test

Change-Id: I0098b7e842ab48b0b4dd2a59142098b098d23d34
2019-09-11 18:32:57 -07:00
Yifan Hong c69029f2a8 libsnapshot: APIs for all partitions
Add CreateCowForUpdate / MapSnapshotDevicesForUpdate
that update_engine and init can call them directly.

Bug: 134536978
Test: libsnapshot_test

Change-Id: If53c48855931db27454fd2893745915c77fd37f8
2019-09-11 18:32:57 -07:00
Yifan Hong c02509b500 fs_mgr: CreateDmTable takes CreateLogicalPartitionParams
It has too many arguments.

Also fixed CreateLogicalPartitionParams::InitDefaults because
it doesn't use the provided partition opener to read metadata
(which breaks tests).

Test: libsnapshot_test
Test: liblp_test

Change-Id: I74cf8d468274f741c6f3743438fe8855b2aef15b
2019-09-11 18:32:57 -07:00
Yifan Hong c42f5d64e4 Merge "libsnapshot: collapse fix sanity checks" 2019-09-10 23:47:15 +00:00
Yifan Hong 24815a49d4 Merge "libsnapshot: MapSnapshot: use device string for inner device" 2019-09-10 23:46:22 +00:00