platform_system_core/fs_mgr/libfiemap
Inseob Kim 651db0935d Revert "Completely migrate init first stage to Soong"
Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Update init_first_stage"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert "Add BOARD_BUILD_SYSTEM_ROOT_IMAGE to config vars"

Revert "Add install_in_root to cc_binary"

Revert "Add ramdisk_available to init_first_stage's deps"

Revert submission 15071196-init_first_stage_soong

Reason for revert: fixes b/192248690
Reverted Changes:
I23cf4f975:Add ramdisk_available to init_first_stage's deps
Icd98c7e24:Add ramdisk_available to init_first_stage's deps
If9da9ba16:Add ramdisk_available to init_first_stage's deps
Ibc8668029:Add ramdisk_available to init_first_stage's deps
I3b4b8c475:Add ramdisk_available to init_first_stage's deps
I59cd149e0:Completely migrate init first stage to Soong
I36d789578:Add ramdisk_available to init_first_stage's deps
I2a0daa612:Add BUILD_USES_RECOVERY_AS_BOOT to soong config
Ic76c325ce:Directly create ramdisk dirs in ramdisk image rule...
I4c5374deb:Add BOARD_BUILD_SYSTEM_ROOT_IMAGE to config vars
I8aab5faf3:Add ramdisk_available to init_first_stage's deps
I9d5a10661:Add ramdisk_available to init_first_stage's deps
Iaa2edeb4a:Add ramdisk_available to init_first_stage's deps
I7cb582ca0:Update init_first_stage
I06091d15e:Add ramdisk_available to init_first_stage's deps
I8bdb8dda3:Add ramdisk_available to init_first_stage's deps
I7436b8dd1:Add ramdisk_available to init_first_stage's deps
I39693fd86:Add ramdisk_available to init_first_stage's deps
I0a9ba90f0:Add ramdisk_available to init_first_stage's deps
Ib66b4c4ea:Add ramdisk_available to init_first_stage's deps
I31ce63d23:Add ramdisk_available to init_first_stage's deps
Icb580f97c:Add ramdisk_available to init_first_stage's deps
I044a075b7:Add ramdisk_available to init_first_stage's deps
I33164a7e7:Fix ndk and aml arch order
Ib8d92904a:Add ramdisk_available to sysprop_library
Ibc3516453:Add install_in_root to cc_binary

Change-Id: I147777bb1c4a3b818bc0118c6cf44ccfbf7970a0
2021-07-01 06:50:40 +00:00
..
include/libfiemap Merge changes Ibffdd9c9,Ief5a16d1 2021-05-06 17:57:46 +00:00
testdata fs_mgr: Move libfiemap back to fs_mgr from system/gsid. 2019-12-16 20:10:26 -08:00
Android.bp Revert "Completely migrate init first stage to Soong" 2021-07-01 06:50:40 +00:00
README.md fs_mgr: Move libfiemap back to fs_mgr from system/gsid. 2019-12-16 20:10:26 -08:00
binder.cpp libfiemap: Add some dependency injection. 2021-02-26 17:28:17 -08:00
fiemap_status.cpp libfiemap: Create/Open returns FiemapStatus 2020-01-07 13:01:58 -08:00
fiemap_writer.cpp fiemap: support FIEMAP_EXTENT_MERGED 2021-02-16 13:22:13 -08:00
fiemap_writer_test.cpp libfiemap: Ignore userdata requirements in fiemap_writer_test when running a DSU. 2020-04-27 21:46:53 +00:00
image_manager.cpp libfiemap: Add some dependency injection. 2021-02-26 17:28:17 -08:00
image_test.cpp Fix allowlist for unreliable pinning. 2020-08-06 15:02:56 -07:00
metadata.cpp Extra logging for metadata errors 2020-02-04 20:49:59 +00:00
metadata.h libfiemap: Add helpers to remove images from recovery. 2019-12-20 16:41:35 -08:00
passthrough.cpp libfiemap: Add some dependency injection. 2021-02-26 17:28:17 -08:00
split_fiemap_writer.cpp Use more inclusive language for #inclusivefixit 2020-07-23 11:13:37 -07:00
utility.cpp Merge "Fix allowlist for unreliable pinning." 2020-08-07 17:57:13 +00:00
utility.h Fix allowlist for unreliable pinning. 2020-08-06 15:02:56 -07:00

README.md

libfiemap

libfiemap is a library for creating block-devices that are backed by storage in read-write partitions. It exists primary for gsid. Generally, the library works by using libfiemap_writer to allocate large files within filesystem, and then tracks their extents.

There are three main uses for libfiemap:

  • Creating images that will act as block devices. For example, gsid needs to create a system_gsi image to store Dynamic System Updates.
  • Mapping the image as a block device while /data is mounted. This is fairly tricky and is described in more detail below.
  • Mapping the image as a block device during first-stage init. This is simple because it uses the same logic from dynamic partitions.

Image creation is done through SplitFiemap. Depending on the file system, a large image may have to be split into multiple files. On Ext4 the limit is 16GiB and on FAT32 it's 4GiB. Images are saved into /data/gsi/<name>/ where <name> is chosen by the process requesting the image.

At the same time, a file called /metadata/gsi/<name>/lp_metadata is created. This is a super partition header that allows first-stage init to create dynamic partitions from the image files. It also tracks the canonical size of the image, since the file size may be larger due to alignment.

Mapping

It is easy to make block devices out of blocks on /data when it is not mounted, so first-stage init has no issues mapping dynamic partitions from images. After /data is mounted however, there are two problems:

  • /data is encrypted.
  • /dev/block/by-name/data may be marked as in-use.

We break the problem down into three scenarios.

FDE and Metadata Encrypted Devices

When FDE or metadata encryption is used, /data is not mounted from /dev/block/by-name/data. Instead, it is mounted from an intermediate dm-crypt or dm-default-key device. This means the underlying device is not marked in use, and we can create new dm-linear devices on top of it.

On these devices, a block device for an image will consist of a single device-mapper device with a dm-linear table entry for each extent in the backing file.

Unencrypted and FBE-encrypted Devices

When a device is unencrypted, or is encrypted with FBE but not metadata encryption, we instead use a loop device with LOOP_SET_DIRECT_IO enabled. Since /data/gsi has encryption disabled, this means the raw blocks will be unencrypted as well.

Split Images

If an image was too large to store a single file on the underlying filesystem, on an FBE/unencrypted device we will have multiple loop devices. In this case, we create a device-mapper device as well. For each loop device it will have one dm-linear table entry spanning the length of the device.

State Tracking

It's important that we know whether or not an image is currently in-use by a block device. It could be catastrophic to write to a dm-linear device if the underlying blocks are no longer owned by the original file. Thus, when mapping an image, we create a property called gsid.mapped_image.<name> and set it to the path of the block device.

Additionally, we create a /metadata/gsi/<subdir>/<name>.status file. Each line in this file denotes a dependency on either a device-mapper node or a loop device. When deleting a block device, this file is used to release all resources.