lpmake should be using the intermediates directory for temporary work
rather than /tmp. Add ability to respect TMPDIR environment as
inherited from TemporaryFile.
Bug: 119313545
Test: manual test
Change-Id: I1a0317538875ee37fb4066602ff7a75e4658d74b
This adds a new MetadataBuilder constructor, NewForUpdate, that can be
used by update_engine to simplify upgrading metadata. It is safe to call
whether or not the device is a retrofit. If the metadata has block
devices assigned to a specific slot, and that slot matches the slot
suffix, it will ensure that an equivalent entry exists for the alternate
slot.
Thus, if the source slot is _a and the target slot is _b, and the
metadata has "system_a" as a block device but not "system_b", this will
automatically add "system_b" as a block device.
Bug: 116802789
Test: liblp_test gtest
Change-Id: Ie89d4dbf4c708b5705e658220227ebf33fcb1930
These broke after recent changes to use IPartitionOpener in more places.
The io_tests must now give block device info to TestPartitionOpener.
Bug: N/A
Test: liblp_test gtest
Change-Id: I0a6505c7223e74507dc13184069fdc34bb6b81e4
On retrofit devices, we need images that can be flashed in the
bootloader for the component "super"partitions. This change rewrites
SparseBuilder so that it generates one sparse image per block device,
and now handles partitions that span across multiple block devices.
A new API function has been added to write the set of sparse images to
an output folder.
Bug: 118887112
Test: manual test, flash split images
Change-Id: Iff56efdcb7bdfd5bc8dd7ff44e8234e091ac2346
On retrofit devices, an OTA package or super_empty.img won't know which
slot it applies to. This is not an issue on devices shipping with
dynamic partitions, since they ship on the "a" slot.
To work around this, partitions and block devices can be flagged as
"auto-slot-suffixed". This indicates that ReadMetadata should
automatically append a slot suffix before returning the metadata. This
flag is added by MetadataBuilder when requested, and will be enabled via
lpmake separately.
After ReadMetadata has applied slot suffixes, it takes care to remove
the slot-suffix flag. This prevents the suffix from being applied twice,
if for example the metadata is then imported into a MetadataBuilder.
Bug: 116802789
Test: liblp_test gtest
retrofit device boots
Change-Id: Ic7de06d31253a8d5b8d15c0d936175ca2939f857
This patch adds a new variable, "super-partition-name", to query the
name of the super partition (with a slot suffix if it has one). The
fastboot flashing tool has been updated to query this variable.
Since the super partition name can no longer be determined without
fastbootd, the presence of super_empty.img is used to test for
dynamic partition support rather than the presence of a super partition.
Bug: 116802789
Test: fastboot flashall on retrofit device
Change-Id: If830768eba6de7f31ac3183c64167fae973c77a4
When updating the super partition, attempt to preserve partitions from
the other slot. If any partition can't be preserved, fail and require a
wipe (-w) to proceed. This allows two bootable builds to be flashed to
both slots.
The preserve operation can fail if the metadata is not compatible with
the old partition layout. For example, if the partition references a
group that no longer exists, or a group changed its capacity, or the
metadata's block device list or list contents changed.
Bug: N/A
Test: liblp_test gtest
fastboot flashall --skip-secondary
Change-Id: I53fdd29bc1f0ef132005a93d3cf1cdcd7f2fc05f
This patch adds another uevent-regeneration pass to the first stage
mount. When the super partition spans multiple block devices, we need
/dev/block/by-name symlinks to have been created before we begin mapping
dynamic partitions.
Bug: 116802789
Test: retrofit device boots
Change-Id: I00bb277e1d81385a457c5b4205a95d8fbe365bb2
Although metadata is only stored on the first block device, we ideally
want to validate all the block devices before flashing. This patch does
that and in the process converts ValidateAndSerializeMetadata to use
IPartitionOpener.
Bug: 116802789
Test: manual test
liblp_test gtest
Change-Id: I3f24cfc6f5607309dc3cded0423326c5ba293d26
This patch allows the block device table in LpMetadataHeader to contain
additional partitions. MetadataBuilder can now resize partitions such
that are allocated across block devices as needed, however, it attempts
to minimize this by grouping free regions by device.
Bug: 116802789
Test: liblp_test gtests
device with super partition flashes and boots
Change-Id: I9cf74c8925faf154703eeba2a26546a152efcaa2
This change introduces an IPartitionOpener abstraction. Methods that
interact with live metadata, such as ReadMetadata, UpdatePartitionTable,
and FlashPartitionTable, now require an IPartitionOpener object. Its
purpose is dependency injection: it will make these methods much easier
to test when the super partition spans multiple block devices.
All non-test consumers should be using PartitionOpener, and as such,
some helper methods have been added that automatically create one.
Bug: 116802789
Test: liblp_test gtest
device with super partition boots
Change-Id: I76725a5830ef643c5007c152c00ccaad8085151f
This patch removes the alignment, block device size, and starting sector
fields from LpGeometry into a new LpMetadataBlockDevice struct. The
metadata now contains a table of these structs, and the table will have
exactly one entry representing the super partition.
This refactoring will make it easier to have logical partitions span
multiple physical partitions. When that happens, the table will be
allowed to have more than one entry, and the first entry of the table
will be considered the "root" of the super partition.
Bug: 116802789
Test: liblp_test gtest
device with logical partitions flashes and boots
Change-Id: I97f23beac0363182cb6ae78ba2595860950afcf0
These will help update_engine clear the target slot before applying an OTA.
Bug: 117182932
Test: liblp_test gtest
Change-Id: I6ad370e617f22f2098245a0028a93488f9ac8674
In preparation for supporting multiple block devices, this factors out
the free-list calculation for resizing partitions.
Additionally, it fixes a bug where space in between the first usable
sector and the first extent wasn't added to the free list.
Bug: 116802789
Test: liblp_test gtest
Change-Id: I965760eef0176020e9a5691ce1be2c8b5e0c8bc8
Align the first usable sector to the logical block size, if no other
alignment was specified. This fixes a bunch of warnings during certain
gtests (ones with unaligned metadata sizes). The warnings were coming
from MetadataBuilder::GrowPartition() which expects the first sector
to always be block-aligned.
Bug: 116802789
Test: liblp_test gtest
Change-Id: I8dcf502aa4c2ba0674c5b4dcb77a274f300ff0a3
Traditionally the first 512 bytes of a partition can be interpreted as
an MBR. To prevent any compatibility issues, we explicitly zero the
first 4096 bytes of the super partition (one logical block, on most
systems).
Bug: 116802789
Test: liblp_test gtest
device with super partition flashes and boots
Change-Id: I29688ca75dbb52442f1464e8ab35893678a4f79e
The group_indices variable was intended to facilitate this, but I forgot
to actually use it.
Bug: 116817738
Test: lpmake, lpdump super_empty.img
Change-Id: Ia0da50b56b6c09e277324ec9d7aea6ce48fdc10a
Now that backup metadata is stored at the start of the super partition,
this field is no longer needed. In actuality, it was not needed even
before then: both it and first_logical_sector exist for convenience,
since they can be re-derived at any time given an LpMetadataGeometry.
Bug: 116802789
Test: liblp_test gtest
device with dynamic partitions flashes and boots
Change-Id: I259a443097e689a0a9db7f822bbf1a52d40076dc
There's no reason to do this, since the fields are already in
LpMetadataGeometry. Removing this also simplifies multiple-block-device
support.
Bug: 116802789
Test: liblp_test gtest
Change-Id: Ib244a98fdd9d36c94a2dffd81bef68a1d5644ab9
Previously, metadata backups were stored at the end of the partition to
make them easy to locate. On older devices where the super partition
could span system/vendor partitions, we may want to leave the end of
each partition free to store an AVB footer. To allow this, we now store
geometry and metadata backups near the start of the partition instead.
They are still positioned at a fixed offset.
Bug: 116802789
Test: device boots after flashing new metadata
Change-Id: Ib173f251a4a13e290adcc4ac5cfbeb030eacda30
Note that this moves SparseBuilder into images.h, and splits
ReadLogicalPartitionGeometry into componenet methods for better
testability.
Bug: 116802789
Test: liblp_test gtest
Change-Id: Ib41a46b633c71623c136a10fcc8017e4de20884c
This patch introduces a new "groups" table in the super partition
metadata. Each entry denotes a named partition group with a maximum
size. All partitions now belong to a group, and the total size of
partitions in a group must not exceed its maximum size. This is enforced
by MetadataBuilder.
There is also a "default" group with no size restriction. This is used
for one-off partitions that aren't restricted by updates, for example,
the scratch partition for overlayfs, or partitions created through
fastbootd.
Bug: 116817738
Test: liblp gtest
Change-Id: I7049ffd35d326e41e25d01b1748cb53a584783a7
If a device has logical partitions but does not use a partition called
"super", we will need this to be configurable. Note that the "slot"
argument is currently unused, but will be necessary for update_engine if
there are A and B super partitions.
Bug: 116608795
Test: super partition works
Change-Id: Ieb548a158faeb423f3561230f26d582325efbc6b
MetadataBuilder::AllocatableSpace() represents the total space
available. Adding MetadataBuilder::UsedSpace() to represent the
space used by the existing set of partitions. This will allow a
caller to predict if a partition can be resized (Grown), or to
create a partition to use up the free space (i.e. scratch).
Test: gTest liblp.UseAllDiskSpace
Bug: 109821005
Change-Id: Iae487d869fe18310c76b48c4281f1d6c1ee333cd
Move test to verify we can't read unwritten metadata from
liblp.UpdateAnyMetadataSlot to liblp.CreateFakeDisk.
Test: gTest liblpg_test
Bug: 109821005
Change-Id: I4714c949e2b2d9de6444996a2af2992a821c4a85
This change enables reading metadata images from memory, for situations
where using file descriptors is not practical (such as fastbootd flash).
Bug: 78793464
Test: liblp_test gtest
Change-Id: I9ad08b0ddd4cbb96e87583237a90785e0f4befa4
This is in preparation for "fastboot flash super", where we want to
verify the validity of a super image before writing it. To do so, we
need to parse the image from the download buffer, and it is useful to do
this from memory rather than a file descriptor.
Bug: 78793464
Test: liblp_test gtest
Change-Id: I7fb1ef4fdf2e8f1d93aea38f75626157fcb4bfc1
Callers of ParseMetadata must manually copy geometry to the final
LpMetadata structure, which is error-prone. Instead, force callers to
pass geometry to ParseMetadata to ensure it is always propagated.
Bug: N/A
Test: liblp_test gtest
Change-Id: I5b24b9d94ab1857db600c40bf6d3c9d8aaa47368
Logical partition metadata has "slots" for AB purposes, but when
flashing or updating the partition table via fastboot, there is no
reason not to synchronize all copies of the metadata. It makes the state
of the super partition much more clear. It also makes the super
partition less likely to break on a slot change from the user, for
example if a "_b" partition is created before changing to the "b" slot.
Bug: 78793464
Test: liblp_test gtest
Change-Id: I3c44f0362f21f87d0bfc3a5c3394e26dc3dd38be
When adding extents to partitions, if the previous extent and new extent
are contiguous, merge them together to avoid allocating unnecessary
device-mapper targets.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I80087df9aea8141c5e16f8d4cdb3dd7da02aee8c
This logs when partition tables update, when partitions resize, and when
partitions are unmapped from device mapper.
Bug: N/A
Test: N/A
Change-Id: I1125332c79fccc3ebc556b3b48856901e2503c47
DM_TABLE_LOAD will reject dm-linear entries if their size is not a
multiple of the backing device's logical block size. For example, a
partition of 10GiB+512 bytes will fail to map in device-mapper if the
logical block size is 4096 bytes. To address this, this patch adds a
few changes to liblp:
The block size given to lpmake is now recorded in LpGeometryMetadata.
The block size must be a multiple of the sector size. In addition,
partiton sizes are now aligned to the block size, and the super
partition must have enough free space to allocate at least one block (in
addition to storing metadata).
GrowPartition now has multiple checks that the block-size invariant is not
violated, to ensure that no invalid partition tables will be created.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I484aac1f9b90ebd92dc1c89ce1e09cd89bbb441e
The partition resize algorithm duplicates a lot of logic because it
handles the final free interval separately from other free intervals.
This is unnecessary and makes it harder to change the actual algorithm.
This change makes GrowPartition() treat the final free space region the
same as free gaps in between partitions. It does this by converting the
extent list into a gap list, and then adds a final gap for the remainder
of the free space. The resize function no longer has to treat the end of
the disk separately.
This patch does not change the way partitions are allocated, it is
purely a refactoring.
Bug: 79173901
Test: liblp_test gtest
Change-Id: I4780f20b23fe021eac62de874b061857712c04fe
This change is to assist with implementing the fastbootd "resize-partition"
command. The GrowPartition and ShrinkPartition functions are now
private.
Bug: 78793464
Test: N/A
Change-Id: Ic66a3052359e2558663272ff6e014704206b197e
This change allows partition data to be included in the sparse image for
the super partition. The data is specified as a map from partition name
to partition image file. The image file can either be a normal file or a
sparse image. If it is sparse, it is unpacked into a temporary file and
then re-packed into the super image.
Bug: 79173901
Test: (1) lpmake -S with a partition image flashes to walleye.
(2) The image can be mounted if it contains a filesystem.
(3) simg2img will successfully unpack the image.
Change-Id: I98e9af3bfec9863143c1b95a1712b1d435942542
David Anderson