Commit Graph

68697 Commits

Author SHA1 Message Date
Elliott Hughes af021e87a6 libutils: remove obsolete README.
This dates from when libutils was still part of frameworks, and talks
about stuff that *didn't* move out of frameworks.

Test: treehugger
Change-Id: Icf22eebc417bcb37abdbfdc89e9a3f14894e18c2
2021-05-11 12:01:26 -07:00
Roman Kiryanov 8371ec25ab Merge "Remove the tech debt from the property_service" 2021-05-06 20:53:39 +00:00
Roman Kiryanov 2d1a401b6e Remove the tech debt from the property_service
emulator migrated to bootconfig, we don't use
the kernel command line to pass userspace properties.

Bug: 182291166
Test: boot
Signed-off-by: Roman Kiryanov <rkir@google.com>
Change-Id: Iff2f627aec64b7ba2240444639acedc76a1aa806
2021-05-06 11:55:29 -07:00
David Anderson cedb057e2b Merge changes Ibffdd9c9,Ief5a16d1
* changes:
  libsnapshot: Add a test case for unexpected snapshots during data resets.
  libfiemap: Add some dependency injection.
2021-05-06 17:57:46 +00:00
David Anderson 53dfaceac4 Merge "Add cow_api_test to presubmit." 2021-05-06 17:36:27 +00:00
Treehugger Robot b212a33c58 Merge "Link libcrypto in host unit tests for libavb" 2021-05-06 14:49:35 +00:00
Treehugger Robot 0d712ad3a2 Merge "Revert "init: adding fallback paths for debug ramdisk"" 2021-05-06 07:05:41 +00:00
Yo Chiang d7115b8463 Merge "Rename system/cote/cpio/ -> system/core/mkbootfs/" 2021-05-06 06:43:37 +00:00
David Anderson 7cdbd8e64d Merge "OverlayFS support for fstab" 2021-05-06 01:28:52 +00:00
David Anderson cbf8420b2a Add cow_api_test to presubmit.
These are unit tests for important VAB functionality. They are very
quick to run and should be included in presubmit testing.

Bug: N/A
Test: th green
Change-Id: I02e24c76df365e9fb2d68f904e930dce60b9bdaf
2021-05-05 15:29:35 -07:00
David Anderson a06eac491c Merge "Ensure vts_libsnapshot_test will pass after being interrupted." 2021-05-05 22:22:33 +00:00
David Ng d926aded73 OverlayFS support for fstab
Add overlayfs (lowerdir) mount entry support to fstab.

  overlay  <final dir>  overlay lowerdir=/1:/2

E.g.
  overlay /vendor overlay lowerdir=/odm/vnd_ovl1/1:/odm/vnd_ovl2

Test: Ensure mounting with fstab overlayfs entry
Change-Id: Ib025e203f8ac1836ab62dfa96fb14e8e108f82fb
2021-05-05 09:30:18 -07:00
Martin Stjernholm 7b5e682da1 Merge "Require libjdwp from the ART APEX." 2021-05-05 14:30:50 +00:00
Mohammad Islam a2de531f2c Merge "Add execute permission to /data/apex/decompressed directory" 2021-05-05 12:31:43 +00:00
Yi-Yo Chiang 3987c0663a Rename system/cote/cpio/ -> system/core/mkbootfs/
Test: Presubmit
Change-Id: I1a55ce97817ebb3d7a7e97cb7a41ba61cbe878e3
2021-05-05 19:03:36 +08:00
Bowgo Tsai 26c1b74998 Revert "init: adding fallback paths for debug ramdisk"
This reverts commit 6cb4311f4d.

Reason for revert: no need to keep the compatibility now

Bug: 186485355
Change-Id: Iffdf3abebc17f32e36f3b3fe2b4f9c2592da9653
Test: TreeHugger
2021-05-05 10:31:29 +00:00
David Anderson 1de6c16ad7 Ensure vts_libsnapshot_test will pass after being interrupted.
We try to clean up previous test runs, but this can crash since we
haven't opened fake-super yet. Refactor the harness so we always open
fake-super if it exists. If it does, we'll delete and recreate it after
cleanup. If it doesn't, we'll create it immediately.

It's still possible that cleanup can fail: If interrupted during a merge,
libsnapshot does not allow cleanup until the merge completes. The test
harness doesn't bother handling this case yet.

Bug: 187151854
Test: vts_libsnapshot_test, ctrl+c, run again
Change-Id: I58a7094336a391cff493a31e4f80d8c8b1b166f8
2021-05-04 22:21:08 -07:00
David Anderson 6939583deb Merge "init: Add tests and document the new fnmatch behavior in ExternalFirmwareHandler." 2021-05-05 00:21:34 +00:00
Steven Moreland 1c65d40154 Merge "libutils: LightRefBase: incStrongRequireStrong" 2021-05-04 23:52:12 +00:00
Christopher Ferris 24b15570b1 Merge "Re-add backtrace note about unreadable elf." 2021-05-04 21:44:53 +00:00
Steven Moreland c2dc7cd31c libutils: LightRefBase: incStrongRequireStrong
Allow LightRefBase to be used with
ANDROID_UTILS_REF_BASE_DISABLE_IMPLICIT_CONSTRUCTION, mainly for
libhwui.

Bug: N/A
Test: libutils_test
Change-Id: I251c874a80f0a069572bc51da45f8f8e74ba6f5b
2021-05-04 21:42:26 +00:00
Eric Biggers 890fd0c239 Merge "init: make reboot_on_failure not apply to manually stopped services" 2021-05-04 21:06:12 +00:00
Treehugger Robot f737614ce9 Merge "Add a small testcase for snapshot writer." 2021-05-04 19:37:19 +00:00
Kelvin Zhang ab42259321 Add a small testcase for snapshot writer.
update_engine heavily relies on snapshot writer, add a testcase

Test: th
Change-Id: I4e3cf47dcff92ec4df0b3b84a0a8c7cb7866e7e7
2021-05-04 13:51:34 -04:00
Andrew Scull ef2864edd5 Link libcrypto in host unit tests for libavb
libavb is moving to use boringssl for crypto operations in user space
rather than using its own implementation. Link with libcrypto to
resolved the new dependencies.

Test: atest --host --host-unit-test-only --test-mapping system/core/fs_mgr/libfs_avb
Bug: 185329132
Change-Id: I412f4ef677aa6e29c5b67ffe5e3e8377640a2847
2021-05-04 15:18:58 +00:00
Tri Vo e23e85bee5 Merge "trusty: Helper library for metrics" 2021-05-04 14:17:05 +00:00
Eric Biggers 1c51525f66 init: make reboot_on_failure not apply to manually stopped services
Add a new service flag SVC_STOPPING which tracks whether a service is
being manually stopped by init, and make the "reboot_on_failure" service
setting not apply when SVC_STOPPING is set.

This is needed for devices that use FDE, because otherwise the device
reboots during the following init script fragment:

    on property:vold.decrypt=trigger_shutdown_framework
        class_reset late_start
        class_reset main
        class_reset_post_data core
        class_reset_post_data hal

... because that stops all services, including apexd which has been
marked with reboot_on_failure since
https://android-review.googlesource.com/c/platform/system/apex/+/1325212.
So init was killing apexd, then rebooting the device because apexd
"failed" due to having been killed.  Making reboot_on_failure not apply
when init stops a service itself fixes the problem.

This is one of a set of changes that is needed to get FDE working again
so that devices that launched with FDE can be upgraded to Android 12.

Bug: 186165644
Test: Tested FDE on Cuttlefish
Change-Id: I599f7ba107e6c126e8f31d0ae659f0ae672a25e4
2021-05-03 21:38:50 -07:00
Akilesh Kailash 362b2b7537 Merge changes from topic "snapuserd-enable-read-ahead"
* changes:
  libsnapshot: snapuserd: Enable read-ahead functionality
  libsnapshot: Bump up the kCowVersionManifest version
2021-05-04 04:28:29 +00:00
Akilesh Kailash b2e125cb1a libsnapshot: snapuserd: Enable read-ahead functionality
This enables read-ahead functionality by having
scratch space in the COW

Bug: 183863613
Test: OTA tests with new COW format

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I7988687c81d0ea239e71695818199db4653ddb80
2021-05-04 02:41:35 +00:00
Akilesh Kailash c67686ecb9 libsnapshot: Bump up the kCowVersionManifest version
kCowVersionManifest will be 2. This should now
be in sync with kCowVersionMajor.

Bug: 183863613
Test: OTA with new COW format (by enabling scratch space option)
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ia6c31e399de723ee83459b59d6b076b48f5c88d5
2021-05-04 02:41:23 +00:00
Akilesh Kailash eebf447fef Merge changes from topic "snapuserd-read-ahead"
* changes:
  libsnapshot:snapuserd:Add unit test for read-ahead code path.
  libsnapshot: Flush data to scratch space only for overlapping regions
  libsnapshot:snapuserd: read-ahead COW copy ops
  libsnapshot: Retrieve COW version from update engine manifest
  libsnapshot:snapuserd: Add 2MB scratch space in COW file
  libsnapshot:snapuserd: mmap + msync header after merge
2021-05-04 00:55:28 +00:00
Tri Vo a8919a2e82 trusty: Helper library for metrics
Bug: 182489121
Test: libtrusty_metrics_test
Change-Id: I7fc45ea3319185a8ac55889e56bd484d4774e928
2021-05-03 17:15:30 -07:00
Christopher Ferris fe751c5a61 Re-add backtrace note about unreadable elf.
When moving to the proto-ized tombstones, the note about unreadable
elf files in a backtrace got lost. This re-adds it and adds a test
to verify that the note properly shows up.

Bug: 185428454

Test: Ran unit tests.
Change-Id: I1150cc737772e1b79fd73ec5c782caadc4629421
2021-05-03 15:21:11 -07:00
Treehugger Robot 4f8a56f15d Merge "Make mkbootfs self-contained" 2021-05-03 16:58:55 +00:00
Yi-Yo Chiang 288f9eec0f Make mkbootfs self-contained
Now that mkbootfs is in prebuilt build tools, make it have no dynamic
dependency so that the binary is portable.

Bug: 184490452
Test: Presubmit
Change-Id: Ida4ee9af3c51ba9d163cf9c1e7b7098fd24e0de1
2021-05-02 19:36:23 +08:00
Treehugger Robot 2de90ca526 Merge "filesystem_config: Add a aid/uid for UWB subsystem" 2021-04-30 19:04:57 +00:00
Akilesh Kailash 0ec13ceda9 Merge "libsnapshot: Add Version field which will be used by update-engine manifest" 2021-04-30 18:08:54 +00:00
Akilesh Kailash 936e9ce79d libsnapshot:snapuserd:Add unit test for read-ahead code path.
Add overlapping copy ops to test the read-ahead logic.

Bug: 183863613
Test: cow_snapuserd_test
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ie96bc644c5f2eaae45cf048d9ba8a206930c3ce8
2021-04-30 06:49:03 +00:00
Akilesh Kailash 150bcbf7c6 libsnapshot: Flush data to scratch space only for overlapping regions
When read-ahead thread caches the data from base device, flush the data
only if there are overlapping regions. If there is crash, subsequent
reboot will not recover the data from scratch space. Rather, data
will be re-constructed from base device.

Additionally, allow batch merge of blocks by the kernel even for
overlapping region given that we have the read-ahead thread
taking care of the overlapping blocks.

Bug: 183863613
Test: 1: Incremental OTA from build 7284758 to 7288239. Merge time
         reduces from ~6 minutes to ~2.5 minutes
      2: Reboot and crash kernel multiple times when merge was in
         progress
      3: Verify read-ahead thread re-constructs the data for overlapping
         region.
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I50e0d828f4fb36a23f0ca13b07a73229ba68874d
2021-04-30 06:46:47 +00:00
Akilesh Kailash 580312bc95 libsnapshot:snapuserd: read-ahead COW copy ops
Introduce read-ahead mechanism for COW copy ops.

1: Read-ahead thread will read from base device
   and store the data in scratch space along with the metadata.
2: Worker threads during merge will retrieve the data
   from read-ahead cache
3: Fixed set of blocks are read during each cycle by the read-ahead
   thread.
4: When the last block in the region is merged, read-ahead thread
   makes forward progress.

Scratch space is set to 2MB and is only used from COW copy operations.
We can extend this to Replace Ops based on performance evaluation.

Performance:

As mentioned in bug 181883791, Incremental OTA of size 55M with
235K copy operations where every block is moved by 4k:

Without read-ahead: 40 Minutes for merge completion
With read-ahead:  21 Minutes for merge completion

Bug: 183863613

Test: 1: Full OTA - no regression observed.
2: Incremental OTA - with older COW format. Daemon will just skip
   the read-ahead feature for older COW format.
3: Incremental OTA - with new COW format.
4: Reboot and crash kernel when multiple times when incremental OTA is in-flight.
   Verify post reboot, read-ahead thread re-constructs the data from scratch
   space.
5: No regression observed in RSS-Anon memory usage when merge in-flight.

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Ic565bfbee3e9fcfc94af694596dbf44c0877639f
2021-04-30 06:46:33 +00:00
Akilesh Kailash d967d01f56 libsnapshot: Retrieve COW version from update engine manifest
update_metadata.proto will have the COW version. Retrieve
that from the manifest and compare it with the COW library.
If the versioning doesn't match, disable VABC.

The primary use case of this is during downgrade tests
in pre-submit. Whenever we have a COW format changes,
we may have to disable VABC for that specific transition
build. At a high level, the flow of version check will be:

1: Create a initial COW version of 1 in manifest (update_metadata.proto)
2: The latest COW version of libsnapshot is 2
3: libsnapshot will return VABC disabled
4: Check-in the CL and changes to manifest
5: Once the CL is baked in and the build is green, bump up the COW version to 2 in the manifest
6: Next set of tests, since both versions match, libsnapshot will enable VABC
7: Downgrade should be done to the build which was checked in at (5)

Bug: 183863613
Test: Apply OTA and verify if VABC is disabled if the versions don't
match
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: Id55f33a90bb31b417e72f4fbe370daf05a68f05a
2021-04-30 06:46:24 +00:00
Akilesh Kailash 771b17f509 libsnapshot:snapuserd: Add 2MB scratch space in COW file
Add 2MB scratch space in the COW file. This is a preparation
patch for read-ahead patch. This just add the buffer
space right after the header. Bump up the version number
in the header in order to distiguish between older and newer
COW formats.

No operation is done on this buffer with this patch-set.

Scratch space option is disabled by default.

Bug: 183863613
Test: 1: Create Full OTA with the new COW format.
2: Incremental OTA with older COW format.
3: vts_libsnapshot_test

Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I42a535a48ec22adb893dfe6f86a4f51650e1f88a
2021-04-30 06:45:57 +00:00
Akilesh Kailash 31e0426489 libsnapshot:snapuserd: mmap + msync header after merge
mmap the CowHeader and use msync to flush only the
first 4k page after merge is complete.

This cuts down ~30 seconds of merge completion time
on a 55M incremental OTA with 235k copy operations.

Although, this isn't a significant gain but this patch
creates a scaffolding for the next set of read-ahead patches.

Bug: 183863613
Test: Incremental and Full OTA
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: I15bfec91ea1d5bdf4390670bcf406e1015b79299
2021-04-30 06:31:12 +00:00
Akilesh Kailash acc1c5d189 libsnapshot: Add Version field which will be used by update-engine
manifest

Bug: 183863613
Test: Build
Signed-off-by: Akilesh Kailash <akailash@google.com>
Change-Id: If4b6a60669a55eb2ea1167fb2cc49ce04c054748
2021-04-30 06:31:01 +00:00
Daniel Rosenberg 82129fd4ce Merge "libsnapshot: Allow inspect_cow to show failed blocks" 2021-04-30 05:31:51 +00:00
Inseob Kim 4664bbc6c7 Merge "Fix precompiled sepolicy logic" 2021-04-30 01:14:15 +00:00
Daniel Rosenberg 05a73c5788 libsnapshot: Allow inspect_cow to show failed blocks
Adds the -b option to show the bad data block that failed to decompress.
If the block is large enough, display the front as though it were a
CowOperation, as this is the most likely culprit.

Change-Id: I287f13e0794a1ca9d647d4b1099ab238a6202b23
Bug: 183985866
Test: inspect_cow -db <COW_FILE>
2021-04-29 21:58:58 +00:00
Roshan Pius e85201bbeb filesystem_config: Add a aid/uid for UWB subsystem
Bug: 186788640
Test: Compiles
Change-Id: Ib3d1b064e8e68e7b9e59b26f356ac7c4da2eaaf8
2021-04-29 14:57:07 -07:00
Eric Biggers 4bb41c0ee4 Merge "init.rc: fix (re)starting apexd on FDE devices" 2021-04-29 20:20:42 +00:00
Martin Stjernholm bcd1d4b670 Require libjdwp from the ART APEX.
There is no direct dependency in platform on this library, but we still
need a link to it from the system namespace, since adbconnection can
load it as a JVMTI agent without a class loader, and that is changing
to use the system namespace in https://r.android.com/1673312.

Test: atest CtsJdwpTestCases
Test: atest CtsJdwpTunnelHostTestCases
Bug: 130340935
Change-Id: Ia06c0f2a80226a056195fcff2f5d4dcab8f38518
2021-04-29 20:40:31 +01:00