Commit Graph

47372 Commits

Author SHA1 Message Date
Mark Salyzyn c21c709f0b Merge "Revert "build: Split out mkfs in BuildImageMkfs"" 2018-11-08 19:28:18 +00:00
Mark Salyzyn 533558ec96 Revert "build: Split out mkfs in BuildImageMkfs"
This reverts commit 2b72b7f01e.

out/target/product/generic_x86/installed-files.txt \${DIST_DIR}/installed-files-rescued.txt; exit 1 ) )" 
2018-11-08 19:12:28 - build_image.py - ERROR : Failed to build out/target/product/generic_x86/obj/PACKAGING/systemimage_intermediates/system.img from out/target/product/generic_x86/system 
Traceback (most recent call last): 
File "build/make/tools/releasetools/build_image.py", line 767, in <module> 
main(sys.argv[1:]) 
File "build/make/tools/releasetools/build_image.py", line 755, in main 
BuildImage(in_dir, image_properties, out_file, target_out) 
File "build/make/tools/releasetools/build_image.py", line 449, in BuildImage 
CheckHeadroom(mkfs_output, prop_dict) 
NameError: global name 'mkfs_output' is not defined 


Bug: 111302946
Reason for revert: Build problem

Change-Id: I786f232e07af653a7207509055df5a07a6d8bb9e
2018-11-08 19:26:50 +00:00
Treehugger Robot 5f2e64aaa4 Merge "build: Split out mkfs in BuildImageMkfs" 2018-11-08 17:33:03 +00:00
Treehugger Robot 9da0bd95e5 Merge "build_image: right size image problems." 2018-11-08 01:06:41 +00:00
Mark Salyzyn 2b72b7f01e build: Split out mkfs in BuildImageMkfs
Separate out BuildImageMkfs from BuildImage, which just makes the
filesystem without any verity, avb, or other decisions.  BuildImage
does all the wrapping for such.  This will hopefully ease maintenance
and drop the issues surrounding BuildImage reentrancy.  Change
right-size estimation path to use BuildImageMkfs, and do so without
verity or avb wrappings.

Test: build
Bug: 111302946
Change-Id: I30e2e2b727f40ecca5164142f34139f5244f6424
2018-11-07 15:51:32 -08:00
Mark Salyzyn 41f781de9e build_image: right size image problems.
Neglected to correct for filesystem size in estimate for verity.

To keep build space optimized and make sure that the existing image
does not confuse the tools, remove the out_file after the test build
image phase.

Test: build
Bug: 111302946
Change-Id: I360775d862d02a721d061bbc43e8b4e2f03bd01e
2018-11-07 14:50:33 -08:00
Tao Bao 68b081fd5a Merge "releasetools: Add tests for VerityImageBuilder." 2018-11-07 20:23:12 +00:00
Treehugger Robot 1c7db31772 Merge "Build docs as part of checkbuild" 2018-11-07 18:21:32 +00:00
Treehugger Robot b359355760 Merge "Add .hpp files to sgrep and treegrep" 2018-11-07 17:15:27 +00:00
Anton Hansson 0f22d9e395 Merge "Make right-sizing less aggressive" 2018-11-07 14:22:45 +00:00
Anton Hansson 0e60f8a42c Make right-sizing less aggressive
This is a rollback of Id5ae6645b9aa2d036e6fefe2fb17672e8f8ef6f0
and the followup I6a0f3919e8d45d0b3e7cd558a6ad4d3799012f2c

The magic constant introduced appears to be too aggressive.
Note that the new constant in I0f32c75e80b5da4d47671055ac274ccc2b485992
doesn't appear to fix the broken builds.

Bug: 111302946
Test: make
Change-Id: I29207265172d293c6fee98212e3266bde85895df
2018-11-07 14:02:42 +00:00
Treehugger Robot fcc971934b Merge "build_image still too aggressive with right sizing (part troix)" 2018-11-07 13:59:38 +00:00
Anton Hansson 20e7b65663 Merge "Set LOCAL_SDK_VERSION := current for RROs" 2018-11-07 11:57:40 +00:00
Mark Salyzyn 2b191894c2 build_image still too aggressive with right sizing (part troix)
Regression introduced by c3d408e3c1

Found a case where 1 + 1/60 reduction in size based on free space
failed when right sizing an ext4 image. Reduced the math to 1 + 1/61.

Test: compile
Bug: 111302946
Change-Id: I0f32c75e80b5da4d47671055ac274ccc2b485992
2018-11-06 15:57:03 -08:00
Tao Bao d0b9758203 Merge "releasetools: Create VerityImageBuilder." 2018-11-06 23:37:15 +00:00
Treehugger Robot 9c3a481b5f Merge "releasetools: Log the exceptions before rethrowing." 2018-11-06 23:28:17 +00:00
Tao Bao 46901fb8ca releasetools: Log the exceptions before rethrowing.
In Append2Simg and Append in verity_utils.py, we catch and rethrow
exceptions as BuildVerityImageError. However, this suppresses the
traceback from the original exception which usually indicates the actual
cause. We can better handle this with the raise statement in Python 3,
which is however unavailable in Python 2.

This CL logs the exception before rethrowing to retain the useful bits.

Test: Inject an error to append2simg. `m -j systemimage` with
      aosp_marlin-userdebug. Check the output.
Change-Id: I0c2f57d6023fa1038256b85fa98d57ad0244a70d
2018-11-06 10:42:35 -08:00
Colin Cross fd5ab7c9b2 Merge "Revert "Use protoc-gen-javalite for java lite protos"" 2018-11-06 18:42:03 +00:00
Tao Bao 4a0d5134e1 releasetools: Add tests for VerityImageBuilder.
`testdata/verity_mincrypt` is generated with the following command.
  $ generate_verity_key \
        -convert testdata/verity.x509.pem \
        testdata/verity_mincrypt

Test: python -m unittest test_verity_utils
Change-Id: I3f2f0cd89da0fd5d2420e0e2b09761cc77f893c5
2018-11-06 10:00:51 -08:00
Tao Bao 7549e5e919 releasetools: Create VerityImageBuilder.
Test: `m dist` with aosp_marlin-userdebug (VB 1.0)
Test: `m dist` with aosp_taimen-userdebug (VB 2.0)
Test: `python -m unittest discover .`
Change-Id: I8c016b7621b73f220fb2a4586cc45a21eeee438d
2018-11-06 10:00:28 -08:00
Treehugger Robot 6a806e76d4 Merge "Only zip2zip bundle resources if they exist" 2018-11-06 16:43:00 +00:00
Aurelio Jargas 67edd15196 Add .hpp files to sgrep and treegrep
There are plenty of .hpp files in the AOSP sources.

    $ find . -name \*.hpp | wc -l
    2430

They were already grepped in cgrep(), but were missing in
sgrep and treegrep.

Test: manual - run the script

Change-Id: I1fb3f576f6cdcc2b492e2beadf11ced98cb3938e
2018-11-06 17:36:46 +01:00
Anton Hansson 0625f9a524 Merge "Move ro.build.characteristics to /product props" 2018-11-06 08:52:02 +00:00
Colin Cross 5e7b9937e3 Merge "Remove old copies of host-libprotobuf-java-full.jar in test suites" 2018-11-06 01:53:58 +00:00
Treehugger Robot 344f2524fc Merge "Fix pre-extracted PDKs" 2018-11-06 00:28:16 +00:00
Colin Cross 3b4417a45e Remove old copies of host-libprotobuf-java-full.jar in test suites
Test suite directories not covered by installclean have an old copy
of host-libprotobuf-java-full in them that causes problems when
libprotobuf-java-full uses an updated ABI, remove them.

Bug: 119049592
Test: manual
Change-Id: Iedf0aaf2a1d1ca2c7332da3cc6d948946946eebe
2018-11-05 15:11:46 -08:00
Dan Shi 8b86ca8827 Merge "Copy shared libraries used by tests to testcases directory" 2018-11-05 21:16:04 +00:00
Dan Willemsen 9b7c1e4393 Fix pre-extracted PDKs
The PDK unzip logic does not place all the files in the same directory
it would place a zipfile, but instead inside a directory of the same
name:

  a/b/platform.zip!pdk.mk

turns into:

  a/b/platform/pdk.mk

So fix the PDK_FUISON_PLATFORM_DIR autodetection to take this into
account. Also don't create touch rules for the pre-extracted java files.

Bug: 118776469
Test: try building the PDK with PDK_FUSION_PLATFORM_ZIP set, and files extracted
Change-Id: Ib27d12aba773a5e6bd41fcbe8945a0bc31b8c281
2018-11-05 10:57:24 -08:00
Colin Cross 7f1ca81e86 Build docs as part of checkbuild
Catch docs breakages in TreeHugger by building docs as part of
checkbuild targets.

Bug: 118815658
Test: m checkbuild
Change-Id: I7e71b176c680f269730d4bf98369a96138c0ee8a
2018-11-05 18:15:24 +00:00
Colin Cross 85fa3def65 Only zip2zip bundle resources if they exist
Only attempt to zip2zip the bundle parts/res.zip if it was created
because of full_classes_jar or PRIVATE_EXTRA_JAR_ARGS.

Bug: 117295826
Test: m dist on unbundled target with no resources or classes
Change-Id: I6072b29c7331201f33d196c24154f9947b4b0053
2018-11-05 10:13:10 -08:00
Anton Hansson 5ac408badd Move ro.build.characteristics to /product props
This property is not device-generic, as it's used for things like
whether the product is phone/tablet/other and whether it has an
sdcard or not.

Bug: 118618261
Test: adb shell getprop ro.build.characteristics for aosp_blueline
Change-Id: I6bbf1d4c36d7f37f92c67bebde58d72aba9d0852
2018-11-05 17:29:28 +00:00
Dan Shi 6fe5bf0adb Copy shared libraries used by tests to testcases directory
This change collects the shared libraries files used by tests in
suite, and deploy the files to testcases/lib*. The shared libraries then
can be zipped in general-tests.zip and device-tests.zip. And the host
test can be run in TradeFed host based on build artifacts.

Bug: 111486845
Test: m -j general-tests; m -j device-tests
Confirm the test can run in testcases directory:
out/host/linux-x86/testcases$ ./net_test_avrcp/x86_64/net_test_avrcp
unzip general-tests.zip, confirm net_test_avrcp can run.

Change-Id: I4f9322118aa4891226ea318cbd1bcee6ca48b050
2018-11-04 19:51:59 -08:00
Colin Cross e4f4c73c6a Revert "Use protoc-gen-javalite for java lite protos"
This reverts commit 84f0a11205.

Change-Id: I0d0d750deefb308eb63c99e517ae1a8622376fac
2018-11-04 17:23:32 -08:00
Colin Cross 4c9399d988 Merge "Use protoc-gen-javalite for java lite protos" 2018-11-05 01:13:15 +00:00
Jaewoong Jung 761b58065b Merge "Produce NOTICE files for static libraries." 2018-11-02 21:59:21 +00:00
Treehugger Robot b966219142 Merge "Revert "Revert "Create bundle modules suitable for bundletool""" 2018-11-02 20:55:30 +00:00
David Brazdil 67828c207c Merge "Remove 'ro.art.hiddenapi.warning' system property" 2018-11-02 10:17:28 +00:00
Jiyong Park 987d91455d Merge "Remove apex_debug_key" 2018-11-02 09:28:24 +00:00
Jiyong Park 90418404d4 Remove apex_debug_key
Remove apex_debug_key, in favor of apex.test.key which is under
/system/apex and is built with soong.

Bug: 118213152
Test: /system/etc/security/apex/testkey exists
Change-Id: Iaa0facea2d26cadd48783778e8915bc3c560de10
2018-11-02 14:36:08 +09:00
Dan Willemsen 2683a137c2 Merge "Exempt Soong from LOCAL_TEST_DATA checks" 2018-11-02 04:22:33 +00:00
Dan Willemsen 2066a5cc27 Exempt Soong from LOCAL_TEST_DATA checks
Soong is already checking for reasonable paths, and it supports
generated test data as well, which this does not.

Test: OUT_DIR=$PWD/out m nothing
Change-Id: I62dd09b74f0f3347930fd51c213e2f66c7dfeb2d
2018-11-01 19:37:03 -07:00
Treehugger Robot a0a7e03ff0 Merge "Link hwasan static library to native tests." 2018-11-02 01:51:02 +00:00
Jaewoong Jung 3d90562b7c Produce NOTICE files for static libraries.
Bug: 36073965
Test: mma + visual check.
Change-Id: I3ff341a0668cc6d6de8a54418a7e0db8af28c85c
2018-11-01 16:40:15 -07:00
Evgenii Stepanov ed90746cbd Link hwasan static library to native tests.
They are executables, but they are not EXECUTABLES.

Bug: 112438058
Test: make SANITIZE_TARGET=hwaddress tests
Change-Id: I0f5d8d6259d7df4196bde50ec553b73099f2c8ac
2018-11-01 15:43:14 -07:00
Treehugger Robot 1f50f75988 Merge "build_image too aggressive with right sizing" 2018-11-01 22:31:34 +00:00
Dan Willemsen e0e0cadb21 Merge "Remove obsolete (CLANG|LLVM)_TBLGEN" 2018-11-01 20:51:11 +00:00
Colin Cross 35eb2e32bb Revert "Revert "Create bundle modules suitable for bundletool""
This reverts commit 3451e0ddf2.

Reason for revert: zip2zip now handles recursive globs better.

Bug: 117295826
Change-Id: Ifa30add28d6dbef03d08f8ef7826bb82ed702af6
2018-11-01 20:34:28 +00:00
Mark Salyzyn 4a6ce93f27 build_image too aggressive with right sizing
Regression introduced by c3d408e3c1

Found a case where 1 + 1/59 reduction in size based on free space
failed when right sizing an ext4 image. Reduced the math to 1 + 1/60.

Test: compile
Bug: 111302946
Change-Id: I6a0f3919e8d45d0b3e7cd558a6ad4d3799012f2c
2018-11-01 12:45:29 -07:00
Anton Hansson 8549417b69 Set LOCAL_SDK_VERSION := current for RROs
For sanity, and to silence the error of not setting
PRIVATE_PLATFORM_APIS or LOCAL_SDK_VERSION.

Exempt framework-res from this, as it can't build against the SDK.
Also tweak the previous whitelisting to be more specifically for
framework-res.

Bug: 118823052
Test: make downstream products
Change-Id: Ia15aaa3b1ea95e02e3483673b668ed2cbca76ba9
2018-11-01 18:48:28 +00:00
Treehugger Robot c03653720f Merge "add flags_health_check binary to system_base.mk" 2018-11-01 18:46:52 +00:00