Commit Graph

85 Commits

Author SHA1 Message Date
Dan Willemsen 09c11073ac Add strict module check to package-modules.mk
The test lists keep getting out of date. Add a per-caller option so that
once they're clean on all builds we can stop them from regressing.

Test: add my_modules_strict := {true,false,,foo} to user
Change-Id: I3e09a8cbe5a07bbbff042b26cea7041c331dde96
2020-01-15 20:54:13 -08:00
Peter Collingbourne bbddfcf7ba Only include the "tools" and "testcases" subdirectories in android-cts.zip.
Without this, we can end up packaging log files from previous CTS runs in
the zip file. If the names of those log files contain whitespace characters,
it can result in a build failure.

Change-Id: Id96d1915a03b6a715acfdc212c9b2d6f28045baa
2019-08-21 12:31:41 -07:00
Tao Bao 2bbb07c53c Build build_image and build_super_image as modules.
Bug: 63866463
Test: TreeHugger
Test: `atest releasetools_test`
Test: `atest releasetools_py3_test`
Change-Id: I2059a4ced709d1b2ee331a9aaaa5ca30db4ebf6b
2019-07-18 10:27:38 -07:00
Tao Bao cf821fc963 Fix an issue in setting up PATH.
`PATH=$(foreach p,$(INTERNAL_USERIMAGES_BINARY_PATHS),$(p):)$$PATH` will
produce a space separated string due to `foreach`, if
$(INTERNAL_USERIMAGES_BINARY_PATHS) has more than one string. We didn't
hit the issue in the past because $(INTERNAL_USERIMAGES_BINARY_PATHS)
had contained only one path in practice.

Test: TreeHugger
Change-Id: I74ef4356668af63d871a81f6bfd4c9324aa1d956
2019-06-24 14:08:36 -07:00
Colin Cross 4f60e366db Make kernel modules into normal installed files
We sometimes see build failures when building platform.zip happens
at the same time as building vendor.img if the vendor.img rule
runs rm -rf $OUT/vendor/lib/modules at the same time that platform.zip
is zipping $OUT/vendor/.  Move the kernel modules into normal
installation rules so that they are in place by the time either
the vendor.img or platform.zip rules run.

This will also cause the kernel modules to show up in
installed-files*.txt.

Test: m vendorimage && ls $OUT/vendor/lib/modules
Change-Id: I178b1d54bfcdb5cf5c29885ace9183ac28fc8826
2019-06-06 12:51:47 -07:00
Dan Willemsen 288bedfc8f Convert more uses of $(ACP) to `cp`
Test: treehugger
Change-Id: Id8365ce4c06dfe731be9b80f805e071fab522909
2019-05-28 15:36:47 -07:00
Dan Willemsen 8e96a794ac Split ALL_MODULES.*.REQUIRED and friends between target/host
So that we actually respect different LOCAL_REQUIRED_MODULES for the
host and device versions instead of unioning them. That got particularly
problematic when LOCAL_SHARED_LIBRARIES is implicitly added to
LOCAL_REQUIRED_MODULES. We also used to walk through device-only modules
when filling out the list of required modules, which triggered even more
extra installations.

This also changes the requirements for PRODUCT_HOST_PACKAGES so that it
no longer accepts target-only phony modules (since we can now
differentiate them). They were all removed in previous patches.

Test: treehugger; diff resulting builds
Test: diff list of product_target_FILES and product_host_FILES
Change-Id: I2ed8950320d31f5693323ad8cef6ec5b6780b7d4
2019-04-10 13:22:23 -07:00
Anton Hansson 8dab0a6f50 Access PRODUCT_ variables directly
This CL simplifies the PRODUCTS.$(INTERNAL_PRODUCT).X accesses of
product variables, and removes unnecessary stripping of them.

Replace: '\$\(PRODUCTS\.\$\(INTERNAL_PRODUCT\)\.([^\)]*)\)' with '$(\1)'
Replace: '\$\(strip\s*\$\(PRODUCT_([^\)]*)\)\)' with '$(PRODUCT_\1)'

A few minor manual tweaks.

Bug: 116769560
Test: presubmit
Change-Id: I70c54f1582e3cc780028535960147d99ebc2e0e1
2019-03-28 15:54:25 +00:00
Nelson Li 1f8357fe7d Revert "Revert "Build System: Solve dependency problem for test""
This reverts commit 6fe7f194b9.

Reason for revert: Fixed all build break.

Change-Id: I0a4842df1225399752515a4cd7a7c14173a5bf7b
2019-03-14 01:05:36 +00:00
Nelson Li 6fe7f194b9 Revert "Build System: Solve dependency problem for test"
This reverts commit 224e103308.

Reason for revert: Build Breakage in git_pi-dev-plus-aosp/docs @5366136
make -j110 docs showcommands dist DIST_DIR=/buildbot/dist_dirs/git_pi-dev-plus-aosp-linux-docs/5366136 checkbuild
FAILED:
 Dependencies in out found with no rule to create them:
 out/target/product/generic/data/app/CtsVerifierTester/CtsVerifierTester.apk
 out/target/product/generic/data/app/TradeFedTestApp/TradeFedTestApp.apk
 out/target/product/generic/data/app/TradeFedUiTestApp/TradeFedUiTestApp.apk
 15:53:46 stopping
and
make -j50 showcommands dist TARGET_PRODUCT=cf_x86_phone DIST_DIR=/buildbot/dist_dirs/git_master-linux-ndk_translation_all/5366149 ndk_translation_all
FAILED: ninja: 'out/target/product/vsoc_x86/data/nativetest/arm/arm_insn_tests_arm_static/arm_insn_tests_arm_static', needed by 'out/target/product/vsoc_x86/obj/PACKAGING/ndk_translation_tests_intermediates/arm_insn_tests_arm_static_result.xml', missing and no known rule to make it
15:55:38 ninja failed with: exit status 1
make: *** [run_soong_ui] Error 1
Return Code: 2

Change-Id: Idf95ef2e06526a0a31690420c923207db627605f
2019-03-11 16:48:29 +00:00
nelsonli 224e103308 Build System: Solve dependency problem for test
1. A test can add a runtime dependent test module by just setting
   LOCAL_REQUIRED_MODULES or LOCAL_TARGET_REQUIRED_MODULES. Then the dependent test
   module will be copied to testcase folder.
2. Do not install to $(TARGET_OUT_DATA) for testcase

BUG: 117224272

Test: 1. (a) vi cts/tests/tests/text/Android.mk
         (b) add LOCAL_REQUIRED_MODULES := CtsPrintTestCases
         (c) m -j CtsTextTestCases
         (d) Then, CtsPrintTestCases should also be built to testcase folder like below.
             ./target/product/generic_arm64/testcases/CtsPrintTestCases

Change-Id: I24ea3783486c54a05cfa9d3d0375b977afc230f8
2019-03-07 14:08:35 +08:00
Julien Desprez e483544cf5 Remove host proto lib from cts jars
The lib is already statically included in tradefed.jar.

Test: run cts-unit-tests
Bug: 118297021
Change-Id: I073b68216870b1e744ee35db82a7e03f11db6782
2018-10-23 09:38:09 -07:00
Julien Desprez 5ec361fff6 Merge "Add a version.txt file to compatibility suite zips"
am: aa2abc3324

Change-Id: I6775b48c36b323c2b7f502acb0a2e12fed04f4c9
2018-07-26 15:09:09 -07:00
Julien Desprez 7a10601e7c Add a version.txt file to compatibility suite zips
This will avoid having to tag jar file with a manifest
implementation version.

Test: make cts, check the zip
Bug: 111834256
Change-Id: Id809217cefb36b4779fdf41c02f5809ba22d03a1
2018-07-26 11:47:46 -07:00
Julien Desprez 5b9602634b Merge "Add loganalysis to cts" am: ee9ea1d601
am: 6d948a6cf5

Change-Id: Id523f239f07720b131a175f61118878b591a1ca0
2018-03-20 03:10:14 +00:00
jdesprez 4619031efe Add loganalysis to cts
Tradefed depends on Loganalysis so it should also be part
of cts. Luckily there was no overlap (cts never called a
part of TF that use loganalysis) so it never failed
with classNotFound before.

Test: build cts
Bug: 75979908
Change-Id: I2542ecd077149f7165aff120d67054943df20b54
2018-03-19 16:47:22 -07:00
Nan Zhang 972bd248f9 Merge "Use BUILD_NUMBER_FROM_FILE instead of BUILD_NUMBER in Make." am: d3bec931cd am: 90b8d087e6
am: 7e634580ba

Change-Id: I514525914843c2cad6591ed6f515aed3aecc286a
2018-02-22 02:36:22 +00:00
Nan Zhang 0abdb5811d Use BUILD_NUMBER_FROM_FILE instead of BUILD_NUMBER in Make.
<Two phase commits> Since internal master code has more places that use
BUILD_NUMBER (mostly in vendor/) than AOSP (conflict). We can't
deprecate BUILD_NUMBER directly. Therefore, we try to switch to
BUILD_NUMBER_FROM_FILE as much as possible at first. Then we will do
a one-off deprecation for BUILD_NUMBER in internal master next step.

Test: m -j
Bug: b/70351683
Change-Id: I14ffee7381933c9fde14c4bde8c0c14e45fe98bf
2018-02-21 14:12:18 -08:00
Colin Cross 426ecd0f60 Merge "Replace build/tools with build/make/tools" am: 973301af79 am: e6dc6b2026
am: eb6c8444a6

Change-Id: I14d033658d219e81790ab1e730b7e477c70f4a13
2017-12-13 03:03:47 +00:00
Colin Cross fdea893833 Replace build/tools with build/make/tools
Test: m checkbuild
Change-Id: I8db6c27a6d2372f142d10568fe92d6f2e9a59252
2017-12-06 16:12:40 -08:00
Siyuan Zhou 1888315487 Added test suite specific libs.
Change-Id: Iab18a8c0e118c957f8ee3dfa258d3a72351ce4db
2017-11-28 10:31:40 -08:00
Bowgo Tsai 0e1ada16ce Merge "Adding Android verified boot 1.0 metadata into ENG builds" am: 5f958ccad5 am: 86c4f8ed54
am: 08a05cd9ae

Change-Id: Ie7471976439616452b0c830b19471be1748a750a
2017-10-15 03:26:24 +00:00
Bowgo Tsai 6ceeb1a8bf Adding Android verified boot 1.0 metadata into ENG builds
Adding verified boot metadata with a "disable magic". The resulting
metadata at the end of each image (e.g., system.img, vendor.img) will
be the same as triggering an "adb disable-verity" on an USERDEBUG image.

This can help simplify the code on fs_mgr, which won't have to check if
current image is an ENG build or not.

Bug: 63056044
Test: boot sailfish eng/userdebug builds
Change-Id: I95d23ac7b76c04d6d4483c9c4dc1de16bf0d9c3a
2017-10-12 10:08:44 +08:00
Bowgo Tsai 90f4777cdd Merge "Support signing custom_images with AVB" am: e09809a32d am: d8c0bb8f61
am: e047421956

Change-Id: I8d22ff2d4a9bd999d99637f61e2dfabd64069978
2017-08-25 08:05:23 +00:00
Bowgo Tsai 480dc7c19d Support signing custom_images with AVB
Current AVB signing for custom images is enabled by either of the
following build variables:
  CUSTOM_IMAGE_AVB_HASH_ENABLE := true
  CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true

A previous change to support chain partition replaced avb_signing_args
with avb_key_path and avb_algorithm. This change updates the
corresponding change for custom_images.

To sign a custom_image as a chain partition, it needs:
  CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem
  CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
  CUSTOM_IMAGE_AVB_ROLLBACK_INDEX := 1

Note that it doesn't support include metadata of custom images into
vbmeta.img. Because custom_images is designed to build multiple files
(e.g., custom1.img, custom2.img, custom3.img, etc) and a device can only
use/mount one of them. The vbmeta.img needs to be generated per each
combination.

Bug:36701014
Test: sign custom image with AVB HASH descriptor (non-chain)
Test: sign custom image with AVB HASH descriptor as chain partition
Test: sign custom image with AVB HASHTREE descriptor (non-chain)
Test: sign custom image with AVB HASHTREE descriptor as chain partition
Change-Id: I492e2ce768e7caec22228b776b2c13a2d37a5b89
2017-08-25 13:28:57 +08:00
Bowgo Tsai 1799d360b5 Merge "custom_images: support verity fec (forward error correction)" am: abf553c9d3 am: bd91ac7740 am: 9a797706b5
am: 5cf6551455

Change-Id: I701c2708b9945f543e03b583d505ca947ebd63f6
2017-07-15 00:10:39 +00:00
Bowgo Tsai bd91ac7740 Merge "custom_images: support verity fec (forward error correction)"
am: abf553c9d3

Change-Id: I4b6e678859538ac91fc2548fc7195677660af37c
2017-07-14 23:58:45 +00:00
Bowgo Tsai 8ec2a1cb08 custom_images: support verity fec (forward error correction)
Bug: 63691195
Test: `make custom_images` with CUSTOM_IMAGE_SUPPORT_VERITY_FEC := true
Test: boot device with the custom image built above

Change-Id: I198fa1e0697cb00712bbfb6f1a717ec623703ede
2017-07-14 21:47:13 +08:00
Howard Chen 133a64829e Merge "Support kernel modules in the odm image" am: 95a8c7e83f am: 1b0da968a0 am: e4b1bdaa9e
am: fd3340ee6f

Change-Id: I62b58c6cc623f1fa591ff2dc2a3eaefa9e30d995
2017-07-11 02:17:27 +00:00
Howard Chen 1b0da968a0 Merge "Support kernel modules in the odm image"
am: 95a8c7e83f

Change-Id: Ib34b1eb969b5fffba4040f40143ea1f3679d4913
2017-07-11 02:10:24 +00:00
Howard Chen 0c5f1e575d Support kernel modules in the odm image
This patch reuses the build-image-kernel-modules macro to build the
odm/lib/modules directory according to the BOARD_ODM_KERNEL_MODULES
which contains list of kernel module files.

Bug: 36012197
Test: android master build on pixel
Change-Id: I2c004132a89e7f230690b4d26c98c3d5b2769f11
2017-07-10 10:44:58 +08:00
Dan Willemsen d2a32cf66a Merge "Support custom my_copy_pairs in package-modules.mk" am: 5b91bd4ccb am: 616764b247 am: b2f7189479
am: 0e59a32e8f

Change-Id: I64201c181653b4a067988909b802c288ab3a515d
2017-06-21 03:40:44 +00:00
Dan Willemsen b2f7189479 Merge "Support custom my_copy_pairs in package-modules.mk" am: 5b91bd4ccb
am: 616764b247

Change-Id: I288d04b2e69c7f231f25905edcff19b31c6cf499
2017-06-21 03:00:30 +00:00
Dan Willemsen f354b178b1 Support custom my_copy_pairs in package-modules.mk
To support extra files in package-modules.mk, allow the user to set
my_copy_pairs to a list of src:dest pairs that will be copied into the
zip file.

Test: build-aosp_arm.ninja is identical before/after
Test: codesearch says that these variables aren't otherwise used
Test: set my_copy_pairs, ensure that they exist in the zip.
Change-Id: Ia80cd136db8ad37a71010baf0552621b281c8bc3
2017-06-20 16:00:48 -07:00
Todd Lee bae7ee5d9e add support for test suites with 'prebuilt tools'
- prebuilt tools copied directly to /tools subdir of zip
- currently only used by wts, noop for other suites

Bug: b/62102556
Change-Id: I796d7490f39f817af8f2ec5bea86593182bfcbe2
2017-06-09 17:58:51 +00:00
Bowgo Tsai b4c268eb7b Merge "Support signing custom images with AVB HASH or AVB HASHTREE" am: fe7aeb1de7 am: 16b64cbb58
am: 3989b738b8

Change-Id: I11393e1833db97301ffdd31ec3db0968e87c1a24
2017-05-31 03:38:24 +00:00
Bowgo Tsai 7ea994b21c Support signing custom images with AVB HASH or AVB HASHTREE
`make custom_images` supports to build different kinds of *non-droid* images,
e.g., odm.img. Adding the support of signing them with either AVB HASH footer
or AVB HASHTREE footer. The user can use HASH for small images and
HASHTREE for large images.

Sample signing configurations:
 * AVB HASH footer:
   - CUSTOM_IMAGE_AVB_HASH_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASH_FOOTER_ARGS := --append_to_release_string my_odm_image

 * AVB HASHTREE footer:
   - CUSTOM_IMAGE_AVB_HASHTREE_ENABLE := true
   - CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS := --fec_num_roots 8

 * Using custom signing key:
   - CUSTOM_IMAGE_AVB_ALGORITHM := SHA256_RSA2048
   - CUSTOM_IMAGE_AVB_KEY_PATH := external/avb/test/data/testkey_rsa2048.pem

Bug: 36701014
Test: `make custom_images` with AVB HASH footer
Test: `make custom_images` with AVB HASHTREE footer
Test: `make droid` to check system.img is still properly signed with AVB HASHTREE
Test: `make droid` to check vendor.img is still properly signed with AVB HASHTREE

Change-Id: I8dc420e12e37e9a631345c0cd883339db05d489f
2017-05-24 14:21:03 +08:00
Sen Jiang c5d5fdee22 Merge "Add support for avbtool in custom_images." am: 8a8ca3438b am: 7cdc14d8b0
am: c900a7cd48

Change-Id: I6612fc8c91ae83ff9da7fa7f78460f6d8249a687
2017-05-16 18:09:45 +00:00
Sen Jiang 5b0fe18feb Add support for avbtool in custom_images.
Set CUSTOM_IMAGE_AVB_ENABLE := true to enable avb, add_hashtree_footer
args can be added in CUSTOM_IMAGE_AVB_ADD_HASHTREE_FOOTER_ARGS.

Bug: 38319818
Test: m custom_images

Change-Id: Ia452dc5ce8b55bcbd3abba9e965b72e78fd8c104
2017-05-15 17:34:42 -07:00
Dan Willemsen 4e9e5d526c Merge "Rewrite link type checking" am: e4d92ef568 am: d76aaed983
am: e834d5f059

Change-Id: I02342b80077a7e1e36e61d1f9384d3b104a982c1
2017-04-20 15:15:19 +00:00
Dan Willemsen b47d4e9cf1 Rewrite link type checking
All the new features are turned off for now, since multiple branches and
products need to be verified before they can be turned on. So everything
should behave the same as today, except for no partition-based
warnings.

Instead of the current link type checks that happen during the build,
run as many as possible immediately after loading all the Android.mk
files. If we're allowing missing dependencies ('mm',
ALLOW_MISSING_DEPENDENCIES, tapas, etc), we'll defer the link type
checks to during the build. If we're not allowing missing dependencies,
we'll produce a better error message to the user about the missing
dependencies.

See core/main.mk for a description of the storage format.

This also remove the partition-based type checking. It hasn't worked all
that well, particularly with ASAN builds. The new VNDK checks will
handle the most pressing cases.

Test: Verify all link_type files and dependencies are the same:
  grep link_type: out/build-aosp_arm64.ninja | sed -E "s/ rule[0-9]+//" | sort
Change-Id: Id643658b9d9e84f99f5db0d526aad88c1f5d3417
2017-04-19 22:41:32 -07:00
Colin Cross 4b6bc2b427 Merge "package-modules: use && instead of ; between copy commands" am: e0e20079c2 am: 8843039368
am: d826201c69

Change-Id: Ib5529cd05e439878a1083756e3b73d4d08940809
2017-03-15 02:28:39 +00:00
Colin Cross f075bcbc98 package-modules: use && instead of ; between copy commands
Using ; to join commands in a rule causes failures to be ignored by
make.  Use && instead, and add true at the end as the second operand
to the last && operator.

Also inline copy-test-in-batch, batching is no longer necessary as
kati will fall back to a shell script if the recipe is too long to
fit in a command line.

Test: builds
Change-Id: I4a2528bf2a15106cfabaae0336662c4a0464271d
2017-03-14 16:58:34 -07:00
Dan Willemsen ff490b6763 Merge "Speed up *TS zip file creation" am: 6d0f949296 am: 83667a3821
am: d6f077092b

Change-Id: Icf328eb0c239b7bd0a0ba602490860cad02f56db
2017-03-13 20:07:18 +00:00
Dan Willemsen c4cf49569f Speed up *TS zip file creation
This also makes the resulting zip files more repeatable -- the file list
is sorted and the entries have static timestamps.

On my machine, this saves ~30 seconds for android-cts.zip, it now takes
less than a second.

Test: m -j cts; compare output from without this change
Change-Id: Ia71e35878ff98ba9775115860530e87eee47739f
2017-03-10 13:39:26 -08:00
Dan Willemsen aa7e67c871 Merge changes I5e684409,I4fa35540,I24015ef0 am: 93de77745c am: d1378dec05
am: ead8af8580

Change-Id: I0608060da7307b9bd9a5bd15e601b1787c510dd4
2017-02-28 21:41:33 +00:00
Dan Willemsen e19ca033c6 Improve warnings for package-modules.mk
Include the calling makefile and package name instead of just a warning
pointing to package-modules.mk

Test: multiproduct_kati, grep logs
Change-Id: I4fa35540b9695b44eea6c23463e137ec37d1d2c9
2017-02-24 15:49:50 -08:00
Alex Deymo df32f43f1e Merge "Include the LOCAL_REQUIRED_MODULES when packaging." am: 06f3258694 am: 2ee71f6946 am: af5554c699
am: 0f9093a31c

Change-Id: Ib02724ee1825334069e48af6b7342364b759fbe4
2017-02-13 16:15:10 +00:00
Alex Deymo 00dc66760e Include the LOCAL_REQUIRED_MODULES when packaging.
When building test modules, dependencies added by the test modules get
build but don't get included in the packaged tests .zip file.

When packaging modules into a .zip file, this patch includes the modules
explicitly listed as a dependency in LOCAL_REQUIRED_MODULES for the
requested modules to package.

If these LOCAL_REQUIRED_MODULES dependencies are not used in the base
system image, they were build as part of the "tests" target but weree
not included in the package nor in the system image. This patch includes
those modules, making it easier to define dependencies of a test
module in the Android.mk file that defined said module, instead of
requiring to re-list all the dependent modules when packaging test
modules.

Bug: 27348226

Change-Id: Ic6f60cf2916b3fae0fa39f84aee8a4f440af9539
2017-01-05 12:12:42 -08:00
Tao Bao eac8bbcb81 Merge "HOST_OUT_EXECUTABLES path is not needed in misc_info file" am: 376c0146b3 am: 7def52bfe8 am: f2cffd6849
am: 70e1bacb1a

Change-Id: I4e484604f6a691f0cb48015192cbba120d3306cc
2016-12-21 18:15:06 +00:00