Commit Graph

16917 Commits

Author SHA1 Message Date
Jiyong Park 2812df4edb Avoid duplicated classes for boot dex jars
When a boot classpath java library is directly or indirectly included in
APEXes, multiple variant of the library exist. When running the
hiddenapi tool, we need to eliminate the duplication, otherwise the tool
will complain.

Previously, we simply choose the platform variant of the java library
among the multiple variants. However, if the java library is marked not
available for the platform (i.e. "//apex_available:platform" is not in
the apex_available property), the platform variant does not exist and
thus it is not fed into the hiddenapi tool, which causes missing
references in the tool.

To solve the problem, the platform variant is selected only for the java
libs that are available for the platform. For those libs that are not
available for the platform, we choose one variant of it using a
heuristic; skip testing APEXes and choose com.android.art.release among
the com.android.art.* variants.

Bug: 128708192
Test: m

Change-Id: I33bf297eb3029696ae3504a011486210708fb2c2
2019-12-16 15:19:13 +09:00
Jiyong Park 505221f6e0 ignore apex_available for host variants
The main purpose of apex_available is to prevent a module from being
installed to the system partition (or being statically linked to
something that is installed to the system partition). It's not the goal
to prevent host tools from using the modules.

However, since both host and device variants are not mutated for
platform when the module is not available to platform, the host tools
that depends on those modules couldn't be built. To solve the problem,
the platform variation is NOT skipped for the host variants.

Bug: 128708192
Test: m

Change-Id: I1d662cd6d165581f344138d872329a15bfc43d17
2019-12-16 15:19:13 +09:00
Jiyong Park 1833ceff09 fix: LOCAL_PATH for modules included in APEX is wrong
This change fixes a bug that LOCAL_PATH for modules included in an APEX
is set to the path of the APEX bundle, not to the path of the embedded
module. For example, LOCAL_PATH of libconscrypt included in
com.android.adbd was set to /system/core/adb instead of
/external/boringssl. This caused a problem that NOTICE file in
/external/boringssl is not tagged to libconscrypt, but the NOTICE file
for adbd is.

Fixing the problem by recording the module directories of the included
modules and emitting it in LOCAL_PATH.

Bug: 145347092
Test: Settings -> About Phone -> Legal Information -> Third-party
license. The license for /apex/com.android.adbd/lib64/libconscrypt.so is
OpenSSL.

Change-Id: I76f1830d5a10af63fa74dcc2a42730ffabb8c4ed
2019-12-16 10:53:44 +09:00
Treehugger Robot 0f41daf682 Merge "Switch the mac back to toybox xargs." 2019-12-15 16:35:05 +00:00
Dan Willemsen 950bf327b7 Merge "Remove obsolete copy headers" 2019-12-15 00:04:15 +00:00
Treehugger Robot 0558ecce51 Merge "Decouple addition of new sdk member types from sdk code" 2019-12-14 14:57:42 +00:00
Dan Willemsen 7842826f49 Remove obsolete copy headers
When LOCAL_COPY_HEADER lines are removed, we just removed the copy rule,
we didn't actually remove the old header, so C/C++ files could still
include the old headers during incremental builds.

While we do consider LOCAL_COPY_HEADERS as near-obsolet, and it'll
disappear as we move everything over to Soong, this could produce some
unfortunate incremental build results while people are attempting to
remove them.

Fix this by ensuring that only the files currently listed in
LOCAL_COPY_HEADERS exist in TARGET_OUT_HEADERS after we run Kati.

Test: Remove a LOCAL_COPY_HEADERS entry; see the header removed
Change-Id: I817305703a6996d50490d552623d7df019b608c9
2019-12-13 19:38:23 -08:00
Treehugger Robot ac68248051 Merge "Add "legacy_android10_support" to "apex"" 2019-12-14 02:07:16 +00:00
Chih-hung Hsieh a2fce65a9e Merge "Install rust tests under nativetest(64)" 2019-12-14 01:14:44 +00:00
Chih-hung Hsieh 738bff96f1 Merge "Add x86 device output for Rust" 2019-12-13 22:54:10 +00:00
Chih-Hung Hsieh 9a4a7bab41 Install rust tests under nativetest(64)
* Now the installation directories match those for C/C++ tests:
  * The relative_install_path refers to path under nativetest(64).
  * Device test files are installed in data/nativetest(64).
  * Automatically generated configuration files and copied test binaries
    are still in the "testcases" directory.
* Change host test configuration to run test binary files
  in testcases/<mutated_module_name>/<arch_type>/<stem_name>

Bug: 140938178
Test: atest --include-subdirs under external/rust/crates
Change-Id: I4b29afb897f4ba8749e87f79857c5b1a959bb2b0
2019-12-13 14:51:32 -08:00
Treehugger Robot edb4e21de5 Merge "Separate sdk membership support out of cc/library.go" 2019-12-13 21:40:21 +00:00
Treehugger Robot e676810de5 Merge "Install flattened/unflattend apexes for GSI" 2019-12-13 21:10:56 +00:00
Paul Duffin 255f18e584 Decouple addition of new sdk member types from sdk code
Previously, adding a new SdkMemberType would require adding a new
sdkMemberListProperty instance to the sdkMemberListProperties as well
as adding a new property into the sdkProperties struct. They are
potential sources of conflict and couple the sdk code with all the
packages that add members to it. This change switched to a
registration model that allows each package to register its sdk
member types decoupling them from the sdk code.

Adds an SdkPropertyName() method to SdkMemberType that specifies the
name of the property to use in the sdk/sdk_snapshot. Also provides
an SdkMemberTypeBase struct to be used by providers of SdkMemberType
implementations.

SdkMemberType instances are registered using the
RegisterSdkMemberType() func which sorts the registered instances
by their SdkPropertyName() to ensure the behavior is consistent and
not affected by order of registration.

When creating a new sdk module a dynamicSdkMemberTypes instance is
created that contains the following:

* A properties struct is created dynamically that contains a field for
  each registered SdkMemberType, corresponding to that type's
  SdkPropertyName().

* A list of sdkMemberListProperty instances is also created, one for
  each registered SdkMemberType.

The dynamicSdkMemberTypes instance is cached using a key that uniquely
identifies the set of registered types just in case new types are
registered after one has been created, e.g. by tests.

Bug: 142918168
Test: m checkbuild
Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
2019-12-13 20:07:33 +00:00
Paul Duffin 2f6bc09fee Separate sdk membership support out of cc/library.go
Bug: 142918168
Test: m checkbuild
Change-Id: I4a7f2af8e0b41feca2b5a431b42ad03bbfe94b0c
2019-12-13 10:43:10 +00:00
Paul Duffin 57b9e1da75 Improve handling of generated include dirs
The exported include dirs includes both source and generated
directories (e.g. containing aidl generated headers). The latter are
always arch specific so if they are present they make all the include
directories arch specific.

This change separates the source and generated include dirs so that
the source include dirs (which are probably not arch specific) can be
optimized separately from the arch specific generated include dirs.

The FilterPathList() func was refactored to extract the more general
FilterPathListPredicate() func.

A number of tests needed to be updated to reflect the more optimal
snapshot creation.

Bug: 142918168
Test: m checkbuild
Change-Id: Id1a23d35a45b250ae2168834f9c2a65c86a5fd77
2019-12-13 09:59:48 +00:00
Paul Duffin a7cd8c8344 Generalize common property extraction
Previously, code that attempted to optimize the generated .bp rules
treated the properties structure as a single entity. So, a single arch
specific value would cause all properties to be treated as arch
specific. Also, that code was specific to one structure type.

This generalizes the optimization to work with any properties structure
which will be helpful for other multi-variant module types. It also
treats each property separately.

The hasArchSpecificFlags field has been removed from nativeLibInfo and
a commonProperties field has been added instead into which the common
values will be found. File path creation that conditionally prefixed a
path with archType has been replaced with general code that relies on
archType being "" for common properties and filepath.Join(..) ignoring
empty string components.

The common and arch variant properties are always processed. The first
within the context of the .bp module's property set and the latter
within an arch specific property set. There are always some properties
that are arch specific, e.g. outputFile, so there is no need to worry
about an empty arch property set being created.

The archSpecificNativeLibInfo type was renamed nativeLibInfoProperties
as it may not be arch specific.

The printExportedDirCopyCommandsForNativeLibs variable was renamed to
addExportedDirCopyCommandsForNativeLibs as it no longer does any
printing.

Bug: 142918168
Test: m checkbuild
Change-Id: Iad45913299c37fd76fe03ed0ca68bdc68ed76431
2019-12-13 09:59:48 +00:00
Treehugger Robot cf96a82fbd Merge "Expose some factories for aidl_test.go" 2019-12-13 05:51:38 +00:00
Chih-Hung Hsieh 1f202e9bbb Add x86 device output for Rust
* Copy current configuration from
  rust/config/x86_64_device.go and
  cc/config/x86_device.go.

Bug: 145608833
Test: build rust modules for aosp_x86-userdebug
Change-Id: Ie4951dd4703596f8e503d813a6c4ac03c1f20980
2019-12-12 19:18:06 -08:00
Treehugger Robot 90872f1c9c Merge "add apex_pubkey and apex_manifest.pb to the bundle module" 2019-12-13 03:16:31 +00:00
Elliott Hughes d329254328 Switch the mac back to toybox xargs.
Test: make sdk
Change-Id: I29a6b0404e248553ecfdfdeb263aa9cd0d2fdb3e
2019-12-12 16:52:06 -08:00
Jooyung Han 214bf37311 Add "legacy_android10_support" to "apex"
APEXes with "legacy_android10_support" will have apex_manifest.json for
compatibility as well as apex_manifest.pb.

Bug: 143951586
Test: m (soong tests)
Change-Id: I019252aee5a9423f4b180ba1026e6e99c9961437
2019-12-13 08:47:16 +09:00
Paul Duffin 9ab556fd8e Added support for using static libraries in sdk snapshot
Parameterized the cc.librarySdkMemberType to allow it to support
both static and shared libraries. Created two instances, one for shared
and one for static libraries. A follow up change will add support for
libraries that can be both.

Added *librarySdkMemberType to nativeMemberInfo as information from
there is needed when generating the snapshot.

Made organizeVariants() func a method of *librarySdkMemberType so that
it can initialize the new field. Moved it to be with all the other
methods of that type.

Added host and device tests for the new module type.

Bug: 142918168
Test: m nothing
Change-Id: I00b1e8424b9d81f7d15edc4883971d10668ec2cc
2019-12-12 11:37:39 +00:00
Paul Duffin c62a5107f8 Discard duplicate operations to copy files to snapshot
Header include directories are copied into the snapshot separately for
each cc library that exports them. However, the same include
directories can be exported by multiple libraries which caused ninja
error because two separate rules (albeit identical) were defined to
create the same files.

This avoids the duplicate ninja rules by detecting and discarding
duplicate copies, i.e. where the source and destination are the same.
It will also report an error if two or more different source files are
copied to the same destination.

Bug: 142918168
Test: m nothing
      added test and verified it produced two identical copy rules
      fixed code and verified duplicate copy rules had been eliminated

Change-Id: I39e37405035bee5093f96e03248e9e29ed30962c
2019-12-12 11:37:39 +00:00
Paul Duffin ac6e608da7 Dedup prebuilt cc library creation
Bug: 142918168
Test: m checkbuild
Change-Id: I82cf3b098f024a6ab0884642b21b2238864942a2
2019-12-12 11:37:39 +00:00
Paul Duffin fa02872b26 Sort sdkProperties by package and then name
Bug: 143678475
Test: m nothing
Change-Id: Ib610b799e7d86b413f4321239535a1caf4775ed9
2019-12-12 11:31:59 +00:00
Treehugger Robot aafc04ea5c Merge "AndroidMk for the hostdex library has separate AndroidMkEntries" 2019-12-12 03:24:03 +00:00
Treehugger Robot 8ce3c2c8df Merge "AndroidMkEntries() returns multiple AndroidMkEntries structs" 2019-12-12 03:23:06 +00:00
Tri Vo 3606b00176 Merge "cc_fuzz: add "data" field" 2019-12-11 23:04:57 +00:00
Ulyana Trafimovich 3fae7662ee Revert "Use boot image extension for framework libraries."
This reverts commit 4d2eeed0da.

Reason for revert: breaks avd/avd_boot_health_check test
  on cf_x86_phone-userdebug_coverage on branch rvc-release
  (the device fails to boot).

Test: m
Test: aosp_walleye-userdebug boots

Bug: 145749668

Exempt-From-Owner-Approval: revert

Change-Id: Ie1d93a200222e26461c1bcd384fdb69b7351e259
2019-12-11 18:29:19 +00:00
Jiyong Park 55bd98b2d9 AndroidMk for the hostdex library has separate AndroidMkEntries
Previously, the -hostdex library was emitted using the ExtraFooters.
Now, it has its own AndroidMkEntries, which can be returned together
with the AndroidMkEntries for the main library.

Bug: 128708192
Test: m
Change-Id: Ic9eb0d3839572ed340ccbc5fc6c4b54241e1cb24
2019-12-11 17:27:07 +09:00
Jiyong Park 0b0e1b9804 AndroidMkEntries() returns multiple AndroidMkEntries structs
AndroidMkEntries now returns multiple AndroidMkEntires so that a module
can emit multiple Make modules if needed.

Bug: 128708192
Test: m

Change-Id: I56b6f76d22943b80329951c5acb80a1b932441ad
2019-12-11 17:25:27 +09:00
Paul Duffin ce482dce02 Exclude META-INF/ from repackaged source jar in snapshot
Prevents the META-INF/MANIFEST.MF file from the source jar from being
copied to the snapshot zip file.

Bug: 143678475
Test: m conscrypt-module-sdk
      manually check contents of generated zip

Change-Id: I6eca1435dfc25b562e49de46b049fa81cf8daf90
2019-12-11 08:13:07 +00:00
Treehugger Robot 32fc94d097 Merge "Add property to apex soong modules to generate an APEX without hashtree" 2019-12-11 01:51:48 +00:00
Elliott Hughes e2d33d52e1 Switch to our hermetic bc.
Test: treehugger
Change-Id: I12142a7f54190edd21e719bbd8f5df831ada6161
2019-12-10 10:12:18 -08:00
Logan Chien e87dae6083 Merge "Run ABI checks for shared libs exported by APEX" 2019-12-10 17:36:55 +00:00
Nikita Ioffe c72b5dd10a Add property to apex soong modules to generate an APEX without hashtree
This will help testing on-device hashtree generation.

Test: see other CL in topic
Bug: 145670581
Change-Id: I96b1b2392302aed8a6e80ed52de2d057a532a288
2019-12-10 11:24:53 +00:00
Treehugger Robot 4858e41719 Merge "Add system_ext_specific" 2019-12-10 01:28:45 +00:00
Alexander Smundak 863d5ded9d Merge "Expand namespaces explanation." 2019-12-10 00:06:20 +00:00
Andrew Chant ed75f18e25 Merge "Revert "soong: upgrade Android platform to clang-r370808"" 2019-12-09 23:37:23 +00:00
Treehugger Robot 455e3a089d Merge "Move/dedupe some host path functions in package_ctx.go." 2019-12-09 23:20:16 +00:00
Treehugger Robot 8fc9eeacc8 Merge "Fix generated headers of snapshot" 2019-12-09 23:15:55 +00:00
Martin Stjernholm 7260d066ea Move/dedupe some host path functions in package_ctx.go.
These functions don't need a PackageContext object.

Test: m
Bug: 145934348
Change-Id: Ia1da2c76cbad292f9ca79617199b2b5b6b265568
2019-12-09 22:04:21 +00:00
Sasha Smundak d42609e15c Expand namespaces explanation.
Test: N/A
Change-Id: Iec7d40e6886285046fbe00aea8e19073c6c1ba73
2019-12-09 13:26:09 -08:00
Nick Desaulniers 862eb4648a Revert "soong: upgrade Android platform to clang-r370808"
This reverts commit 4f49e35902.

Causes test failures in ndk_translate.
Bug: 145807809
Bug: 145827049
Test: N/A
Change-Id: I8f42a15fec44475b74b1926c70bc98e03fe66f52
Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
2019-12-09 19:16:08 +00:00
Paul Duffin 593b3c9fb0 Ensure prebuilt modules have same visibility as source modules
Exports visibility and package mutator registration functions so they
can be used in sdk testing. Updates sdk test to support visibility and
package modules.

Adds EffectiveVisibility(...)[]string function to make the effective
visibility rules available to sdk snapshot creation.

Extracts compositeRule.Strings() []string from compositeRule.String()
method so that it can be used by above func.

Adds visibility property to sdk snapshot and prebuilt modules along
with a test to ensure it works properly.

Adds dir parameter to CheckSnapshot so that it can check the snapshot
generated for a non-root package. That is required in order to ensure
that visibility of :__subpackages__ on a source module in package
<pkg> is resolved to an effective visibility of
//<pkg>:__subpackages__ on its corresponding prebuilt.

Test: m conscrypt-module-sdk
Bug: 143678475
Change-Id: Icaacac5b9c04726d28e6fec93e49715ac45df7f4
2019-12-09 13:32:28 +00:00
Paul Duffin 9d8d609fcd Pass SdkMember to AddPrebuiltModule instead of the name
This is needed for a follow up change that makes sure that the
prebuilt modules have the same visibility as the source modules.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I9461c8c094ab19ee9ececb5e5fd50565789f2fa2
2019-12-09 13:28:36 +00:00
Jooyung Han b90e491e2b Expose some factories for aidl_test.go
Because aidl_interface depends on some ndk modules, their factories are
exposed so that aidl_test.go can use them.

Bug: n/a
Test: m
Change-Id: I98d282cc77310d5896a7abaf3936456a14d56ccf
2019-12-09 18:21:48 +09:00
Inseob Kim d110f878a7 Fix generated headers of snapshot
For VNDK snapshot and SDK snapshot, deps files have been used to capture
generated headers. But exported deps might contain intermediate phony
files instead of actual header files, which are for optimization of
ninja. To correctly capture all headers, exported generated header files
are gathered separately.

Bug: 65377115
Test: m nothing
Change-Id: Ia03fa69186490a818578190e3c0bfb0261d1fd6e
2019-12-09 12:38:34 +09:00
Jiyong Park 53ae334130 add apex_pubkey and apex_manifest.pb to the bundle module
The two were missing.

Bug: 145678884
Test: m out/dist/mainline_modules_arm64/com.android.tzdata-base.zip and
inspect the content

Change-Id: I7e244561f59e5adce56b6a64f363a413faa106f2
2019-12-08 02:06:24 +09:00