Commit Graph

315 Commits

Author SHA1 Message Date
Paul Duffin b9e7a3ca7a Make licenseModule SdkAware
Making licenseModule SdkAware caused two breakages in the build. The
breakages were both caused by having an SdkAware module that was
depended upon by a versioned sdk snapshot but which was not itself
versioned and so did not have the member_name property set.

That occured because some default licenses have been added to the
packages containing prebuilts, e.g. prebuilts_runtime_license in
prebuilts/runtime/Android.bp. They apply to both the versioned and
unversioned members.

Once license support has been added to the sdk most of those will be
removed and replaced with properly versioned license modules. However,
in the meantime it is necessary to support that.

This change avoids the issue by checking to see whether the module is
itself versioned before relying on the member_name property. It also
improves the error message when a panic is recovered to make it easier
to identify where it originates.

Bug: 181569894
Test: m nothing
Change-Id: I0e7da2e0c4a30a6f814c2faab821b185aaed2135
2021-05-06 23:13:06 +01:00
Paul Duffin 525a590565 Make sdk tests more realistic
The tests use <sdk>_<module>_<version> as the format for a versioned
sdk member name but the format should be <sdk>_<module>@<version>. This
change corrects it and also fixes a similar issue in an error message.

Bug: 181569894
Test: m nothing
Change-Id: I8be0db4bcd0b6f4d6fbdf9e402ef7257fae8e18b
2021-05-06 23:13:06 +01:00
Mathew Inwood 7e554ec2de Merge "Ensure current.zip is put in the right place." 2021-05-06 15:21:04 +00:00
Mathew Inwood 60770e2250 Ensure current.zip is put in the right place.
Using the InstallFile as the snapshot ensure that the artifact
in out/soong/mainline-sdks/ is built when building the sdk
target.

Test: m ipsec-module-sdk
Change-Id: I45ce6001dbae3a7a9c4cf50f8d7d5d67f94dbcb3
2021-05-06 13:16:27 +00:00
Paul Duffin 62035b5991 Add baseline test for sdk snapshot env variables
Bug: 157884619
Test: m nothing
Change-Id: I56c14ddfec1c33b60fe70f56e4d59b90639657fe
2021-05-06 10:50:00 +01:00
Paul Duffin f23bc472b0 Move configuration checks from getBootImageJar
The getBootImageJar function will be removed once the boot image
creation has been moved to the platform_bootclasspath and
bootclasspath_fragment module types. However, the consistency checks
that it performs are still useful so this change moves them out
first.

The ART boot image related checks are now performed in the
bootclasspath_fragment module type. A previous change accidentally
disabled the checks when the contents property was not empty which has
been fixed. Also, the error messages have been tweaked to make it clear
that the art-bootclasspath-fragment is now the source of truth as to
its contents not the configuration.

The framework boot image related checks are now performed in the
platform_bootclasspath module type.

Initially, this change included an extra check to make sure that
UpdatableBootJars comes from updatable APEXes but that broke because
framework-wifi and framework-tethering are not currently marked as
updatable in AOSP.

Bug: 177892522
Test: m nothing
Change-Id: I80fb600fa2c7cec4566b3461c6a33c4c6f0743f4
2021-04-30 12:06:28 +01:00
Paul Duffin 895c7140a9 Add stub_libs properties to bootclasspath_fragment snapshot
Bug: 177892522
Test: m nothing
Change-Id: I7644122c4abed8e8bc2cc7b58f408bee3bdc91f6
2021-04-28 22:02:00 +01:00
Paul Duffin 2ac45f019b Remove special handling of frameworks/base/config/boot-profile.txt
Previously, the method was run in repo manifests that did not include
the frameworks/base repository so it had to handle the file being
missing. However, now that this is being called from
platform_bootclasspath which is only defined in frameworks/base that
is no longer required.

Bug: 177892522
Test: m droid
Change-Id: I77fa5a204d1426a6be24a6f0b48e18f48f3dd908
2021-04-28 09:03:46 +01:00
Paul Duffin eee466e439 Pass module to SdkMemberTypeDependencyTag.SdkMemberType
This change allows an SdkMemberTypeDependencyTag to select the
SdkMemberType to use to add a dependency based on the module.

Bug: 177892522
Test: m nothing
Change-Id: I2d6e51b615636dc7cd41da65808ef851dd8ae6ac
2021-04-27 23:23:20 +01:00
Paul Duffin 2dc665bb48 bootclasspath_fragment: Always output contents property in snapshot
Bug: 177892522
Test: m art-module-sdk
      - check generated snapshot contains contents property
Change-Id: I122dedba6600a199bfd83b01988da36ddfd09d63
2021-04-26 21:21:58 +01:00
Paul Duffin e95b53a55d Automatically add bootclasspath_fragment contents to sdk
Previously, both a bootclasspath_fragment and its contents had to be
explicitly added to the sdk. This change means that adding a
bootclasspath_fragment to and sdk will automatically add its contents
as if they were added using java_boot_libs.

Bug: 177892522
Test: m nothing
Change-Id: I8f7e70649f272c9a109d4606571a2d12c44b7904
2021-04-26 21:21:58 +01:00
Paul Duffin cd06467124 Generate snapshot member list properties from dependencies
Previously, the snapshot member list properties were simply copied from
the sdk variants. That worked because the snapshot member list
properties must include all exported (i.e. visible outside the sdk
snaphot) members and only members explicitly specified in the sdk
member list properties were exported. However, a preceding change
allowed members that were transitively added (and so not explicitly
specified on the sdk) to be exported.

This change ensures that those exported members are added to the
snapshot member properties which means that they will now work properly
with the sdk version mutators.

This does not add tests for this because no existing SdkMemberType
exports its transitive dependencies and there is currently no way to
register an SdkMemberType for testing only. A following change will
exercise this functionality.

Bug: 186290299
Test: m nothing
Change-Id: Iadbdd86f6dcdd183fa99f647222a43d412271501
2021-04-24 22:33:04 +01:00
Paul Duffin a720811c38 Allow exporting of sdk members to be done per tag
Previously, every module added to an sdk directly through one of the
SdkMemberType specific properties, e.g. java_libs, was exported and
every module added automatically via a transitive dependencies was not
exported. This change allows that behavior to be customized per tag.

Bug: 186290299
Test: m art-module-sdk
      - verify that this change does not affect its contents.
Change-Id: I563b5bcd823e61c23cdb706cfcbb13337963d550
2021-04-24 22:30:26 +01:00
Paul Duffin 2d1bb89fe3 Extract the os specific snapshot module properties handling
Previously, the code was inlined in the addSnapshotModule() method
which made it difficult to follow, did not have clear separation
between the different pieces and duplicated functionality. This
change combines the dynamic and static properties together and then
separates the processing into collation, optimization and property
set updating methods.

Bug: 186290299
Test: m nothing
Change-Id: I311eebca2337cb67634e63d71da12305f4370c07
2021-04-24 22:30:18 +01:00
Paul Duffin fbe470e7d3 Dedup snapshot module property set initialization
Previously, the code to add the compile_multilib property was
duplicated. This change moves the logic into the existing
addMemberPropertiesToPropertySet method, renames it to reflect new
behavior and renames and promotes the variantProperties struct from the
addSnapshotModule() method to the package level scope.

Bug: 186290299
Test: m nothing
Change-Id: I832a7fd89a3cf95467a1ff0807902c9be34c50b9
2021-04-24 16:50:47 +01:00
Paul Duffin 7b0259ff62 Make order of properties in snapshot module consistent
Previously the compile_multilib property was added after the member
list properties at the top level but before them in the target.os
level. This change makes it always be added before them.

Bug: 186290299
Test: m nothing
Change-Id: I01fe272933fd11f7f730cc379f60fb2ce70f2114
2021-04-24 16:50:47 +01:00
Paul Duffin cc3132e496 Clarify behavior around handling of sdk members
Renames some types and adds some documentation to try and clarify how
the sdk handles the members.

Bug: 186290299
Test: m nothing
Change-Id: I4038276f5f4c473a96fe06aedc43050c90416877
2021-04-24 16:49:26 +01:00
Paul Duffin 2182726990 Rename sdkMemberRef to sdkMemberVariantDep
Previously, there were two types sdkMemberRef and sdkMember that seemed
as though the former was referring to the latter but that was not the
case. The former was actually information about the dependency the sdk
had on a specific member variant. The latter is all the information
about a specific member, including all its variants.

This change attempts to clarify that relationship, both through a
better type name and through some improved documentation.

Bug: 186290299
Test: m nothing
Change-Id: I1dcc0ddf3333da9e797bb23ae945d04729839014
2021-04-24 16:48:16 +01:00
Paul Duffin 26197a65da Extract addSnapshotModule from buildSnapshot
Bug: 186290299
Test: m nothing
Change-Id: I297d63366d71f692572e8e40f1e510b6e41859e6
2021-04-24 16:48:16 +01:00
Paul Duffin 023dba0a3f Add exported_bootclasspath_fragments to prebuilt_apex/apex_set
This is needed to allow a prebuilt_bootclasspath_fragment to be used
interchangeably with a bootclasspath_fragment in the
platform_bootclasspath module.

The platform_bootclasspath module depends on APEX specific variants of
bootclasspath_fragment modules. That works because the
bootclasspath_fragment modules are part of an apex and so have an APEX
specific variant which the platform_bootclasspath can specify.

Using a prebuilt_bootclasspath_fragment in place of a
bootclasspath_fragment requires that the prebuilt also has an APEX
specific variant.

Specifying exported_bootclasspath_fragments on a prebuilt_apex/apex_set
will cause it to create an APEX variant for the named module whcih will
allow it to be selected by the platform_bootclasspath module.

Bug: 186034565
Bug: 177892522
Test: m nothing
Change-Id: I7ddacc6498ec3a4a9f26c5f78b7f9a033e494d78
2021-04-23 12:12:37 +01:00
Paul Duffin 7c95555d79 bootclasspath_fragment: Add hidden API flag files to snapshot
These are needed at the moment to ensure the hidden API processing
generates the same set of flags whether it is being generated from
source or prebuilts. Soon these will be needed to ensure that the
hidden API flags generated for the individual modules are compatible
with previous releases.

Bug: 179354495
Test: m art-module-sdk and check snapshot zip contains the files
      and the generated Android.bp references them.
Change-Id: I9a3334cc48faa381bbbcbbb59479db719042796a
2021-04-21 23:54:22 +01:00
Paul Duffin a57835e8e5 bootclasspath_fragment: Add contents to snapshot
Bug: 177892522
Test: m nothing
Change-Id: I54fe0537b758a0e3dacd34b139ef3eb21b8841fd
2021-04-21 23:54:22 +01:00
Paul Duffin 0b28a8d356 Replace boot image with bootclasspath fragment in sdk package
Bug: 177892522
Test: m nothing
Change-Id: I67589bccbde426de2caf513bd643d9484df86c1e
2021-04-21 23:54:17 +01:00
Treehugger Robot 1cd2357dd3 Merge "Do not output two trailing newlines in generated bp files." 2021-04-20 23:41:00 +00:00
Martin Stjernholm ee9b24e6d2 Do not output two trailing newlines in generated bp files.
Test: m platform-mainline-sdk
  Check Android.bp in the generated platform-mainline-sdk-current.zip.
Bug: 156286550
Change-Id: Ifb99288f3b85c3b986270d48e2dc7d9e15fa7536
2021-04-20 15:54:21 +01:00
Paul Duffin 1267d875b6 java_sdk_library: Make dex stub jars available for hiddenapi
The hidden API processing needs access to dex jars for the API stubs in
order to determine which dex members are part of an API surface. The
dex stubs used for the monolithic file are provided by normal
java_library modules for legacy reasons. However, the APEXes that
contribute to the bootclasspath, and so need to perform hidden API
processing, typically provide stubs created by a java_sdk_library.

This change adds support to java_sdk_library/_import to make the dex
stub jars available when requested, that involves:
1. Adding compile_dex property to java_sdk_library_import and
   propagating it down the the java_import modules for the stubs. That
   is already handled for java_sdk_library.
2. Propagating the java_sdk_library compile_dex property to the
   java_sdk_library_import in the generated snapshot.
3. Refactoring and wiring to make the dex stubs jar available to other
   parts of Soong.

Bug: 179354495
Test: m nothing
Change-Id: I5895d4f2ba0b684870862b9429b2364865e4afc6
2021-04-16 18:48:20 +01:00
Paul Duffin e1381887ae Migrate sdk tests away from checkAndroidBpContents
Replaces a single call to checkAndroidBpContents(...) with separate
calls to check the versioned and unversioned Android.bp files.

Test: m nothing
Bug: 179354495
Change-Id: I270bf73909958d97b2f298e8d7f6d10a1f75ae71
2021-04-16 18:48:20 +01:00
Jingwen Chen 2f6a21e957 Export OsTypeList through a function.
This is a cleanup CL to make the OsTypeList accessor consistent with
ArchTypeList by creating a function that creates a copy of the
underlying slice.

Test: TH
Change-Id: I346c5c3937bbdb8a373e4d7135b896a5274cc1f1
2021-04-05 07:35:06 +00:00
Treehugger Robot 25c47a43a5 Merge ""current" is implicitly added to stubs.versions" 2021-04-01 04:53:55 +00:00
Jiyong Park d4a3a137ed "current" is implicitly added to stubs.versions
So far, when a library `libfoo` has `stubs.versions: ["10", "11"]`, then
`shared_libs: ["libfoo"]` is linked to the version 11 of the stub.

This requires the author of `libfoo` to manually update the property
whenever a new version is introduced. Otherwise, clients are not able
to use the newly added APIs because the latest stub is for an old
version.

This change eliminates the need for manual updating. "current" version
is always implicitly added to `stubs.versions`. It is added even when
nothing is set on the property, if `stubs.symbol_file` is set. i.e.

```
cc_library {
    name: "libfoo",
    stubs: {
        symbol_file: "libfoo.map.txt",
	// no versions: [...] needed
    },
}

cc_library {
    name: "a_client",
    shared_libs: ["libfoo"],
    apex_available: ["myapex"],
    min_sdk_version: "29",
}

apex {
    name: "myapex",
    native_shared_libraries: ["a_client"],
    min_sdk_version: "29",
}
```

`a_client` links to the "current" stub of `libfoo` that has all symbols
shown in the map file.

Note that, above doesn't mean that the client has unlimited access to
APIs that are introduced even after the min_sdk_version of the client
(29 in this example). The use of such APIs still has to be guarded with
`__builtin_available` check.

Bug: N/A
Test: m
Change-Id: I70bb1600c18e74d36c6b24c3569d2149f02aaf96
2021-04-01 09:58:53 +09:00
Paul Duffin 55e740e9a6 Remove varargs from RunTest(t *testing.T)
Use GroupFixturePreparers instead.

Bug: 182885307
Test: m nothing
Change-Id: Iaedb0ddc9d6a704f4d41363e705f3025a1291dc8
2021-03-31 16:03:59 +01:00
Colin Cross 045ed0f684 Merge changes I71a83e3a,I66101c0c,Ie387c8c4,Iea742e75
* changes:
  Strengthen metalava sandbox support using sbox
  Move metalava's output files into a subdirectory
  Fix lint warnings in droidstubs.go
  Split droidstubs out of droiddoc.go
2021-03-26 15:14:53 +00:00
Colin Cross cb77f75aae Move metalava's output files into a subdirectory
Move all of the output files of the metalava rule into a single
subdirectory to make it compatible with sandboxing in sbox.

Test: TestDroidstubs
Change-Id: I66101c0c224dee702c8175e61c12cc9cd1aa8b93
2021-03-25 11:11:36 -07:00
Paul Duffin bb9ff5108b Filter duplicate modules in platform_compat_config_singleton
Previously, unpacking a snapshot containing a
prebuilt_platform_compat_config into a source build would cause build
failure because of duplicate ids because the singleton would collate
ids from both prebuilts (versioned and unversioned) and source.

This change filters out versioned prebuilts and only uses prebuilts
that are preferred and source modules that have not been replaced by a
prebuilt.

Bug: 182402754
Test: m nothing
Change-Id: Idacbb34444e5156370df70bf88c6e8a7e2d67890
2021-03-25 12:53:25 +00:00
Paul Duffin db462dd987 Disallow non-existent paths in sdk package
Test behavior was changed a while ago so that tests by default ignore
non-existent source paths (unless they explicitly check for/rely on
them). Prior to that CheckSnapshot() could detect when files were
missing from the snapshot but it no longer can.

This change disallows non-existent source files in all the sdk tests
which means that they are disallowed when processing the snapshots as
they use the same preparers as were used to process the sources.

This caused a test failure which has been temporarily ignored and has
a TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I969d8515d20ef5ae515f2b5f93d8ed4e4f8ede75
2021-03-25 12:53:22 +00:00
Paul Duffin 1822a0a371 Improve sdk snapshot testing
The sdk produces snapshots that are expected to be unpacked in one of
the Android repos. Often that can lead to issues due to conflicts
between the source and prebuilts. This change attempts to avoid those
conflicts by testing the different ways that those files can be used.

With the existing test to cover adding the snapshot
This change will cause the sdk tests to check the following:
1) Snapshot on its own (already done).
2) Snapshot plus original source where the original source is
   preferred.
3) Snapshot plus original source where the snapshot is preferred.

It also adds the ability for tests to provide their own custom checkers
to verify the result of each of the previous tests.

This change reveals a number of bugs already present. Rather than
attempt to fix them this change adds the ability to specify error
handlers for the two cases that mix source and snapshot to allow those
errors to be temporarily ignored while allowing the majority of the
tests to benefit from this improvement. Each of those failures has a
TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I105233195074dbe7a6422b6dfc5486e74398ea15
2021-03-25 12:50:18 +00:00
Paul Duffin 76e5c8a37f Convert test that disallows non existent paths to use fixtures
This change needed to add some additional files to the registered
files for PrepareForTestWithJavaDefaultModules because otherwise they
would fail when "TestAllowNonExistentPaths = false". Those files were
being added by the TestJavaLintRequiresCustomLintFileToExist (albeit in
some cases in different locations to that required by the default
modules but as the files are needed by the modules defined in
PrepareForTestWithJavaDefaultModules they should be defined in it.

A couple of other places also provided some files so moving them into
PrepareForTestWithJavaDefaultModules caused some conflicts which needed
to be resolved.

Bug: 183184375
Test: m nothing
Change-Id: I76ce9f1673c1c1c4000635b76b8377d582224bf1
2021-03-24 22:08:05 +00:00
Paul Duffin ac94726ef2 Remove testSdkWithJava
Bug: 181070625
Test: m nothing
Change-Id: I9ef906a386cc87f69b166ec88e0b6c7388c3d06a
2021-03-24 15:47:28 +00:00
Paul Duffin 9ec86b14e7 Convert java_sdk_test.go tests to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I6a1fabdd2c8385e5fbaef6985047f9d0bdceb209
2021-03-24 15:47:28 +00:00
Paul Duffin 001b2342f7 Add platform_compat_config to sdk
Bug: 182402754
Test: m nothing
Change-Id: Ife3f4f64fc116d62eb7c3cc10c50e00f19d1d81c
2021-03-22 19:13:40 +00:00
Paul Duffin 04b4a19fe6 Add TestBasicSdkWithBootImage
Added to reproduce the conditions that lead to the error reported in
the bug so they can be fixed. There were a number of issues that were
fixed in previous changes and this test verifies that they have been
fixed.

Bug: 182992071
Test: m nothing
Change-Id: I2197899b284a99973e698db314b15812f602b141
2021-03-22 19:13:40 +00:00
Paul Duffin 4a1d451405 Convert boot_image_sdk_test.go to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I8c9cb399b15d04ec8465a3b3bb4b43561aef46fd
2021-03-22 19:13:22 +00:00
Paul Duffin cf3ee2f87e Allow sdk package tests to optimize their test setup
Extracts the setup for apex from prepareForSdkTest to allow it to be
reused without using all of it. That will allow tests to optimize their
test setup.

Bug: 182638834
Test: m nothing
Change-Id: I2056103b15c2737a616ee29ff890c6af0722e6d2
2021-03-22 18:31:53 +00:00
Paul Duffin c93c98e315 Use test fixtures in CheckSnapshot()
Using the preparer(s) that were used to run the test to verify the
integrity of the generated snapshot ensures that it will be verified in
the same environment as the snapshot was generated. This ensures that
as sdk tests are migrated to use fixtures that are optimized for each
test that they will benefit from those optimizations when checking the
snapshot.

Bug: 183184375
Test: m nothing
Change-Id: I62b383f9a1d9a77d1cabb101d9d1e4a976170fe3
2021-03-22 18:31:53 +00:00
Paul Duffin 89648f98fa Remove usages of FixtureFactory from misc packages
These packages have already been migrated to use per test build
directory so have no need for a FixtureFactory.

Bug: 183235980
Test: m nothing
Change-Id: I667d1d992caaf0f615de91f89efdae11c44986c2
2021-03-22 18:31:53 +00:00
Paul Duffin 6bef6fee3c Merge "Ensure that DepIsInSameApex is not called for ExcludeFromApexContentsTag" 2021-03-22 18:30:10 +00:00
Paul Duffin c737facd2b Merge "Replace AssertPanic with AssertPanicMessageContains" 2021-03-22 08:58:28 +00:00
Paul Duffin 4c3e8e2d67 Ensure that DepIsInSameApex is not called for ExcludeFromApexContentsTag
The ExcludeFromApexContentsTag marker interface was added to avoid
every implementation of DepIsInSameApex() from having to deal with the
special tags, like PrebuiltDepTag. Unfortunately, when adding that
not all calls to DepIsInSameApex() were protected which meant that the
BootImageModule, which panics if it doesn't recognize a tag, was
causing failures. This change documents the need and improves the
consistency.

A follow up change will add a test for this.

Bug: 182992071
Test: m nothing
Change-Id: If0bf9a7447ebf7a0bb0c88e91951a7220d4af45c
2021-03-22 08:43:55 +00:00
Paul Duffin d14cbc1e69 Add prepareForSdkTestWithJava
Bug: 182638834
Test: m nothing
Change-Id: I86cb08a640c7e39c1eaaf54926699f32a32ccc9b
2021-03-21 11:01:13 +00:00
Paul Duffin 9f4b3bbb7c Replace AssertPanic with AssertPanicMessageContains
Bug: 182885307
Test: m nothing
Change-Id: Idffa314285c90080796cc3df391de9c314eaa422
2021-03-20 12:08:51 +00:00