Commit Graph

65 Commits

Author SHA1 Message Date
Martin Stjernholm 8410eb1551 Merge "Do not allow duplicate deapexer dependencies." into sc-dev 2021-07-12 09:01:43 +00:00
Paul Duffin 6e89b54780 Generate boot images for host from prebuilts
Previously, when building from prebuilts boot no rules were created to
produce the boot image files for the host, i.e. the OS on which the
build was running. That caused problems with checkbuilds. No rules were
produced as there was no host variant of a prebuilt apex to provide
them.

This change restructures the code to allow the prebuilt bootclasspath
fragment to build the host variants of the files from the dex files
provided by the prebuilt APEX. The generated files will not be the same
as they would be if built from source as there is no boot image profile
to use but it should be sufficient to satisfy the checkbuild target and
allow for host side testing.

Bug: 192575099
Test: m SOONG_CONFIG_art_module_source_build=false droid dist checkbuild
Merged-In: I6af00f19bb71aa18dd462f5eac6aa38e3e721023
Change-Id: I6af00f19bb71aa18dd462f5eac6aa38e3e721023
(cherry picked from commit a56be7d7815ad164cdd854f8fc6b1cbc3bbf1918)
2021-07-06 22:20:24 +01:00
Paul Duffin d564c32818 Switch boot image generation to use GetGlobalSoongConfig(ctx)
Previously, boot image generation used GetCachedGlobalSoongConfig(ctx)
in order to get access to the GlobalSoongConfig. That required that
some other part of the code had called GetGlobalSoongConfig(ctx) to
initialize the cached value. That was left over from when the boot
image generation was done in a singleton which could not call
GetGlobalSoongConfig(ctx) directly. That is no longer true.

This change switches the uses of GetCachedGlobalSoongConfig(ctx) to
GetGlobalSoongConfig(ctx) and removes the now unnecessary call to
GetGlobalSoongConfig(ctx) from outside the functions.

Bug: 192575099
Test: m nothing
Merged-In: I34b7b1526d072d8b09fda7caa96e381695888e16
Change-Id: I34b7b1526d072d8b09fda7caa96e381695888e16
(cherry picked from commit 8fc51a8eb6ca20dbb4b65822123201568bb297f1)
2021-07-06 22:20:24 +01:00
Paul Duffin bb2e205603 Generate boot zip file from prebuilt_bootclasspath_fragment
Previously, the boot zip file, containing all the boot image files for
all the supported architectures, was only created from source. It was
not created when building from a prebuilt_bootclasspath_fragment. That
lead to build failures when building from ART prebuilts.

This change pulls the boot zip file creation out so that it can be done
for both source and prebuilt bootclasspath_fragment modules as well as
for the platform_bootclasspath module.

Bug: 192575099
Test: m out/target/product/generic_arm64/boot.zip
      m SOONG_CONFIG_art_module_source_build=false out/target/product/generic_arm64/boot.zip
      - Compare the output of the first command from before the change
        with the output from them both after and confirm that when the
        ART prebuilts are up to date with the source that there are no
        differences.
Merged-In: Ie7dd5e2ca4a865d06fd9ebf87320cf68c4d05bc3
Change-Id: Ie7dd5e2ca4a865d06fd9ebf87320cf68c4d05bc3
(cherry picked from commit 56afb27fb099cb79c1537c661628db1776f1fcc3)
2021-07-06 13:34:26 +01:00
Paul Duffin aba5275676 Use both module name and stem name to filter updatable boot jars
Sometimes the stem property is set to change both the installed file
name and the name use to filter the configured module list, e.g. when
adding a test library to replace the standard library, e.g. test_foo
instead of foo.

Sometimes it is used just to change the installed file name.

This change uses both to filter the updatable boot jars and not just
the stem.

Bug: 180105615
Test: m nothing
Merged-In: I6c459fc3597b1e4f062bc9a4e52843305b538c5f
Change-Id: I6c459fc3597b1e4f062bc9a4e52843305b538c5f
(cherry picked from commit 56c93e899a355311463d41e988c9ba511af14278)
2021-07-06 13:34:25 +01:00
Martin Stjernholm 95994067b1 Do not allow duplicate deapexer dependencies.
Without these errors, the last encountered deapexer would silently be
used, and we wouldn't know if it was taken from the prebuilt APEX that
actually get installed in the system image.

With this error check there may be only one enabled prebuilt_apex or
apex_set for each apex_name (which defaults to the module name). E.g.
if there are both prebuilt com.android.foo and com.google.android.foo,
it is necessary to disable one of them in the .bp file.

Merged-In is set from https://r.android.com/1745454, a change that has
gone into AOSP and internal master, as well as sc-dev-plus-aosp, but
specifically not sc-dev. This change cannot merge into sc-dev-plus-aosp
and others, because they may or may not have the com.google.android.art
prebuilt APEX present depending on manifest, and with this check
exactly one of com.android.art and com.google.android.art prebuilts has
to be present and enabled. It'll be cherry-picked to AOSP along with a
full fix for b/192006406, when it can be safely enabled everywhere.

Test: m nothing SOONG_CONFIG_art_module_source_build=false
Test: m nothing SOONG_CONFIG_art_module_source_build=true
Test: m nothing
  with enabled:true for the com.android.art prebuilt APEX - check that
  it fails with an "ambiguous duplicate deapexer" error
Bug: 192006406
Bug: 192542393
Change-Id: I44566fd26b12f82a8a67fe4a69e56303460756d0
Merged-In: Id2410b4e38a78ec2146a42298840954381a7c472
2021-07-02 00:31:01 +01:00
Paul Duffin dad8880399 Retry: Make bootclasspath_fragments always perform hidden API processing
Previously, bootclasspath_fragment modules would only perform hidden
API processing if they provided some stub libraries and fragments. That
was needed because the bootclasspath_fragment modules were added before
Soong supported hidden API processing on all the different modules and
before they all provided the necessary information that hidden API
processing required.

This change stops hidden API being conditional as it is no longer
required as it has been enabled on all existing bootclasspath_fragment
modules.

Bug: 179354495
Test: m nothing
Change-Id: Ibf81a7de63b888a3ebf445528326fa6101fdb1ba
2021-07-01 17:22:50 +01:00
Paul Duffin 71e98fa303 Revert "Make bootclasspath_fragments always perform hidden API processing"
This reverts commit 2b255d7f6b.

Reason for revert: build breakage http://b/192554546

Bug: 179354495
Change-Id: Id758af3f59bc83b02d8af4b1b83b3f279a208595
2021-07-01 15:27:27 +00:00
Paul Duffin 2b255d7f6b Make bootclasspath_fragments always perform hidden API processing
Previously, bootclasspath_fragment modules would only perform hidden
API processing if they provided some stub libraries and fragments. That
was needed because the bootclasspath_fragment modules were added before
Soong supported hidden API processing on all the different modules and
before they all provided the necessary information that hidden API
processing required.

This change stops hidden API being conditional as it is no longer
required as it has been enabled on all existing bootclasspath_fragment
modules.

Bug: 179354495
Test: m nothing
Change-Id: I0cbf11986adff1f2f967b96f86e6bfe0e9b8b1ef
2021-07-01 11:56:41 +01:00
Paul Duffin fb8f07bc0e Calculate widest stub dex jars per module
Previously, the stub dex jars for each HiddenAPIScope was created by
merging the stub dex jars provided by each module for that scope. Then
the widest stub dex jars were chosen. So, if module A provided public,
system and test stub dex jars and module B provided only public then
the stub dex jars for each scope would be:
* public -> A,B
* system -> A
* test -> A

So, the widest API scope for which there are stub dex jars is "test"
and so the widest stub dex jars would just come from module A and not
module B. So, when "hiddenapi list" is run for module C which depends
on modules A and B it only gets given stub dex jars for module A which
means that it cannot resolve all the types that C may use which can
lead to incorrect flags being generated.

This change does not merge the stub dex jars from each module together
and instead keeps them separate by module. The widest stub dex jars
list is constructed by asking each module in turn for their widest stub
dex jars. e.g. Given the above example we would have:

Module A:
* public
* system
* test <- widest

Module B:
* public <- widest

So, the widest stub dex jars will be A's test and B's public stub dex
jars.

Bug: 179354495
Test: m out/soong/hiddenapi-flags.csv
      - make sure that this does not change the file.
Merged-In: Ib137825ebffe94b2bf220732bae6077f7b7ac6db
Change-Id: Ib137825ebffe94b2bf220732bae6077f7b7ac6db
(cherry picked from commit 280a31aac39697a069f2f2f2ee471ffafb52d2a3)
2021-06-28 16:05:05 +01:00
Paul Duffin b3821fea38 Support hidden API processing for modules that use platform APIs
Previously, hidden API processing could only be done by those
bootclasspath_fragment modules that either did not depend on any other
fragments (e.g. art-bootclasspath-fragment) or only depended on APIs
provided by other fragments (e.g. i18n-bootclasspath-fragment). That
meant that modules like com.android.os.statsd-bootclasspath-fragment
that depended on APIs provided by parts of the platform which are not
yet part of another bootclasspath_fragment could not perform hidden
API processing.

This change adds support for a bootclasspath_fragment to specify the
additional stubs needed to perform hidden API processing. It adds a new
additional_stubs property that can be used to specify the additional
stub libraries.

Most bootclasspath_fragments that need to use the property will need
access to the APIs provided by the android-non-updatable.* libraries.
Rather than have each fragment explicitly specify the correct module
for each scope it treats "android-non-updatable" as if it was a
java_sdk_library that can provide different jars for each scope.
Soong will handle mapping that to the correct android-non-updatable.*
module.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv \
        out/soong/hiddenapi/hiddenapi-index.csv \
        out/soong/hiddenapi/hiddenapi-stub-flags.txt \
        out/soong/hiddenapi/hiddenapi-unsupported.csv
      - make sure that this change does not change the contents.
      m TARGET_BUILD_APPS=Calendar nothing
Merged-In: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
Change-Id: Ia8b79830ed0e6d42100de03d76b0c51b7f6c8ade
(cherry picked from commit 5cca7c44e51d0d08a5ea842d0f9870a772529dec)
2021-06-25 21:59:28 +01:00
Paul Duffin 05bbff90bd Add HiddenAPIScope to replace use of SdkKind
Previously, the hidden API processing used SdkKind to identify the API
scopes, e.g. public, system, etc. that are of interest for hidden API
processing. Unfortunately, there is a mismatch between the SdkKind and
what hidden API processing needs. e.g. SdkKind includes values that are
not used by hidden API processing and hidden API processing needs
additional API scope specific information not provided by SdkKind. The
apiScope struct used in sdk_library.go is also not a suitable
representation for similar reasons.

This change adds the HiddenAPIScope (following a similar approach as
apiScope) that rectifies that and uses it as a replacement for SdkKind
in most parts of the hidden API processing. The SdkKind is still used
for retrieving information from java_sdk_library[_import] modules.

Follow up changes will extend the HiddenAPIScope with more information.

Bug: 179354495
Test: m out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that this change has no effect on the generated flags.
Merged-In: I97968f58535121652852b8d25217aa288afd2bfd
Change-Id: I97968f58535121652852b8d25217aa288afd2bfd
(cherry picked from commit 31fad800a7a44ef2edda5d4051335f28d514139a)
2021-06-25 16:44:30 +01:00
Paul Duffin 034196d9fe Simplify deapexer support
Uses the apex relative path to the file as the identifier that is used
to obtain the path to the corresponding file extracted from the apex.
That is instead of a special constructed string id.

Bug: 177892522
Test: m nothing
Merged-In: I5dc77c8fb272bac289b8891d1eac801e541af1f5
Change-Id: I5dc77c8fb272bac289b8891d1eac801e541af1f5
(cherry picked from commit b4bbf2ca10cc8509e3ae0ab104e9e3b55861831b)
2021-06-21 10:54:22 +01:00
Paul Duffin 7db57e0a77 Remove javalib special case in createDeapexerModuleIfNeeded
Removes the special case in createDeapexerModuleIfNeeded for handling
java libraries and just get the Import and SdkLibraryImport module
types to implement RequiredFilesFromPrebuiltApex instead.

Bug: 177892522
Test: m nothing
Merged-In: I5cc341b5b4168b8eb196f72273a00d498de6856f
Change-Id: I5cc341b5b4168b8eb196f72273a00d498de6856f
(cherry picked from commit fef5500a766d1f515ff19038e8e0e8f606e07287)
2021-06-21 10:54:22 +01:00
Paul Duffin b508405e08 Export boot image files from prebuilt_apex/apex_set
Previously, the prebuilt art-bootclasspath-fragment did not provide any
boot image files. That meant it was impossible to build any module that
requires access to those files from prebuilts, e.g. any module that
needs to be dexpreopt-ed.

This change enables that module to retrieve those files from the
prebuilt_apex.

Bug: 177892522
Bug: 189298093
Test: m nothing
      m droid SOONG_CONFIG_art_module_source_build=false SKIP_BOOT_JARS_CHECK=true
      - the previous command does not work but this change does fix one
        of the issues reported.
Merged-In: I1d4d9545172d79282918130df6b9aa55471bffc1
Change-Id: I1d4d9545172d79282918130df6b9aa55471bffc1
(cherry picked from commit 5466a3699c550d1fa67066db48951b765156be4f)
2021-06-18 12:11:17 +01:00
Paul Duffin a0089ec78c Copy boot dex jars from prebuilt art-bootclasspath-fragment if preferred
Previously, the boot dex jars were only copied to the predefined
locations used by the build from the source art-bootclasspath-fragment
if it was preferred, otherwise no files were copied. That caused build
failures when attempting to build with ART prebuilts. This change
copies the files from prebuilts too.

Bug: 177892522
Bug: 189298093
Test: m nothing
      m droid SOONG_CONFIG_art_module_source_build=false SKIP_BOOT_JARS_CHECK=true
      - the previous command does not work but this change does fix one
        of the issues reported.
Merged-In: I35b37355170546daf6ecac2134d1ca9a73d0e3bc
Change-Id: I35b37355170546daf6ecac2134d1ca9a73d0e3bc
(cherry picked from commit ce918b0278628afe4de73b09f73bac777d545b7e)
2021-06-18 12:11:17 +01:00
Paul Duffin 360f8359f3 Cleanup bootclasspathFragmentInitContentsFromImage
Removes some unnecessary logic left over from previous changes.

Bug: 177892522
Test: m nothing
Merged-In: Id11e2ea8a647283af1b152b33757132e44dcd6c3
Change-Id: Id11e2ea8a647283af1b152b33757132e44dcd6c3
(cherry picked from commit 58e7586b3d8fffa63b869c421a36c69fdd30a264)
2021-06-18 12:11:17 +01:00
Paul Duffin c75bbcecdb Combine hidden API encoding with flag generation
Previously, the rules to perform hidden API encoding were generated
separately to the rules to perform hidden API flag generation. This
change combines them within the (renamed) produceHiddenAPIOutput()
method and makes the paths to the encoded dex files an output of the
generateHiddenAPIBuildActions method alongside the paths to the
generated flag files.

As encoded dex jars are now an output of the produceHiddenAPIOutput()
method which is implemented for both prebuilts and source
bootclasspath_fragment modules that necessitated the prebuilt also
providing paths to encoded dex files. That in turn required updates to
some of the tests to provide dex files from prebuilt_apex modules.

Similarly, as the produceHiddenAPIOutput() method may not be called
for some bootclasspath_fragment modules as they do not yet provide all
the information needed to perform hidden API encoding then it is
necessary to extract the encoded dex files produced by the modules
themselves. That also required a few changes to tests that did not
previously provide dex files.

Bug: 177892522
Test: m com.android.art
      - check that this change does not change the contents of the apex
        file, i.e. is byte-for-byte identical.
Merged-In: I60996a34d06ed1c87ed244ab3509621999ad86ec
Change-Id: I60996a34d06ed1c87ed244ab3509621999ad86ec
(cherry picked from commit e521881bd45de0306bc2e80d5c746ae361d6ffe2)
2021-06-18 12:11:17 +01:00
Paul Duffin 98ea0d4926 Avoid passing around []hiddenAPIModule
Previously, an []android.Module was converted to an []hiddenAPIModule
that was then used to retrieve boot dex jars. That was ok when
obtaining the dex jar files from source modules for
bootclasspath_fragment but does not work well for other use cases as
it would require doing that conversion in multiple places.

This change pushes the use of hiddenAPIModule down to the methods that
retrieve information from it which makes the methods more flexible and
easier to reuse.

Bug: 177892522
Test: m nothing
Change-Id: Ib84aaf03d8f5a63b48232036fe4589646fc23352
Merged-In: Ib84aaf03d8f5a63b48232036fe4589646fc23352
(cherry picked from commit dd5993f6d41efa932c06afef71c40af446c85b4e)
2021-06-16 11:07:58 +01:00
Paul Duffin 3e2db5c10b Make copyBootJarsToPredefinedLocations simpler and less fragile
Previously, copyBootJarsToPredefinedLocations relied on all its
parameters having the same length and the same order. That made it
quite fragile as changes to one of the parameters without corresponding
changes to the other would cause failures. It also combined the
retrieval of the boot dex jars from the modules, handling of missing
boot dex jar files and the generation of the rules to copy the files.

This change separates the retrieval of boot dex jars and handling of
missing files from the copying of those files while at the same time
making the function less fragile by replacing the three ordered
parameters with two maps that shared common keys.

Bug: 179354495
Test: m nothing
Merged-In: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
Change-Id: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
(cherry picked from commit 5f148ca7cf2e9a9478922577b7ed70b1caacb55e)
2021-06-16 03:06:35 +01:00
Paul Duffin c45a86a65d Export hidden api related types and fields
This will export some hidden api related types and fields so they can
be used from outside the java package. This is needed to allow a follow
up change to move the TestPlatformBootclasspath_Fragments from the java
to the apex package.

Bug: 179354495
Test: m nothing
Merged-In: Ib69eea9d79cc83b8e3fc29919a29f071e1ec17b5
Change-Id: Ib69eea9d79cc83b8e3fc29919a29f071e1ec17b5
(cherry picked from commit 524c82c01acbd1f45a401d3444354cd69dd342e3)
2021-06-16 03:06:33 +01:00
satayev b75cd4ed7a Add exemption for framework-sdkextensions-classpaths boot jar.
As long as we have boot jars defined in make, we should try to keep
global.UpdatableBootJars as a source of truth when it comes to classpath
configuration. However, for testing purposes there is a need to add
java libraries to bootclasspath that aren't present in
PRODUCT_UPDATABLE_BOOT_JARS variable.

Ideally, we would unconditionally add all contents to the config for
test apexes, however, I don't fully understand apex merging to support
it.

Bug: 180105615
Test: atest sdkextensions_classpaths_e2e_tests
Change-Id: I552f93289c27b3713e75dcda1faed909b0bfcef2
Merged-In: I552f93289c27b3713e75dcda1faed909b0bfcef2
(cherry picked from commit 1b75a3ce623357020ec6f1446edff974a0dc283a)
2021-06-08 15:29:19 +01:00
Colin Cross cbb2b8172d Treat java libraries in classpath fragments as directly in apex
Coverage is applied to java libraries that are directly in an apex.
Mark java libraries that are in an apex through a bootclasspath_fragment
or a systemserverclasspath_fragment as directly in the apex by
implementing CopyDirectlyInAnyApexTag on the dependency tags used for
their contents.

Bug: 183759446
Test: TestApexJavaCoverage
Change-Id: I0116f5f415083b5194000988cb257454ef115200
Merged-In: I0116f5f415083b5194000988cb257454ef115200
(cherry picked from commit c33e5216f1eb3f837dee89cc65208eff058e9c5e)
2021-05-27 13:14:33 -07:00
satayev 07753d8467 Use stem when filtering boot jars.
For testing purposes, a boot jar may be provided by a test java_library
that has a different content name, but sets "stem" property to match
the original java_library.

Given that Stem() returns either the property value or module name,
it is safe to replace all content names by their stems.

Bug: 180105615
Test: atest CtsClasspathsTestCases sdkextensions_e2e_tests
Change-Id: Ic519ffa0c5b616abddf15b41c934421dfac2e78a
2021-05-25 21:16:34 +01:00
Artur Satayev 5cb569794e Revert "Partial Revert "Populate individual classpath_fragments'..."
Revert submission 14717811-revert-populate-platform-bootclasspath

Reason for revert: retry with a fix
Reverted Changes:
Ib58cd0211:Revert "Add bootclasspath_fragments to platform-bo...
I13b622d6c:Partial Revert "Populate individual classpath_frag...

Bug: 180105615
Test: atest sdkextensions_e2e_tests

Change-Id: I7b6b6b980a4c6430a70394e85222f3b35c4efd5f
2021-05-25 21:00:31 +01:00
Anton Hansson bbe2d3db4a Merge "Partial Revert "Populate individual classpath_fragments' classpaths.prot..."" into sc-dev 2021-05-25 14:18:42 +00:00
satayev a3fc817771 Partial Revert "Populate individual classpath_fragments' classpaths.prot..."
Reason for revert: test breakage b/189114287
Bug: 180105615
Bug: 189114287
Test: atest sdkextensions_e2e_tests

Change-Id: I13b622d6c61ea392bfcc8a40535045c87fa3a7b5
2021-05-25 15:12:23 +01:00
Paul Duffin 588e22a6bc Tighten bootclasspath_fragment property validation
Previously, due to legacy reasons, the property validation did not
require a contents property and allowed the image_name to be either
"art" or "boot". Those reasons no longer apply and so this change
requires a contents property and only allows the image_name to be set
to "art" if specified.

Bug: 177892522
Test: m nothing
Merged-In: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
Change-Id: I8855d6e5365ef0b55490e90e7b6c0081cf070ee5
(cherry picked from commit 8018e50ddb141dea7891918c4178a7a394a788d8)
2021-05-25 09:30:20 +01:00
Paul Duffin 0788cd6038 Only export boot image files to APEX if they are actually created
Previously, the generation of the rules to create the boot image files
was separate from the code to provide those files to the APEX which
meant that it was possible for the APEX to try and use files that had
no rules to create them.

This changes avoids that by only exporting the files once the rules
have been created.

This necessitated a few changes to tests that were relying on the
previous behavior. Including removing completely the test that used an
image_name: "boot" as that is no longer a valid configuration name as
its functionality has been replaced by platform_bootclasspath.

A follow up change will make the validation of the properties of
bootclasspath_fragment stricter to prevent "boot" being used as the
image_name.

Bug: 177892522
Bug: 188680624
Test: lunch qemu_trusty_arm64-userdebug
      m droid dist
      - verify that before this change it fails and after this change
        it works.
      m com.android.art
      - verify that this change does not change the APEX contents with
        either qemu_trusty_arm64-userdebug or aosp_arm64-userdebug
Merged-In: I0497a151eb0731cbe6a1a7e7bbbb1e4dda75898f
Change-Id: I0497a151eb0731cbe6a1a7e7bbbb1e4dda75898f
(cherry picked from commit 58e0e769c9b1f4dbad95acfd1792dd91e716e456)
2021-05-25 09:30:20 +01:00
Paul Duffin 67c1e57551 Support hidden API processing for fragments with dependencies
Previously, a bootclasspath_fragment that depended on classes provided
by another bootclasspath_fragment did not support hidden API processing
as it would not supply information about those dependencies.

This change adds support for that as follows. Each fragment:
1. Exports the transitive sets of stub dex jars for each of the public,
   system, test and core_platform APIs (where relevant).
2. Adds dependencies onto its dependent fragments.
3. Retrieves the API stubs dex jars from its dependent fragments and
   passes them to the "hiddenapi list" tool which will use them to
   resolve dependencies but will not output them to the generated
   flags.

Once the flags are generated the existing encoding functionality
encodes the flags into the dex files of the bootclasspath_fragment's
content modules which are then packaged into the apex.

Bug: 179354495
Test: m com.android.sdkext
      - verify that this does not change the contents of the apex files
Merged-In: I3e82a6dbb437f1e417e5d7e25aeb212e378603d0
Change-Id: I3e82a6dbb437f1e417e5d7e25aeb212e378603d0
(cherry picked from commit f1b358cb5764039ff7502e9f73ad0abfdc91c7aa)
2021-05-24 20:26:02 +01:00
Paul Duffin 6a58cc94d1 Remove bootclasspathApiInfo
Previously, the bootclasspathApiInfo was only used for tests and follow
up changes will need to provide the stub dex jars. This change moves
the stubJarsByKind into HiddenAPIInfo and removes bootclasspathApiInfo
and the corresponding provider.

Bug: 179354495
Test: m nothing
Merged-In: I5459c56de561c053ed671dc9d5cb3ee4820c0ee8
Change-Id: I5459c56de561c053ed671dc9d5cb3ee4820c0ee8
(cherry picked from commit 18cf19745e2fffb184a7b42f50609ea98d4adb1d)
2021-05-24 18:20:09 +01:00
Paul Duffin 699a00489e Fix hidden API flags in com.android.i18n
Change 70cfdff3da2ea07cd5cb7f7b91474f6fa0c248e5 changed the hidden API
flags in com.android.i18n as it stopped the i18n-bootclasspath-fragment
from making the hidden API flag files available for use by
platform-bootclasspath.

This change fixes that by exporting the flag files even if hidden API
flag generation is skipped.

Bug: 179354495
Test: m com.android.i18 out/soong/hiddenapi/hiddenapi-flags.csv
      - make sure that the flags in
        packages/modules/RuntimeI18n/apex/hiddenapi/hiddenapi-max-target-o-low-priority.txt
        are reflected in the core-icu4j dex files in the apex.
Merged-In: I9b5c7c74bd996ab447bc0e0452da5fd49191a35d
Change-Id: I9b5c7c74bd996ab447bc0e0452da5fd49191a35d
(cherry picked from commit 62370923911827d0a9cf6103e47652f40ca2cd25)
2021-05-24 18:20:08 +01:00
Paul Duffin 71955b4bc3 Rename hiddenAPIFlagFileInfo to HiddenAPIInfo
This reflects that it has expanded from its initial purpose to include
more than just flag files. It is exported for use in tests in other
packages.

Bug: 179354495
Test: m nothing
Merged-In: I9f780b20e18ce3a774e4aa04a276463070a64c34
Change-Id: I9f780b20e18ce3a774e4aa04a276463070a64c34
(cherry picked from commit af99afa919790cc76a9b1a1947d473065ecb65a5)
2021-05-24 18:20:08 +01:00
Paul Duffin 5aadef8ab5 Separate input to flag generation from hiddenAPIFlagFileInfo
Encapsulating the information needed by hidden API processing in a
struct makes it easy to add additional information in future and allows
the code to populate that struct from various different sources to be
grouped together.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Merged-In: I53805737dff36a3ae87aca5aad51cf46ae1361fe
Change-Id: I53805737dff36a3ae87aca5aad51cf46ae1361fe
(cherry picked from commit 1352f7c4715d3d311a2644b9337e4b1028d82cb0)
2021-05-24 18:20:08 +01:00
Paul Duffin cad63842d2 Separate output of flag generation from hiddenAPIFlagFileInfo
HiddenAPIFlagOutput encapsulates the paths to the files produced by the
hidden API flag generation of a single bootclasspath_fragment. It is
returned from hidden API flag generation and is embedded within the
hiddenAPIFlagFileInfo so they can be passed to other modules.

Unlike the fields it replaces in hiddenAPIFlagFileInfo the fields in
HiddenAPIFlagOutput are of type Path not Paths which makes it easier to
use.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Merged-In: I7373ba1001cac3a75eb54a23e62fa52f5013ee7f
Change-Id: I7373ba1001cac3a75eb54a23e62fa52f5013ee7f
(cherry picked from commit 1e6f5c4e633bab71d82eace7bad0f52494ef1839)
2021-05-24 18:20:07 +01:00
Paul Duffin 9e3b906581 Separate monolithic hidden API processing from hiddenAPIFlagFileInfo
The hiddenAPIFlagFileInfo was being used for both the input and output
of bootclasspath_fragment and platform_bootclasspath and also to pass
information around to various hidden API rule methods. Supporting
multiple different uses in this way made it hard to reason about.

This change creates a separate structure for use by the
platform_bootclasspath. Follow up changes will split out other
functionality into separate types.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Merged-In: Ia5c5f65ae5645486c42819c669a8601588217f88
Change-Id: Ia5c5f65ae5645486c42819c669a8601588217f88
(cherry picked from commit 438eb57a2744b9b0bd38a5526e67cacf43c42b31)
2021-05-24 18:20:07 +01:00
Paul Duffin b40610ad00 Perform hidden API encoding in bootclasspath_fragment
Previously, the apex content info was populated with hidden API encoded
dex jars retrieved directly from the java module. This change retrieves
the unencoded dex jars from the java module, encodes them and then
stores the result in the apex content info.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Merged-In: Ib1b6eb8b62ac50e03b9e0d07c877ca70bb6f6d25
Change-Id: Ib1b6eb8b62ac50e03b9e0d07c877ca70bb6f6d25
(cherry picked from commit 54c98f5b4af1b47b922bd76a4dbf0aca50c2a453)
2021-05-23 18:41:47 +01:00
Paul Duffin 7555fcbab3 Store dex jar paths in bootclasspath_fragment's apex content info
Previously, the DexBootJarPathForContentModule(module) simply called
directly through to the module to retrieve the dex jar path. This
change changes it so the bootclasspath_fragment retrieves the dex
jars from the module and stores them in the info structure for this
method to retrieve directly.

This makes it easier for the bootclasspath_fragment to stop retrieving
hidden API encoded dex jars from the module and perform the encoding
itself.

Bug: 179354495
Test: m com.android.art com.android.ipsec com.android.os.statsd com.android.conscrypt
      - verify that this does not change the contents of the apex files
Merged-In: Ic79dea080f10f4017f1a75d6d1fb5a3bfe04c2ce
Change-Id: Ic79dea080f10f4017f1a75d6d1fb5a3bfe04c2ce
(cherry picked from commit 1a8010a24171c4ac1928b659f48dc680ee8b0353)
2021-05-21 09:34:53 +01:00
satayev 95bfbb169f Populate individual classpath_fragments' classpaths.proto configs.
To avoid duplicates on *CLASSPATH environ variables at runtime, remove
split entries from platform-*classpath, i.e. all updatable jars that
have their own classpath fragments should not appear in the
platform-*classpath's classpaths.proto config.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
2021-05-20 19:07:29 +01:00
satayev 72ede0fac5 Move classpaths.proto related info into a separate provider.
The new info struct can be easily shared with systemserverclasspath
fragments.

Bug: 180105615
Test: m nothing
Merged-In: I9986e64fdf19f4168da63c156de3dc9bcafac8d8
Change-Id: I9986e64fdf19f4168da63c156de3dc9bcafac8d8
(cherry picked from commit 14e49130bbeba7d222c8851e2f59710ac0f6eb71)
2021-05-18 10:37:51 +01:00
Paul Duffin 460952c7f4 Add temporary restriction on hidden API processing
Hidden API processing of a bootclasspath_fragment requires the fragment
provides information, such as dependencies on other fragments and flag
files to override the default flags. Failing to do so will cause hidden
API generation to either fail or to generate different flags to that
generated by the hidden API processing done by platform_bootclasspath
which will cause the build to fail.

Previously, this was handled by only performing hidden API processing
for those modules that provide stub libs and relied on there only being
bootclasspath_fragments defined for ART (which already supports hidden
API processing), and Conscrypt and I18n neither of which provide stubs.
Unfortunately, that can no longer be relied upon due to a couple of
recent changes:
1. A java_sdk_library in a bootclasspath_fragment's content property is
   automatically treated a stub library. That avoids duplication for
   most bootclasspath_fragments that provide the implementation and
   stub libraries through the same java_sdk_library. It does not affect
   either ART, conscrypt or i18n as they all define the implementation
   separately to the stubs.

2. bootclasspath_fragment modules have been defined for a number of
   android modules as they are needed for reasons other than hidden API
   processing.

In combination this meant that rules to perform hidden API processing
were being created but they were not currently being used which is
good because they fail.

However, adding the fragment to the platform-bootclasspath will cause
those rules to be used as the platform_bootclasspath module performs a
consistency check on the hidden API flags generated by each of the
fragments to ensure that they are consistent with those it generates
itself.

The dynamic bootclasspath work will need to add fragments to the
platform_bootclasspath and without this change that would be blocked
until all fragments had been switched to generating hidden API flags
properly.

This change adds a new fragments property to the bootclasspath and
disables hidden API processing for everything other than ART and tests
if the fragments property or stub libs is empty.

Bug: 179354495
Test: - Before making this change.
      - Add com.android.os.statds-bootclasspath-fragment to platform-bootclasspath.fragments
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - hidden API processing fails in statsd as it cannot find java.lang.Object.
      - After making this change, run the above command again and it should pass.

Merged-In: Ifbb362f8fcfb2c06595fbd5ae39421b536e329ef
Change-Id: Ifbb362f8fcfb2c06595fbd5ae39421b536e329ef
(cherry picked from commit 70cfdff3da2ea07cd5cb7f7b91474f6fa0c248e5)
2021-05-17 21:30:03 +01:00
Paul Duffin 21f62ef867 Merge "Generate monolithic hidden API files direct from class jars" 2021-05-14 18:52:11 +00:00
Paul Duffin 79fd3d728c Allow shared libraries on bootclasspath
A previous change treated this as an error in order to try and detect
issues with misconfigured java_sdk_library modules and/or
bootclasspath_fragment modules but unfortunately this is not always an
error, e.g. when migrating a library that was a shared library to the
bootclasspath.

This change stops treating that as an error.

Bug: 179354495
Test: m nothing
Change-Id: I4a833ab5f4caf86c6cd340090fc65d2c2f141512
2021-05-14 16:14:17 +01:00
Paul Duffin 537ea3d04c Generate monolithic hidden API files direct from class jars
Previously, the monolithic hidden API files, e.g. hiddenapi-index.csv
file, were generated in two steps. First, each module created its own
files using the information in its class jars. Then, the monolithic
files were created by merging those module specific files into a larger
file.

This change switches to generating the monolithic files directly from
the class jar files and bypassing the intermediate files.

In order to ensure that this change did not change the monolithic files
it is necessary for the hiddenapi-metadata.csv to go through a
reformatting step. Hopefully, this will be able to be removed in a
follow up change.

Bug: 179354495
Test: verified that the monolithic out/soong/hiddenapi/... files are
      unchanged by this change
Change-Id: I5a78e747516014b7c0f402a4b4431b14be6a84b2
2021-05-14 15:57:04 +01:00
Paul Duffin 34827d4c0e Use java_sdk_library in bootclasspath_fragment contents as stubs
A java_sdk_library specified in the bootclasspath_fragment contents
must be providing APIs for the bootclasspath_fragment and so must be
treated as if they were specified in the stub_libs. This avoids having
to specify them in both contents and stub_libs.

Bug: 179354495
Test: m nothing
Change-Id: I535065ee1a79b439e2676f35e06a75d4626adcaf
2021-05-14 01:49:19 +01:00
Paul Duffin 2fef136885 Generate hidden API flags for a bootclasspath_fragment
This change adds support for generating the hidden API flags for the
contents of a bootclasspath_fragment. Currently, it will only work for
the art-bootclasspath-fragment as it has no support for creating
dependencies between bootclasspath_fragment modules which will be
needed for handling any other bootclasspath_fragment.

The hidden API flag generation added by this change is completely
separate to the normal hidden API processing and is not as yet encoded
in dex jars so will have no effect on the runtime.

The generated files are provided for use by other modules and copied
into the sdk snapshot. That is needed to allow the build to verify that
the hidden API flags generated by the individual bootclasspath_fragment
modules are consistent with the flags generated for the whole
bootclasspath, whether building from source or prebuilts.

Bug: 179354495
Test: m art-module-sdk
      m out/soong/.intermediates/art/build/boot/art-bootclasspath-fragment/android_common_apex10000/modular-hiddenapi/all-flags.csv
      m out/soong/hiddenapi/hiddenapi-flags.csv
      - test that the former file is a subset of the latter and that
        where they overlap they are identical.
Change-Id: Ie27303e2960953db1b7abe95510e3bca4411b09a
2021-05-14 01:48:51 +01:00
Paul Duffin f4600f6e6a Disallow shared libraries in bootclasspath_fragment contents
Bug: 177892522
Test: m nothing
Change-Id: I78c8ef8664ec1eb0fe3456a2de2cb956162ca0da
2021-05-14 00:39:24 +01:00
Paul Duffin a10bd3c127 Add java_sdk_library in bootclasspath_fragment contents to sdk
A java_library specified in a bootclasspath_fragment's contents
property will be automatically added to the sdk containing that
bootclasspath_fragment. Previously, if that was attempted with a
java_sdk_library it would be added to the sdk as if it was a normal
java_boot_libs which would prevent the sdk from containing the
API specific artifact such as current.txt files and stub libraries
and sources.

This change fixes that and adds a java_sdk_library as a java_sdk_libs
module.

Bug: 177892522
Test: m nothing
Change-Id: Ided57b846ce5b8940c7e898c786fd77602582ea2
2021-05-13 21:25:42 +01:00
Paul Duffin 2fc82ad044 Build boot images in bootclasspath_fragment/platform_bootclasspath
Moves the building of boot images from the dexpreopt_bootjars singleton
to the bootclasspath_fragment and platform_bootclasspath.

The art boot image is generated by the art-bootclasspath-fragment
module and the framework boot image by the platform-bootclasspath
module.

This does temporarly duplicate the generation of an identical boot
profile for each image. As part of the work to modularize the boot
image profile each image will have its own custom default boot profile.

Bug: 177892522
Bug: 186455808
Test: m droid and TreeHugger
Change-Id: I23cf05ec7648749b21c7cf6fcba282b46649a981
2021-05-12 21:51:27 +01:00
Paul Duffin 7ebebfd5f8 Move copying of dex files from dexpreopt_bootjars singleton
The art dex files are copied in the bootclasspath_fragment and the
non-updatable and updatable dex files are copied in the
platform_bootclasspath.

Bug: 177892522
Test: m nothing
Change-Id: I5d3d533d1a7a9f8e7ae20c12eb33029a898a2cd6
2021-05-12 21:51:19 +01:00