Commit Graph

188 Commits

Author SHA1 Message Date
Liz Kammer 9ba460fd8f Add property api_levels_jar_filename to droidstubs
The default is android.jar to maintain current behavior but allows users
to specify a different filepath to support generating api-versions for
docs other than the platform (e.g. auto).

Bug: 162552181
Test: m doc
Test: go test java_test
Merged-In: I9e51abaf7d5451d68ca782157d9b739f76c5da82
Change-Id: I9e51abaf7d5451d68ca782157d9b739f76c5da82
2020-08-05 20:14:37 +00:00
Ramy Medhat abe1a1aa3e Add sandbox property to the javadoc rule.
The sandbox property indicates whether metalava should only read
inputs explicitly specified on the command line. This CL adds the
property and sets the appropriate configuration for RBE depending
on whether the sandbox is set or not.
Bug: b/156613606
Test: built aosp_crosshatch-userdebug with/without RBE_METALAVA.
Change-Id: I7256d29f18e0af18dbe65d1c7dbbf62fd3d65f4c
Merged-In: I7256d29f18e0af18dbe65d1c7dbbf62fd3d65f4c
2020-07-06 20:00:13 +00:00
Paul Duffin 4c21463f04 Merge "Fix build breakages when WITHOUT_CHECK_API=true" into rvc-dev 2020-06-22 12:39:41 +00:00
Paul Duffin 86672f6f4e Fix build breakages when WITHOUT_CHECK_API=true
Bug: 158578354
Test: export WITHOUT_CHECK_API=true
      m checkbuild
Change-Id: I7b5fff40b870c9d754f779ff1c3314bac7e42440
2020-06-19 19:15:29 +01:00
Colin Cross 1e28e3c615 Add support for running Android lint on java and android modules.
Add a rule that runs Android lint on each java and android module
and produces reports in xml, html and text formats.

Bug: 153485543
Test: m out/soong/.intermediates/packages/apps/Settings/Settings-core/android_common/lint-report.html
Change-Id: I5a530975b73ba767fef45b257d4f9ec901a19fcb
Merged-In: I5a530975b73ba767fef45b257d4f9ec901a19fcb
(cherry picked from commit 014489c1e6)
2020-06-18 11:01:47 -07:00
Colin Cross f5f663b0c3 Remove most paths from java.TestConfig
Now that tests don't need to specify every path passed to
PathForSource or PathForModuleSrc, remove most of them from
java.TestConfig.  Leave a few that are globbed by lots of tests,
and move a few that are globbed by a single test into the tests.

Bug: 153485543
Test: all soong tests
Change-Id: Ica91d7203a6a7dbca0fd4fed84c78f149b8699e1
Merged-In: Ica91d7203a6a7dbca0fd4fed84c78f149b8699e1
(cherry picked from commit 238c1f3903)
2020-06-15 09:46:31 -07:00
Ramy Medhat 7f2006c091 Turn on metalava sandbox warning in all droiddoc metalava invocations.
Bug: b/156613606
Test: built aosp crosshatch userdebug with RBE_METALAVA=1 and dependency
file support.

Change-Id: I5ca637984429df12fe05bb07f5db154be9b24f97
Merged-In: I5ca637984429df12fe05bb07f5db154be9b24f97
2020-06-11 17:57:22 +00:00
Ramy Medhat 8e9b63b6ff Add support for Metalava implicit dependencies for remote execution.
Bug: b/156613606
Test: built aosp crosshatch userdebug with RBE_METALAVA=1
Change-Id: Ic64d98785e34717ef9bdad62b4885085f84f132a
Merged-In: Ic64d98785e34717ef9bdad62b4885085f84f132a
2020-06-11 17:57:12 +00:00
Ramy Medhat 46bad760ec Add sourcepath to inputs of remoteable metalava action.
Bug: b/156613606
Test: presubmit
Change-Id: Idd77abddac0a676302226eb62883c74d5d7489af
Merged-In: Idd77abddac0a676302226eb62883c74d5d7489af
2020-06-11 17:55:36 +00:00
Ramy Medhat 1fb3cd8496 Add support for the remote execution of metalava actions.
Bug: b/156613606
Test: built aosp crosshatch userdebug with RBE_METALAVA=1
Change-Id: I3d42d75b4522f99ff95ce8c997ead782e4322f6e
Merged-In: I3d42d75b4522f99ff95ce8c997ead782e4322f6e
2020-06-11 00:52:07 -04:00
Paul Duffin 174b26e7d3 Switch droiddoc to use SdkHeaderJars()
Previously, droiddoc was using SdkImplementationJars() to get a JAR to
add to the classpath in order to resolve unknown references in the
source.

Given that SdkHeaderJars() returns jars created by Turbine which are
(for compilation at least) functionally identical to the implementation
jars there is no point in using the implementation jars.

Bug: 148080325
Test: m nothing
Merged-In: I2b718cc1445c04e849dcb0b1f53bba2b0bd90c95
Change-Id: I2b718cc1445c04e849dcb0b1f53bba2b0bd90c95
(cherry picked from commit 649dadfb87)
2020-05-27 07:41:18 +01:00
Paul Duffin 533f9c78cd Retry: "java_sdk_library: Improve consistency with ..._import"
The scopePaths struct is used by both java_sdk_library and its prebuilt
but was not populated in the same way. This change addresses those
discrepancies in preparation for a follow up change which will allow
access to some of those fields through OutputFileProvider.

Changes:
* Document the scopePaths field and struct.
* Switch those fields that may not be fully populated from Paths to
  OptionalPath to make that 100% clear and protect against unchecked
  use.
* Switch java_sdk_library_import to use the dependency extraction
  mechanism driven by the dependency tag. This should actually have
  been part of the change that added that mechanism.
* Only create prebuilt_stubs_sources if sources have been provided.
* Add dependencies from java_sdk_library_import on its stubs source
  child modules if sources have been provided. That will ensure the
  stubsSrcJar field is updated.
* Updates current/removedApiFilePath if provided for the scope in
  java_sdk_library_import.
* Extracts ApiStubsSrcProvider from ApiStubsProvider to allow it to
  be implemented by PrebuiltStubsSources so that it can provide access
  to the stubs src jar that it creates.

Test: m nothing
Bug: 148080325
Bug: 155164730
(cherry picked from commit 0f8faffdc0)
Change-Id: Ib56ca608d6fc1357d3d89e9c4cfed6ff8da11735
2020-05-25 10:40:50 +01:00
Anton Hansson c5e13279ad Don't require merge_annotations_dir for annotations_enabled
As far as I can tell, annotations_enabled does not require use of
merge_annotations_dir (although it may have done in the past).

Bug: 157010342
Test: m
Change-Id: I6705522774203b4566f58679bbc2f0c17422c1dd
2020-05-21 11:29:22 +01:00
Anton Hansson 18a289560d Fix API lint copy command
The space after the backtick prevents it working when copy-pasting it
into the shell.

Test: create lint error => m => cp/pasted command runs successfully
Merged-In: I42b8b60d01dcaed34dbde0390eb1691d656a32cf
Change-Id: I42b8b60d01dcaed34dbde0390eb1691d656a32cf
(cherry picked from commit 3361a294c0)

// Added for rvc-dev. Used this bug as it is the bug that needs all the build changes.
Bug: 155164730
Change-Id: I42b8b60d01dcaed34dbde0390eb1691d656a32cf
2020-05-16 11:03:46 +00:00
Paul Duffin 8986cc947c Enable api_lint in java_sdk_library
Adds api_lint.enabled property to control whether API linting should be
performed.

Test: m checkapi
Bug: 156126315
Merged-In: I87ca5a942228cf6af1a9939f0334d6fc46c39a63
Change-Id: I87ca5a942228cf6af1a9939f0334d6fc46c39a63
(cherry picked from commit 160fe41c39)
2020-05-13 11:27:44 +01:00
Makoto Onuki ba15f14271 Merge "Merge the "big 3" metalava invocations into one" into rvc-dev 2020-05-08 21:23:53 +00:00
Paul Duffin 75dcc80396 Export API files as part of the java_sdk_library
Bug: 153443117
Test: m nothing

Merged-In: I9d6f5b91a7cc25019e2eb9e3c138f0874d2831de
Change-Id: I9d6f5b91a7cc25019e2eb9e3c138f0874d2831de
(cherry picked from commit 1fd005d5b3)
2020-05-08 19:38:57 +01:00
Paul Duffin f488ef2a3c Export stub sources as part of the java_sdk_library
Minor refactoring of checkMergeZip(string) -> checkMergeZips(...string)
to allow testing of multiple merge zips.

Bug: 153443117
Test: m nothing
Merged-In: I8db00f611ced15f8476ba16f2834a72e8c913596
Change-Id: I8db00f611ced15f8476ba16f2834a72e8c913596
(cherry picked from commit 3d1248ceb6)
2020-05-08 19:34:30 +01:00
Makoto Onuki b850a9deaa Merge the "big 3" metalava invocations into one
This CL merges the biggest three metalava invocations --
stub and signature file generation, api lint, and "check released" --
into one call.

For each of the existing separate rules for api lint and check released this change:
1) Stopped creating a separate rule containing its own metalava invocation.
2) Moved the code for creating the rule earlier so it can add extra flags to the combined metalava command.
3) Moved the message handling into metalava, allowing the message to be passed in.
4) Added a flag to record whether the specific check was needed so that you can separate the timestamp file creation out to run after the merged metalava command.

A couple of minor bug fixes and clarifications.

Bug: 151160048
Test:  "m" and treehugger

Test: Apply https://android-review.googlesource.com/c/platform/frameworks/base/+/1295541 and run `NINJA_ARGS="-k 999" m checkapi`
-> Make sure there are 4 kinds of lint failures ("AddedFinal" and "VisiblySynchronized") with the valid error messages, from:
- api-stubs-docs
- system-api-stubs-docs
- test-api-stubs-docs
- module-lib-api-

Test: Run the 4 cp commands shown by lint
-> Make sure the following 4 baseline files are updated:
frameworks/base/api/lint-baseline.txt
frameworks/base/api/module-lib-lint-baseline.txt
frameworks/base/api/system-lint-baseline.txt
frameworks/base/api/test-lint-baseline.txt

Test: Then run `NINJA_ARGS="-k 999" m checkapi` again.
-> Make sure the API lint errors are gone
-> Make sure "api-stubs-docs" shows the "You have tried to change the API from what has been
previously released in ..." error. (because of "AddedFinal")
-> Make sure it the other 3 API surfaces show the "You have tried to change the API from
what has been previously approved" error.

Test: Remove "final" from Intent.java and run "NINJA_ARGS="-k 999" m checkapi" again.
-> This time, all the 4 API surfaces say "You have tried to change the API from
what has been previously approved" error."

Test: Run the 4 "make *-update-current-api" commands shown in the above errors and "NINJA_ARGS="-k 999" m checkapi"
-> The exact command is:
$ make \
  module-lib-api-update-current-api \
  api-stubs-docs-update-current-api \
  test-api-stubs-docs-update-current-api \
  system-api-stubs-docs-update-current-api

-> Make sure frameworks/base/api/*-current.txt are properly updated.

Test: Run "m update-api"
-> Make sure no text signature files get updated.

Test: Run "m droid".
-> Make sure everything builds fine.

Change-Id: Ia1a5510b57d58748b4655473d4585636deb7f45e
Merged-in: Ia1a5510b57d58748b4655473d4585636deb7f45e
2020-05-08 10:47:12 -07:00
Anton Hansson e714565ce8 Treat lint warnings as errors
Warnings get lost in the build spam and never get fixed. Right now is
the best time to do this, because we only track the warnings/errors
since the last released API. The API has just been frozen, so the number
of warnings is at an all-time low.

Whitelist the car lib and test stubs for now, which produce a lot of
warnings.

Bug: 154317059
Test: presubmit
Change-Id: I9aa7cb1b947c6c664f15e2bd5835d76eaac1237a
2020-05-04 21:47:34 +01:00
Anton Hansson 34ced82d2a Fix soong crashing when there's no baseline file
Soong was referencing the baseline file regardless of whether it was
valid or not.

Also update the lint message to match the one metalava itself prints,
which only suggests whitelisting the error if it's not possible to fix
it properly.

Bug: 154616909
Test: m (with lint error)
Change-Id: I4c6ec327a59aefe74825b4c4719393dd267ba3aa
2020-05-01 17:12:36 +01:00
Makoto Onuki b52c8eafaa Don't use metalava for "check-current"
Because we already generate the latest current.txt and removed.txt
during the "main" metalava invocation (i.e. stub generation),
we don't have to use metalava for current API check.

Just use diff instead.

Bug: 151160048

Test: Remove @hide from Intent.EXTRA_INSTALL_RESULT, and do `m out/soong/.intermediates/frameworks/base/api-stubs-docs/android_common/check_current_api.timestamp`.
Result is:
```
FAILED: out/soong/.intermediates/frameworks/base/api-stubs-docs/android_common/check_current_api.timestamp
( true && diff -u -F '{ *$' frameworks/base/api/current.txt out/soong/.intermediates/frameworks/base/api-stubs-docs/android_common/api-stubs-docs_api.txt && diff -u -F '{ *$' frameworks/base/api/removed.txt out/soong/.intermediates/frameworks/base/api-stubs-docs/android_common/api-stubs-docs_removed.txt && touch out/soong/.intermediates/frameworks/base/api-stubs-docs/android_common/check_current_api.timestamp ) || ( echo -e "\n******************************\nYou have tried to change the API from what has been previously approved.\n\nTo make these errors go away, you have two choices:\n   1. You can add '@hide' javadoc comments (and remove @SystemApi/@TestApi/etc)\n      to the new methods, etc. shown in the above diff.\n\n   2. You can update current.txt and/or removed.txt by executing the following command:\n         make api-stubs-docs-update-current-api\n\n      To submit the revised current.txt to the main Android repository,\n      you will need approval.\n******************************\n" ; exit 38 )

Change-Id: Ib391ca2afb296b5c5174d755c7ffb66d3fa5810c

--- frameworks/base/api/current.txt     2020-04-16 17:50:21.911899599 -0700
+++ out/soong/.intermediates/frameworks/base/api-stubs-docs/android_common/api-stubs-docs_api.txt       2020-04-16 17:51:35.211792638 -0700
@@ -10415,6 +10415,7 @@ public class Intent implements java.lang
     field public static final String EXTRA_INDEX = "android.intent.extra.INDEX";
     field public static final String EXTRA_INITIAL_INTENTS = "android.intent.extra.INITIAL_INTENTS";
     field public static final String EXTRA_INSTALLER_PACKAGE_NAME = "android.intent.extra.INSTALLER_PACKAGE_NAME";
+    field public static final String EXTRA_INSTALL_RESULT = "android.intent.extra.INSTALL_RESULT";
     field public static final String EXTRA_INTENT = "android.intent.extra.INTENT";
     field public static final String EXTRA_KEY_EVENT = "android.intent.extra.KEY_EVENT";
     field public static final String EXTRA_LOCAL_ONLY = "android.intent.extra.LOCAL_ONLY";

******************************
You have tried to change the API from what has been previously approved.

To make these errors go away, you have two choices:
   1. You can add '@hide' javadoc comments (and remove @SystemApi/@TestApi/etc)
      to the new methods, etc. shown in the above diff.

   2. You can update current.txt and/or removed.txt by executing the following command:
         make api-stubs-docs-update-current-api

      To submit the revised current.txt to the main Android repository,
      you will need approval.
******************************

ninja: build stopped: subcommand failed.
```

Test: Remove @removed and @SystemApi from Intent.ACTION_MASTER_CLEAR, and do `m out/soong/.intermediates/frameworks/base/system-api-stubs-docs/android_common/check_current_api.timestamp`
Result is:
```
( true && diff -u -F '{ *$' frameworks/base/api/system-current.txt out/soong/.intermediates/frameworks/base/system-api-stubs-docs/android_common/system-api-stubs-docs_api.txt && diff -u -F '{ *$' frameworks/base/api/system-removed.txt out/soong/.intermediates/frameworks/base/system-api-stubs-docs/android_common/system-api-stubs-docs_removed.txt && touch out/soong/.intermediates/frameworks/base/system-api-stubs-docs/android_common/check_current_api.timestamp ) || ( echo -e "\n******************************\nYou have tried to change the API from what has been previously approved.\n\nTo make these errors go away, you have two choices:\n   1. You can add '@hide' javadoc comments (and remove @SystemApi/@TestApi/etc)\n      to the new methods, etc. shown in the above diff.\n\n   2. You can update current.txt and/or removed.txt by executing the following command:\n         make system-api-stubs-docs-update-current-api\n\n      To submit the revised current.txt to the main Android repository,\n      you will need approval.\n******************************\n" ; exit 38 )
--- frameworks/base/api/system-removed.txt      2020-04-16 17:50:21.911899599 -0700
+++ out/soong/.intermediates/frameworks/base/system-api-stubs-docs/android_common/system-api-stubs-docs_removed.txt     2020-04-16 17:53:45.319602905 -0700
@@ -55,7 +55,6 @@ package android.content {

   public class Intent implements java.lang.Cloneable android.os.Parcelable {
     field @Deprecated public static final String ACTION_DEVICE_INITIALIZATION_WIZARD = "android.intent.action.DEVICE_INITIALIZATION_WIZARD";
-    field @Deprecated public static final String ACTION_MASTER_CLEAR = "android.intent.action.MASTER_CLEAR";
     field @Deprecated public static final String ACTION_SERVICE_STATE = "android.intent.action.SERVICE_STATE";
     field @Deprecated public static final String EXTRA_CDMA_DEFAULT_ROAMING_INDICATOR = "cdmaDefaultRoamingIndicator";
     field @Deprecated public static final String EXTRA_CDMA_ROAMING_INDICATOR = "cdmaRoamingIndicator";

******************************
You have tried to change the API from what has been previously approved.

To make these errors go away, you have two choices:
   1. You can add '@hide' javadoc comments (and remove @SystemApi/@TestApi/etc)
      to the new methods, etc. shown in the above diff.

   2. You can update current.txt and/or removed.txt by executing the following command:
         make system-api-stubs-docs-update-current-api

      To submit the revised current.txt to the main Android repository,
      you will need approval.
******************************
```

Test: Add `baseline_file` to `check_api.current` and run `m`
FAILED: out/soong/build.ninja
out/soong/.bootstrap/bin/soong_build -t -l out/.module_paths/Android.bp.list -b out/soong -n out -d out/soong/build.ninja.d -globFile out/soong/.bootstrap/build-globs.ninja -o out/soong/build.ninja Android.bp
error: frameworks/base/StubLibraries.bp:86:1: module "api-stubs-docs" variant "android_common": current API check can't have a baseline file. (module %s): api-stubs-docs
ninja: build stopped: subcommand failed.

Test: With the above changes, run "m api-stubs-docs-update-current-api system-api-stubs-docs-update-current-api", make sure the API files got updated, and then run "m"
Merged-in: I47b146c6fe4caa65775ecf5425ab09b57f43f839
Change-Id: I3045bd8dc95d5f70893729ed65426d8cfc88c5ec
2020-04-27 12:13:35 -07:00
TreeHugger Robot 416b811742 Merge changes Ib8025019,I7d58d104,I9dd4312c,I995ed389,Iba2a8a5a, ... into rvc-dev
* changes:
  Allow droidstubs to not generate any stubs
  Remove conscrypt.module.intra.core.api.stubs from apex white list
  Ignore PrebuiltDepTag when processing APEX contents
  Stop requiring apex_available on java_library members of sdks
  Add dependency tags to apex available errors
  Extract DepIsInSameApex and RequiredSdks interfaces
2020-04-23 15:19:41 +00:00
TreeHugger Robot 4586da2aae Merge changes Ie883eb45,If46a2f74,I8f4c5395,Ic4eb169d,I2b2408ef, ... into rvc-dev
* changes:
  Replace references to droiddoc with droidstubs
  Allow walkPayloadDeps visitor to control walk flow
  Add apex_available to sysprop_library
  Improve missing apex_available message
  Split TestApexAvailable into separate tests
  Use reflect.Zero(type) to get value to clear field
  Sdk snapshot set compile_multilib per OsType
  Remove old SdkMemberType API for creating snapshot modules
  Improve consistency of handling java snapshot properties
  Make new module creation API more flexible
  Add abstraction for tracking compile multilib usages
  Add support for using cc_library in sdk/module_exports
  Extract the osTypeSpecificInfo code from module creation loop
  Extract archTypeSpecificInfo code from module creation loop
  Clean up the main module creation loop
  Add support for cc_prebuilt_library
  Refactor prebuilt to use srcs supplier function
  Output properties before sets in snapshot module
  Remove SdkMemberType.FinalizeModule
  Follow up a review comment that was missed
  Copy shared_libs and system_shared_libs to module snapshot
  Support extracting common values from embedded structures
  Refactor common value extraction
  Copy white listed apex available settings into snapshot
  Disable installation for sdk snapshot versioned prebuilts
  Remove special handling of test_ apexes
  Remove special handling of com.android.art.debug/release
  Allow sdk members to vary by os type
  Add support for multiple os types
  Copy sdk_version to cc library snapshots
  Refactor java_library/java_test snapshot processing
  Refactor snapshot module creation
  Enable androidmk processing in sdk testing
  Add a nice install paths for module SDKs and exports.
  Copy apex_available properties to snapshot modules
  Improve documentation of CompileMultiTargets and related properties
  Only check copy rules into the snapshot directory
  Simplify java library sdk member code
  Add CommonOS variant for sdk
  Fix bug in error reporting when adding duplicate properties
  Avoid invoking sdk member to add empty list of dependencies
  Add windows to the list of available OS's in sdk tests
  Add support for cc_library_headers in sdk/module_exports
  Prune any empty property sets from the modules before transforming
  Simplify cc library sdk snapshot handling of include dirs/headers
  Add support for transforming a property set after its contents
  Fix issues with bp transformation
2020-04-23 15:19:26 +00:00
Anton Hansson 0ee713aa0e Remove unused arguments in droiddoc
Nothing passes these arguments, so it's dead code.

There is a lot of other cruft in this file that is effectively
dead code, but required further cleanup.

Bug: 152479829
Test: m
Change-Id: Ib2630d4d49f21003b17e2331b0900e8df0a27e66
2020-04-22 21:09:46 +01:00
Paul Duffin 455b0bf922 Allow droidstubs to not generate any stubs
Needed to optimize the handling of the module_lib API surface which
currently has to be generated with two separate droidstubs instances,
one to generate the stubs and another to generate the .txt file. This
allows the module generating the .txt file to avoid also wasting time
generating stubs that are not used.

This change:
* Adds a generate_stubs property that defaults to true to allow the
  behavior to be customized on a per module basis.
* Uses either the stubs srcjar or the api .txt file as the OutputFile
  for the AndroidMkEntries to ensure that they get written out properly.
* Rearranges the code for generating stubs to make it easier to
  turn it off.

Bug: 146727827
Bug: 153306490
Test: m droid
      Check output dir of framework-sdkextensions-api-module_libs_api
	  to make sure it does not contain any sources or srcjars.
Merged-In: Ib8025019f8a7a8cf5fa8765d76b5ad470af20006
Change-Id: Ib8025019f8a7a8cf5fa8765d76b5ad470af20006
2020-04-22 12:51:53 +01:00
Paul Duffin 93520edca2 Remove old SdkMemberType API for creating snapshot modules
Migrates system modules and droid stubs over to use the new API for
creating the snapshot modules and removes the old API.

Bug: 153306490
Test: m nothing
Merged-In: Ia825767f1f7ee77f68cfe00f53e09e6f6bfa027f
Change-Id: Ia825767f1f7ee77f68cfe00f53e09e6f6bfa027f
2020-04-22 12:51:45 +01:00
Automerger Merge Worker 82f316b8db Add support for order-only dependencies to RuleBuilder
Test: TestRuleBuilder
Change-Id: I1609a790dd4d0a03c8308b6e552622fe33fa2499
Bug: 153071808
Merged-In: Icfa98d6840b1dc2e273ba29c33011635d1cf93b1
2020-04-09 08:17:32 -07:00
Aurimas Liutikas 2d732e0070 Do not include any documentation when building stubs
We only need documentation (javadocs) when building docs stubs,
for regular stubs, we can drop printing out all the docs.

Tested with:
touch frameworks/base/core/java/android/view/View.java && time make -j framework

Before (with just r.android.com/1254909)
Run #1 2m4.779s
Run #2 2m6.672s
Run #3 2m4.179s

Average: 125.2 seconds

After (with this patch and r.android.com/1254909)
Run #1 1m58.682s
Run #2 1m58.820s
Run #3 1m57.724s

Average: 118.4 seconds

Speedup ~6.8 seconds (5.4%)

Bug: 151160048
Test: make checkapi
Change-Id: I27eafb5a61bb9d26a74ca25f84c9daca17a8394f
Merged-in: I27eafb5a61bb9d26a74ca25f84c9daca17a8394f
2020-03-19 14:46:17 -07:00
Paul Duffin 83a2d967de Allow java_system_modules_import to replace java_system_modules
Previously, there were some places where a java_system_module_import
could not be used in place of a java_system_module. That was because
the code assumed a *SystemModules type not a *systemModulesImport type.
This change introduces a SystemModulesProvider interface that is used
instead and is implemented on both types.

Bug: 142940300
Test: m nothing
      ran new tests before changes to make sure they detected the issue
	  and after to make sure the changes fixed the issue.
Change-Id: I7b16ac5708880bdf61e6f5b1e6616c986f0ed763
2020-02-19 14:24:15 +00:00
Jaewoong Jung 3aff5787e2 Simple refactoring of prefix related functions.
Rename the two prefix-in-list funcs so that their usages are clearer.
Also find and replace all the code that essentially does either.

This introduces additional loops in some places, but I think the added
readability and simplicity outweighs the performance degradation, which
should be negligible anyway.

Test: m nothing
Test: TreeHugger
Change-Id: I37e2276ca0c815105ed0031f23c0b1264b480e4f
2020-02-14 14:33:49 -08:00
Anton Hansson 26bf49b0b2 Fix sdk aidl include for droiddoc
Prior to this change droidstubs modules that set sdk_version
did not get framework.aidl added to its aidl includes.

Bug: 149138391
Test: patch CL in bug && m system_aidl_test-droidstubs
Change-Id: I92ab344c8a4311e10c1e5c8ebf525fa2dc704075
(cherry picked from commit f278ca60e06da86c67f6a3865c290f8451657ce9)
2020-02-08 20:37:07 +00:00
Mathew Inwood abd49ab4df Build rules for compat config docs generation.
We add a compat config build rule to extract the merged config, and
then update the droiddoc build rule to consume that.

Test: m -j offline-sdk-docs
Bug: 144927670
Change-Id: Ib1e85f97895c89227882e665572bda9bfc2a8cba
Exempt-From-Owner-Approval: ag/10097965 approved internally, Colin requested to patch to aosp
2020-02-04 16:28:22 +00:00
Jiyong Park 6a927c4e6a Abstract sdk_version string using sdkSpec type
The value format that sdk_version (and min_sdk_version, etc.) can have
has consistently evolved and is quite complicated. Furthermore, with the
Mainline module effort, we are expected to have more sdk_versions like
'module-app-current', 'module-lib-current', etc.

The goal of this change is to abstract the various sdk versions, which
are currently represented in string and is parsed in various places,
into a type called sdkSpec, so that adding new sdk veresions becomes
easier than before.

The sdk_version string is now parsed in only one place 'SdkSpecFrom', in
which it is converted into the sdkSpec struct. The struct type provides
several methods that again converts sdkSpec into context-specific
information such as the effective version number, etc.

Bug: 146757305
Bug: 147879031
Test: m
Change-Id: I252f3706544f00ea71c61c23460f07561dd28ab0
2020-01-25 21:56:43 +09:00
Colin Cross 8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Paul Duffin e602918294 Restrict SdkMemberTypes that can be used with sdk/sdk_snapshot
By default SdkMemberTypes are only supported on module_exports module
type. Support for sdk module type has to be explicitly specified.

The java_header_libs, native_shared_libs and stubs_sources are
supported on sdk. The latter is required to provide the stubs source
for an API specified in java_header_libs as they should be kept in
sync.

Bug: 146341462
Test: m nothing
Change-Id: I19b9e60792780a797458d4a9e489506602b13144
2019-12-31 15:08:59 +00:00
Paul Duffin 9b478b0831 Generate .srcjar for prebuilt_stubs_sources
Changes prebuilt_stubs_sources to generate a .srcjar from its input
instead of just exposing the srcs it is given. This ensures that it can
be used as a drop in replacement for a droidstubs module.

Updates the test for prebuilt_stubs_sources to be more representative
of the actual use made of it by sdk snapshot which outputs a directory
not a glob pattern. Added some documentation of the
prebuilts_stubs_sources srcs property to make it clear that it is
supposed to be a set of directories.

Extracts common code from sdk/testing.go for normalizing path/paths
for testing.

Bug: 143678475
Test: m conscrypt-module-sdk conscrypt-module-host-sdk conscrypt-module-test-sdk
      unzip those in place of external/conscrypt
	  build core-current-stubs-source which expects it to provide a .srcjar.
Change-Id: I8204a022557a9b0b45e19eac79ecba98ff95213d
2019-12-20 08:13:45 +00:00
Paul Duffin 884363e782 Dedup droiddoc module type registration
Some tests were changed as they were using the wrong name for the
droiddoc_exported_dir module type.

Test: m checkbuild
Bug: 146540677
Change-Id: If57404760d2df3757c93e237696359355cf0c96e
2019-12-19 19:19:40 +00:00
Paul Duffin 255f18e584 Decouple addition of new sdk member types from sdk code
Previously, adding a new SdkMemberType would require adding a new
sdkMemberListProperty instance to the sdkMemberListProperties as well
as adding a new property into the sdkProperties struct. They are
potential sources of conflict and couple the sdk code with all the
packages that add members to it. This change switched to a
registration model that allows each package to register its sdk
member types decoupling them from the sdk code.

Adds an SdkPropertyName() method to SdkMemberType that specifies the
name of the property to use in the sdk/sdk_snapshot. Also provides
an SdkMemberTypeBase struct to be used by providers of SdkMemberType
implementations.

SdkMemberType instances are registered using the
RegisterSdkMemberType() func which sorts the registered instances
by their SdkPropertyName() to ensure the behavior is consistent and
not affected by order of registration.

When creating a new sdk module a dynamicSdkMemberTypes instance is
created that contains the following:

* A properties struct is created dynamically that contains a field for
  each registered SdkMemberType, corresponding to that type's
  SdkPropertyName().

* A list of sdkMemberListProperty instances is also created, one for
  each registered SdkMemberType.

The dynamicSdkMemberTypes instance is cached using a key that uniquely
identifies the set of registered types just in case new types are
registered after one has been created, e.g. by tests.

Bug: 142918168
Test: m checkbuild
Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
2019-12-13 20:07:33 +00:00
Paul Duffin 9d8d609fcd Pass SdkMember to AddPrebuiltModule instead of the name
This is needed for a follow up change that makes sure that the
prebuilt modules have the same visibility as the source modules.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I9461c8c094ab19ee9ececb5e5fd50565789f2fa2
2019-12-09 13:28:36 +00:00
Paul Duffin 1387957727 Parameterize the sdk member processing
Extracts the type specific functionality into the SdkMemberType
interface which has to be implemented by each module type that can
be added as a member of the sdk. It provides functionality to add
the required dependencies for the module type, check to see if a
resolved module is the correct instance and build the snapshot.

The latter was previously part of SdkAware but was moved because
it has to be able to process multiple SdkAware variants so delegating
it to a single instance did not make sense.

The custom code for handling each member type specific property,
e.g. java_libs, has been replaced with common code that processes
a list of sdkMemberListProperty struct which associates the
property (name and getter) with the SdkMemberType and a special
DependencyTag which is passed to the SdkMemberType when it has to add
dependencies.

The DependencyTag contains a reference to the appropriate
sdkMemberListProperty which allows the resolved dependencies to be
grouped by type.

Previously, the dependency collection methods would ignore a module if
it was an unsupported type because they did not have a way of
determining which property it was initially listed in. That meant it
was possible to add say a droidstubs module to the java_libs property
(and because they had the same variants) it would work as if it was
added to the stubs_sources property. Or alternatively, a module of an
unsupported type could be added to any property and it would just be
ignored.

However, the DependencyTag provides information about which property
a resolved module was referenced in and so it can detect when the
resolved module is of the wrong type and report an error. That check
identified a bug in one of the tests where the sdk referenced a
java_import module (which is not allowed in an sdk) instead of a
java_library module (which is allowed). That test was fixed as part
of this.

A list of sdkMemberListProperty structs defines the member properties
supported by the sdk and are processed in order to ensure consistent
behaviour.

The resolved dependencies are grouped by type and each group is then
processed in defined order. Within each type dependencies are grouped
by name and encapsulated behind an SdkMember interface which includes
the name and the list of variants.

The Droidstubs and java.Library types can only support one variant and
will fail if given more.

The processing for the native_shared_libs property has been moved into
the cc/library.go file so the sdk package code should now have no type
specific information in it apart from what is if the list of
sdkMemberListProperty structs.

Bug: 143678475
Test: m conscrypt-module-sdk
Change-Id: I10203594d33dbf53441f655aff124f9ab3538d87
2019-12-06 12:16:59 +00:00
Paul Duffin b645ec8e34 Add model to represent generated snapshot .bp file
Having each module type generate the contents of the snapshot's .bp
file results in lots of duplicated code. This adds an intermediate
model for use by the module types and then generates the .bp file
contents from that.

This not only removes the duplicated formatting code but it also
allows consistent handling of shared properties such as name further
reducing duplication. It also makes it possible to duplicate the
versioned and unversioned prebuilt modules from the same model.

Extracts generatedContents from generatedFile to allow the contents
to be populated without creating an output file, for testing.

Cleans up unused code.

Bug: 143678475
Test: m nothing
Change-Id: If21b84db0ef3fdfb5dc11ea0973ce6cb73603ea3
2019-12-02 12:59:38 +00:00
Paul Duffin b1a8813520 Dedup code for generating prebuilt_stubs_sources in snapshot
Removes duplicated code for generating a versioned and non-versioned
form of the prebuilt_stubs_sources module in the snapshot.

Bug: 143678475
Test: m nothing

Change-Id: Ic431b6fcb6f3c7c85f1a04cb6f087424aaf0cffa
2019-11-26 22:30:27 +00:00
Paul Duffin 9154718303 Adds droidstubs support to sdk module
Adds stubs_sources property to sdk and unzips the droidstubs srcjar
into the snapshot directory.

Adds an UnzipToSnapshot method to the SnapshotBuilder which creates
a rule that uses zip2zip to repackage the supplied zip content into a
temporary zip file that matches what the required snapshot structure.
e.g. if the supplied zip contains foo/Foo.java and that needs to be in
the snapshot directory java/foo/stubs then it will create a zip that
contains java/foo/stubs/foo/Foo.java.

The temporary zip that is the output of that rule is added to the
zipsToMerge field for merging later.

If the zipsToMerge is empty then the snapshot zip is created as
before. Otherwise, a temporary zip file is created. That is then
merged with the other zip files in zipsToMerge to create the final
snapshot zip.

Adds prebuilt_stubs_sources for use by the generated .bp module.

Bug: 143678475
Test: added conscrypt sdk module and attempted to build it
Change-Id: Ie274263af3a08e36a73c61c0dbf0c341fd6967e2
2019-11-22 20:52:59 +00:00
Treehugger Robot 80df439072 Merge "Enforce hidden apis usage in product(soong)" 2019-11-11 02:05:16 +00:00
Paul Duffin 13a9dd604b Remove unnecessary constraint on previous_api
The constraint requires the previous_api property to be specified
unless validating nullness. However, there is no reason within the
Soong code why that should be.

The only use of previous_api is as the argument to the metalava
--migrate-nullness option which is only added when previous_api is
specified. So, there is no reason in the Soong code for the
constraint.

Metalava also does not require the --migrate-nullness option to be
specified unless validating nullness. So, there is no reason in
metalava for the constraint.

It is therefore safe to remove the constraint.

Bug: 142113521
Test: m checkbuild
Change-Id: I189071e215e928fdf43a39a03d540732743b7a32
2019-11-08 08:23:14 +00:00
Adrian Roos 109f64766e Merge "API Lint: Archive lint errors" 2019-11-06 12:35:55 +00:00
Jeongik Cha 2cc570dc63 Enforce hidden apis usage in product(soong)
Only if PRODUCT_ENFORCE_PRODUCT_PARTITION_INTERFACE is set,
every app and java library in product cannot use hidden APIs anymore.

checkSdkVersion() checks if sdk_version of app and library is narrow enough,
checkLinkType() checks every library that app links agianst

Bug: 132780927
Test: m
Test: set PRODUCT_ENFORCE_PRODUCT_PARTITION_INTERFACE, and check whether build
error occurs.
Change-Id: Ic630503b875040f730feda4fef826ed6d71da111
2019-11-06 19:42:42 +09:00
Adrian Roos 3b8f1cda45 API Lint: Archive lint errors
Adds reported lint errors to dist. This lets us track them and
build tooling on top.

Bug: 138440867
Test: make dist droid; ls -l out/dist/apilint
Change-Id: I75629e3d9e1a53d161d844dfac121a5998429e04
2019-11-04 19:04:52 +01:00
Colin Cross 6cef481ee7 Use java language 1.9 for sdk_version: "current"
Java language was set for 1.8 for anything building against the
current SDK because the stubs were not built in Soong, so the
system modules could not be built.  The stubs have been built
in Soong since Iabd32b30954b3f4a6d9a779fde52a032b684807e, but
I5e870c34dd0ebc8ae3f888ec627da590c846a76f missed updating
this TODO.

Use 1.9 when building against the stubs, but continue using
1.8 for unbundled builds until we have prebuilt system modules.

Always use the core-current-stubs-system-modules to avoid
splitting android.* packages between the system modules and
the classpath, which would cause new classes in android.*
packages in classpath jars to be ignored.  Add a new
java9Classpath field to sdkDep that will contain the stubs jar
when targeting Java language level 1.9, and plumb it through
to javac and turbine.  Rename the modules field to bootclasspath.

Bug: 142896162
Test: m checkbuild
Change-Id: Icfd32d0a863b2303a997c7cf03cb3708aade4724
2019-10-29 13:35:39 -07:00