Commit Graph

322 Commits

Author SHA1 Message Date
Paul Duffin 70d3bee3e0 Remove emptyFixtureFactory from apex and java
Bug: 183235980
Test: m nothing
Change-Id: I350b45e2f57430fb158f4141a566e75de17208cd
2021-03-22 18:31:53 +00:00
Colin Cross d2db7953a0 Merge changes Iab4e09d9,Icf2f24dd,I15be5ef1,Ic0db9619
* changes:
  Run lint actions in sbox
  Support sbox-in-RBE
  Move android package on top of remotexec
  Support sandboxing inputs in RuleBuilder
2021-03-19 17:25:14 +00:00
Paul Duffin 54a5e9a467 Merge "Remove javaMockFS()" 2021-03-19 14:18:46 +00:00
Paul Duffin 89fbdb2387 Merge "Fix TestTurbine to work without javaMockFS()" 2021-03-19 11:11:42 +00:00
Paul Duffin 903a8a08ab Merge "Convert TestClasspath to test fixtures" 2021-03-19 11:08:17 +00:00
Colin Cross 1661aff8be Run lint actions in sbox
Run lint actions in sbox with RuleBuilder.SandboxInputs.  This
copies all input files into the sandbox, which prevents the lint
tool from finding nearby source files that were not presented to it.

Using SandboxInputs requires use of PathForInput or PathForOutput
anywhere a path is used outside of the RuleBuilderCommand methods
that take paths so that they can be translated to the paths that
will be used in the sandbox.

Bug: 181681346
Test: lint_test.go
Test: m lint-check dist
Test: m USE_RBE=true RBE_LINT=true lint-check dist
Test: m USE_RBE=true RBE_LINT=true RBE_LINT_EXEC_STRATEGY=remote lint-check dist
Change-Id: Iab4e09d961891ef182643583d4d456e413bc5e39
2021-03-18 16:17:34 -07:00
Paul Duffin c3f89bece1 Merge "Clean up java package use of python build components" 2021-03-18 17:02:44 +00:00
Paul Duffin 1efdb30182 Remove javaMockFS()
Also, removes the workaround that allowed preparers to override files
added in javaMockFS().

The performance of the java tests was determined by running:
  m nothing
  rm -fr out/soong/.bootstrap/soong-java/test/
  time m nothing

Prior to starting this work that returned:
  real  1m30.060s
  user  33m3.028s
  sys   13m25.413s

Immediately prior to this change that returned:
  real  1m2.777s
  user  14m14.352s
  sys   9m40.745s

After applying this change that returned:
  real  0m20.282s
  user  3m7.429s
  sys   0m43.601s

So, this reduced the real time cost of running the java tests from 90
seconds to 20 seconds.

Bug: 182638834
Test: m nothing
Change-Id: I30490fd40ddb0684a5a4975ce1811bab5dc79e4e
2021-03-18 08:02:38 +00:00
Paul Duffin 3d9f268437 Fix TestTurbine to work without javaMockFS()
Bug: 182638834
Test: m nothing
Change-Id: Idf400b5f0d171a504acf006425cbfc514b391e74
2021-03-18 08:02:38 +00:00
Paul Duffin 76101fa5ec Convert TestClasspath to test fixtures
The main complication with this change was that many of the tests check
paths that are provided by the default java modules. The location of
them is different in the test fixtures that it is when using
testConfig() and so the test needed to be changed.

Bug: 182638834
Test: m nothing
Change-Id: I6d325dbd3ba39e5de7e53c576d7cfe07bd95a965
2021-03-18 08:02:38 +00:00
Paul Duffin 220ddd7a6b Clean up java package use of python build components
Bug: 182885307
Test: m nothing
Change-Id: I206ebc075c568a09008ec8b63122922dd87ba318
2021-03-17 23:56:58 +00:00
Paul Duffin afeee22c17 Add emptyFixtureFactory in java package
Bug: 182402754
Test: m nothing
Change-Id: I6a01b846a8aee5a0ace9e6ac056ee439bf6f12f7
2021-03-17 17:50:22 +00:00
Yuexi Ma b96a04ba49 Merge "Export a few utility methods" 2021-03-17 01:40:51 +00:00
Yuexi Ma 627263fece Export a few utility methods
Add a few utility methods to support generating TradeFed test plan in C-Suite

Test: go test .
Change-Id: I86a505068b5d5aa0f5a37b3e3a2465839a7480ef
2021-03-16 15:57:59 -07:00
Paul Duffin 163043d615 Fix TestJavaSdkLibrary... tests to work without javaMockFS()
Bug: 182638834
Test: m nothing
Change-Id: I140b88b3fdf2c532d266a196fee84675c518d859
2021-03-16 14:25:21 +00:00
Paul Duffin 22b77cd538 Convert JavaSdkLibrary... tests in java package to test fixtures
These tests rely on files provided by javaMockFS() so need to are being
converted to test fixtures to allow them to remove that dependency
which will allow javaMockFS() to be removed.

Bug: 182638834
Test: m nothing
Change-Id: If736b95d134fe73c0a0ccd438ec31e93568fb929
2021-03-16 14:25:21 +00:00
Paul Duffin 05f72de083 Convert TestJavaSdkLibraryEnforce to test fixtures
Bug: 182638834
Test: m nothing
Change-Id: I99efe76008f9ca0996177f47923274ca594a6858
2021-03-15 17:17:43 +00:00
Paul Duffin aa6caa7fa8 Create sub tests for each test in TestJavaSdkLibraryEnforce
Bug: 182638834
Test: m nothing
Change-Id: I509b071ee4e470da45159b2174aae7acf1b0eb1a
2021-03-15 17:17:43 +00:00
Paul Duffin bf028b533d Extract PrepareForTestWithJavaSdkLibraryFiles from javaMockFS()
Needed for tests in both java and sdk packages.

Bug: 182638834
Test: m nothing
Change-Id: Ia8c5e7db47d8b63bf8dcf2965b3af60a4d35a9e5
2021-03-14 01:08:26 +00:00
Paul Duffin 6bac49c560 Convert testJavaError to test fixtures
Bug: 182638834
Test: m nothing
Change-Id: If6de91cd60fea36c945f2b0d51a04e26c838e4af
2021-03-14 01:08:26 +00:00
Paul Duffin db284be913 Separate system modules tests into their own file
java_test.go is getting too big.

Bug: 182402568
Test: m nothing
Change-Id: I9c3818f1d747a230fcb66687319a3a14d557a57c
2021-03-11 08:21:49 +00:00
Paul Duffin 95bdab4000 Support test fixtures in java package
Restructures the java package test setup code to create FixturePreparer
instances for setting up a test fixture and converts some tests to
use it.

The goal with this change is not to switch all the java tests over to
directly using the new model but instead to ensure that the majority of
the java tests run with the new model, to allow existing tests to
easily switch to the new model when needed and to allow dependent
packages to be switched to the new model.

Bug: 181070625
Test: m nothing
Change-Id: I1c9d96ddbc973aaf9733dcd7fa0479f79b0f471f
2021-03-10 10:13:09 +00:00
Paul Duffin d6ceb8600c Clean up cc.RegisterRequiredBuildComponentsForTest()
Changes this function so it only registers components from the cc
package by pushing the call to genrule.RegisterGenruleBuildComponents()
down into those packages whose tests need it.

This will make it easier to migrate cc package tests to test fixtures
as the RegisterRequiredBuildComponentsForTest() no longer overlaps with
preparers from the genrule packages.

Bug: 181070625
Test: m nothing
Change-Id: Ic00c7e480dc738d7a88d038aca6ab95a1502a24a
2021-03-05 18:20:33 +00:00
Paul Duffin ec0fe1775b Avoid hiddenapi ignoring prebuilt with missing dex implementation jar
Previously, when a prebuilt was preferred but did not provide a
suitable boot dex jar both the source and the prebuilt were silently
ignored which meant that the "hiddenapi list" command was not given a
complete set of boot dex jars. That could either lead to incorrect
hiddenapi flags being set or the "hiddenapi list" command failing if it
could not find a class. Debugging the cause of either of those cases
can be very time consuming so this change fails early and makes the
cause very explicit.

Bug: 181267622
Test: m nothing
Change-Id: I6763ddb9ba90ed2e501d0cf7984f6655237e905d
2021-03-01 12:58:28 +00:00
Paul Duffin eafc16bf14 Allow pre-singleton types to be registered in RegistrationContext
Bug: 181070625
Test: m droid
Change-Id: I708b78ed0b42ec55b0442307f40531cfe1233c2b
2021-02-24 10:17:19 +00:00
Treehugger Robot 277303f042 Merge "Add ctx to AndroidMkExtraEntriesFunc" 2021-02-22 22:40:31 +00:00
Colin Cross aa2555387d Add ctx to AndroidMkExtraEntriesFunc
Add a ctx parameter to AndroidMkExtraEntriesFunc to allow them to
access providers.

Test: m checkbuild
Change-Id: Id6becc1e425c3c3d8519248f8c0ce80777fac7cc
Merged-In: Id6becc1e425c3c3d8519248f8c0ce80777fac7cc
2021-02-19 23:05:40 +00:00
Pedro Loureiro 5d190cc24e Add support for lint baseline files
Test: m droid
Test: go test ^TestJavaLint # (from soong/build/java)

Change-Id: I249a0a0597b0bf8495460ed283b476ad2eb36edc
2021-02-18 11:15:30 +00:00
Paul Duffin 635aa0834f Make dexpreopt tools available for all java tests
Bug: 177892522
Test: m nothing
Change-Id: I594bddae692ef76dc8c3da504934d0151b13d4c9
2021-01-25 19:15:19 +00:00
Paul Duffin 9f04524673 Allow createGlobalSoongConfig() to be used from tests
Previously, the createGlobalSoongConfig() function was explicitly
prevented from being used in tests because it would fail. However, it
turns out that is no longer the case and it does now work.

That allows the following changes to be made:
* Tests no longer need to use GlobalSoongConfigForTests() to
  prepopulate the cache.
* GlobalSoongConfigForTests() is only needed in the dexpreopt
  package.

Bug: 177892522
Test: m nothing
Change-Id: Ifcbb1a44254c5d2d10c1d02ab23227488d1d1ed1
2021-01-21 15:44:30 +00:00
Paul Duffin c059c8c9a0 Add java.RegisterRequiredBuildComponentsForTest function
Insulate tests that exercise code in the java package from having to
register the build components provided by the java package by providing
a single function that registers them all. This follows the pattern
currently used in the cc and rust packages.

This change is in preparation for switching the dex_bootjars singleton
from a singleton, which does not require a module definition in order
to be instantiated, to a singleton module which does. That will require
adding a module definition into java.GatherRequiredDepsForTest() and
this change ensures that the required components will have been
registered in every test.

Bug: 177892522
Test: m nothing
Change-Id: I6475db8240894947dd07c89a940a3e4f201aa598
2021-01-21 11:47:32 +00:00
JaeMan Park 90e75350d7 Make TestJavaSdkLibraryEnforce faster
TestJavaSdkLibraryEnforce is too slow because it tests all
combinations of options. Change TestJavaSdkLibraryEnforce to
run test on specific test cases, not all combinations.

Bug: 177323052
Test: go test -timeout 10s -run ^TestJavaSdkLibraryEnforce$ android/soong/java
Change-Id: Ie7fe4e22b570a3e25259a6ad4bd37936805c6604
2021-01-15 13:24:09 +09:00
Jooyung Han e197d8b174 Add "aidl.flags:" property for cc_/java_ modules
The property can be used to pass additional flags to the AIDL compiler.
For example,

  cc_library {
    ..
    srcs: ["Foo.aidl"],
    aidl: {
      flags: [
        "-Werror",      // warnings as error
        "-Weverything", // turn on all warnings
      ],
    },
  }

Bug: 168028537
Test: soong test
Change-Id: I8120eeae7cd7b1acdd34c554af996a29e760a368
2021-01-05 10:40:22 +09:00
Jaewoong Jung 8bfb63c5a9 Merge "Break up app.go." 2020-12-28 17:50:44 +00:00
Jaewoong Jung f9b44657c8 Break up app.go.
Test: m nothing + TreeHugger
Change-Id: I64c6d7f10530c424bc282d8111dfaf9159426f00
2020-12-21 12:31:51 -08:00
Anton Hansson dff2c78a20 Add attribute to disable last-api compat tracking
Setting this to true by default is dangerous as it can mask bugs. Create
a dedicated attribute for java_sdk_library to enable this behavior
instead. The default will be flipped in a future CL when all the current
offenders have been fixed.

Fix all the tests to have the right API files.

Bug: 176092454
Test: m nothing
Change-Id: Ieab94bcb74abf8d018365a56fb447fe3dbd46957
2020-12-21 17:25:30 +00:00
JaeMan Park a4d314c91c Merge "Add java sdk library enforcement flag" 2020-12-02 04:39:10 +00:00
Colin Cross e5218b6be6 Merge "Annotate dependency tags for dependencies of installed files" 2020-11-24 16:01:35 +00:00
JaeMan Park ff71556a53 Add java sdk library enforcement flag
Add java sdk library enforcement for inter-partition library
dependency, for ensuring backward-compatible libraries for
inter-partition dependencies.

Test: m nothing
Bug: 168180538
Change-Id: I6bfac54c3499b03003a3bc6c2bb62b165b4ce5f9
2020-11-24 17:56:15 +09:00
Colin Cross e9fe2949b8 Annotate dependency tags for dependencies of installed files
Relands Ic22603a5c0718b5a21686672a7471f952b4d1017 with a minor
change to track libc++ dependencies for python hosts and after
a fix to an internal genrule that depended on transitively
installed java libraries (ag/13068670).

Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.

Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.

This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.

Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: I725871249d561428e6f67bba6a7c65b580012b72
2020-11-23 18:06:08 -08:00
Colin Cross c9fe10f5b8 Remove restriction on exported plugins that generate APIs
hilt_android requires seven separate annotation processors, which
is only feasible to support using exported_plugins to avoid having
to list all seven in every module that uses it.  Unfortunately they
all set generates_api: true.  Turbine is already disabled for modules
that directly use a plugin that sets generates_api: true, because
turbine doesn't run annotation processors.  Also add support for
disabling turbine if a module transitively uses a plugin that
generates APIs via exported_plugins.

Bug: 173397767
Test: TestExportedPlugins
Change-Id: If70354a3dd67efb4ce88bc9c934d41ccb6241b28
2020-11-23 11:42:26 -08:00
Paul Duffin b479459ac9 Merge changes from topic "fix-stubs-source-snapshot"
* changes:
  Fix prebuilt_stubs_sources to work with no stubs sources
  Revert "Use glob for java_sdk_library_import stub_srcs"
2020-11-19 18:39:02 +00:00
Treehugger Robot e63ab5ea02 Merge "java link time error improve" 2020-11-19 18:38:16 +00:00
Paul Duffin 1a39332cf6 Fix prebuilt_stubs_sources to work with no stubs sources
The framework-sdkextension java_sdk_library module defines an API for
public, system and module_lib API surfaces but the public API is empty.
The empty public API results in an empty .srcjar being repackaged and
merged into the sdkextension-sdk snapshot and results in no directory
for the public API stubs sources being created. Unfortunately, the
Android.bp file in the snapshot is created by Soong and it does not
know that the public API will be empty and so it creates an Android.bp
file that references the directory into which the stubs sources should
be added but which ends up not existing in the snapshot. Referencing a
non-existent directory causes a build failure.

This change fixes that issue by using PathForModuleSrc with no path
components to get the path to the module directory (which must exist)
and then resolving the module relative local src directory against
that. The local src directory is globbed to find all the files, which
will return an empty set of paths if the directory does not exist.
Finally, the file paths are passed as an rsp file to soong_zip to avoid
exceeding any command line limits.

Many other different approaches were considered:
* Adding a property to the java_sdk_library to indicate that the public
  API was actually empty. That would require extra maintenance by
  developers and would require some extra checks to be performed after
  generating the stubs source to ensure that it was empty which would
  complicate the build process.
* Creating a directory with some placeholder file (empty directories
  don't work well with git) that would force the creation of the
  directory. That file would most likely be created whether the API
  was empty or not, would need to be stored in git alongside the source
  and could be quite confusing to reviewers.

Bug: 173508731
Test: m nothing - to run new tests
      Build sdkextension-sdk, unpack it and then build the .srcjar
      files for the public, system and module_lib API surfaces.
      Without this change the build failed, reporting that the
      stubs_sources directory for the public API did not exist.
      With this change the build succeeded.
      Checked the contents of the resulting .srcjar files and made
      sure that the public one was empty and the others contained
      the SdkExtensions.java class and a package-info.java file.
Change-Id: Ia468a3f37349f2dbc21db67744bda6461498d515
2020-11-19 12:42:18 +00:00
Treehugger Robot b08a091502 Merge changes Iba57c949,Ief43ff51,Ib1809a4d,I2ab64f36
* changes:
  Store ndkKnownLibs in the config
  Register the kythe singleton on the Context instead of globally
  Store ninja file deps from PackageVarContext in the config
  Store SingletonMakeVarsProviders in the config
2020-11-18 20:00:31 +00:00
Steven Moreland 0029898a84 java link time error improve
Asked about error message including "against private API.Adjust
sdk_version", which is a bit hard to parse due to missing space. Also
tried to make error message less verbose, so that it is more clear what
to do, and fixed a grammar mistake.

Bug: N/A
Test: N/A
Change-Id: Ib9a30d86b5cb0e9b3b7d5576ecb9498a9b316042
2020-11-17 21:57:23 +00:00
Colin Cross 3d68051218 Move genrule on top of RuleBuilder
In preparation for more complicated sandboxing support in sbox, use
a single implementation of the sbox sandboxing by moving genrule to
use RuleBuilder's sbox support instead of creating an sbox rule
directly.

Also move genrule's input list hash support into RuleBuilder.

Test: genrule_test.go
Test: rule_builder_test.go
Change-Id: I292184d02743c7e6887ebbcd232ba565db2ab0cc
2020-11-17 11:23:45 -08:00
Colin Cross 06fa588d1a Store SingletonMakeVarsProviders in the config
Store SingletonMakeVarsProviders in the config instead of a global
variable to avoid races between tests running in parallel.

Test: all soong tests
Change-Id: I2ab64f368b5ac673fd985399d4421ed018abc562
2020-11-17 10:47:24 -08:00
Ulyana Trafimovich d8d8e726a5 Merge changes Iebfbf2ff,Ibd974268
* changes:
  Rename fields and methods to reflect class loader context changes.
  Do not add dependencies of shared SDK libraries to manifest_fixer.
2020-11-17 10:22:00 +00:00
Colin Cross e3d308b5a5 Merge "Revert "Annotate dependency tags for dependencies of installed files"" 2020-11-17 06:32:57 +00:00