Makes Path.RelativeToTop the source of truth as to what `relative to
top` means.
Uses it in PathRelativeToTop so that it will be tested by the existing
tests that use AssertPathRelativeToTop.
Also adds RelativeToTop() to WritablePaths and Paths.
Bug: 183650682
Test: m droid
Change-Id: I88dc56afd1314e51b4e41795a2448cab4ce8a899
First, the buildDir() method was renamed to getBuildDir() to avoid
clashing with the buildDir field.
Then, a buildDir was added to both `OutputPath` and `InstallPath` but
not to `PhonyPath` as it does not contain any path components. Instead
the `PhonyPath.getBuildDir()` was changed to simply return "".
Bug: 183650682
Test: m droid
Change-Id: I12e1854c829b980c5c01205753c62c00dc0a4774
This change needed to add some additional files to the registered
files for PrepareForTestWithJavaDefaultModules because otherwise they
would fail when "TestAllowNonExistentPaths = false". Those files were
being added by the TestJavaLintRequiresCustomLintFileToExist (albeit in
some cases in different locations to that required by the default
modules but as the files are needed by the modules defined in
PrepareForTestWithJavaDefaultModules they should be defined in it.
A couple of other places also provided some files so moving them into
PrepareForTestWithJavaDefaultModules caused some conflicts which needed
to be resolved.
Bug: 183184375
Test: m nothing
Change-Id: I76ce9f1673c1c1c4000635b76b8377d582224bf1
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.
It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.
Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.
Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
This is achieved by writing soong.environment.used in Main() instead of
as a side effect of a singleton. This makes a difference because build
actions are not generated when GENERATE_BAZEL_FILES=true is set,
therefore the side effect did not happen.
Arguably, Main() is made worse by this change, but I don't want to
tackle the problem of readably determining which mode soong_build is
running in in this change.
Test: Presubmits + the additional test.
Change-Id: I66af2429aedf008762173eaaa55b828b4cf4328b
The methods returns a copy of TestingBuildParams with every usage of a
path that is relative to the temporary test build directory with a path
relative to a notional top. Similar to how PathRelativeToTop does.
Bug: 182885307
Test: m nothing
Change-Id: I6ec20fc52ed76748138f0d48b4df80f765dfcfdc
When searching for an TestingBuildParams this change will compare the
supplied path against the value returned by PathRelativeToTop() as well
as the String() and Rel() values. In the event that it cannot find the
file it also formats the available outputs, one per line with both rel
and path relative to top.
Bug: 182885307
Test: m nothing
Change-Id: Icbd37018b9a028abf37e27d1bea12a407316e031
Allows the removal of the provider parameter and makes it easier to add
new functionality to baseTestingComponent.
Bug: 182885307
Test: m nothing
Change-Id: Ie8ac600cbce982f2c5fc24a22968efe0c2a29d0c
The existing NormalizePathForTesting function does not handle make
install paths very well, as it returns a relative path with a leading
"../" which is very confusing. It also does not clearly differentiate
between the different paths.
These functions return paths that are basically what are seen in a
normal developer build, i.e.
* <source path>
* out/soong/<soong output path>
* out/<make output path>
That makes tests that use them easier to understand.
Follow up changes will clean up the existing usages of the
Normalize... functions.
Bug: 182885307
Test: m nothing
Change-Id: I17ddc996bef5bbbf4a62da8334ea6ce29e306109
This change tries to distinguish between the preparations needed for an
integration test and the preparations needed for more unit-testy tests.
Integration tests exercise most, if not all of Soong (at least the
parts that are available when running tests in a specific package) and
as a result can take a long time. Other tests, while being as realistic
as possible, should only use a subset of the preparations in order to
reduce run time.
Registering module types has very little overhead so it is simplest to
try and register those all together. However, care needs to be taken
for singletons and mutators as they can do a lot of work. Especially
with tests that include lots of default module definitions such as are
created by java.GatherRequiredDepsForTest().
Once it is possible to use test fixtures in tests it will be much
easier to customize tests individually and so avoid just adding new
test specific content to the common deps.
Bug: 181070625
Test: m nothing
Change-Id: Ia0a676297224d90df947054746e7f99563ffb323
It is no longer necessary to use preparers in a specific order now that
the test infrastructure enforces a consistent registration order.
So, this change:
* Sorts preparers alphabetically to make them easier to maintain.
* Creates a single visiblity preparer for all the visibility mutators.
* Adds PrepareForTestWithPackageModule
Bug: 181070625
Bug: 181953909
Test: m nothing
Change-Id: I9e65dd182be664d8d0d5db7ccc0b63fa315129c8
Previously, the TestContext.Register() method stored all the components
that were available, rather than just the ones that were used by the
tests. That was practically useless for debugging. This change just
stores the ones used by the test.
Bug: 181953909
Test: m nothing
Change-Id: I8589b9b4423133709ab0cfd763bd284a7f47a24a
In preparation for allowing the TestContext to enforce an ordering on
the registration of singletons and pre-singletons in the same way as it
does for mutators this defers the registration of them into the
underlying Context.
Bug: 181953909
Test: m nothing
Change-Id: I2d9652122bb6387b6b47ca4761e811885d15c2b6
This change uses the order in which mutators are registered at runtime
to define the order in which mutators must be registered in tests to
improve test reliability and prevent issues like bug 181974714.
Generally, it simply sorts the test mutators into the same order as
used at runtime. However, if the test includes a mutator that is not
used at runtime then it cannot sort them because it does not have
enough information to know where it should appear in that order. So,
instead it simply checks the order and makes sure that it matches.
Allowing relationships between mutators to be explicitly defined, e.g.
mutator X should come after mutator Y but before mutator A would fix
that information gap and allow them to be sorted but that is outside
the scope of this piece of work.
The code here is written generically for a sortableComponent as
follow up changes will sort singletons and pre-singletons in the same
way.
Bug: 181953909
Test: m nothing
Change-Id: Ib7d421f578e25f6dccaaff4f73b69838d1b54b00
This separates the collation of mutators from the registration of them
to allow the test infrastructure to sort the mutator order to match
that used at runtime.
Bug: 181953909
Test: m nothing
Change-Id: I01a073289d44417f327b0815c09eb1c033d464f2
In preparation for following changes that will ensure the order of
registration (and so execution) for mutators, singletons and
pre-singletons in tests match the order in runtime this change creates
the sortableComponent interface to allow those to be sorted in the same
way without having to resort to reflection.
By moving the registration code into each component type this change
also eliminates some unnecessary duplication of that code.
Bug: 181953909
Test: m nothing
Change-Id: I597b461b966c84faaeb13e7dff765f1fadd99981
A lot of existing tests that expect errors to be reported do not check
the returned result. This change treats failures of the error handler
in that situation as fatal.
The default error handler already behaved that way. The others did not.
FixtureExpectsAllErrorsToMatchAPattern uses
CheckErrorsAgainstExpectations and it was safe to just make that func
treat any unmatching errors as fatal as all the existing usages are at
the end of test functions.
FixtureExpectsAtLeastOneErrorMatchingPattern uses the
FailIfNoMatchingErrors function which is used in a number of places
(including CheckErrorsAgainstExpectations) that do not want to treat
a test failure as fatal. So, that was modified to return false if no
matching error was found and the error handler treated that as fatal.
Bug: 181070625
Test: m nothing
Change-Id: I6e4df53f93250348bc050d4ff098134e6314ae30
FixturePreparers is a bad name for a method that creates a collection
of FixturePreparers for a couple of reasons:
* Conventionally it would be used as the name for []FixturePreparer if
it is necessary to add behavior to that.
* There are many different types of collection, particularly when order
matters.
Bug: 181070625
Test: m nothing
Change-Id: I55394ff369375dcac2d7b72e4d803a4818762d36
Adds the test fixture support and converts a few tests to exercise the
code and show how it works.
Bug: 181070625
Test: m nothing
Change-Id: I0a2b40fff93b6041f9aa8c4ef0aba91da1bc8bf3
Add a ctx parameter to AndroidMkExtraEntriesFunc to allow them to
access providers.
Test: m checkbuild
Change-Id: Id6becc1e425c3c3d8519248f8c0ce80777fac7cc
Merged-In: Id6becc1e425c3c3d8519248f8c0ce80777fac7cc
This currently expands all globs, still need to support converting glob
syntax.
Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
The previous implementation relied on the implicit registration of Bp2Build mutators, resulting in test non-hermeticity. Refactor bp2build tests to explicitly specify the bp2build mutators under test.
Test: Soong tests
Test: TH
Change-Id: I9b9674bad1ea533b3bd31b07077a9e02c99b4c1d
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.
See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.
Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.
Change-Id: I33a96365bd439043b13af6db9e439592e9983188
Dexpreopt and boot jars package check all require access to dex
implementation jars created for java_library and java_sdk_library. They
were available when building from source but not when building from
prebuilts, even though they are embedded within the .apex files that
are referenced from prebuilt_apex.
This changes adds support to prebuilt_apex to export the dex
implementation jars and updates java_import to use those exported dex
implementation jars.
In a source build dexpreopt/boot jars package check access the apex (or
platform) specific variant of a java_library, e.g. core-oj, from which
it retrieves the dex implementation jar path.
After this change in a prebuilt build dexpreopt/boot jars package check
behave in the same way except in this case they retrieve the dex
implementation jar path from the apex (or platform) specific variant of
the java_import, e.g. core-oj.
The work to export files from a `.apex` file for use by other modules
is performed by a new `deapexer` module type. It is not used directly
in an `Android.bp` file but instead is created implicitly by
`prebuilt_apex`,
In order to do that this contains the following changes:
* Adds a new `dexapexer` module type to handle the exporting of files
from the `.apex` file.
* Adds an exported_java_libs property to prebuilt_apex to specify the
set of libraries whose dex implementation jars need exporting.
* Creates apex specific variants of the libraries listed in the
exported_java_libs property.
* Adds the set of exported files to the ApexInfo to make them available
to the apex specific variants.
* Prevents the prebuilt_apex variants from being merged together as
they will not be compatible.
* Modifies java_import to use the exported file for variants of a
prebuilt_apex.
* Adds a ninja rule to unpack (using deapexer) the contents of the
prebuilt_apex's apex file, verify that the required files are present
and make them available as outputs for other rules to use.
* Some minor refactorings to support these changes.
* Adds tests to cover prebuilt only, prebuilt with source preferred,
and prebuilt preferred with source.
Test: m nothing
Bug: 171061220
Change-Id: Ic9bed81fb65b92f0d59f64c0bce168a9ed44cfac
The build has some implicit dependencies (via the boot jars
configuration) on a number of modules, e.g. core-oj, apache-xml, that
are part of the java boot class path and which are provided by mainline
modules (e.g. art, conscrypt, runtime-i18n) but which are not otherwise
used outside those mainline modules.
As they are not needed outside the mainline modules adding them to
the sdk/module-exports as either java_libs, or java_header_libs would
end up exporting more information than was strictly necessary. This
change adds the java_boot_libs property to allow those modules to be
exported as part of the sdk/module_exports without exposing any
unnecessary information.
Some points to note:
* The java_import has to have a valid file for the src property
otherwise it will be disabled.
* The src property is supposed to reference a jar file but the
java_boot_libs property will make it reference an empty file (not
an empty jar) so that any attempt to use that file as a jar, e.g.
compiling against it, will cause a build failure.
* The name of the file passed to the src property should make it
clear that the file is not intended to be used.
* The test makes sure that only the jar file is copied to the
snapshot.
Test: m nothing
Bug: 171061220
Change-Id: I175331e4c8e3874ab70a67cdc2f76ed1576e41eb
A SingletonModule is halfway between a Singleton and a Module. It has
access to visiting other modules via its GenerateSingletonBuildActions
method, but must be defined in an Android.bp file and can also be
depended on like a module.
Bug: 176904285
Test: singleton_module_test.go
Change-Id: I1b2bfdfb3927c1eabf431c53213cb7c581e33ca4
Store SingletonMakeVarsProviders in the config instead of a global
variable to avoid races between tests running in parallel.
Test: all soong tests
Change-Id: I2ab64f368b5ac673fd985399d4421ed018abc562
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.
Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
Similarly to Output, we return the list of Rules that have been
generated for TestingModule. This helps debugging failing tests.
Test: m nothing
Change-Id: I3542f4e4632f94fb84208c2e48e629271a373fd4
APEX variants that share the same SDK version and updatability
almost always use identical command line arguments to build but
with different intermediates directories. This causes unnecessary
build time and disk space for duplicated work.
Deduplicate APEX variants that would build identically. Create
aliases from the per-APEX variations to the new shared variations
so that the APEX modules can continue to depend on them via the
APEX name as the variation.
This has one significant change in behavior. Before this change,
if an APEX had two libraries in its direct dependencies and one
of those libraries depended on the other, and the second library
had stubs, then the first library would depend on the implementation
of the second library and not the stubs. After this change, if
the first library is also present in a second APEX but the second
library is not, then the common variant shared between the two
APEXes would use the stubs, not the implementation.
In a correctly configured set of build rules this change will
be irrelevant, because if the compilation worked for the second
APEX using stubs then it will work for the common variant using
stubs. However, if an incorrect change to the build rules is
made this could lead to confusing errors, as a previously-working
common variant could suddenly stop building when a module is added
to a new APEX without its dependencies that require implementation
APIs to compile.
This change reduces the number of modules in an AOSP arm64-userdebug
build by 3% (52242 to 50586), reduces the number of variants of the
libcutils module from 74 to 53, and reduces the number of variants
of the massive libart[d] modules from 44 to 32.
This relands I0529837476a253c32b3dfb98dcccf107427c742c with a fix
to always mark permissions XML files of java_sdk_library modules as
unique per apex since they contain the APEX filename, and a fix
to UpdateUniqueApexVariationsForDeps to check ApexInfo.InApexes
instead of DepIsInSameApex to check if two modules are in the same
apex to account for a module that depends on another in a way that
doesn't normally include the dependency in the APEX (e.g. a libs
property), but the dependency is directly included in the APEX.
Bug: 164216768
Test: go test ./build/soong/apex/...
Change-Id: I2ae170601f764e5b88d0be2e0e6adc84e3a4d9cc
Only print the list of variants of the matching module when
ctx.ModuleForTesting finds the module but not the variant.
Test: all soong tests
Change-Id: I51ffdde4645db39ec1d37ec018e0dea11d74280e
Previously, while sdk tests would pass the sdk code would often fail
in androidmk processing. This change makes the tests more realistic
and will catch the errors earlier.
Bug: 142935992
Test: m nothing
Change-Id: Ifd0b2d7cf24e941c919f6b6e0beb2403a67d4308
This is for use to add dependencies between specific module variants,
without getting the dependencies potentially duplicated as more
variants are created.
Moved genrule tool dependency registration to this phase, to avoid
potential splitting of its dependencies, and for consistency with the
same kind of dependency on dex2oat in https://r.android.com/1205730.
Test: m nothing
Bug: 145934348
Change-Id: I1f9e75354e359cef5f41d14e5016b0a67d7655d7
Some pre arch mutators are hard coded into mutator.go and so could not
share code for registering those mutators between tests and runtime.
This change adds a new HardCodedPreArchMutators(RegisterMutatorFunc)
method to RegistrationContext which allows hard coded mutators to be
registered alongside other build components during init() and testing.
The method is treated as a noop on the InitRegistrationContext and
behaves just like the PreArchMutators(RegisterMutatorFunc) method on
the TestingContext.
Bug: 146540677
Test: m nothing
Change-Id: I6f8b1e2d54d9dc4e86f951ced61d1ee7b0fe4b2e
Make android.AddLoadHook wrap blueprint.AddLoadHook. Also pass
the config object to ParseBlueprintsFiles.
Test: all soong tests
Change-Id: I60c988b717d395f52498ec23ef7c9046d9861a6e
Changes prebuilt_stubs_sources to generate a .srcjar from its input
instead of just exposing the srcs it is given. This ensures that it can
be used as a drop in replacement for a droidstubs module.
Updates the test for prebuilt_stubs_sources to be more representative
of the actual use made of it by sdk snapshot which outputs a directory
not a glob pattern. Added some documentation of the
prebuilts_stubs_sources srcs property to make it clear that it is
supposed to be a set of directories.
Extracts common code from sdk/testing.go for normalizing path/paths
for testing.
Bug: 143678475
Test: m conscrypt-module-sdk conscrypt-module-host-sdk conscrypt-module-test-sdk
unzip those in place of external/conscrypt
build core-current-stubs-source which expects it to provide a .srcjar.
Change-Id: I8204a022557a9b0b45e19eac79ecba98ff95213d
The filesystem object was available through ModuleContext.Fs(), but
gives too much access to the filesystem without enforicing correct
dependencies. In order to support sandboxing the soong_build
process move the filesystem into the Config. The next change will
make it private.
Bug: 146437378
Test: all Soong tests
Change-Id: I5d3ae9108f120fd335b21efd612aefa078378813
AndroidMkEntries now returns multiple AndroidMkEntires so that a module
can emit multiple Make modules if needed.
Bug: 128708192
Test: m
Change-Id: I56b6f76d22943b80329951c5acb80a1b932441ad
Avoid having to pass ModuleFactoryAdaptor to every call to
RegisterModuleType in a test by wrapping RegisterModuleType.
Test: all soong tests
Change-Id: If8847d16487de0479cc3020b728256922b3cadba
Paul Duffin