Commit Graph

93 Commits

Author SHA1 Message Date
Liz Kammer 4562a3b218 Add bp2build arch-specific paths mutator
Adds deps for properties tagged `android:"path"` within arch, multilib,
and target properties.

Test: build/bazel/ci/bp2build.sh
Test: m nothing
Bug: 185217298
Change-Id: I0230da399d2c4e984b837f69523fa09eadba3ff1
2021-04-21 18:21:50 -04:00
Paul Duffin b00c7ed2e8 Merge "Convert TestApexPermittedPackagesRules to use test fixtures" 2021-03-31 21:40:09 +00:00
Liz Kammer acacbc1166 Merge "Handle product_variable asflag for cc_object." 2021-03-31 18:43:55 +00:00
Paul Duffin 45338f05e3 Convert TestApexPermittedPackagesRules to use test fixtures
Adds PrepareForTestWithNeverallowRules to make it easy to test
neverallow rules. Avoid exporting any unnecessary neverallow related
methods from the android package.

Bug: 181070625
Test: m nothing
Change-Id: Idfc6955cb23f1a4d1790be7879388154b03f3980
2021-03-31 16:16:18 +01:00
Liz Kammer a060c4521e Handle product_variable asflag for cc_object.
cc_object crtbrand sets product_variable.platform_sdk_version.asflag
and will not compile correctly within mixed builds without it.

Only handles product_variables that expand product variables.

Bug: 181794963
Test: ~/aosp/build/bazel/scripts/milestone-2/demo.sh full
Change-Id: I293fcb18032aa51f63bb7b3de94abd6d1ec38180
2021-03-30 15:16:43 -04:00
Jingwen Chen 12b4c2706d bp2build: add allowlist for package-level conversions.
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.

It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.

Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.

Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
2021-03-24 02:27:19 -04:00
Paul Duffin c05b034e95 Separate the collation of mutators from registration
This separates the collation of mutators from the registration of them
to allow the test infrastructure to sort the mutator order to match
that used at runtime.

Bug: 181953909
Test: m nothing
Change-Id: I01a073289d44417f327b0815c09eb1c033d464f2
2021-03-07 12:33:07 +00:00
Paul Duffin 1d2d42f8e9 Add sortableComponent abstraction
In preparation for following changes that will ensure the order of
registration (and so execution) for mutators, singletons and
pre-singletons in tests match the order in runtime this change creates
the sortableComponent interface to allow those to be sorted in the same
way without having to resort to reflection.

By moving the registration code into each component type this change
also eliminates some unnecessary duplication of that code.

Bug: 181953909
Test: m nothing
Change-Id: I597b461b966c84faaeb13e7dff765f1fadd99981
2021-03-07 12:33:07 +00:00
Liz Kammer fc46bc1ee4 Refactor BazelTargetModule
This eliminates the need to remove quotes, delete attributes, and
re-checking that name has correct prefix. Additionally, this allows
assignment directly to the BazelTargetModuleProperties struct, which
allows defaulting unused fields and clarity of which field is being set.

Test: go test soong tests
Test: ran ./build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ia9bfcce76234c793a4ddd5f29a661150f83341c9
2021-02-23 08:29:40 -05:00
Jingwen Chen fb4692a743 bp2build: refactor BazelTargetModule naming boilerplate.
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.

Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
2021-02-08 04:10:18 -05:00
Jingwen Chen 1fd14691dd bp2build: Refactor CreateBazelTargetModule API.
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.

Test: soong tests

Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
2021-02-05 09:24:55 -05:00
Liz Kammer 356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Jingwen Chen 041b184c92 bp2build: flatten *_defaults properties.
This CL brings the defaults mutators into bp2build, reusing the
implementation of flattening defaults from *_defaults module deps into
the modules themselves.

This is an approach to converge defaults semantics with Bazel, which
doesn't have existing equivalent abstractions, other than macros, which
require more infrastructure to be built.

Test: TH
Test: build_conversion_test.go
Change-Id: Ia1443f8e6c3eb277fa861bb96625425a22cfc6d7
2021-02-01 00:33:30 -05:00
Jingwen Chen 7385067640 bp2build: framework for generating BazelTargetModules.
This CL creates the framework necessary for generating
BazelTargetModules from regular Soong Android modules.
BazelTargetModules are code-generated into Bazel targets in BUILD files.

See the follow-up CL for examples of creating filegroup/genrule
BazelTargetModules.

Test: GENERATE_BAZEL_FILES=true m nothing # creates out/soong/bp2build
with no BUILD files, because there are no BazelTargetModules in the
module graph.

Change-Id: I33a96365bd439043b13af6db9e439592e9983188
2021-01-21 22:46:11 -05:00
Bob Badour 37af046002 Revert^2 "Add ability to declare licenses in soong."
df98d3e4a5

Change-Id: Ie7e1695ecb8cd943124426a0cad2c0d1db2b46ed
2021-01-06 20:48:24 -08:00
Jerome Gaillard df98d3e4a5 Revert "Add ability to declare licenses in soong."
Revert submission 1377717-metalics

Reason for revert: This has broken renderscript_mac target for aosp-master, see b/176909442

Reverted Changes:
I26ac54ca9:Define the standard license_kind rules.
I656486070:Export soong license data to make.
If9d661dfc:Export soong license data to make.
I97943de53:Add ability to declare licenses in soong.
Icaff40171:Rough-in license metadata support to make.
Ib8e538bd0:Add variables for notice deps, license kinds etc.

Change-Id: Iddf5468e0175831ddb642011f2768f87a5e4fcf2
2021-01-06 19:00:05 +00:00
Bob Badour 8a36d94714 Add ability to declare licenses in soong.
See: http://go/android-license-checking-in-soong-v2-design

Bug: 151953481
Bug: 151177513
Bug: 67772237

Change-Id: I97943de53b071cd9918679c17015ed3397c172e9
2021-01-05 08:42:48 -08:00
Jingwen Chen 4133ce6912 Add bp2build mode to soong_build.
This CL adds a new CONVERT_TO_BAZEL env var, and a bp2build goal. It
reuses the queryview architecture, but registers no mutators before
converting to BUILD files. This gives us a blank slate of the module
graph to work with, and create a shadow/alternate pipeline of mutators
to converge the Bazel BUILD graph to be semantically equivalent with the
Android.bp graph (after apply its current set of mutators).

The command to do so is:

    $ CONVERT_TO_BAZEL=true m bp2build

To not clobber with queryview, the generated files are in
out/soong/bp2build.

Test: CONVERT_TO_BAZEL=true m bp2build && bazel query --config=bp2build //...
Test: m queryview && bazel query --config=queryview //..
Test: soong tests
Test: TH presubmit
Fixes: 174465461
Signed-off-by: Jingwen Chen <jingwen@google.com>
Change-Id: I4dd0ccc73abc345d70a50ca2803d6f400cd8c863
2020-12-03 23:36:57 -05:00
Colin Cross d27e7b8e45 Add providers support
Propagate the providers methods from
https://github.com/google/blueprint/pull/309 to Soong.

Test: m checkbuild
Change-Id: Iad7a9023df4421cd01dbb0518be0e85382097481
2020-09-17 18:57:03 -07:00
Colin Cross 4f1dcb0e40 Return new dependencies from BottomUpMutatorContext.Add*
Propagate new return values of the BottomUpMutatorContext.Add*
methods from https://github.com/google/blueprint/pull/316 to
Soong.

Test: m checkbuild
Change-Id: I5378db46adf2f31cc4222b4619cc39db53791610
2020-09-16 21:06:00 -07:00
Colin Cross 9f35c3d6dd Copy documentation from Blueprint interfaces into Soong interfaces
Copy documentation from Blueprint for interface methods that are
wrapped into Soong interfaces.

Test: m checkbuild
Change-Id: Ib58c9201b8e7dfbf900a252e709f0a3cfed73286
2020-09-16 21:06:00 -07:00
Colin Cross 617b88a288 Create os and arch variants for bootstrap Go modules
AddFarVariationDependencies was broken, which allowed modules to add
dependencies on bootstrap Go modules using os and arch variations
even though boostrap Go modules did not have any variations.  Mutate
bootstrap Go modules in the os and arch mutators so that they have the
expected os and arch variations.

This relands Ida7bc75a51ab1d2d38a6be11f574399097380cc9 with fixes
to also mutate *bootstrap.goPackage modules so that they can find a
matching *bootstrap.goBinary module when creating their pluginFor
dependency, and to create a variation for linux_bionic when it is
enabled so that linux_bionic builds can package Go binaries.

Also requries https://github.com/google/blueprint/pull/320 to
support multiple variants of bootstrap Go modules.

Test: all soong tests
Change-Id: Ibc627aa262c298a076d171a4c21b0b9c2a7b0ada
2020-09-09 14:44:57 -07:00
Colin Cross 1b9604b463 Add CreateAliasVariation to BottomUpMutatorContext
Propagate CreateAliasVariation from Blueprint to Soong.

Bug: 164216768
Test: m checkbuild
Change-Id: I635bf67481092b83a8d26ef1cb06e2e3e21a595f
2020-08-13 17:50:05 -07:00
Paul Duffin 80342d72d0 Restrict replacements of source dependencies with prebuilts
Previously, when java_sdk_library_import was preferred over a
java_sdk_library the latter ends up depending on the prebuilt child
modules created by the java_sdk_library_import instead of the source
child modules that it created itself. That was because all dependencies
on those source child modules were replaced with the corresponding
prebuilt child modules.

This change prevents those dependencies from being replaced to preserve
the dependencies from java_sdk_library onto its source child modules by
making the replacement conditional depending on the tag used. It also
updates the affected test.

Bug: 159902351
Test: m nothing
Change-Id: I4441b901dedfd44b9769df1ac2e248b94834cf85
2020-07-08 16:46:16 +01:00
Paul Duffin 44f1d8404b Stop java_sdk_library_import from depending on source modules
Previously, java_sdk_library_import added the dependencies on its child
components in the deps mutator after prebuilts without a matching
source module are renamed to the source module name. That meant that
the java_sdk_library_import has to use the source module name and ended
up depending on the source module unless it was preferred.

This change adds a new component deps mutator that runs before the
PrebuiltRenameMutator so that the java_sdk_library_import can add
dependencies onto the prebuilt modules. It also updates an affected
test.

Bug: 159902351
Test: m nothing
Change-Id: I3576c4873302743e51aff547ea1497bef6d748ac
2020-07-08 16:46:16 +01:00
Paul Duffin afa9fa104d Add hook to be called after defaults have been applied
Previously, the only way for a module type to create modules (other
than defining its own mutator) was to register a LoadHook. However,
that is called before defaults are applied so any properties used in
that hook cannot take advantage of defaults, e.g. java_sdk_library
cannot use defaults to set its sdk_version property and have that
affect its child modules.

This change adds a new SetDefaultableHook() to DefaultableModule to
register a hook that is called after any defaults have been applied.

Also adds some tests to ensure that errors in the visibility property
introduced in a DefaultableHook are reported in the gather phase of
visibility processing.

A follow up change will switch java_sdk_library to use that instead
of the AddLoadHook() mechanism.

Bug: 155295806
Test: m checkapi
Change-Id: I13df3115f9e225f7324b6725eaeb16a78cc2538a
2020-05-06 08:55:19 +01:00
Paul Duffin c988c8e202 Move prebuilts mutators after defaults have been applied
A follow up change will add a mechanism for a module to register a hook
that is called after any defaults have been applied. That is intended
for use by modules like java_sdk_library that create child modules that
are dependent upon properties that could be supplied by defaults.

Creating those child modules after prebuilts mutators are run will
cause problems if those child modules clash with prebuilts modules.
Moving the prebuilts mutators after the defaults mutators will fix
that.

Tests are currently being run with the mutators in different orders so
this change also cleans that up so they are consistent with the actual
code that is being run.

Bug: 155295806
Test: m checkbuild
Change-Id: I825c6df09058fb3a45db196661959eb332aca2f3
2020-05-06 08:55:16 +01:00
Paul Duffin cdfcec98bb Simplify package by using LoadHook instead of mutators
This was not previously possible due to LoadHooks being run after the
module was registered.

Bug: 155462403
Test: m nothing
Change-Id: Ia8383b9d1272bb12c8a83948753a0e4b0d98a650
2020-05-05 20:47:04 +01:00
Paul Duffin aa4162e71e Document preArch mutators
Bug: 155295806
Test: m checkbuild
Change-Id: I4cd6647a8707f49363d297a3b3082ef19ba93868
2020-05-05 11:35:43 +01:00
Artur Satayev c5570ac9b1 Allow defining neverallow in packages other than android.
This exposes test func on ValueMatcher and helpers relevant for testing neverallows.

Bug: 153333044
Test: m
Change-Id: Ied07cd33afa537f9d7aa3c33e59f4ac985901a0f
2020-04-15 12:06:13 +01:00
Colin Cross 43e789d667 Follow input changes to proptools.CloneEmptyProperties
Test: m checkbuild
Change-Id: I1fd53d03722d134009f7ed663f05bd6dc5980dd1
2020-01-28 12:17:06 -08:00
Martin Stjernholm 710ec3aef2 Add FinalDeps mutator phase, where CreateVariations is disallowed.
This is for use to add dependencies between specific module variants,
without getting the dependencies potentially duplicated as more
variants are created.

Moved genrule tool dependency registration to this phase, to avoid
potential splitting of its dependencies, and for consistency with the
same kind of dependency on dex2oat in https://r.android.com/1205730.

Test: m nothing
Bug: 145934348
Change-Id: I1f9e75354e359cef5f41d14e5016b0a67d7655d7
2020-01-21 11:40:02 +00:00
Colin Cross 31a738bda0 Use blueprint's load hooks
Make android.AddLoadHook wrap blueprint.AddLoadHook.  Also pass
the config object to ParseBlueprintsFiles.

Test: all soong tests
Change-Id: I60c988b717d395f52498ec23ef7c9046d9861a6e
2020-01-06 10:57:26 -08:00
Colin Cross 87f790c509 Delete arch hooks
They are not used anywhere.

Test: m checkbuild
Change-Id: Ice7d68b97ca894c0c4c9efc6dc55c0870a132b9c
2020-01-03 17:21:04 +00:00
Colin Cross fb0c16e95a Move imageMutator before archMutator
Run the imageMutator between osMutator and archMutator so that
different arch variants can be set for the different partitions.

Bug: 142286466
Test: m checkbuild
Change-Id: I65d05714b75aa462bf9816da60fdc2deda4de593
Merged-In: I65d05714b75aa462bf9816da60fdc2deda4de593
(cherry picked from commit 9c8f687584)
2019-12-17 14:45:11 -08:00
Paul Duffin 593b3c9fb0 Ensure prebuilt modules have same visibility as source modules
Exports visibility and package mutator registration functions so they
can be used in sdk testing. Updates sdk test to support visibility and
package modules.

Adds EffectiveVisibility(...)[]string function to make the effective
visibility rules available to sdk snapshot creation.

Extracts compositeRule.Strings() []string from compositeRule.String()
method so that it can be used by above func.

Adds visibility property to sdk snapshot and prebuilt modules along
with a test to ensure it works properly.

Adds dir parameter to CheckSnapshot so that it can check the snapshot
generated for a non-root package. That is required in order to ensure
that visibility of :__subpackages__ on a source module in package
<pkg> is resolved to an effective visibility of
//<pkg>:__subpackages__ on its corresponding prebuilt.

Test: m conscrypt-module-sdk
Bug: 143678475
Change-Id: Icaacac5b9c04726d28e6fec93e49715ac45df7f4
2019-12-09 13:32:28 +00:00
Colin Cross ae6c5207cc Move ImageMutator after archMutator
Move the ImageMutator to be registered just after the archMutator
in preparation for moving it between osMutator and archMutator.
Requries updating variants in a few tests that now run the
ImageMutator.

Bug: 142286466
Test: no change to build.ninja
Test: all soong tests
Change-Id: Ia9d2a7bc0e225bedec3c9a83ea04f471a931bf47
2019-12-06 12:37:14 -08:00
Colin Cross 43b92e0d0f Make CreateVariations return []android.Module
Test: soong tests
Change-Id: I9bc40642c4902392eb8193bf73e32b7f1c18c14e
2019-11-19 10:33:08 -08:00
Jaewoong Jung 9f88ce26a8 Add AliasVariation to BottomUpMutatorContext.
Test: TreeHugger
Change-Id: Ib9c5db8f671ed8f0742183fc533dcc582269d17a
2019-11-15 10:57:34 -08:00
Colin Cross a195f91471 Split out osMutator from archMutator
Split the archMutator into two mutators, osMutator to create
android/linux/darwin variants, and archMutator to then further
split them into arm/arm64, etc.  When combined with alias
variants in Blueprint this will allow us to depend on the
"android" variant without having to know what architectures
were used to build it.

Test: m checkbuild
Change-Id: I0f51d60fc440cf5ae503904ce7849ec797af5fe2
2019-10-16 15:27:23 -07:00
Colin Cross 18c46807c9 Filter product variable property structs
For each module, take the default product variable properties and
create a new type that contains only the properties that exist in
the current module.  This will provide better errors when attemping
to set product variable properties that will never get used for
a module, and fixes errors when a module attempts to use a product
variable that also contains properties that don't apply to the
current module.

Fixes: 79249983
Test: TestProductVariables
Test: m checkbuild
Change-Id: I4dc24f7781b8ce798651cfc8bf3563005a92c13d
2019-09-26 15:03:07 -07:00
Colin Cross e003c4abdd Make CreateModule return the newly created module
Allow mutators to modify properties of the newly created module
by returning it.

Test: m checkbuild
Change-Id: I682caca86c0c8f7c3ae815a494d4c75962c8e2e8
2019-09-26 15:03:07 -07:00
Colin Cross 84dfc3d331 Make CreateModule take an android.ModuleFactory
Reduce the boilerplate required to call CreateModule by taking an
android.ModuleFactory instead of a blueprint.ModuleFactory.

Test: m checkbuild
Change-Id: I1259d2dd3f7893b5319c333bc180727ac40f9e91
2019-09-26 17:19:26 +00:00
Steven Moreland 349cd474ef CreateModule also inherits variableProperties.
Bug: N/A
Test: manual
Change-Id: Ic5ecbeac04841d4016c0caa8710a6adae1ec2bd6
2019-09-06 18:28:58 -07:00
Jiyong Park 1d1119f4bd Fix sanitizer dep
This change fixes a problem in sanitizerMutator where a module is linked
with of non-sanitized variant of a lib at build-time, but is linked with
the sanitized variant of the lib at run-time.

This happened because, for each sanitizer type, every shared libs are
split into non-sanitized and sanitized variants, and then either of the
variants are suppressed from Make so that it isn't installed to the
device.

This change fixes the problem by NOT splitting for shared libs; only the
sanitized variant is created if needed. Header libs, static libs and
shared libs for a few sanitizer types (asan/fuzzer) are however split
into two. This is because the static and headers libs become part of the
depending module, and asan/fuzzer require that the depending module and
the dependant module should be compiled for the same sanitizer.

Bug: 138103882
Bug: 138426065
Test: m com.android.runtime.debug
Check that libziparchive exists under both
/system/apex/com.android.runtime/[lib|lib64]

Change-Id: Ia447785c485c0d049e19477b32bc638bfe6f1608
2019-08-08 01:51:26 +09:00
Steven Moreland 12f23e0883 Automatically inherit common properties.
Before, the values were often simply ignored. If bad values are provided
now, we should be able to catch them.

Bug: 119771576
Test: relying on this for AIDL/HIDL
Change-Id: I8d2ff26da0b2d01ebe7f78c26d69c7b618a65367
2019-07-30 22:42:19 +00:00
Colin Cross 9a36223d59 Add a String() method to android.Module
Add a String() method to android.Module for use in debugging.  Store
the name and variations of the module as they are mutated.

Test: TestModuleString
Bug: 136473661
Change-Id: I74e393703dcfc96ed4e21ac4a4419a7858b59216
2019-07-03 15:37:11 -07:00
Colin Cross cb55e089f0 Add MutatorName to mutator contexts
Follow https://github.com/google/blueprint/pull/255.

Bug: 136473661
Test: m checkbuild
Change-Id: Ia74d442c925689fa1e06a70c2acfed925063f844
2019-07-03 14:16:42 -07:00
Paul Duffin e2453c705f Allow default visibility to be set per package
Adds a package module type with a default_visibility property. The
package module type can only be specified once per package.

Bug: 133290645
Test: m droid
Change-Id: Ibb2fb499c9ea88ecaa662d3cd2cbde478e4b9a4b
2019-06-27 13:24:50 +01:00
Colin Cross dc35e21124 Consolidate *MutatorContext and ModuleContext into BaseModuleContext
Following 99bdb2ab4fd5168a71a20aecf10611425be47ec4 in build/blueprint,
move more methods into BaseModuleContext.

This reapplies I9f8df94f1ae2b55d3cccf7b9468247f3e7cb2ebd after fixing
missing errors thrown for missing defaults modules when
AllowMissingDependencies == true.

Test: m checkbuild
Test: defaults_test.go
Change-Id: Ia17b2bcbf2bac6889c419b2e73953946f6aa40ad
2019-06-10 16:30:35 -07:00