Commit Graph

54 Commits

Author SHA1 Message Date
Jingwen Chen 8b086655cd Merge "Revert "Add bp2build support for cc_library_static."" 2021-03-25 06:14:03 +00:00
Jingwen Chen f9410540a8 Revert "Add bp2build support for cc_library_static."
Revert submission 1594391-bp2build-cc_library_static

Reason for revert: Broke the build on aosp-master
Reverted Changes:
Ib16ccf31a:Add cc_library_static macro to help with bp2build ...
I37c856be2:Add bp2build support for cc_library_static.

Change-Id: Ie94d5bc6da81758cd4e0461c08a810a29643c971
2021-03-25 06:11:59 +00:00
Rupert Shuttleworth d3e10104b0 Merge "Add bp2build support for cc_library_static." 2021-03-25 05:39:39 +00:00
Rupert Shuttleworth 21e743df41 Add bp2build support for cc_library_static.
Also refactor bp2build-related code for cc_library_headers.

Test: Added unit test.

Test: bp2build-sync.py write; bazel build //bionic/... still works (but bp2build is disabled for most cc_library_static targets for other reasons)

Change-Id: I37c856be20a47b154909338a22a7dba1ab55693f
2021-03-24 13:56:44 +00:00
Jingwen Chen 12b4c2706d bp2build: add allowlist for package-level conversions.
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.

It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.

Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.

Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
2021-03-24 02:27:19 -04:00
Liz Kammer a4aa43072b Handle include_build_directory prop in bp2build
Test: go test cc tests
Test: generate bp2build, sync bp2build, bazel build //bionic/...
Bug: 181794963
Change-Id: I8dcef585e3025ef6f44d22430ed67b1e0429dca0
2021-03-19 12:33:34 -04:00
Liz Kammer bdc609972c Support autoconverted modules in mixed builds
modules converted with bp2build_available are will also be available to
be used in mixed builds.

Test: build/bazel/scripts/milestone-2/demo.sh full
Test: go tests
Change-Id: I49f16ec3ba5bb11dfed8066af069c27eb04371fb
2021-03-16 15:11:49 -04:00
Liz Kammer ba3ea16f14 Use handcrafted build targets in bp2build
If both bp2build_available and label are specified, label will be
preferred.

Initially, we copy the entire BUILD.bazel file. Eventually we may move
this to use bazel query for a more accurate result.

Test: go test *
Test: build/bazel/scripts/milestone-2/demo.sh full
Test: GENERATE_BAZEL_FILES=true m nothing
      edit bionic/libc/tools/BUILD.bazel
      GENERATE_BAZEL_FILES=true m nothing and verify changes picked up
Bug: 180516554
Change-Id: I43025583300e6b10d2c18032cd4a76237b578d59
2021-03-15 13:09:25 -04:00
Jingwen Chen 13b9b42f8f bp2build: add python_binary_host converter.
This CL adds a simple python_binary_host to native py_binary converter
for standalone modules that don't depend on libs. It also adds support
for the conditional py2/py3 build target based on the version prop.

Test: milestone-2/demo.sh full
Test: bazel query 'kind("py_binary rule", //bionic/...)'
Test: bazel run //bionic/libc/tools:genfunctosyscallnrs
Test: bazel run //bionic/libc/tools:genseccomp
Test: go tests
Fixes: 182236395
Change-Id: Ibe5ec6cd0dc12a61b3a449a8c723d80b891fae42
2021-03-09 18:58:42 -05:00
Jingwen Chen 5d8644990b bp2build: add configurable attribute (select) support.
This CL adds a basic framework to support configurable string_list
attributes, selecting on the Arch variant (x86, x86_64, arm, arm64).

It offers fine-grained controls to map individual configurable
properties (arch_variant) to configurable Bazel attributes, starting
with the string_list type for the copts property for cc_object.

This design is primarily motivated to have minimal boilerplate in
bp2build mutators, allowing anyone to opt-in configurable attributes,
and modify intermediate states before passing them on into the
CreateBazelTargetModule instantiator.

Fixes: 178130668

Test: go tests
Test: build/bazel/scripts/milestone-2/demo.sh

Change-Id: Id6f04d7c560312a93e193d7ca4e1b7ceb6062260
2021-02-26 05:17:54 -05:00
Jingwen Chen 1251bb5775 Merge "bp2build: add support for cc_object's objs and exclude_srcs properties." 2021-02-25 14:09:08 +00:00
Jingwen Chen 88ae408a0f queryview: add "size" to ignored prop.
Test: m queryview && bazel query //... --config=queryview
Change-Id: Ia84be6e2fc5b302b4a147f94bfa219a729b67be1
2021-02-24 19:55:50 -05:00
Jingwen Chen db12024524 bp2build: add support for cc_object's objs and exclude_srcs properties.
objs contains module references to other cc_objects that will be used
for linking later. This maps to cc_library deps.

Also support exclude_srcs, and added tests.

Test: bp2build generate, sync, bazel build //bionic/...

Change-Id: I21200ff73f24bcf5357d9df8dcb5519cde532a77
2021-02-24 01:08:57 -05:00
Treehugger Robot bbfd5ab606 Merge "Move bazel.Properties to a BazelModuleBase" 2021-02-24 05:53:25 +00:00
Liz Kammer ea6666fba9 Move bazel.Properties to a BazelModuleBase
This allows more direct access to the bazel label and whether the module
is bp2build available.

Test: go test *
Change-Id: I23375835d20fa53d7d94127b2dc2d5bb20487bfb
2021-02-23 10:58:57 -05:00
Liz Kammer fc46bc1ee4 Refactor BazelTargetModule
This eliminates the need to remove quotes, delete attributes, and
re-checking that name has correct prefix. Additionally, this allows
assignment directly to the BazelTargetModuleProperties struct, which
allows defaulting unused fields and clarity of which field is being set.

Test: go test soong tests
Test: ran ./build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ia9bfcce76234c793a4ddd5f29a661150f83341c9
2021-02-23 08:29:40 -05:00
Jingwen Chen ba369adff1 bp2build: Unexport GenerateBazelTargetsForDir.
Resolving a comment from aosp/1595678

Test: build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ia7aba2b149fc09615fbafee67838632c71320f1b
2021-02-22 10:22:58 -05:00
Jingwen Chen 164e0867fc bp2build: add a simple UI to report migration progress.
Sample output:

[bp2build] cc_library_headers: 5 targets
[bp2build] cc_object: 5 targets
[bp2build] filegroup: 4 targets
[bp2build] genrule: 4 targets
[bp2build] sh_binary: 1 targets
[bp2build] Generated 19 total BUILD targets from 39270 Android.bp modules.

This CL adds an additional CodegenMetrics return value to
GenerateBazelTargets calls, which are called from bp2build, queryview,
and their tests. For this UI, we only want to use it for bp2build, and
not queryview or tests, since it's not useful for the former, and can
pollute the CLI for the latter.

Test: build/bazel/scripts/milestone-2/demo.sh
Change-Id: Ic84307a1ed1a25e360c9b23459e5449d932bc2e7
2021-02-19 09:30:11 -05:00
Jingwen Chen 8c1b97e01b bp2build: cc_object converter.
This CL introduces a basic bp2build converter for cc_object modules.
cc_objects maps cleanly to cc_library targets, but with -fnoaddrsig.

This CL also demonstrates generating include deps within a macro to
allow the cc_object compilation to depend on a relative-include header
within an include dir.

e.g. if "foo.cc" includes "android/log.h" and the latter is located at
"include/android/log.h", the autogenerated header deps would export
"android/log.h" correctly to the foo.cc upstream target.

Test: GENERATE_BAZEL_FILES=true m nothing && bp2build-sync write && bazel build //bionic/libc:crtbegin_so1

Change-Id: Ifd9e097051ec184ab0a1929d07918f0ff4f24d98
2021-02-18 03:27:28 -05:00
Rupert Shuttleworth a0516f76ab Merge "Add cc_library_headers support to bp2build." 2021-02-17 08:57:19 +00:00
Rupert Shuttleworth 54e7841b90 Add cc_library_headers support to bp2build.
Test: Added cc_conversion_test.go.
Change-Id: Id4459d2c2fa02687a374cd9fb25d687e9678218c
2021-02-16 23:07:01 +00:00
Liz Kammer ebfcf672ef Omit bazel.LabelList attrs if no path is specified
Test: go test build_conversion_test.go
Change-Id: Ibcd81049788e99422569fe74e5f3834df4a05122
2021-02-16 15:00:05 -05:00
Rupert Shuttleworth fb97fdee6b Move sh_binary tests for bp2build into their own file.
Test: go test passes for refactored tests.
Change-Id: Ie094031f4b1f190aa711a37d40bacb182e277b26
2021-02-11 04:41:38 +00:00
Treehugger Robot b68036f44b Merge "Add LOCAL_LICENSE_KINDS to build/soong" 2021-02-10 07:18:29 +00:00
Treehugger Robot 6f833664f0 Merge "Add sh_binary support to bp2build converter." 2021-02-09 16:29:55 +00:00
Rupert Shuttleworth a1a56e8f82 Add sh_binary support to bp2build converter.
Test: Added a unit test in this CL.
Change-Id: I6e9b3c833a257414daf82e603b733e459535d206
2021-02-09 11:53:00 +00:00
Jingwen Chen bb483cbcf5 bp2build: rename generated files to BUILD
The checked in files are named BUILD.bazel, while bp2build files are
named BUILD.

Test: GENERATE_BAZEL_FILES=true m nothing
Change-Id: If226f581b5075088f774dc4f5f9ee88259821835
2021-02-09 04:26:18 +00:00
Treehugger Robot 198158b9f7 Merge "bp2build: refactor BazelTargetModule naming boilerplate." 2021-02-09 03:21:44 +00:00
Jingwen Chen ff8f1ce7d7 Merge "bp2build: make generated BUILD files publicly visible." 2021-02-09 00:38:03 +00:00
Treehugger Robot 41905bf733 Merge "bp2build: add bazel_module: { bp2build_available } prop." 2021-02-08 18:14:20 +00:00
Jingwen Chen 033bfa6e29 Merge "queryview: blocklist package module type." 2021-02-08 15:18:19 +00:00
Jingwen Chen fb4692a743 bp2build: refactor BazelTargetModule naming boilerplate.
This CL replaces the "__bp2build__" name prefix boilerplate with a props
creation function, and centralizes the prefixing in there.

Test: TH
Test: soong tests
Change-Id: Ic963199ab60dcce0d3361abff111cfa9acd4c21b
2021-02-08 04:10:18 -05:00
Jingwen Chen 96af35bec6 queryview: blocklist package module type.
The package module type was recently introduced, and its module name is
the full package path. queryview uses the module name as the eventual
target name in the generated BUILD files, so this causes a crash as seen
in b/179605762.

Workaround this by not generating package modules in the queryview
directory.

Test: m queryview && bazel query --config=queryview //...
Fixes: 179605762
Change-Id: Iac2bd79d02bec47d6271583d106c184fb2f16e68
2021-02-08 00:49:32 -05:00
Bob Badour 02040de891 Add LOCAL_LICENSE_KINDS to build/soong
Added SPDX-license-identifier-Apache-2.0 to:
  Android.bp
  android/Android.bp
  android/soongconfig/Android.bp
  androidmk/Android.bp
  apex/Android.bp
  bazel/Android.bp
  bp2build/Android.bp
  bpf/Android.bp
  bpfix/Android.bp
  cc/Android.bp
  cc/config/Android.bp
  cc/libbuildversion/Android.bp
  cc/libbuildversion/tests/Android.bp
  cc/ndk_api_coverage_parser/Android.bp
  cc/ndkstubgen/Android.bp
  cc/symbolfile/Android.bp
  cmd/dep_fixer/Android.bp
  cmd/diff_target_files/Android.bp
  cmd/extract_apks/Android.bp
  cmd/extract_jar_packages/Android.bp
  cmd/extract_linker/Android.bp
  cmd/fileslist/Android.bp
  cmd/host_bionic_inject/Android.bp
  cmd/javac_wrapper/Android.bp
  cmd/merge_zips/Android.bp
  cmd/multiproduct_kati/Android.bp
  cmd/path_interposer/Android.bp
  cmd/pom2bp/Android.bp
  cmd/pom2mk/Android.bp
  cmd/sbox/Android.bp
  cmd/soong_build/Android.bp
  cmd/soong_env/Android.bp
  cmd/soong_ui/Android.bp
  cmd/zip2zip/Android.bp
  cmd/zipsync/Android.bp
  cuj/Android.bp
  dexpreopt/Android.bp
  dexpreopt/dexpreopt_gen/Android.bp
  env/Android.bp
  etc/Android.bp
  filesystem/Android.bp
  finder/Android.bp
  finder/cmd/Android.bp
  genrule/Android.bp
  jar/Android.bp
  java/Android.bp
  java/config/Android.bp
  kernel/Android.bp
  linkerconfig/Android.bp
  linkerconfig/proto/Android.bp
  makedeps/Android.bp
  partner/Android.bp
  phony/Android.bp
  python/Android.bp
  python/tests/Android.bp
  remoteexec/Android.bp
  rust/Android.bp
  rust/config/Android.bp
  scripts/Android.bp
  sdk/Android.bp
  sh/Android.bp
  shared/Android.bp
  symbol_inject/Android.bp
  symbol_inject/cmd/Android.bp
  sysprop/Android.bp
  tradefed/Android.bp
  ui/build/Android.bp
  ui/logger/Android.bp
  ui/metrics/Android.bp
  ui/metrics/proc/Android.bp
  ui/status/Android.bp
  ui/terminal/Android.bp
  ui/tracer/Android.bp
  xml/Android.bp
  zip/Android.bp
  zip/cmd/Android.bp

Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD to:
  finder/fs/Android.bp
  third_party/zip/Android.bp

Bug: 68860345
Bug: 151177513
Bug: 151953481

Test: m all

Exempt-From-Owner-Approval: janitorial work

Change-Id: Ia47ca14f16b8c9f84f9d533a07e5b00e2c04e8d4
2021-02-06 04:23:21 +00:00
Jingwen Chen 1c2317315c bp2build: make generated BUILD files publicly visible.
This is obviously not good build hygiene, but since Soong modules are in a global namespace, making BUILD files fully public will increase our velocity during the initial migration phases.

We'd want to lock visibility down to private by default as soon as possible, but for the purposes of migration, these will be public.

Test: bazel build //bionic/...
Change-Id: I99d62f3f8826712240decd325928435333870352
2021-02-05 09:38:15 -05:00
Jingwen Chen 77e8b7b6d2 bp2build: add bazel_module: { bp2build_available } prop.
This CL adds a per-target allowlist to instruct bp2build on which modules it should generate Bazel targets for.

Test: soong tests
Change-Id: I869e66fce405c2c6689b381569b8cc0118cbcf76
2021-02-05 09:26:22 -05:00
Jingwen Chen 1fd14691dd bp2build: Refactor CreateBazelTargetModule API.
This CL refactors the CreateBazelTargetModule API to minimize boilerplate, and to establish a well defined function signature for the expected metadata about a BazelTargetModule.

Test: soong tests

Test: TH
Change-Id: I474ff5a2b0db8deeed49ba4ca73b416ccb494fdd
2021-02-05 09:24:55 -05:00
Liz Kammer 356f7d45c1 bp2build: convert paths/module refs to Bazel label
This currently expands all globs, still need to support converting glob
syntax.

Test: go build_conversion_test
Test: GENERATE_BAZEL_FILES=true m nothing
Test: m nothing
Bug: 165114590
Change-Id: If7b26e8e663d17566fad9614ca87a8da1f095284
2021-02-04 13:45:56 -05:00
Jingwen Chen 4d2c0878a3 bp2build: rename method for readability.
Rename GenerateSoongModuleTargets to GenerateBazelTargets to reflect its return type.

Test: TH
Change-Id: I5359c8f4365750790837a9d412946d0b7fae64b7
2021-02-02 07:06:56 -05:00
Jingwen Chen 041b184c92 bp2build: flatten *_defaults properties.
This CL brings the defaults mutators into bp2build, reusing the
implementation of flattening defaults from *_defaults module deps into
the modules themselves.

This is an approach to converge defaults semantics with Bazel, which
doesn't have existing equivalent abstractions, other than macros, which
require more infrastructure to be built.

Test: TH
Test: build_conversion_test.go
Change-Id: Ia1443f8e6c3eb277fa861bb96625425a22cfc6d7
2021-02-01 00:33:30 -05:00
Jingwen Chen 40067de675 bp2build: support Starlark rules and load statements.
This CL adds support to bp2build for declaring the location of the
Starlark rule definition when creating BazelTargetModules. This is
needed for non-native rules that needs to be loaded from .bzl files
somewhere in the tree.

Since load statements are aggregated at the top of the BUILD file, away
from the targets that actually use them, this CL also introduces an
abstraction to group BazelTargets together and compute their load
statements and target string representations separately, allowing load
statements to be decoupled and written into a BUILD file before the
targets themselves.

Test: soong tests
Test: TH
Test: GENERATE_BAZEL_FILES=true m nothing && build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...
Fixes: 178531760

Test: TH
Change-Id: Ie5f793a00006eb024eaef07ddd9fde7aaefc054e
2021-01-26 22:46:20 -05:00
Jingwen Chen a42d6417b3 Make bp2buildMutators registration local to TestContext.
The previous implementation relied on the implicit registration of Bp2Build mutators, resulting in test non-hermeticity. Refactor bp2build tests to explicitly specify the bp2build mutators under test.

Test: Soong tests

Test: TH
Change-Id: I9b9674bad1ea533b3bd31b07077a9e02c99b4c1d
2021-01-26 21:57:27 -05:00
Jingwen Chen 885ee7ad86 bp2build: support genrule $(location) and $(locations)
Soong genrules support $(location) and $(locations) cmd variable
shortcuts without labels. The shortcut is for the location of the first
tool module from the concatenation of the tools and tool_files
properties.

Bazel doesn't support this shortcut, so the bp2build mapping needs to
support it.

Documentation:
https://cs.android.com/android/platform/superproject/+/master:build/soong/genrule/genrule.go;l=95-96;drc=316e07c593ab66599c74725cf482dedbb32b2875
Code: https://cs.android.com/android/platform/superproject/+/master:build/soong/genrule/genrule.go;l=236-246;drc=316e07c593ab66599c74725cf482dedbb32b2875

Test: build_conversion_test.go
Test: TH

Change-Id: I8aa98ae460af3a3fbb0a7835572518680dc7ade1
2021-01-26 09:15:01 -05:00
Jingwen Chen dcc329af9f Implement String() method for CodegenMode.
Test: build_conversion_test.go
Change-Id: I17bf5c9b366b0e7bda59f0b1e18f6272153a6ed7
2021-01-26 09:14:55 -05:00
Jingwen Chen 33832f96e0 bp2build: introduce CodegenMode.
The CodegenMode enum helps to differentiate between bp2build and queryview modes of generating BUILD files from the Soong module graph.

bp2build is used for generating BUILD files from an unconfigured Soong module graph for Soong->Bazel conversion, whereas QueryView is a front-end for querying the existing Soong configured module graph with Bazel query.

Test: go test

Test: TH
Change-Id: I5709f591f1ae08b2770adf272d8ce60c9ee1f46a
2021-01-26 00:30:30 -05:00
Jingwen Chen f92e14b08d Merge "bp2build: automatically convert all genrules." 2021-01-25 22:18:42 +00:00
Jingwen Chen 467f8b6c46 Merge "queryview: prefix internal attribute names." 2021-01-25 22:18:32 +00:00
Jingwen Chen 288e2babde queryview: prefix internal attribute names.
There's a module type in internal main that uses the module_deps
property, which clashes with QueryView's module_deps attribute. A fix is
to prefix the attribute as soong_module_deps to avoid the conflict.

While we're at this, rename the other internal attribute names to be
consistent as well.

Test: soong tests

Test: m queryview && bazel query //... --config=queryview

Fixes: 178351366
Change-Id: I84961c8f58c0c0fac1a1c2cbc6bc625499ac895e
2021-01-25 04:46:07 -05:00
Jingwen Chen 4e4756d720 bp2build: rename variables for readability.
Test: go test
Change-Id: I103bbed32844bf7d24c63e7d908630294f7ab20a
2021-01-24 21:13:56 -05:00
Jingwen Chen 316e07c593 bp2build: automatically convert all genrules.
Results of `cquery //bionic/...`: https://paste.googleplex.com/5608885982330880?raw

Note that this CL does not contain the global module name mapping to
fully qualified label lookup mechanism, so while the BUILD targets are
queryable, they are not buildable (yet).

Test: GENERATE_BAZEL_FILES=true m nothing && ./build/bazel/scripts/bp2build-sync.sh write && bazel cquery //bionic/...

Change-Id: I6f2316222723d8b612d9b041ca82a188cd05bac9
2021-01-24 20:51:18 -05:00