Commit Graph

34874 Commits

Author SHA1 Message Date
Paul Duffin 863ecfb7ba Merge "Disallow non-existent paths in sdk package" 2021-03-25 18:09:06 +00:00
Paul Duffin cf0bc53bc3 Merge "Improve sdk snapshot testing" 2021-03-25 18:08:26 +00:00
Colin Cross d11cf62ec7 Use interface for $(location) values in genrules
Use an interface instead of a string to retrieve the value for
$(location) or $(locations) expansions in genrules to allow
delaying the evaluation until the RuleBuilderCommand is available.
This allows using helpers like RuleBuilderCommand.PathForInputs
to properly rewrite the values for sandboxing.

Also remove the standalone SboxPathFor* methods that don't operate
on a specific RuleBuilderCommand that are now unnecessary.

Test: genrule_test.go
Change-Id: I8bb2647332ef118204a216cead23d062517e2b8c
2021-03-25 11:06:45 -07:00
Colin Cross 7ee54ffd70 Replace ANDROID_SDK_HOME with ANDROID_PREFS_ROOT for metalava
Fixes warnings:
Warning: Using ANDROID_SDK_HOME for the location of the '.android' preferences location is deprecated, please use ANDROID_PREFS_ROOT instead.
Support for ANDROID_SDK_HOME is deprecated and will be removed in 6.0

Test: m android_stubs_current
Change-Id: Ie8721dcda0578c670dfc796675ba43cda16883f6
2021-03-25 11:06:45 -07:00
Colin Cross 5bedfa2d45 Simplify lint rules using improved RuleBuilder rsp support
With the improved RuleBuilder rsp support a manual resources.list
file is not necessary, use FlagWithRspFileInputList instead.

The switch to RBE support in RuleBuilder in
Iab4e09d961891ef182643583d4d456e413bc5e39 obsoleted tracking remoteInputs
and remoteRSPInputs, remove them.

writeLintProjectXML was written to allow it to be applied to a separate
rule than the one that ran lint, but it is not used that way.  Using
the same rule for both means that manual tracking of the input
dependencies described by the project.xml rule but read by the lint
rule is not necessary, just treat them as inputs to the single rule.

Test: m lint-check
Test: m USE_RBE=true RBE_LINT=true RBE_LINT_EXEC_STRATEGY=remote lint-check
Change-Id: If1827b9dede3ebcd0792b6b4b8114d3199f6570b
2021-03-25 11:06:45 -07:00
Colin Cross ce3a51dc96 Support multiple rsp files in RuleBuilder
The lint rule is manually creating a second rsp file because Ninja
only supports on per rule.  Move the support into RuleBuilder so
that it can apply the same rewrites that it does to the primary
one.

Test: TestRuleBuilder_Build
Change-Id: Iec250a2d60e74ccf1b4ad085a960fec6867ea559
2021-03-25 11:06:45 -07:00
Colin Cross e55bd423df Pass rsp files into sbox and rewrapper
The current implementation causes inputs listed in an rsp file used with
sbox to be duplicated 3 times in the build.ninja file; once as a
dependency of the rule, once in the rspfile_content field of the rule
with the paths rewritten to be relative to the sandbox, and once in the
rule to write the sbox manifest.  When RBE is enabled it also gets a
fourth copy in the list of files to be treated as inputs by rewrapper.

Reduce this to a single copy by using "$in" for the rspfile_content so
that the files only have to be listed in the input dependencies of the
rule, and then add support to sbox to rewrite the rsp file while copying
it into the sandbox, and pass it to rewrapper as well.

Test: m lint-check
Change-Id: I3f46f61119508d39a8bb231c99fc130153fb6f04
2021-03-25 11:06:45 -07:00
Colin Cross 045bfd9640 Add test for sbox input sandboxing
Add a test that was dropped in Ic0db961961b186e4ed9b76246881e3f04971825c.

Test: rule_builder_test.go
Change-Id: Idf136919939ad28eb5260dd8d686abe6948f47c7
2021-03-25 11:06:45 -07:00
Colin Cross a4eafddc41 Support multiple rsp files in REParams
rewrapper supports a comma separate list of rsp files, replace
REParams.RSPFile with REParmas.RSPFiles.

Test: remoteexec_test.go
Change-Id: I7850c071c23d368d6fad4480dd527d146c13c6d3
2021-03-25 11:06:45 -07:00
Colin Cross fd708b5651 Move response file handling to a separate package
sbox is going to need to read and write response files, move ReadRspFile
to its own package.

Test: response_test.go
Change-Id: Iecb5486b4aaeb2531828743ad8ef784df675e18e
2021-03-25 11:06:45 -07:00
Paul Duffin bbb0f8fae4 Make TestingBuildParams.RelativeToTop() modify (Writable)Path(s)
Bug: 183650682
Test: m droid
Change-Id: I5cb150aa28cf1a06c353b67443fd2128aff4bb5d
2021-03-25 18:03:00 +00:00
Paul Duffin 6c968b1f0f Merge "Fix TestDroiddoc's reliance on absolute paths" 2021-03-25 18:02:26 +00:00
Paul Duffin 301deecd6c Merge "Add Path.RelativeToTop" 2021-03-25 18:01:31 +00:00
Pirama Arumuga Nainar 4c0b11a16e Merge "[pgo] Do not explicitly link profile runtime" 2021-03-25 17:57:49 +00:00
Ivan Lozano 63bb7680c9 rust: Add whole_static_libs, revert static_lib
Revert the static_lib behavior to the previous behavior (pass static
libs to the linker rather than via to rustc using `-lstatic=<lib>`). To
bundle static libraries into libraries, provide the whole_static_libs
property which retains the current static_libs behavior.

Passing all static libraries via -lstatic was resulting in odd bloat
where in some cases static symbols were duplicated in binaries and
libraries. This split makes it possible to be explicit about when static
libraries should be bundled.

Bug: 183182230
Test: mma system/bt; mma system/security/keystore2; mma external/rust

Change-Id: Ic2dde5d1542dca5ce145aa3a3fbd9ea54440d991
2021-03-25 13:50:17 -04:00
Roland Levillain 737ad608ab Merge "Fix a typo in the definition of `varApexNames` in `java/dexpreopt_bootjars.go`." 2021-03-25 17:25:44 +00:00
Liz Kammer 8206d4ff98 Don't use custom mixed build logic for cc_object
Starlark implementation of cc_object now performs partial linking, which
better matches Soong's implementation of cc_object.

Bug: 181794963
Test: generate build files with bp2build, mixed build droid
Change-Id: I5a325aa1c608981deaee3671ac28c6014e6cbf08
2021-03-25 12:00:30 -04:00
Rupert Shuttleworth 095081c6d7 Add bp2build support for cc_library_static.
Also refactor bp2build-related code for cc_library_headers.

(Retry of previous CL after the presubmit failed to detect a merge conflict)

Test: Added unit test.

Test: bp2build-sync.py write; bazel build //bionic/... still works (but bp2build is disabled for most cc_library_static targets for other reasons)
Change-Id: I2f4405c2fea305623bbc6daaaf62808b0c074216
2021-03-25 15:16:11 +00:00
Jingwen Chen 6400f20430 Merge "bp2build: add a per-module denylist." 2021-03-25 15:02:57 +00:00
Paul Duffin 618eeff010 Merge "Remove config from basePath" 2021-03-25 14:45:43 +00:00
Paul Duffin 85484e344a Merge changes I12e1854c,I08f7dba4
* changes:
  Add buildDir to WritablePath implementations
  Add srcDir to SourcePath
2021-03-25 14:44:51 +00:00
Paul Duffin 4dd6cdb3bc Fix TestDroiddoc's reliance on absolute paths
TestDroiddoc compares the value returned by OutputFiles(""), which will
usually be absolute paths including the temporary buildDir, against
paths returned from TestingBuildParam.RelativeToTop(), which does not
currently change the Path contents and so will include absolute
temporary paths. However, a follow up change to this will make the
TestingBuildParam.RelativeToTop() also change the Path contents at
which point this test would be comparing relative to absolute paths.

So, this change makes sure that they are all converted to relative to
top paths before comparison.

Bug: 183650682
Test: m droid
Change-Id: Ia4478f527af27a920945f5849525e5031cc5b8b6
2021-03-25 13:14:37 +00:00
Paul Duffin 85d8f0dcd9 Add Path.RelativeToTop
Makes Path.RelativeToTop the source of truth as to what `relative to
top` means.

Uses it in PathRelativeToTop so that it will be tested by the existing
tests that use AssertPathRelativeToTop.

Also adds RelativeToTop() to WritablePaths and Paths.

Bug: 183650682
Test: m droid
Change-Id: I88dc56afd1314e51b4e41795a2448cab4ce8a899
2021-03-25 13:14:37 +00:00
Paul Duffin bb9ff5108b Filter duplicate modules in platform_compat_config_singleton
Previously, unpacking a snapshot containing a
prebuilt_platform_compat_config into a source build would cause build
failure because of duplicate ids because the singleton would collate
ids from both prebuilts (versioned and unversioned) and source.

This change filters out versioned prebuilts and only uses prebuilts
that are preferred and source modules that have not been replaced by a
prebuilt.

Bug: 182402754
Test: m nothing
Change-Id: Idacbb34444e5156370df70bf88c6e8a7e2d67890
2021-03-25 12:53:25 +00:00
Paul Duffin db462dd987 Disallow non-existent paths in sdk package
Test behavior was changed a while ago so that tests by default ignore
non-existent source paths (unless they explicitly check for/rely on
them). Prior to that CheckSnapshot() could detect when files were
missing from the snapshot but it no longer can.

This change disallows non-existent source files in all the sdk tests
which means that they are disallowed when processing the snapshots as
they use the same preparers as were used to process the sources.

This caused a test failure which has been temporarily ignored and has
a TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I969d8515d20ef5ae515f2b5f93d8ed4e4f8ede75
2021-03-25 12:53:22 +00:00
Ulya Trafimovich dd622951a4 Fix AAPT path in dexpreopt.
Dexpreopt uses AAPT to parse `targetSdkVersion` from the manifest, so it
the generated dexpreopt.sh script that calls AAPT must depend on it,
otherwise AAPT might not be built by the time it is used. Tool
dependencies are handled via the global Soong config and the
DEXPREOPT_GEN_DEPS variable that Soong generates for Make. This config
always uses Soong tool paths, like out/soong/host/linux-x86/bin/aapt
rather than out/host/linux-x86/bin/aapt.

This CL fixes a mistake in dexpreopt rule that used context-dependent
AAPT path (so, when called from Make, the dependency was on Soong tool,
but a Make tool was actually used, so it failed sometimes).

Bug: 132357300
Test: lunch aosp_cf_x86_64_phone-userdebug && m
Change-Id: I1f0ab4afac98e6239f324e7f3571d670fd7a36cd
2021-03-25 12:52:49 +00:00
Paul Duffin 1822a0a371 Improve sdk snapshot testing
The sdk produces snapshots that are expected to be unpacked in one of
the Android repos. Often that can lead to issues due to conflicts
between the source and prebuilts. This change attempts to avoid those
conflicts by testing the different ways that those files can be used.

With the existing test to cover adding the snapshot
This change will cause the sdk tests to check the following:
1) Snapshot on its own (already done).
2) Snapshot plus original source where the original source is
   preferred.
3) Snapshot plus original source where the snapshot is preferred.

It also adds the ability for tests to provide their own custom checkers
to verify the result of each of the previous tests.

This change reveals a number of bugs already present. Rather than
attempt to fix them this change adds the ability to specify error
handlers for the two cases that mix source and snapshot to allow those
errors to be temporarily ignored while allowing the majority of the
tests to benefit from this improvement. Each of those failures has a
TODO and bug associated with it.

Bug: 183184375
Test: m nothing
Change-Id: I105233195074dbe7a6422b6dfc5486e74398ea15
2021-03-25 12:50:18 +00:00
Ulyana Trafimovich 2258cbbe42 Merge "Allow using updatable boot jars in dexpreopt (but don't use them yet)." 2021-03-25 10:13:45 +00:00
Jingwen Chen 5d72cba25c bp2build: add a per-module denylist.
This speeds up development of experimental converters.

Test: TH
Change-Id: I1623d3dbf63c8ed777646d64560d77b38edf9213
2021-03-25 09:29:26 +00:00
Paul Duffin 74abc5d4ee Remove config from basePath
Bug: 183650682
Test: m droid
Change-Id: I60eb2b8ed0317e874ecdafd5cb37d7c997d41bb8
2021-03-25 09:18:29 +00:00
Paul Duffin d65c58b204 Add buildDir to WritablePath implementations
First, the buildDir() method was renamed to getBuildDir() to avoid
clashing with the buildDir field.

Then, a buildDir was added to both `OutputPath` and `InstallPath` but
not to `PhonyPath` as it does not contain any path components. Instead
the `PhonyPath.getBuildDir()` was changed to simply return "".

Bug: 183650682
Test: m droid
Change-Id: I12e1854c829b980c5c01205753c62c00dc0a4774
2021-03-25 09:18:29 +00:00
Paul Duffin 580efc8716 Add srcDir to SourcePath
Bug: 183650682
Test: m droid
Change-Id: I08f7dba449538d289a6c4d6b5325c1a833cff834
2021-03-25 09:18:29 +00:00
Paul Duffin 2b1d2b6e4d Merge "Cleanup some issues left over from previous reviews" 2021-03-25 09:16:40 +00:00
Paul Duffin e742ee14f8 Merge "Convert test that disallows non existent paths to use fixtures" 2021-03-25 09:16:20 +00:00
Thiébaud Weksteen e8b0ee73f4 rust: Add features list to rust_project.json
Bug: 183679729
Test: SOONG_GEN_RUST_PROJECT=1 m nothing
Test: manually check v1 feature resolution for libprofcollectd
Change-Id: Ib59b63372865f7f4545fe327e92d1642d76ce9c9
2021-03-25 09:59:49 +01:00
Treehugger Robot f7e79a591c Merge "Enable safe ICF for all devices" 2021-03-25 08:59:00 +00:00
Jingwen Chen 8b086655cd Merge "Revert "Add bp2build support for cc_library_static."" 2021-03-25 06:14:03 +00:00
Jingwen Chen f9410540a8 Revert "Add bp2build support for cc_library_static."
Revert submission 1594391-bp2build-cc_library_static

Reason for revert: Broke the build on aosp-master
Reverted Changes:
Ib16ccf31a:Add cc_library_static macro to help with bp2build ...
I37c856be2:Add bp2build support for cc_library_static.

Change-Id: Ie94d5bc6da81758cd4e0461c08a810a29643c971
2021-03-25 06:11:59 +00:00
Rupert Shuttleworth d3e10104b0 Merge "Add bp2build support for cc_library_static." 2021-03-25 05:39:39 +00:00
Paul Duffin 3cb2c06b44 Cleanup some issues left over from previous reviews
Bug: 183235980
Test: m nothing
Change-Id: I187765c56b889445de3a4729405ee745e5f73729
2021-03-24 22:08:05 +00:00
Paul Duffin 76e5c8a37f Convert test that disallows non existent paths to use fixtures
This change needed to add some additional files to the registered
files for PrepareForTestWithJavaDefaultModules because otherwise they
would fail when "TestAllowNonExistentPaths = false". Those files were
being added by the TestJavaLintRequiresCustomLintFileToExist (albeit in
some cases in different locations to that required by the default
modules but as the files are needed by the modules defined in
PrepareForTestWithJavaDefaultModules they should be defined in it.

A couple of other places also provided some files so moving them into
PrepareForTestWithJavaDefaultModules caused some conflicts which needed
to be resolved.

Bug: 183184375
Test: m nothing
Change-Id: I76ce9f1673c1c1c4000635b76b8377d582224bf1
2021-03-24 22:08:05 +00:00
Paul Duffin 958408d921 Merge "Cleanup usages of Dex2oatDepTag" 2021-03-24 22:04:58 +00:00
Jingwen Chen 95c6eb3959 Merge "bp2build: add allowlist for package-level conversions." 2021-03-24 21:38:07 +00:00
Nicolas Geoffray ed35a90d2c Merge "Fix "filename too long" issue on Luci caused by test name" 2021-03-24 20:45:38 +00:00
Paul Duffin 688ba5bc2b Merge "Remove testSdkWithJava" 2021-03-24 17:59:09 +00:00
Paul Duffin c8ad86479f Merge "Convert java_sdk_test.go tests to fixtures" 2021-03-24 17:57:47 +00:00
Lukács T. Berki f2926e751c Merge "Make null builds always be null builds." 2021-03-24 15:48:02 +00:00
Paul Duffin ac94726ef2 Remove testSdkWithJava
Bug: 181070625
Test: m nothing
Change-Id: I9ef906a386cc87f69b166ec88e0b6c7388c3d06a
2021-03-24 15:47:28 +00:00
Paul Duffin 9ec86b14e7 Convert java_sdk_test.go tests to fixtures
Bug: 182638834
Test: m nothing
Change-Id: I6a1fabdd2c8385e5fbaef6985047f9d0bdceb209
2021-03-24 15:47:28 +00:00
Paul Duffin a35178b105 Fix "filename too long" issue on Luci caused by test name
Stop including fields in the test name for TestJavaSdkLibraryEnforce to
reduce its length to avoid the filename too long limit.

Test: m nothing
      Check the lengths before (~240) and after (~74) to make sure
      there was a sizeable reduction.
Change-Id: I275a1110e5102b8ea8376759f28c7c6333a5efee
2021-03-24 15:09:37 +00:00