Commit Graph

38 Commits

Author SHA1 Message Date
Paul Duffin 709e0e32c2 Add TestingBuildParams.RelativeToTop()
The methods returns a copy of TestingBuildParams with every usage of a
path that is relative to the temporary test build directory with a path
relative to a notional top. Similar to how PathRelativeToTop does.

Bug: 182885307
Test: m nothing
Change-Id: I6ec20fc52ed76748138f0d48b4df80f765dfcfdc
2021-03-22 22:09:53 +00:00
Paul Duffin 30ac3e7ca7 Remove uses of FixtureFactory from android package
Bug: 183235980
Test: m nothing
Change-Id: I72898ada020ee1a73fd534c61afb5c22fa00c1e5
2021-03-22 18:31:53 +00:00
Colin Cross ab020a7333 Support sandboxing inputs in RuleBuilder
When RuleBuilder.SandboxInputs() is called configure sbox to copy
all the input files into the sandbox directory and then change the
working directory there when running the command.

Copying input files into the sandbox directory gets tricky when
the input file is the output file from another rule, and could
be at an arbitrary, possibly absolute path based on the value
of OUT_DIR.  They will need to be copied to a directory in the
sandbox using the path relative to OUT_DIR.

RSP files need special handling, they need to both be copied into
the sandbox as an input, rewritten to contain paths as seen in the
sandbox, and references to them on the command line need to use
sandbox paths.

Bug: 182612695
Test: rule_builder_test.go
Change-Id: Ic0db961961b186e4ed9b76246881e3f04971825c
2021-03-18 16:17:32 -07:00
Paul Duffin d250ff620a Convert android/rule_builder_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: I6b8f4c419810e3766abaadfea403967c6714a484
2021-03-17 16:18:17 +00:00
Colin Cross b70a1a900a Remove unescaped spans support from RuleBuilder
Now that rsp files use an explicit path instead of $out.rsp the
unescaped spans and NinjaEscapedCommands() support can be removed
from RuleBuilder.

Bug: 182612695
Test: rule_builder_test.go
Change-Id: I6705151ff0528c59aa1af56ee52c294da15a7a66
2021-03-16 16:53:20 -07:00
Colin Cross 70c4741215 Add explicit rspfile argument to RuleBuilderCommand.FlagWithRspFileInputList
Using $out.rsp as the rsp file adds extra complexity around keeping
the $ unescaped.  Make callers to FlagWithRspFileInputList provide
an explicit path for the rsp file instead.

Bug: 182612695
Test: rule_builder_test.go
Change-Id: I3f531d80c1efa8a9d09aac0a63790c5b11a9f0c6
2021-03-16 16:52:56 -07:00
Colin Cross ba9e403703 Sandbox genrule tools
This relands I38393900677c5dbe8e72fe06a7dd7d71f3c09f82 after
I8038ed5c6f05c989ac21ec06c4552fb3136b9a7a, which makes the ASAN
variants of libraries have PackagingSpecs so they can be copied
into sandboxed genrules.

Copy tools used by genrules into the sandbox directory.  This
ensures correct dependencies on all files used by tools, and
is one step closer to enabling genrules inside unselected
namespaces.

Bug: 124313442
Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I57c0d5fc8bba216fac4deb972d0d2098593e8963
2020-12-17 10:02:56 -08:00
Colin Cross 0096047180 Revert "Sandbox genrule tools"
This reverts commit 3a150d6951.

Reason for revert: b/175743249

Bug: 124313442
Bug: 175743249
Change-Id: Id1695df7b83bf057511f856b40cf7d5413030270
2020-12-16 02:52:32 +00:00
Colin Cross 3a150d6951 Sandbox genrule tools
Copy tools used by genrules into the sandbox directory.  This
ensures correct dependencies on all files used by tools, and
is one step closer to enabling genrules inside unselected
namespaces.

Bug: 124313442
Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I38393900677c5dbe8e72fe06a7dd7d71f3c09f82
2020-12-10 17:01:50 -08:00
Colin Cross f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Colin Cross e16ce36818 Reland: Rewrite sbox to use a textproto manifest
This relands I3b918a6643cea77199fd39577ef71e34cdeacdb1 with a fix
to create the directory for the output depfile if doesn't exist.

In preparation for more complicated sandboxing that copies tools
and/or inputs into the sandbox directory, make sbox use a textproto
input that describes the commands to be run and the files to copy
in or out of the sandbox.

Bug: 124313442
Test: m checkbuild
Test: rule_builder_test.go
Test: genrule_test.go
Change-Id: I8af00c8c0b25a92f55a5032fcb525715ae8297c2
2020-11-20 15:45:04 -08:00
Colin Cross 619b9ab260 Revert "Rewrite sbox to use a textproto manifest"
This reverts commit 151b9ff0cf.

Reason for revert: broke builds

Change-Id: I69b3b8795d5a36b4fa0debb1af2d433be3c15d6c
2020-11-20 18:44:31 +00:00
Colin Cross 151b9ff0cf Rewrite sbox to use a textproto manifest
In preparation for more complicated sandboxing that copies tools
and/or inputs into the sandbox directory, make sbox use a textproto
input that describes the commands to be run and the files to copy
in or out of the sandbox.

Bug: 124313442
Test: m checkbuild
Test: rule_builder_test.go
Test: genrule_test.go
Change-Id: I3b918a6643cea77199fd39577ef71e34cdeacdb1
2020-11-18 13:02:22 -08:00
Colin Cross 3d68051218 Move genrule on top of RuleBuilder
In preparation for more complicated sandboxing support in sbox, use
a single implementation of the sbox sandboxing by moving genrule to
use RuleBuilder's sbox support instead of creating an sbox rule
directly.

Also move genrule's input list hash support into RuleBuilder.

Test: genrule_test.go
Test: rule_builder_test.go
Change-Id: I292184d02743c7e6887ebbcd232ba565db2ab0cc
2020-11-17 11:23:45 -08:00
Colin Cross ae8600b507 Pass Config to NewTestContext instead of ctx.Register
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.

Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
2020-11-12 10:07:49 -08:00
Jingwen Chen ce679d29ec Add symlink_outputs support to Soong.
This CL adds symlink_outputs to various locations in Soong that creates
actions that creates symlink outputs, and explicitly mark them as such.

Test: m
Bug: 160568334
Change-Id: I322751bada52a9f49011c74731d84761586e03e7
2020-10-19 01:26:50 -04:00
Colin Cross 405af07859 Revert "Make lots of tests run in parallel"
This reverts commit 323dc60712.

Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests

Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
2020-10-09 18:34:24 -07:00
Colin Cross 323dc60712 Make lots of tests run in parallel
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.

Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
2020-10-06 15:12:22 -07:00
Colin Cross da71eda20d Add support for order-only dependencies to RuleBuilder
Test: TestRuleBuilder
Change-Id: I1609a790dd4d0a03c8308b6e552622fe33fa2499
2020-02-28 18:16:13 +00:00
Colin Cross 98be1bb00f Move filesystem into Config
The filesystem object was available through ModuleContext.Fs(), but
gives too much access to the filesystem without enforicing correct
dependencies.  In order to support sandboxing the soong_build
process move the filesystem into the Config.  The next change will
make it private.

Bug: 146437378
Test: all Soong tests
Change-Id: I5d3ae9108f120fd335b21efd612aefa078378813
2019-12-18 08:19:10 -08:00
Colin Cross 4b49b768a2 Make TestContext.RegisterModuleType take an android.ModuleFactory
Avoid having to pass ModuleFactoryAdaptor to every call to
RegisterModuleType in a test by wrapping RegisterModuleType.

Test: all soong tests
Change-Id: If8847d16487de0479cc3020b728256922b3cadba
2019-11-25 10:59:44 -08:00
Dan Willemsen c89b6f1981 Rewrite depfile from sbox to stay reproducible
sbox will generate a random directory for the output root, and most
tools will encode that directory name in the output target of the
depfile.

So embed the library from dep_fixer into sbox so that it can rewrite the
output filename to a static (reproducible) value. Ninja doesn't care
what that value is, so it's just "outputfile".

Also fix up rule_builder to actually tell sbox about the depfile.

Test: mmma system/iorap; check the contents of:
out/soong/.intermediates/system/iorap/libiorap-binder/android_arm_armv7-a-neon_core_static/gen/aidl/system/iorap/binder/com/google/android/startop/iorap/IIorap.cpp.d

Change-Id: I3640a2e8b0c034f143a35e398a8418a6d621b265
2019-08-29 14:47:40 -07:00
Colin Cross 0cb0d7b1c5 Add rspfile support to RuleBuilder
Allow RuleBuilderCommands to use an rspfile with FlagWithRspFileInputList.
This requires being more careful with ninja escaping, as the rspfile
will be $out.rsp, and the value for $out may not be known yet so it
must be inserted as a ninja variable that must not be escaped.

Test: rule_builder_test.go
Change-Id: Ifa91e24a0bb8f0ceeb5c9bfa5689be2a4ff3b9cd
2019-07-16 11:12:04 -07:00
Colin Cross 2ffb9a8d7d Share buildDir for android/soong/android tests
There is no need to create a separate buildDir for each test
file, use TestMain to create a global one for the package.

Test: all soong tests
Change-Id: I435ee7aa88b7e0bb8ccc1ba79f82833a7accf3e9
2019-06-10 15:51:06 -07:00
Dan Willemsen 633c502295 Support RuleBuilder.Sbox to wrap commands in sbox
This essentially allows you to declare that everything in a directory
will be created by the rule, and we'll ensure that your command actually
writes out all of the claimed outputs, and remove any other files that
previously existed in that directory.

Test: built-in tests
Change-Id: I990dce2b3a0d89ebd2736ac1a0cadfb5864c6e73
2019-04-15 14:52:05 -07:00
Colin Cross 1d2cf0494a Add depfile support to RuleBuilder
Allow rules built with RuleBuilder to use depfiles.  Ninja only
supports a single depfile with single output.  If there are
multiple outputs in a rule, move all but the first to implicit
outputs.  If multiple depfiles are specified, use new support
in dep_fixer to combine additional depfiles into the first depfile.

Test: rule_builder_test.go
Change-Id: I7dd4ba7fdf9feaf89b3dd2b7abb0e79006e06018
2019-03-29 16:35:06 -07:00
Colin Cross 92b7d584c8 Add RuleBuilderCommand.Flags
Test: rule_builder_test.go
Change-Id: I7887a67aaef33bc591d83fade9175da3e401529f
2019-03-29 16:35:06 -07:00
Colin Cross baa676f671 Fix RuleBuilder.Restat
RuleBuilder.Restat wasn't being propagated to the rule.

Test: rule_builder_test.go
Change-Id: Ie64c8076692ea10a0c47ed5a8832e3f553bea0fc
2019-03-05 03:17:33 +00:00
Colin Cross 4c83e5ccd4 Support testing Rules in Modules and Rules and Builds in Singletons
Add support for TestingModule to return RuleParams for rules created
by the module.

Refactor TestingModule to use helpers, and use the helpers to
implement a similar TestingSingleton.

Use the new functionality to test RuleBuilder's module and singleton
rules.

Test: none
Change-Id: I8348c56ff5086d0c49401f5a00faf7c864e6b6f3
2019-03-05 03:17:26 +00:00
Colin Cross 69f59a3327 Make RuleBuilder methods take Paths
There are no more Make paths being used in Soong now that
dexpreopting and hiddenapi are in Soong. Use the Path types
in the inputs to RuleBuilder, and fix all users of RuleBuilder.

This reapplies I886f803d9a3419a43b2cae412537645f94c5dfbf with
fixes to disable preopt for Soong-only builds when the global
dexpreopt.config doesn't exist.

Test: all soong tests
Test: m checkbuild
Change-Id: I4dae9ecd5de22f062f9478ec8f0747f099cf8190
2019-02-20 22:06:09 -08:00
Colin Cross ab898dc4a4 Revert "Make RuleBuilder methods take Paths"
This reverts commit acdd694071.

Reason for revert: broke ndk build

Change-Id: I5655e48c15eb8f5f0267afdd853fbc25765b8623
2019-02-21 05:03:00 +00:00
Colin Cross acdd694071 Make RuleBuilder methods take Paths
There are no more Make paths being used in Soong now that
dexpreopting and hiddenapi are in Soong. Use the Path types
in the inputs to RuleBuilder, and fix all users of RuleBuilder.

Test: all soong tests
Test: m checkbuild
Change-Id: I886f803d9a3419a43b2cae412537645f94c5dfbf
2019-02-20 14:23:37 -08:00
Colin Cross deabb94380 Add RuleBuilder.Installs().String()
Add a RuleBuilderInstalls type for a slice of RuleBuilderInstalls,
and give it a String() method that returns the list of installs
in the format that is convenient for passing to Make.

Test: rule_builder_test.go
Change-Id: I2e9cd9abf4dfb0ad312d0a6662f1567baf9cd222
2019-02-13 08:03:29 -08:00
Colin Cross c7ed004dd3 Add RuleBuilderCommand.FlagForEachArg()
Add an analog to FlagForEachInput that takes non-path arguments.

Test: rule_builder_test.go
Change-Id: Ifdf5a16079018bfff9b06ce48b13b104a93fddd7
2019-02-12 17:07:35 -08:00
Colin Cross 5cb5b093d1 Add Temporary and DeleteTemporaryFiles to RuleBuilder
Temporary marks an output path as a temporary file that is not
necessary after the rule completes, removing it from the result of
Outputs.  DeleteTemporaryFiles adds a command to the command line
that deletes all files that have been marked with Temporary.

Test: rule_builder_test.go
Change-Id: Ie509ed800992962747fb287858e148e975eee54a
2019-02-05 13:28:43 -08:00
Colin Cross 758290d7ff Improve RuleBuilder documentation and methods
Add a few convenience methods, document all the methods and add
examples that would show up in the godoc if we were to actually
generate it.

Test: rule_builder_test.go
Change-Id: I270fed605ffec34e6f5b36fde0dc9ca52694b741
2019-02-05 13:28:43 -08:00
Colin Cross 786cd6dc13 Allow RuleBuilder to be used with SingletonContext
Make RuleBuilder.Build take a subset of ModuleContext and
SingletonContext, and dynamically call PathForModuleOut only
if it is available.

Test: rule_builder_test.go
Change-Id: Id825cb75236acf187e9d4a36353a47abcac71927
2019-02-05 13:28:43 -08:00
Colin Cross feec25b084 Move dexpreopt.Script to android.RuleBuilder
Move dexpreopt.Script to android.RuleBuilder so that the builder
style can be used in more places.  Also add tests for it.

Test: rule_builder_test.go
Change-Id: I92a963bd112bf033b08899e930094b908acfcdfd
2019-02-05 13:28:43 -08:00