This reverts commit 062b011502.
Reason for revert: This change is accidentally opting in a bunch of people into RBE because we aren't respecting USE_RBE variable in rvc-dev branch (possibly missing cherry-pick of some earlier fix).
Merged-In: I416e8da75f84aa2b53995f525cf50501488dc972
Change-Id: Ic678b63b165deb3ac4ec3d3c5cbc166621d8eec3
If the env variables are loaded after the config object is created, they aren't set in the Environment object inside the config object which causes RBE to not work
Bug: b/198045383
Test: Tested by running a build against rvc-dev branch and ensuring it works.
Change-Id: I509289467df81e5535b4cbbfe4e5eb336c0e4e59
- Rename the "BuildX" variables to "RunX"
- Remove redundant comments
- Inline all the "what to do" based on config in build.go
- Inline some constants only used in one place
Bug: 189187214
Test: m nothing
Test: build/soong/build_test.bash
Change-Id: I111a69e642212d7938d4971283545e0d9acbb01a
I6dca478f356f56a8aee1e457d71439272351390b replaced calling
build/blueprint/bootstrap.bash with running minibp directly,
but didn't propagate the EMPTY_NINJA_FILE environment variable
to minibp. Since everything that uses EMPTY_NINJA_FILE is
executed directly from soong_ui now, replace the EMPTY_NINJA_FILE
environment variable with a config bool and pass it to minibp.
Bug: 189148777
Test: build/soong/build_test.bash --dist --incremental --shard-count=300 --shard=1 && du out/multiproduct
Change-Id: I4d64275ce02c5d68948012f71ac4dc3795af9e85
Merged-In: I4d64275ce02c5d68948012f71ac4dc3795af9e85
(cherry picked from commit f3bdbcbea3fb07d41ff7cf554e0f4db645890944)
Use a concurrency-safe writer in runWithTimeout to avoid data races
on the bytes.Buffer passed in during tests.
Bug: 181095653
Fixes: 187149270
Test: Test_runWithTimeout
Test: go test -race ./cmd/run_with_timeout
Change-Id: I57a889765cb9ee7b42983f0906313e0c2d1e414e
Add a command that can be used to wrap actions with a timeout, and
optionally run an extra debugging command on timeout.
Bug: 181095653
Test: run_with_timeout_test.go
Change-Id: I91df5c3fb5277968717815a4ad4612113766dab1
This refactors bazel-build mode determination logic in soong_ui so it's
clearer which of three possible modes are being used in a given
invocation (NO_BAZEL, GENERATE_BUILD_FILES, or MIXED_BUILDS).
Test: bootstrap tests
Change-Id: I41d2baebf8d560c2cc42db8daa8b936101d453e3
It's under out/soong/workspace and is a symlink forest that merges BUILD
files generated by bp2build and the source tree.
bazel-* symlinks and $OUTDIR are currently ignored.
Test: Presubmits.
Change-Id: If14df23bd0f6a6f2d7d1140f95213f2aee66d652
The full command line run inside sbox can be very long, and printing it
after the errors printed by the failing command can hide the error
messages. Buffer the output of the command and print the failing command
line before the output if it fails.
Bug: 185516277
Test: m out/soong/.intermediates/frameworks/base/system-api-stubs-docs-non-updatable/android_common/metalava/api_lint.timestamp with lint error
Change-Id: I893f3dd01f1baf195e182111c5c49e92eb82f3b0
Error messages printed by failing commands may reference output files
that were created by the command, for example printing a command line
to copy and paste to update a baseline file. Copy output files in the
sandbox to their final locations, ignoring missing files, so that the
messages are valid.
Bug: 185516277
Test: m out/soong/.intermediates/frameworks/base/system-api-stubs-docs-non-updatable/android_common/metalava/api_lint.timestamp with lint error
Change-Id: I604a11c9b54e409ca5bc5c016cd04b3133f74a60
Update the path bootstrap_test.sh uses to check for bpglob reruns
to follow the changes made for sharding globs.
Use bootstrap.GlobFileListFiles to get dependencies during bpbuild.
Test: tests/bootstrap_test.sh
Bug: 159845846
Change-Id: Ibaa7c4360881ed6b666a811bf34b19ea0cdcafe9
It was only appended to the return value, which can be done by the
caller just as well.
Test: Presubmit.
Change-Id: I962696e0dbd4c3496a0159d01d2a911675fd4217
It now handles adding .bp files and changing globs.
In order to do this, depfiles are now written separately from RunBlueprint.
This is necessary due to the confluence of a number of seemingly
unrelated factors:
1. The glob filelist dependencies are discovered in globSingleton
2. Singletons need to be registered because otherwise singleton module
types panic
3. Singletons don't work because they require mutators bp2build does not
run
Due to (1), we would need to run the glob singleton. However, due to (2)
and (3), we can't run singletons and have to run Blueprint with
StopBeforeGeneratingBuildActions, which is when the build actions
writing glob files would be generated. So what happens is:
1. When bp2build is run, the glob singleton is disabled
2. At the end of bp2build, the list of glob files is artifically added
to the depfile of the workspace marker file
3. When build.ninja is generated, the Ninja file containing the glob
list file is written by the now-active glob singleton
Test: Presubmits.
Change-Id: I3c5898d8c57c554a93520276c64a952afc912dbe
This is done by setting the INTEGRATED_BP2BUILD environment variable
when invoking the build.
Even though the name of the marker file insinuates that a Bazel
workspace is already created, this is not the case yet.
An issue that remains is that a .d file is not written for the marker
file so it won't be rebuilt if a .bp file changes. Fixing this requires
delicate surgery because writing the .d file is the result of delicate
interplay between Soong and Blueprint.
There are also a number of semi-related fixes:
- The name of soong.environment.{used,available} is now on the command
line of soong_build (soong_docs is still special cased because its
command line in the Ninja file is taken from the os.Args of
soong_build so it's not trivial to remove the --{available,used}_env
from it
- bp2build writes a separate soong.environment.used file
- I had to call SetAllowMissingDependencies() separately when creating
the android.Context for bp2build so that bp2build runs in the
integration tests (it was not obvious how not to do this)
- Fixed a number of integration tests where a command with an expected
exit code of 1 was used as the last one in a test case, thereby
breaking the test suite
Test: Presubmits.
Change-Id: Ibeb61c26022cf801dcb98505b4039151b3409873
Properly plumb the available environment to the configuration instead of
going through the global variable originalEnv.
Test: Presubmit.
Change-Id: Ia1ea753d5e72c02a0dcaa4c0b43bd6e24fc47cec
This is so that the way soong_build is invoked is simpler, which is in
turn useful so that it can be invoked multiple times within a single
build, which in turn will be used to invoke bp2build routinely (as
opposed to manually when needed)
Test: Presubmits.
Change-Id: Iddaebb05ff7bcedc0db8273192bb31284b739920
Usage: SOONG_DUMP_MODULE_GRPH_JSON=<filename> m nothing
Test: The new test case in bootstrap_test.sh .
Change-Id: I69005a75d47dff915d27187645d0cd1cbb3467ef
This makes it more easy to see what's happening under which modes. Modes
are:
- Regular builds
- Mixed mode builds
- bp2build
- soong_docs
Test: Presubmit + bootstrap_test.sh
Change-Id: I06e3fcb84986b919be1cd9eb456531259ede1183
Previously min_sdk_version was only set for *_library build rules, and
not for *_import targets. This made min_sdk_version default to
sdk_version, which is usually "current", which is not correct for these
libraries.
Test: pom2bp -default-min_sdk_version 24 for androidx
Change-Id: I3b029315abeb559e66efad847ae664efb79f4509
Sbox preserves the permissions of input files when copying them into the
sandbox. A read-only file copied into the sandbox multiple times causes
a permission denied error on the second write. Building in Bazel results
in more read-only files, which triggers the issue on existing sbox rules
with duplicate input files. Remove the destination file when copying if
it exists.
Bug: 184113103
Test: m USE_BAZEL=true
Change-Id: I7edf92d82b766100e3cbbd90d22428269d7d0167
Previously 24 was hardcoded (for somewhat unclear reasons). Update
the tool to accept specifying the correct value on the cmdline.
Test: pom2bp -default-min-sdk-version 24
Change-Id: I39c23dd4bca91684a2a68d7b963cf3cb66eb1f33
The current implementation causes inputs listed in an rsp file used with
sbox to be duplicated 3 times in the build.ninja file; once as a
dependency of the rule, once in the rspfile_content field of the rule
with the paths rewritten to be relative to the sandbox, and once in the
rule to write the sbox manifest. When RBE is enabled it also gets a
fourth copy in the list of files to be treated as inputs by rewrapper.
Reduce this to a single copy by using "$in" for the rspfile_content so
that the files only have to be listed in the input dependencies of the
rule, and then add support to sbox to rewrite the rsp file while copying
it into the sandbox, and pass it to rewrapper as well.
Test: m lint-check
Change-Id: I3f46f61119508d39a8bb231c99fc130153fb6f04
sbox is going to need to read and write response files, move ReadRspFile
to its own package.
Test: response_test.go
Change-Id: Iecb5486b4aaeb2531828743ad8ef784df675e18e
Colin Cross