Some people have run into issues where /tmp is on a smaller partition
than their main source tree, and if it fills up (or only has a few GB
free), their builds can start failing.
There may also be a subset of people whose /tmp may be slower than their
out directory -- if they put their src and out trees on a SSD, but keep
/tmp on a spinning HDD.
Bug: 71755844
Test: check out/soong.log for TMPDIR
Test: do a build, watch out/soong/.temp
Change-Id: Ib749ec736ad0f6c8ce7453861a91b3e99ddbd92d
If Kati ever wrote a line over 64k characters, the scanner in soong_ui
would error, and Kati would hang trying to write into the pipe. Now if
the Scanner errors out, fall back to copying directly from the pipe to
the output.
Test: Add $(warning $(shell find frameworks)), does not hang
Change-Id: I86412ad2f53f2fe3cbda901ee673abb904d56d3c
Soong_UI will update timestamp to build_date.txt, and export variables
to kati/ninja.
Test: m -j32
Bug: b/70351683
Change-Id: I153897afdf2d3f39a32d757d4c3ae7515caea52d
We were previously setting GOROOT to "prebuilts/go/linux-x86" during the
ninja executions when we were running Soong. But we can also run Soong
during the main ninja execution, were GOROOT was unset. When the GOROOT
was unset, the default GOROOT in our Go installation is
"./prebuilts/go/linux-x86" (note the extra ./).
This would cause g.bootstrap.goRoot to change between some soong runs,
causing us to rebuild all go programs (and anything depending on them)
more often than necessary.
So instead, keep GOROOT undefined when running Soong. Everything that
matters is using runtime.GOROOT(), which will fall back to the default.
Continue setting $GOROOT for bootstrap.bash, otherwise it fails when
there is no system provided go binary. What we give bootstrap.bash
doesn't really matter, since we don't actually use the blueprint wrapper
in Android.
Test: m blueprint_tools; touch bionic/libc/tzcode/new.c;
m blueprint_tools <doesn't rebuild everything>
Change-Id: I82f30c7c3b5d25e5cbf28fe37a97fdb776c4a164
This ensures that the current locale supports UTF-8, and that we're
getting a consistent (but still supported by the system) locale for
every configuration except user-facing messages. This should eliminate
any reproducibility problems around sorting, formatting, etc for all
built products, while still showing localized error messages where
available.
Bug: 71573630
Test: LANG=es_ES LANGUAGE=es: m (check env in soong.log)
Change-Id: If33311899eaed8c44573113ee35c5a71cee503a0
This way we strip out ANSI codes when using dumb terminals. It's likely
overkill to use katiRewriteOutput, but we should rarely see any output
on stderr while dumping variables. This also lets us turn on kati_stats.
Bug: 71729611
Test: lunch missing-eng (colored error)
Test: lunch missing-eng 2>&1 | cat (non-colored)
Test: TERM=dumb lunch missing-eng (non-colored)
Change-Id: Ic63fd42d82a4a64e5c68aecd9ae0f242a0d703f1
This architecture only existed for unbundled use, but even the NDK is
removing support in their r17 release, so just remove support for it.
Test: build/soong/build_test.bash -only-soong
Change-Id: I4bd23babf567128d2d242cbdee3311abb198dd7c
This is cherry-picked from attempt 3, which was reverted
due to http://b/70862583.
Before this CL topic, the build toolchain for .java source files
used OpenJDK 8, targeting 1.8 (v52 class files) by default.
This CL topic switches the default to OpenJDK 9, but still
targeting 1.8 (v52 class files) by default. If USE_ERROR_PRONE
is set to true, then the default remains OpenJDK 8.
Code in the Android platform should generally be unaffected,
but if host tools that are now compiled and run using
OpenJDK 9 are causing problems for your team, then let me
know.
To manually switch back to the old behavior for now (continue
using OpenJDK 8), run this command in your shell:
export EXPERIMENTAL_USE_OPENJDK9=false
Bug: 69449021
Test: Treehugger
Test: art/test/testrunner/run_build_test_target.py -j40 art-interpreter
Test: dalvik/dx/tests/run-all-tests
(cherry picked from commit 13f23a2753)
Change-Id: I57abae73f9bdb21ef004a5118ff0e4ef70418ed9
Merged-in: Iac78122f58df0ebbb55134d55021ce6c57351b5f
Previously, these variables were exported only by makevars.go,
but those values are not available to config.mk. This CL adds
the variable to ui/build/config.go, which also makes it
available to config.mk.
Test: Treehugger
Bug: 70862583
Bug: 70521453
Change-Id: Ib54660e4b08ab751265b30004630cf1bb8c0041b
After this revert CL topic, the default toolchain for Android
goes back to being OpenJDK 8.
This revert is being prepared ahead of time in case of
any problems with CL topic:
https://r.android.com/#/q/topic:bug69449021_attempt2
Bug: 69449021
Test: Treehugger
Change-Id: I9ead8d569226bd487baee3c6d5be9ec7033eb56a
(This is cherry-picked from the first attempt to submit this CL
topic, which was reverted after 3 hours because of bug 70286093;
robolectric 3.{1.1,4.2} now stick with OpenJDK 8 to avoid that bug).
Before this CL topic, the build toolchain for .java source files
used OpenJDK 8, targeting 1.8 (v52 class files) by default.
This CL topic switches the default to OpenJDK 9, but still
targeting 1.8 (v52 class files) by default. If USE_ERROR_PRONE
is set to true, then the default remains OpenJDK 8.
Code in the Android platform should generally be unaffected,
but if host tools that are now compiled and run using
OpenJDK 9 are causing problems for your team, then let me
know.
To manually switch back to the old behavior for now (continue
using OpenJDK 8), run this command in your shell:
export EXPERIMENTAL_USE_OPENJDK9=false
Bug: 69449021
Test: Treehugger
Test: Running robolectric tests succeeds on internal-master
after cherry-picking this CL topic, using the command
line from http://b/70286093#comment1
(cherry picked from commit 0ae8b548af)
Change-Id: Ide6a7e55126d919a44f89ef8e0bd14fb12ff470e
to enable other subprojects to use their own instance of
Blueprint in other directories.
Files named Android.bp are still autodetected throughout the
tree like previously.
Bug: 64363847
Test: mkdir -p subdir \
&& echo "syntax error" > Blueprints \
&& m -j nothing
Test: build/soong/scripts/diff_build_graphs.sh \
--products=aosp_arm \
'build/soong:work^' 'build/soong:work'
Change-Id: I8199f12b68dc1699bf44682b86169def37b53a5e
After this revert CL topic, the default toolchain for Android
goes back to being OpenJDK 8.
This revert is being prepared ahead of time in case of
any problems with original change topic,
https://r.android.com/#/q/topic:bug_69449021
Bug: 69449021
Test: Treehugger
This reverts commit 0ae8b548af.
Change-Id: Ief7646a94f1a264085cd299b4327d244b78a1537
Before this CL topic, the build toolchain for .java source files
used OpenJDK 8, targeting 1.8 (v52 class files) by default.
This CL topic switches the default to OpenJDK 9, but still
targeting 1.8 (v52 class files) by default. If USE_ERROR_PRONE
is set to true, then the default remains OpenJDK 8.
Code in the Android platform should generally be unaffected,
but if host tools that are now compiled and run using
OpenJDK 9 are causing problems for your team, then let me
know.
To manually switch back to the old behavior for now (continue
using OpenJDK 8), run this command in your shell:
export EXPERIMENTAL_USE_OPENJDK9=false
Bug: 69449021
Test: Treehugger
Test: "make core-oj", checked that compilation now uses
OpenJDK 9 javac -target 1.8
Test: Checked that this is still compiled using OpenJDK 8.
export EXPERIMENTAL_USE_OPENJDK9=false
make core-oj
Change-Id: Ic87e9bb2a2e5da0ff13a2e51845b5365901c1507
This mostly reverts commit 178d5fefc0
and mostly reapplies change I6d3e52ef62c4cabe85b9a135a54de0e1a6aab29c .
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
--products=aosp_arm \
'build/blueprint:work^ build/soong:work^' \
'build/blueprint:work build/soong:work'
# and see that the only changes were:
# 1. adding some new files
# 2. changing some line numbers
Test: m -j nothing # which runs unit tests
Change-Id: I32baae00277a547fdcdd1c2219fe6625ee0e45d7
This reverts commit 63a250a336.
Reason for revert: Some failures:
namespace_test.go:648: dir1/Blueprints:2:4: a namespace must be the first module in the file
such as New Build Breakage: aosp-master/build_test @ 4475274
Change-Id: I1b5db8eb934e51ff22241bfca44199d886b1393b
Bug: 65683273
Test: build/soong/scripts/diff_build_graphs.sh \
--products=aosp_arm \
'build/blueprint:work^ build/soong:work^' \
'build/blueprint:work build/soong:work'
# and see that the only changes were:
# 1. adding some new files
# 2. changing some line numbers
Test: m -j nothing # which runs unit tests
Change-Id: I6d3e52ef62c4cabe85b9a135a54de0e1a6aab29c
This reverts commit fb941913a3.
Reapplies I4933187e8b72f2ef0c32d18ffea756e2c6fa417c with fixes
to disable the check for mac builds, where many modules are
disabled and the check is just going to cause more problems.
Test: m checkbuild
Change-Id: If6712c90ececd5d015fcdcdeefe0c3d4f5590711
Build target test_mapping will create a tarball of all TEST_MAPPING
files in source code based on the cached TEST_MAPPING.list
Bug: 69678490
Test: m -j blueprint_tools && cat out/.module_paths/TEST_MAPPING.list
Change-Id: Iba3479363137609d6631cf1851a8ce4613078f61
This fourth possible value currently has the same semantics
as a default/unset EXPERIMENTAL_USE_OPENJDK9, but allows
people to explicitly switch back to the old semantics when
the default changes.
Test: make showcommands core-oj (in environments with
EXPERIMENTAL_USE_OPENJDK9 set to "", "1.8", and "true").
Bug: 38177295
Change-Id: I25accf14344a05349a6e97572d7c2c1f6a7f2063
Get a list of leaf nodes in the dependency graph
from ninja, and make sure none of them are in the
output directory. This ensures that there are no
rules that depend on a file in the output directory
that doesn't have rule to generate it. The check
will catch a common set of build failures where
a rule to generate a file is deleted (either by
deleting a module in an Android.mk file, or by
modifying the build system incorrectly). These
failures are often not caught by a local incremental
build because the previously built files are still
present in the output directory.
Bug: 36843214
Bug: 68062417
Test: manual
Change-Id: I4933187e8b72f2ef0c32d18ffea756e2c6fa417c
This way config.mk no longer needs to check which java is in PATH and
fix it. It'll be consistent for all build steps under soong_ui.
Also unify handling of ANDROID_JAVA_HOME / JAVA_HOME with
OVERRIDE_ANDROID_JAVA_HOME / EXPERIMENTAL_USE_OPENJDK9.
Test: m nothing
Test: build/soong/soong_ui.bash --make-mode nothing (w/o envsetup.sh)
Test: aosp_arm ninja files are the same before/after
Test: before/after ninja files match with OVERRIDE_ANDROID_JAVA_HOME
Test: before/after ninja files match with EXPERIMENTAL_USE_OPENJDK9
Change-Id: Icdb65093d9c346524074de239a4f895e4230a24d
build/soong/soong_ui.bash --dumpvars-mode \
--vars="..." \
--abs-vars="..." \
--var-prefix="..." \
--abs-var-prefix="..."
is similar to the previous:
CALLED_FROM_SETUP=true BUILD_SYSTEM=build/core \
make -f build/core/config.mk dump-many-vars \
DUMP_MANY_VARS="..." \
DUMP_MANY_ABS_VARS="..." \
DUMP_VAR_PREFIX="..." \
DUMP_ABS_VAR_PREFIX="..."
and
build/soong/soong_ui.bash --dumpvar-mode [--abs] VAR
is similar to the previous:
CALLED_FROM_SETUP=true BUILD_SYSTEM=build/core \
make -f build/core/config.mk dumpvar-[abs-]-VAR
But uses soong_ui and ckati, so that we use a consistent make parser and
sandboxing configurations.
One major output difference between the pure make implementation and
this one is that report_config in Go is implemented using embedded
newlines in single quotes, while the make implementation uses `` with
embedded echo commands. This seems to work fine for both bash and zsh,
and report_config isn't meant to be machine-parsed anyways.
Test: build/soong/soong_ui.bash --dumpvar-mode report_config
Test: build/soong/soong_ui.bash --dumpvar-mode TARGET_DEVICE
Test: build/soong/soong_ui.bash --dumpvar-mode --abs PRODUCT_OUT
Test: build/soong/soong_ui.bash --dumpvar-mode --abs ALL_PRODUCTS
Test: build/soong/soong_ui.bash --dumpvars-mode --vars="report_config TARGET_DEVICE" --abs-vars="ALL_PRODUCTS"
Test: build/soong/soong_ui.bash --dumpvars-mode --vars=TARGET_DEVICE --abs-vars=PRODUCT_OUT --var-prefix=v_ --abs-var-prefix=a_
Change-Id: I0fbd0732bbf6fcfcd24084cf3c830a91a4b6bfc2
This reverts commit a3e6c520ac. I fixed
a few heavy stack users, and we've got better debugging.
Bug: 36182021
Test: m dist
Change-Id: Iefd8589185720e44c40a2e0ceff5fd1438211b86
Pass --dump-stats to ckati, but filter out the '*kati*' lines so that
they only end up in our verbose output. That way we've always got access
to the statistics.
Bug: 36182021
Test: m nothing; cat out/soong.log
Change-Id: Iaf7a814fc67f3e475c913faf69924a7f4e2ae3b3
This speeds up dumping make variables from ~380ms using make to ~220ms
using ckati. It also means that we're consistently using the same parser
for builds (with the same .KATI_READONLY/etc extensions).
envsetup.sh (lunch) / other scripts still use make, changing those to go
through soong_ui will be a future change.
Test: m clean; m nothing
Test: USE_GOMA=true m nothing
Test: m PRODUCT-aosp_x86-sdk
Test: m APP-Calculator
Test: build/soong/build_test.bash -only-config (on AOSP and internal master)
Change-Id: I6ca554de8de4955fb869001d06d29969b75751cc
They don't really affect anything general in android except for the
output path.
Bug: 65453318
Test: TARGET_BUILD_TYPE=debug m
Change-Id: I98c17fefae61f9f7936e184bd1f8441086bb7d23
Instead of pairing the reading of CleanSpec.mk files with the reading of
Android.mk files, split them into separate, individually cachable steps.
This way we only read Android.mk files once after a clean/sync. We'll
still read the CleanSpec.mk files multiple times, but that's
significantly faster than reading all the Android.mk files.
This adds about 50ms if kati doesn't need to reread the CleanSpec.mk
files. Reading all the CleanSpec.mk files takes about a second.
Bug: 35970961
Test: m clean; m nothing; m nothing
Test: Add CleanSpec.mk line, see it executed.
Change-Id: I83bad15c50709510959d5b8b673a907b8aa7de82
Kati sets detect_leaks=0 by default now.
Test: unset ASAN_OPTIONS; SANITIZE_HOST=address m nothing
Change-Id: I0cbf04ae9ed40de520abca3fb35c65e7f543682d
We can call directly into the blueprint bootstrap.bash using values that
soong_ui has already calculated.
Instead of calling into blueprint.bash, build minibp with microfactory,
and directly run ninja. This allows us to get individual tracing data
from each component.
Test: m -j blueprint_tools
Test: m clean; m -j blueprint_tools
Change-Id: I2239943c9a8a3ad6e1a40fa0dc914421f4b5202c
The Finder runs roughly 200ms faster than findleaves.py in aosp,
and runs roughly 400ms faster in internal master.
Bug: 64363847
Test: m -j
Change-Id: I62db8dacc90871e913576fe2443021fb1749a483
This way we only have one way to start a build, which always has logging
/ tracing / etc, even if we don't need Kati.
There's two ways to use this:
As a direct replacement for mkdir out; cd out; ../bootstrap.bash;
./soong -- as long as --skip-make is always passed, we'll never run
Kati, and Soong will run outside of it's "make" mode. This preserves
most of the speed, and allows full user control over the Soong
configuration.
A (experimental, dangerous) way to temporarily bypass the product
variable and kati steps of a build. As long as a user is sure that
nothing has changed from the last build, and they know exactly which
Ninja targets they want to build (which may not be the same as the
arguments normally passed to 'm'), this can lead to shorter build
startup times.
Test: rm -rf out; m --skip-make libc
Test: rm -rf out; m libc; m --skip-make libc
Test: rm -rf out; mkdir out; cd out; ../bootstrap.bash; ./soong libc
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: Ic0f91167b5779dba3f248a379fbaac67a75a946e
Instead of calling SetNinjaBuildDir, pass it to bootstrap.bash, so that
the bootstrap package can set it consistently during bootstrapping and
normal execution.
Bug: 63720725
Test: m -j nothing
Test: mkdir o; ../bootstrap.bash; ./soong
Change-Id: Ica88d2d5f1461b5be49bfe6316c6ec4ef4d89d49
Dan Willemsen