This was caught while running the soong_build docs action under Bazel as
ninja executor. Since the config file is not an explicit input to the
action, it's not present in Bazel's execution root during action
execution time, leading to a missing file error. To fix it, add the file
as an input to the action, if it's configured in product variables.
New soong_build statement:
build out/soong/docs/soong_build.html: s.writedocs.soongDocs $
out/.module_paths/Android.bp.list out/soong/soong.variables $
out/soong/dexpreopt.config | out/soong/.bootstrap/bin/soong_build
outDir = out/soong/docs
Test: m
Test: USE_BAZEL=1 m
Bug: 161958189
Fixes: 161958189
Change-Id: I2bf264e848bb29ac91f4196fbe9c4ecf3cea8a2c
There is no need for multiproduct_kati to run the tests for
every product, they don't vary by product config. --skip-soong-tests
can also be used for local development to run soong_build even if
the tests don't pass.
Bug: 156428456
Test: m --skip-soong-tests nothing
Change-Id: I9c00e3d1b6e51d17bb290339c3f124d4d1c9e69f
We need to capture the user build command to understand what additional
arguments is passed in to Soong. This data is useful to know which
targets are being executed and have statistics on top build targets
are specified.
Bug: b/169453712
Test: ran "m blueprint_tools", "m nothing", "lunch 1" and checked
in soong_metrics proto.
Change-Id: I7ab94dcf037211f1abffe6afd012b03ffbf99756
Several structs and files are named time which records the total
execution of a traceable event. Soon, resource information of an
executed process is collected during a build event so a more generic
name is applied.
Bug: b/169453825
Test: go test and m nothing. Checked the soong_metrics protobuf.
Change-Id: I65a782603530d51018d6cc1192c18cfeef7566a0
We would like to collect the resource usage of processes such as
kati, soong_build and ninja on the resource usage such as memory
and context switching. This changelist contains the defined fields
in the protobuf file of the resource information to be collected
from interested processes.
Bug: b/169453825
Test: N/A
Change-Id: I4b3cbbc3e7b78900875c10d34d574425f8b11730
Test:
m USE_GOMA=true GOMA_DIR=$(goma_ctl goma_dir) toybox
m FORCE_USE_GOMA=true GOMA_DIR=$(goma_ctl goma_dir) toybox
both result in error message being printed.
Change-Id: I94ccaf217dba71aca990d88d205bad669a49a2f5
This CL adds symlink_outputs to various locations in Soong that creates
actions that creates symlink outputs, and explicitly mark them as such.
Test: m
Bug: 160568334
Change-Id: I322751bada52a9f49011c74731d84761586e03e7
Propagate the subprocess stats from ninja to build.trace.gz.
There is too much data here to put into separate counters for each
statistic, so put them into args on each duration event for now.
We would like to track max RSS for each subprocess, but the Linux
kernel inherits the max RSS of the ninja process in each subprocess,
which sets a lower bound on the measurable max RSS to the size of
the ninja process. The ninja process is large due to the multi-GB
build.ninja files.
Bug: 170701554
Test: examine build.trace.gz
Change-Id: I8aaaafe627a57f1a500af098c097c6381c583ba5
Clean the odm_dlkm and vendor_dlkm staging directories in installclean
to get consistent results in incremental builds on the build servers.
Test: treehugger
Change-Id: Id431d7457d85e0be68437a2034fd544d74cc3709
Collect the total number of CPU cores and available RAM for each build
as part of the system resource info. This is to know what kind of
build machines that the developers are using.
Bug: b/169453825
Test: m nothing and ran printproto on soong_metrics to validate the
data.
Change-Id: I8617bdb9490b37d7689abd27d349af1a651533db
This retriggers soong_build whenever a new bzl, WORKSPACE, or
BUILD.bazel file is changed or added.
Test: Manually verified on bionic/libc genrules with manual changes to
related BUILD/bzl/WORKSPACE files -- these all retrigger builds.
Test: Updated finder_test.go
Change-Id: I634384f88781a6b6db32f5d6bf9c07e179e14c39
Added additional instruction in the regen.sh script to build the
blueprint_tools and aprotoc in case the regen.sh script fails.
Bug: b/169395325
Test: ./regen.sh
Change-Id: Icb38c907fb47febc17eef1067d06d0588dd0b7ba
The hostname is collected during a build to determine where the user
is build from (from office workstation, cloudtop, etc...). This is a
useful metrics for RBE related builds.
Bug: b/169395325
Test: m nothing, ran printproto to verify that soong_metrics has the
hostname.
Change-Id: I3e5d64f948ae43c5dda60de1d4a52b187e36dff8
Tradefed packages are created in out/host/common/obj/PACKAGING
and should be removed during installclean so that they get an
accurate embedded build number.
Fixes: 170308394
Test: treehugger
Change-Id: Ib7a57377b5c9512584bc93f10da316eadf786979
This reverts commit 323dc60712.
Reason for revert: Possible cause of test instability
Bug: 170513220
Test: soong tests
Change-Id: Iee168e9fbb4210569e6cffcc23e60d111403abb8
Prints the following warning messsage:
WARNING:
Setting OUT_DIR to a path other than out may result in slow RBE builds.
See http://go/android_rbe_out_dir for a workaround.
Bug: b/169676232
Change-Id: Id1ad06c4dc672c24373642e0b624833eb6a0dbcf
Putting t.Parallel() in each test makes them run in parallel.
Additional t.Parallel() could be added to each subtest, although
that requires making a local copy of the loop variable for
table driven tests.
Test: m checkbuild
Change-Id: I5d9869ead441093f4d7c5757f2447385333a95a4
FORCE_USE_GOMA is needed to be collected as a metric for the
deprecation of GOMA on Android builds.
Bug: b/169423400
Test: m nothing, ran printproto command
Change-Id: I12300389f1f27239d8f21875d1b6e4ad069d95d7
We should override log-dir with the environment variables if its set.
This is needed in our integration tests and generally to allow for
user-overrides.
Test: RBE_log_dir=`pwd`/rbelogs mmma external/boringssl
Change-Id: I3d316a2d60122c42f5c3b5dfede4037362a7b5f1
This is an initial CL that collects the environment variables
USE_GOMA and USE_RBE. This is a needed metric to determine how
many users are currently using GOMA so we can help them migrate
to RBE.
Bug: b/163598127
Test: m nothing and checked soong_metrics
Change-Id: I39536eabad01aafbb3bd7182515d72230b36f658
This hasn't worked for a couple years, and continues to bitrot. Just
remove it.
Adds a bpfix rule so that we can eventually remove the
product_variables.pdk definition, which is now always a no-op.
Test: treehugger
Change-Id: I830b54d419b59f6db1d4617b45e61a78234f57a7
Merged-In: I830b54d419b59f6db1d4617b45e61a78234f57a7
There are cases where the developer is not building a product so
the metadata information is not being set. For now, set the OS
name so we can track which OS the user is using it.
Fixes: b/163118212
Test: m nothing and checked the blueprint
Change-Id: I9b521eee2d266b50625aa8e916ddaacb4e2feaea
"exec format error" showed up on Mac when running the rbe_test.go.
The generated bash scripts did not have a newline at the end of
the file.
Bug: b/162600461
Fixes: b/162600461
Test: "m blueprint_tools" on Android Mac builder.
Change-Id: I425fa5f428733b6569c3fcdaa312023f9761d020
Disable the error test cases to allow developers to build
AOSP on Mac.
Bug: b/162600461
Test: m nothing
Change-Id: Ic8c6fe04743a8e4fe197a1bb71259036fb749edc
Disable the error test cases to allow developers to build
AOSP on Mac.
Bug: b/162600461
Test: m nothing
Change-Id: Iac4373dcb10710d8a2325eeb3ee20a6eeca6143f
Pass -n to blueprint to make it use validation dependencies for tests,
which will let them run in parallel with soong_build.
Bug: 156428456
Test: m nothing
Change-Id: I706c14330543e2a7728e5dc385976900c67cb2f5
Jingwen Chen