Make sure every exported function or type has a godoc comment.
Also makes minor changes like unexporting functions that are not used
outside the package and fixing minor style warnings.
Bug: 173449605
Test: m checkbuild
Change-Id: I533a595d02035aae8b2b603590be639826d2d4c8
PackagingBase is the base struct that implements the basic packaging
functionalities. Here, packaging means grouping dependencies into an
output file where the built artifacts from the dependencies are placed
in a directory-like structure in the output file. The exact format of
the output file is irrelevant; it could be a filesystem image, tar.gz,
zip, or whatever.
PackagingBase is responsible for traversing the dependencies and copying
their outputs under the package root directory, which is expected to be
provided by the module type that includes PackagingBase. Then the
concrete module type is expected to do final step of converting the
package root directory into the output file of specific format.
Bug: 159685774
Bug: 172414391
Test: m dist out/dist/cvd-host_package.tar.gz
Change-Id: I5446eee4834ce3b6f0f5843d93bb330a26d42fe3
In preparation for more complicated sandboxing that copies tools
and/or inputs into the sandbox directory, make sbox use a textproto
input that describes the commands to be run and the files to copy
in or out of the sandbox.
Bug: 124313442
Test: m checkbuild
Test: rule_builder_test.go
Test: genrule_test.go
Change-Id: I3b918a6643cea77199fd39577ef71e34cdeacdb1
* changes:
Store ndkKnownLibs in the config
Register the kythe singleton on the Context instead of globally
Store ninja file deps from PackageVarContext in the config
Store SingletonMakeVarsProviders in the config
Currently, installation of a module is defined as an action of copying
the built artifact of the module to an install path like out/soong/host
(for host modules) and out/target/product/<device>/<partition> (for
device modules). After the modules are installed, the installed files
are further processed to create packages like system.img, vendor.img,
cvd-host-package.tar.gz, etc.
This notion of installation seems to have originated from the old time
when system.img is the primary product of the entire build process
(modulo a few more like root.img). Packaging the installed files as the
filesystem image was considered as a post-build step then.
However, this model doesn't seem to fit well to the current and future
environment where we have a lot more filesystem images (system, vendor,
system_ext, product, ...). The filesystem images themselves are even
grouped together to form a higher-level filesystem image like super.img.
Furthermore, things like cvd-host-package.tar.gz requires us to be able
to group some of the host tools in a format that isn't filesystem image.
Lastly, we are expected to have more filesystem images that are subsets
of system.img (and their friends) for the Android-like mini OS that will
be running on on-device virtual machines. These all imply that the
packaging (which we call installation today) is not a global post-build
step, but a part of the build rules for creating the package-like
modules.
A model better fits to the new sitatuation might be this; a module
specifies its built artifact and the path where it should be placed. The
latter path is not rooted at out/. It's a relative path to the root
directory which will be determined by another module that implements the
packaging. For example, cc_library will have ./lib (or ./lib64), not
out/target/product/<device>/<partition>/lib as the path. Then packages
like system.img, cvd-host-package.tar.gz, etc. are explicitly modeled as
modules and they have deps to other modules. Then the modules are placed
at the relative path under the package root, and the entire root
directory finally is packaged as the output file (be it img, tar.gz, or
whatever).
PackagingSpec is the first step to implement the new model. It abstracts
a request to place a built artifact at a certain path in a package. It
has extra information about whether the path should be a symlink or not,
and whether the path is for an executable. It currently is created when
InstallFiles (and its friends) are called, and can be retrieved via
the new method PackagingSpecs().
In this CL, no one is using PackagingSpec. The installation is still
done by the existing rules created in InstallFiles, etc. and the
structs are not used for the filesystem images like system.img.
Bug: 159685774
Bug: 172414391
Test: m
Change-Id: Ie1dec72d1ac14382fc3b74e5c850472e9320d6a3
This removes the need to source bazelenv.sh for USE_BAZEL_ANALYSIS, and
unifies mixed builds to use the checked in tools/bazel and bazelrc.
It also unifies all bazel-related output to be in out/bazel.
Without aosp/1502095, this change still requires toplevel_output_directories to be an empty
list, otherwise there'll be this error:
ERROR: Directories specified with toplevel_output_directories should be
ignored and can not be used as sources.
Test: With aosp/1441774: rm -rf out/ && lunch aosp_cf_x86_auto && USE_BAZEL_ANALYSIS=1 m libc && prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-aosp_cf_x86_auto.ninja -t commands libc | grep bazel-out | wc -l # 2 results
Change-Id: I69b217ec88da531415792bb6e04b6a194ca4718d
The documentation java_genrule_host states that it creates a single
variant, which would make it work with the single variant fallback
in AddDependency used by the data property, but it actually has a host
and a host-cross windows variant. Modify osMutator to take the
OS-specific enabled properties into account to skip creating variants
that will immediately be disabled so there is a single variant.
Test: m checkbuild
Change-Id: Ic2daab29f4fa3a3797d7a08348fbfcf1036ec5dc
The HostOrDeviceSupported enums have become unwieldy, replace them
with bitfields to simplify checking if a specific trait is set.
Test: Soong arch tests
Test: m checkbuild
Change-Id: I884ddd092d4c6f740def611b6a055007dc24bd6e
This allows mixed builds to continue functioning even given the toplevel
WORKSPACE file containing toplevel_output_directories with out/.
Test: Manually verified on aosp_flame building libc.
Change-Id: I80fc4853421317e2d2c7f03d92d58286df1342ce
In preparation for more complicated sandboxing support in sbox, use
a single implementation of the sbox sandboxing by moving genrule to
use RuleBuilder's sbox support instead of creating an sbox rule
directly.
Also move genrule's input list hash support into RuleBuilder.
Test: genrule_test.go
Test: rule_builder_test.go
Change-Id: I292184d02743c7e6887ebbcd232ba565db2ab0cc
Store ninja file deps from ExistentPathForSource on a
PackageVarContext in the config instead of the PackageContext, as
the PackageContext may be shared between multiple tests running
in parallel.
Test: all soong tests
Change-Id: Ib1809a4dd4a82696e0fe48a87eac21a44684ecb5
Store SingletonMakeVarsProviders in the config instead of a global
variable to avoid races between tests running in parallel.
Test: all soong tests
Change-Id: I2ab64f368b5ac673fd985399d4421ed018abc562
The android.WriteFile rule takes careful escaping to produce the
right contents. Wrap it in an android.WriteFileRule that handles
the escaping.
Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.
Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.
This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.
Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: Ic22603a5c0718b5a21686672a7471f952b4d1017
Prepare for using Config when adding singletons by passing
Config to NewTestContext and NewContext instead of to ctx.Register.
This will enable a followup change to store SingletonMakeVarsProviders
registered on the Context in the Config, which is necessary to run
multiple tests in parallel without data races.
Test: all soong tests
Change-Id: Id229629a4e42ff4487d317241673837726c075fc
If SOONG_DELVE_PATH=<path> is set, Soong will use that as the path to
dlv. Otherwise, it will use the one on $PATH as usual.
Test: Manual.
Change-Id: Id4334f05d077a911750f601865bd3096d852d69c
Bazel overlay is an experimental feature. This renames the feature to 'queryview' to better describe its purpose, and also move away from the already overloaded 'overlay' term in Android.
Test: m queryview && bazel query --package_path=out/soong/queryview //...
Change-Id: I8b5068c7db46cb61a03a8e87af9c7c9077ebeff9
This was caught while running the soong_build docs action under Bazel as
ninja executor. Since the config file is not an explicit input to the
action, it's not present in Bazel's execution root during action
execution time, leading to a missing file error. To fix it, add the file
as an input to the action, if it's configured in product variables.
New soong_build statement:
build out/soong/docs/soong_build.html: s.writedocs.soongDocs $
out/.module_paths/Android.bp.list out/soong/soong.variables $
out/soong/dexpreopt.config | out/soong/.bootstrap/bin/soong_build
outDir = out/soong/docs
Test: m
Test: USE_BAZEL=1 m
Bug: 161958189
Fixes: 161958189
Change-Id: I2bf264e848bb29ac91f4196fbe9c4ecf3cea8a2c
Adds a singleton that traverses the module variants finding the ones
that are in the list (updatable and non-updatable) of boot jars and
add a ninja rule to ensure that they only contain packages from an
allowed list.
Replaces a hack that ignored any prebuilt boot jars supplied as dex
file with an equivalent one to ensure that they are still ignored.
A follow up change that switches to checking dex jars will allow the
hack to be removed.
The boot jars check can be strict or lax. If strict then all the boot
jars listed in the configuration must be found, otherwise it will only
check the ones it finds. It is strict by default unless
TARGET_BUILD_UNBUNDLED=true or ALLOW_MISSING_DEPENDENCIES=true.
Moves the script and data file from build/make.
Test: m check-boot-jars - for failing and passing cases
SKIP_BOOT_JARS_CHECK=true - no check-boot-jars target created
ALLOW_MISSING_DEPENDENCIES=true - not strict
TARGET_BUILD_UNBUNDLED=true - not strict
verified manually that apart from path differences the same
files (same check sum) were checked in both old make checks and
the new Soong ones
EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m check-boot-jars
Bug: 171479578
Change-Id: I9d81d6650ba64fc0d48d2dab4ba5a3ba8dd03dec
This adds an extra step to mixed builds: creating a master BUILD/bzl
file pair to facilitate running a single cquery command to analyze
all soong->bazel edges in a single request.
Test: Mixed build tested with aosp/1441774, verified ninja outputs
depend on `bazel-out/` intermediates
Test: Manually verified contents of master BUILD and bzl files
Change-Id: I04803bcc91ac4182578f505b3f42893061ddd167
As a result, one can enable bazel-as-ninja-executor separately from
mixed builds.
Test: Manually verified building image with and without bazelenv.sh.
Change-Id: Ia97806fb41e1de850a80b9483ed8a5ff50d9dab2
By making the Append and RemoveList methods return a new list instead
of modifying the existing list it makes the ConfiguredJarList usages
easier to reason about and safer to use, especially considering that
they are primarily used in global configuration.
Added some tests for Append/RemoveList to ensure that they work and
do not modify the original or result in newly created lists sharing
storage with the original which would lead to corruption.
Bug: 171756871
Bug: 171479578
Test: m nothing
EMMA_INSTRUMENT=true EMMA_INSTRUMENT_FRAMEWORK=true m nothing
Change-Id: I541c4686ecdd45c6a0c8b1c93fedf0fcd5952e2b
After previous refactorings the CreateConfiguredJarList function is now
only used in tests and are supplied with a PathContext that will cause
ReportPathErrorf() to panic. So, this change removes the ctx parameter,
calls panic directly on any error and renames the method to make it
clear that it is for testing only.
Bug: 171479578
Test: m nothing
Change-Id: Icfb4bdfe720afa855b64ecf0e74a0b030882d029
This change:
* Switches BootJars/UpdatableBootJars fields of config.productVariables
from []string to ConfiguredJarList.
* Updates BootJars() method to simply concatenate the jars list from
the BootJars/UpdatableBootJars fields.
* Adds an UnmarshalJSON(..) method to ConfiguredJarList to support
unmarshalling from a single string array to avoid having to change the
format of the JSON file from which the configuration is loaded.
* Adds some additional calls to ConfiguredJarList(..) in tests to
convert from []string to ConfiguredJarList. They pass nil as the
ctx argument as there is no suitable PathContext which will cause any
errors to be thrown using panic. That is reasonable for hard coded
values in tests. A follow up change will clean up the calls to
ConfiguredJarList(..).
Bug: 171479578
Test: m nothing
Change-Id: I59b94dafb479ccd8f0471ed802be175af57be271
This change:
* Changes splitConfiguredJarPair to return an error rather than
reporting it through the context.
* Extracts the splitting of a list of pairs into a pair of lists into a
new splitListOfPairsIntoPairOfLists() method for reuse in a follow up
change.
Bug: 171479578
Test: m nothing
Change-Id: I824ed1bc673b47757424563fc37e1190adff2ed1
Colin Cross