This causes Soong to put the outputs of each genrule into a temporary
location and copy the declared outputs back to the output directory.
This gets the process closer to having an actual sandbox.
Bug: 35562758
Test: make
Change-Id: I8048fbf1a3899a86fb99d71b60669b6633b07b3e
Support using strings as product variable substitutions, and
add tests for printfIntoProperty.
Test: varaible_test.go
Change-Id: I06cfadfb1d3fc81da72fb71323706df20426c8b7
The base module handles all the common functionalites, such as version
compatibilty check, version variations split, source file format check,
source/data file duplicate check.
The library/binary module focuses on how to generate binary build actions,
such as setting up stub script, zipping, filling in __init__.py in
runfiles dir tree.
Bug: b/31676493
Test: go test under python package
Change-Id: I06608369f350f7195873d459e1c8d1bdb811e77e
header-abi-dumper: dumps abi exported by source files for Vndk.
header-abi-linker: links abi dumps produced by header-abi-dumper.
header-abi-diff: compares linked dumps.
Test: mm -j64 showcommands > make_log in bionic/libc.
This produced linked dumps in out/soong/.intermediates.
Copied these dumps to
prebuilts/abi-dumps/ndk/current/arm64/source-based/.
Changed the abi and re-ran mm -j64 showcommands > make_log
confirmed that the build reported compatibility breakge without
actually failing (advisory mode).
Change-Id: Iccad6908fe68a80f47230751671d156893b96ead
Nothing changes if BOARD_VNDK_VERSION isn't set.
When the VNDK is enabled (BOARD_VNDK_VERSION in Make), this will split
/system and /vendor modules into two different variant spaces that can't
link to each other. There are a few interfaces between the two variant
spaces:
The `llndk_library` stubs will be available in the /vendor variant, but
won't be installed, so at runtime the /system variant will be used.
Setting `vendor_available: true` will split a module into both variants.
The /system (or "core") variant will compile just like today. The
/vendor ("vendor") variant will compile against everything else in the
vendor space (so LL-NDK instead of libc/liblog/etc). There will be two
copies of these libraries installed onto the final device.
Since the available runtime interfaces for vendor modules may be
reduced, and your dependencies may not expose their private interfaces,
we allow the vendor variants to reduce their compilation set, and export
a different set of headers:
cc_library {
name: "libfoo",
srcs: ["common.cpp", "private_impl.cpp"],
export_include_dirs: ["include"],
target: {
vendor: {
export_include_dirs: ["include_vndk"],
exclude_srcs: ["private_impl.cpp"],
srcs: ["vendor_only.cpp"],
},
},
}
So the "core" variant would compile with both "common.cpp" and
"private_impl.cpp", and export "include".
The "vendor" variant would compile "common.cpp" and "vendor_only.cpp",
and export "include_vndk".
Bug: 36426473
Bug: 36079834
Test: out/soong/build.ninja, out/soong/Android- only changes due to _core addition and
.llndk -> .vendor
Test: attempt to compile with BOARD_VNDK_VERSION:=current
Change-Id: Idef28764043bf6c33dc0d2e7e2026c38867ff769
Instead of using the NDK headers and libraries, add LL-NDK specific
headers and library stubs for VNDK users. This allows us to provide an
expanded liblog interface.
Test: aosp_arm; m -j
Test: Enable BOARD_VNDK_VERSION on aosp_arm; m -j
Test: Inspect out/soong/build.ninja before/after (w/o vndk)
Change-Id: Ic85f07fa10c695b5baab10c41f5e0ad38700bf3d
The NDK stub library generator needs to know what unreleased API
levels exist and what the ordering is between them. This singleton
will later be expanded to generate api-level.h to cut down on
duplication.
Test: make out/soong/api_levels.json && cat out/soong/api_levels.json
Bug: None
Change-Id: I53126dd2cacb67c331c44f7d7c77c98b176b93cd
This reverts commit 4c48f724e1. It's
causing test failures, warnings and complaints, so backing it out and
we'll resolve those before putting it back in.
Bug: 33681361
Test: m -j
This adds a toolchain definition for LinuxBionic that only supports
Clang/64-bit. It pulls pieces from the x86_linux_host and x86_64_device
configs, and uses the android clang triple, with some manual overrides.
To enable building this, set your soong.config file to:
{"Host_bionic": true}
Bug: 31559095
Test: out/soong/{Android,make-vars}-aosp_arm64.mk the same with or
without host bionic turned on
Test: No change to out/soong/build.ninja before/after this change
Change-Id: Id97dda8bd9aa670c32aed31fbe6aaa8175e70b59
Shared libraries are now installed to different directories depending on
their types.
* NDK libraries: /system/lib/ndk
* VNDK libraries: /system/lib/vndk
* VNDK-ext libraries: /system/lib/vndk-ext
* Framework-only libraries: /system/lib
* Vendor-only libraries: /vendor/lib
* Same-process HALs: /vendor/lib/sameprocess
In addition, a new module type vndk_ext_library is added. It is almost
identical to cc_shared_library but it introduces another attribute
'extends'. This is use to reference the vndk library that this vndk-ext
library is extending.
For example, in order to extend a vndk library libFoo:
cc_library {
name: "libFoo",
srcs: [...]
}
---------------------
vndk_ext_library {
name: "libFoo-extended",
srcs: [...]
extends: "libFoo"
}
Then, libFoo will be installed as /system/lib/vndk/libFoo.so and
libFoo-extended will be installed as /system/lib/vndk-ext/libFoo.so.
Note that file name of the latter is libFoo.so, not libFoo-extended.so:
file name of an extending module is automatically set to that of the
extended module.
Bug: 33681361
Test: build & run. Libraries must be in the correct directories.
Change-Id: Ia1eb3940605d582a252c78da0f3a5b36fdab062b
This is configured the same as make -- a global NATIVE_COVERAGE=true
flag to allow native coverage, then COVERAGE_PATHS=path1,path2,... to
turn it on for certain paths.
There are .gcnodir files exported to Make and saved in $OUT/coverage/...
files which are `ar` archives containing all of the compiler-produced
.gcno files for a particular executable / shared library.
Unlike the Make implementation, this only passes links the helper
library (automatically through --coverage) when one of the object files
or static libraries being used actually has coverage enabled.
Host support is currently disabled, since we set -nodefaultlibs, which
prevents libclang_rt.profile-*.a from being picked up automatically.
Bug: 32749731
Test: NATIVE_COVERAGE=true COVERAGE_PATHS=system/core/libcutils m -j libbacktrace libutils tombstoned
$OUT/coverage/system/lib*/libcutils.gcnodir looks correct (self)
$OUT/coverage/system/lib*/libbacktrace.gcnodir looks correct (static)
$OUT/coverage/system/lib*/libutils.gcnodir doesn't exist (shared)
$OUT/coverage/system/bin/tombstoned.gcnodir looks correct (executable)
Test: NATIVE_COVERAGE=true COVERAGE_PATHS=external/libcxxabi m -j libc++
Confirm that $OUT/coverage/system/lib*/libc++.gcnodir looks correct (whole_static_libs)
Change-Id: I48aaa0ba8d76e50e9c2d1151421c0c6dc8ed79a9
The "required" field within "phony" module in .bp file has to be
non-empty. And "phony" module doesn't contain any "src" files.
Bug: b/32981153
Test: make -j out/soong/build.ninja
Change-Id: Iaa2009467c44702572b97ca9cbd02c1d368c8d83
Right now this mostly just copies what Make is doing in
build/core/ninja.mk and build/core/soong.mk. The only major feature it
adds is a rotating log file with some verbose logging.
There is one major functional difference -- you cannot override random
Make variables during the Make phase anymore. The environment variable
is set, and if Make uses ?= or the equivalent, it can still use those
variables. We already made this change for Kati, which also loads all of
the same code and actually does the build, so it has been half-removed
for a while.
The only "UI" this implements is what I'll call "Make Emulation" mode --
it's expected that current command lines will continue working, and
we'll explore alternate user interfaces later.
We're still using Make as a wrapper, but all it does is call into this
single Go program, it won't even load the product configuration. Once
this is default, we can start moving individual users over to using this
directly (still in Make emulation mode), skipping the Make wrapper.
Ideas for the future:
* Generating trace files showing time spent in Make/Kati/Soong/Ninja
(also importing ninja traces into the same stream). I had this working
in a previous version of this patch, but removed it to keep the size
down and focus on the current features.
* More intelligent SIGALRM handling, once we fully remove the Make
wrapper (which hides the SIGALRM)
* Reading the experimental binary output stream from Ninja, so that we
can always save the verbose log even if we're not printing it out to
the console
Test: USE_SOONG_UI=true m -j blueprint_tools
Change-Id: I884327b9a8ae24499eb6c56f6e1ad26df1cfa4e4
Allow tests to specify a data property that lists files or filegroup
modules that will be packaged alongside the test. Also add a path
property to filegroup modules to allow shifting the path of the
packaged files, and add ExpandSourcesSubDir to expand the filegroup
sources while including a shifted relative path in the Paths objects.
Test: soong tests, manually adding data to a module
Change-Id: I52a48942660e12755d313ef13279313361b4fc35
filegroup is a module that contains a list of files, and can be used
to export files across package boundaries. filegroups (and genrules)
can be referenced from srcs properties of other modules using the
syntax ":module".
Test: m -j
Change-Id: I3d6fc4819c0b4225b474e0ad42f0d947f55a5961
Move some cc ctx.Host() / ctx.Device() checks over to using
ctx.toolchain().Bionic(). There will be more changes, these are just the
obvious ones dealing with host_ldlibs / crt / system libraries.
Bug: 31559095
Test: out/soong/build.ninja is identical
Change-Id: Ibba60483b4ab0e01f6996eb7d883120e4acc1830
Move Soong's globbing-with-dependencies support into Blueprint so it can
be used for subdirs= lines in Android.bp files.
Blueprint has a slight change in behavior around subname= lines, it now
always uses the subname and doesn't fall back to Blueprints. To support
the Blueprints files in build/blueprint, use them directly with build=.
Test: build, add source file that matches glob, rebuild
Change-Id: Ifd0b0d3bc061aae0a16d6c7ca9a1cd8672656b4d
For every file which we can run clang-tidy (C/C++ clang-built), we add a
new build node that depends on the object file (since clang-tidy does
not export a depfile), and is depended on by the link step. This is
better than how we're doing it in make, since calling tidy can be turned
on or off without needing to rebuild the object files.
This does not attempt to port WITH_TIDY_ONLY from Make, since the way
that it works is broken (due to the lack of a depfile).
Bug: 32244182
Test: WITH_TIDY=true mmma -j bionic/libc
Test: ./soong (Setting ClangTidy: true)
Change-Id: I40bbb5bb00d292d72bf1c293b93080b5f9f6d8ea
.proto files are translated to .pb.cc and .pb.h files, which are then
compiled normally.
Bug: 32286026
Test: mmma -j system/extras/perfprofd
Change-Id: I538071424d667aacf35b4b8bfebe217f5f092726
Use a temporary directory as the build directory during tests so files
don't get written to the source tree.
Also add a few more tests for prebuilts with no file specified.
Bug: 31800129
Test: m -j, make sure .soong.environment is not written to the source tree
Change-Id: I623bc114b2ff534c8df9fb3ce273e804711f8f05
The prebuilt tests are accidentally writing a .soong.environment file to
the build/soong/android directory, disable them until TestConfig() is
fixed to prevent writing to the source tree.
Bug: 31800129
Test: m -j, make sure .soong.environment is not written to the source tree
Change-Id: I3b7d47e42111ecababe275b7a9ff0d08bcebfc7a
Support prebuilt shared libraries as an initial proof-of-concept of
prebuilts. Future changes will support binaries and static libraries,
and the ability to select which to use based on something besides
blueprint properties.
Test: TestPrebuilts run during m -j
Change-Id: I6e84da667e9005ae11844bad01d25cbe4ced1ce3
Mutator registration is tightly coupled with the android package, move
all registration from the soong package to the android package.
Test: build.ninja identical
Change-Id: Ie183d0b52cc7431c9e05b231934d189208ef1efe
This only applies to shared libraries on the device, and like stripping,
we'll let make do the actual packing if we're embedded in Make.
Change-Id: I1585d74ecfc41e18dcbb5ffb70005adf007cc941
Replace PropertyCustomizer with a more extensible hooks mechanism.
Instead of passing an object that satisifies an interface, pass a
function pointer that takes a context interface as an argument. Callers
can use lambdas to capture any other necessary parameters like property
structs.
Also add two new hooks, arch hooks that occur after splitting modules
into arch variants, and install hooks that occur each time a file is
installed to the output directory.
Change-Id: I3a3e34aa97f1a92d3a31e5004b4b1ba68869d242
Allow any module factory to insert a PropertyCustomizer on the module,
which will be called before any other mutators. The PropertyCustomizer
can append or prepend to any properties, allowing module types to extend
other module types by modifying the public, stable interface provided by
the properties.
Change-Id: Idff02be80d939a70df1c6bbccffdd1f04ff975d2
Add DeviceConfig to store per-device configuration information. Put a
OncePer object inside Config and DeviceConfig, which computes a value
once per key per object to allow build logic to store arbitrary
per-build or per-device computed values.
Change-Id: I1a38b426f29d223ef5e803e0d4d9604500de2fd2
This compresses multiple files in parallel, and will split up larger
files (5MB+) into smaller chunks (1MB) to compress in parallel.
There is a small size overhead to recombine the chunks, but it's only a
few bytes per chunk, so for a 1MB chunk, it's minimal.
Rough numbers, with everything in the page cache, this can compress
~4GB (1000 files) down to 1GB in 6.5 seconds, instead of 120 seconds with
the non-parallel soong_jar and 150 seconds with zip.
Go's DEFLATE algorithm is still a bit worse than zip's -- about 3.5%
larger file sizes, but for most of our "dist" targets that is fine.
Change-Id: Ie4886c7d0f954ace46e599156e35fea7e74d6dd7
This doesn't do any decompression / recompression, but just copies over
the already compressed contents. So it's similar to zip -U, but allows
rewriting of the paths.
The first expected usecase is to replace img_from_target_files during
the build, since it does the equivalent of this:
zip2zip -i <target-files.zip> -o <img.zip> OTA/android-info.txt:android-info.txt IMAGES/*:.
Except it decompresses and recompresses the images, which takes over a
minute instead of a few seconds.
Change-Id: I88d0df188635088783223873f78e193272dbdf1c
For example , instead of trying to have libraryLinker inherit from
baseLinker and libraryCompiler inherit from baseCompiler, create a
single decorator object that wraps both baseLinker and baseCompiler.
Test: Builds, no unexpected changes to build.ninja
Change-Id: I2468adaea8466c203a240259ba5694b8b1df7a52
Split cc.go into files per module type, plus files for the compiler,
linker, and installer stages.
Change-Id: Id44c03f42fcd180950ccd008d4de0c144ea3597b
The list of migrated libraries is currently empty. Libraries will be
migrated as follow up patches.
Test: Migrated libc to this system and everything still builds.
build.ninja shows libraries being built and used and headers are
collected for the sysroot.
Bug: http://b/27533932
Change-Id: Iaba00543c1390f432befe0eed768ed3fbb8a9b96
And install the tools into a more obvious location. soong_env is not
moved, since we need it to exist early, so that we can use it in
soong.bash in case there's a build failure.
Change-Id: I9bd1fa320d84d180b2cf3deb90782d380666f7a6
Improve the androidmk makefile parser based on ideas from go/ast and
friends:
- Use type switching instead of the As* mess
- Don't store endPos for every node, compute it based on the last
known position in the node plus the length of the last token
- Store positions as only the offset into the file, and then unpack
them into Line/Column scanner.Position objects later
Change-Id: I87eb6661859951e6c2ea5a85db6229fa5561d615
Some checks for common errors with user-provided compiler and linker
flags:
* Using -I instead of include_dirs
* Using -l<lib> in ldflags instead of host_ldlibs (or shared_libs)
* Using -L in ldflags
* Splitting a multi-word flag into two flags
* Combining two flags into one list entry
* Using a path that could search outside the source or output trees
* Using a non-whitelisted library in host_ldlibs
Maybe some of the flag checks should happen during a static analysis
pass, but we don't have one right now, and this only adds ~1/2 second to
our 73 second Mega_device runs (recompile the changed code, run
soong_build, then report unknown target).
Change-Id: Icea7436260f1caa62c0cec29817a1cfea27b3e7c
To allow other singletons to use new environment variables, move env.go
to the end of the source list, so that its singleton is run last.
Change-Id: Ic92308f72fc0c92a0fedbc5546e337940e32eaab
For configuration like TARGET_GLOBAL_CFLAGS, it would be good to have
Make and Soong use the same values. This change adds an interface for
packages like cc to verify the current value of make variables match the
Soong equivalents, and once they're satisfied with the results, remove
the make definition and use the value from Soong.
The cc implementation exports a few variables currently that I expect to
match between both implementations in all cases. It also checks
{CLANG,}{HOST,TARGET}_GLOBAL_{C,LD}FLAGS, which should be mostly similar
between make and soong now.
Bug: 23566674
Change-Id: Idc8582ef31ace11a8baefcf525c3683f08a573aa
Add a new feature to cc modules that adds the cflags and libraries
necessary to implement AddressSanitizer, ThreadSanitizer, and
UndefinedBehaviorSanitizer.
Change-Id: Ibe5ffadc7ece56080a2521f2c7c00da9ef712584
Refactor all of cc in order to use composition instead of inheritance.
All cc module types exported by cc are now *cc.Module objects, with
compilation, linking, and installing steps delegated to different
objects in order to form the full module type. Additional features that
modify dependencies and flags can be inserted in a features object list,
and custom module types can be created by adding a Customizer object
that can modify properties.
Change-Id: Ie1283d14920f7856f6947b0530606b2f4d58fab0
This centralizes verification and common operations, like converting the
path to a source file to the path for a built object.
It also embeds the configuration knowledge into the path, so that we can
remove "${SrcDir}/path" from the ninja file. When SrcDir is '.', that
leads to paths like './path' instead of just 'path' like make is doing,
causing differences in compiled binaries.
Change-Id: Ib4e8910a6e867ce1b7b420d927c04f1142a7589e
This defines another mutator between HostOrDevice and Arch that will
expand host modules into a module for each host type
(Darwin/Linux/Windows) that is currently being built.
Change-Id: I4c8ac6b616c229f6bd45ad8a35902652fb6a4fff
androidbp is gone, replaced with combining kati ninja output for
makefiles with blueprint ninja output for soong.
Change-Id: I00b9e28877abf2ceb2223d3ccf0637bc1b7571bb
Add a cc_defaults module, which will prepend any matching properties to
modules that reference it with a "defaults" property.
Change-Id: I5908dd98d204f71b29ad95a4ab85403aa1621ca2
EarlyMutators are identical to BottomUpMutators, except they run before
DynamicDependencies. DynamicDependencies can be replaced with a
BottomUpMutator. Replace both EarlyMutators and DynamicDependencies
with BottomUpMutators, which allows setting the order between all
mutators through registration order.
Change-Id: Id1305d798d3d2da592061c89d7c10a71780b71a3
Blueprint has a generic AppendProperties/AppendMatchingProperties now,
use it, and replace all bool properties that might be modified by a
mutator with *bool, which provides the correct replace-if-set semantics
for append.
Also remove uses of ContainsProperty except when explicitly checking if
a property was set in a blueprints file.
Change-Id: If523af61d6b4630e79504d7fc2840f36e98571cc
Instead of using this hardcoded list, use the blueprint plugin
infrastructure to find them at build time and load them.
Bug: 23567252
Change-Id: Iae87e8c9bf020f4beb42c418b9a6bf8b0fe8b601
Allow modules to vary their properties based on product variables.
For now, DEVICE_USES_LOGD, DEVICE_USES_JEMALLOC, and DEVICE_USES_DLMALLOC,
and BOARD_MALLOC_ALIGNMENT are supported.
Product variables can provide a value (only bool and int supported for
now), and if any of the product variable properties contains a "%d"
then Sprintf will be called with the property value as the format
and the product variable value convert to an int as the only argument.
For example:
product_variables: {
dlmalloc_alignment: {
cflags: ["-DMALLOC_ALIGNMENT=%d"],
},
},
will cause -DMALLOC_ALIGNMENT=16 to be added to any top level
properties called "cflags".
Change-Id: I74882a6ab4914d3e222f8d06cfac371b7b829ae5
Instead of putting all the blueprint registrations in soong_build,
put them all in init() functions. This puts the registration next
to the implementation.
Change-Id: Ide1a749518f5e9d1367a18ab3bb1d91da3310c76
During mutation, make deep copies of the blueprint modules, then start
modifying the property lists to apply the static/shared properties.
Change-Id: Idc7063f4d1cf7d173ae10418e3437f3e2b914f59
Translation is getting complicated because the expressions supported
by Blueprint are difficult to support in Make. Modify androidbp
to use context aware parsing so it can evaluate all expressions at
parse time, so it only needs to deal with constant values.
Change-Id: I57047645fb48475baecd0361f78a93ec0a26011e
Build a map of blueprint directory to modules built from that
directory, and then add phony rules to build.ninja that emulate
the behavior of mma in the current build system.
Also fixes checkbuild to depend on checkbuild files and installable
files, but not installed files.
Change-Id: I8bad6e93387940df7439dbd4554f6d79f924c65f
Rename module definition files to Android.bp to avoid conflicts
with another project called Blueprint.
Change-Id: I105a07555eb7890f56120deab9036cf9ae5d8525
Jeff Gaston