The "required" field within "phony" module in .bp file has to be
non-empty. And "phony" module doesn't contain any "src" files.
Bug: b/32981153
Test: make -j out/soong/build.ninja
Change-Id: Iaa2009467c44702572b97ca9cbd02c1d368c8d83
Right now this mostly just copies what Make is doing in
build/core/ninja.mk and build/core/soong.mk. The only major feature it
adds is a rotating log file with some verbose logging.
There is one major functional difference -- you cannot override random
Make variables during the Make phase anymore. The environment variable
is set, and if Make uses ?= or the equivalent, it can still use those
variables. We already made this change for Kati, which also loads all of
the same code and actually does the build, so it has been half-removed
for a while.
The only "UI" this implements is what I'll call "Make Emulation" mode --
it's expected that current command lines will continue working, and
we'll explore alternate user interfaces later.
We're still using Make as a wrapper, but all it does is call into this
single Go program, it won't even load the product configuration. Once
this is default, we can start moving individual users over to using this
directly (still in Make emulation mode), skipping the Make wrapper.
Ideas for the future:
* Generating trace files showing time spent in Make/Kati/Soong/Ninja
(also importing ninja traces into the same stream). I had this working
in a previous version of this patch, but removed it to keep the size
down and focus on the current features.
* More intelligent SIGALRM handling, once we fully remove the Make
wrapper (which hides the SIGALRM)
* Reading the experimental binary output stream from Ninja, so that we
can always save the verbose log even if we're not printing it out to
the console
Test: USE_SOONG_UI=true m -j blueprint_tools
Change-Id: I884327b9a8ae24499eb6c56f6e1ad26df1cfa4e4
Allow tests to specify a data property that lists files or filegroup
modules that will be packaged alongside the test. Also add a path
property to filegroup modules to allow shifting the path of the
packaged files, and add ExpandSourcesSubDir to expand the filegroup
sources while including a shifted relative path in the Paths objects.
Test: soong tests, manually adding data to a module
Change-Id: I52a48942660e12755d313ef13279313361b4fc35
filegroup is a module that contains a list of files, and can be used
to export files across package boundaries. filegroups (and genrules)
can be referenced from srcs properties of other modules using the
syntax ":module".
Test: m -j
Change-Id: I3d6fc4819c0b4225b474e0ad42f0d947f55a5961
Move some cc ctx.Host() / ctx.Device() checks over to using
ctx.toolchain().Bionic(). There will be more changes, these are just the
obvious ones dealing with host_ldlibs / crt / system libraries.
Bug: 31559095
Test: out/soong/build.ninja is identical
Change-Id: Ibba60483b4ab0e01f6996eb7d883120e4acc1830
Move Soong's globbing-with-dependencies support into Blueprint so it can
be used for subdirs= lines in Android.bp files.
Blueprint has a slight change in behavior around subname= lines, it now
always uses the subname and doesn't fall back to Blueprints. To support
the Blueprints files in build/blueprint, use them directly with build=.
Test: build, add source file that matches glob, rebuild
Change-Id: Ifd0b0d3bc061aae0a16d6c7ca9a1cd8672656b4d
For every file which we can run clang-tidy (C/C++ clang-built), we add a
new build node that depends on the object file (since clang-tidy does
not export a depfile), and is depended on by the link step. This is
better than how we're doing it in make, since calling tidy can be turned
on or off without needing to rebuild the object files.
This does not attempt to port WITH_TIDY_ONLY from Make, since the way
that it works is broken (due to the lack of a depfile).
Bug: 32244182
Test: WITH_TIDY=true mmma -j bionic/libc
Test: ./soong (Setting ClangTidy: true)
Change-Id: I40bbb5bb00d292d72bf1c293b93080b5f9f6d8ea
.proto files are translated to .pb.cc and .pb.h files, which are then
compiled normally.
Bug: 32286026
Test: mmma -j system/extras/perfprofd
Change-Id: I538071424d667aacf35b4b8bfebe217f5f092726
Use a temporary directory as the build directory during tests so files
don't get written to the source tree.
Also add a few more tests for prebuilts with no file specified.
Bug: 31800129
Test: m -j, make sure .soong.environment is not written to the source tree
Change-Id: I623bc114b2ff534c8df9fb3ce273e804711f8f05
The prebuilt tests are accidentally writing a .soong.environment file to
the build/soong/android directory, disable them until TestConfig() is
fixed to prevent writing to the source tree.
Bug: 31800129
Test: m -j, make sure .soong.environment is not written to the source tree
Change-Id: I3b7d47e42111ecababe275b7a9ff0d08bcebfc7a
Support prebuilt shared libraries as an initial proof-of-concept of
prebuilts. Future changes will support binaries and static libraries,
and the ability to select which to use based on something besides
blueprint properties.
Test: TestPrebuilts run during m -j
Change-Id: I6e84da667e9005ae11844bad01d25cbe4ced1ce3
Mutator registration is tightly coupled with the android package, move
all registration from the soong package to the android package.
Test: build.ninja identical
Change-Id: Ie183d0b52cc7431c9e05b231934d189208ef1efe
This only applies to shared libraries on the device, and like stripping,
we'll let make do the actual packing if we're embedded in Make.
Change-Id: I1585d74ecfc41e18dcbb5ffb70005adf007cc941
Replace PropertyCustomizer with a more extensible hooks mechanism.
Instead of passing an object that satisifies an interface, pass a
function pointer that takes a context interface as an argument. Callers
can use lambdas to capture any other necessary parameters like property
structs.
Also add two new hooks, arch hooks that occur after splitting modules
into arch variants, and install hooks that occur each time a file is
installed to the output directory.
Change-Id: I3a3e34aa97f1a92d3a31e5004b4b1ba68869d242
Allow any module factory to insert a PropertyCustomizer on the module,
which will be called before any other mutators. The PropertyCustomizer
can append or prepend to any properties, allowing module types to extend
other module types by modifying the public, stable interface provided by
the properties.
Change-Id: Idff02be80d939a70df1c6bbccffdd1f04ff975d2
Add DeviceConfig to store per-device configuration information. Put a
OncePer object inside Config and DeviceConfig, which computes a value
once per key per object to allow build logic to store arbitrary
per-build or per-device computed values.
Change-Id: I1a38b426f29d223ef5e803e0d4d9604500de2fd2
This compresses multiple files in parallel, and will split up larger
files (5MB+) into smaller chunks (1MB) to compress in parallel.
There is a small size overhead to recombine the chunks, but it's only a
few bytes per chunk, so for a 1MB chunk, it's minimal.
Rough numbers, with everything in the page cache, this can compress
~4GB (1000 files) down to 1GB in 6.5 seconds, instead of 120 seconds with
the non-parallel soong_jar and 150 seconds with zip.
Go's DEFLATE algorithm is still a bit worse than zip's -- about 3.5%
larger file sizes, but for most of our "dist" targets that is fine.
Change-Id: Ie4886c7d0f954ace46e599156e35fea7e74d6dd7
This doesn't do any decompression / recompression, but just copies over
the already compressed contents. So it's similar to zip -U, but allows
rewriting of the paths.
The first expected usecase is to replace img_from_target_files during
the build, since it does the equivalent of this:
zip2zip -i <target-files.zip> -o <img.zip> OTA/android-info.txt:android-info.txt IMAGES/*:.
Except it decompresses and recompresses the images, which takes over a
minute instead of a few seconds.
Change-Id: I88d0df188635088783223873f78e193272dbdf1c
For example , instead of trying to have libraryLinker inherit from
baseLinker and libraryCompiler inherit from baseCompiler, create a
single decorator object that wraps both baseLinker and baseCompiler.
Test: Builds, no unexpected changes to build.ninja
Change-Id: I2468adaea8466c203a240259ba5694b8b1df7a52
Split cc.go into files per module type, plus files for the compiler,
linker, and installer stages.
Change-Id: Id44c03f42fcd180950ccd008d4de0c144ea3597b
The list of migrated libraries is currently empty. Libraries will be
migrated as follow up patches.
Test: Migrated libc to this system and everything still builds.
build.ninja shows libraries being built and used and headers are
collected for the sysroot.
Bug: http://b/27533932
Change-Id: Iaba00543c1390f432befe0eed768ed3fbb8a9b96
And install the tools into a more obvious location. soong_env is not
moved, since we need it to exist early, so that we can use it in
soong.bash in case there's a build failure.
Change-Id: I9bd1fa320d84d180b2cf3deb90782d380666f7a6
Improve the androidmk makefile parser based on ideas from go/ast and
friends:
- Use type switching instead of the As* mess
- Don't store endPos for every node, compute it based on the last
known position in the node plus the length of the last token
- Store positions as only the offset into the file, and then unpack
them into Line/Column scanner.Position objects later
Change-Id: I87eb6661859951e6c2ea5a85db6229fa5561d615
Some checks for common errors with user-provided compiler and linker
flags:
* Using -I instead of include_dirs
* Using -l<lib> in ldflags instead of host_ldlibs (or shared_libs)
* Using -L in ldflags
* Splitting a multi-word flag into two flags
* Combining two flags into one list entry
* Using a path that could search outside the source or output trees
* Using a non-whitelisted library in host_ldlibs
Maybe some of the flag checks should happen during a static analysis
pass, but we don't have one right now, and this only adds ~1/2 second to
our 73 second Mega_device runs (recompile the changed code, run
soong_build, then report unknown target).
Change-Id: Icea7436260f1caa62c0cec29817a1cfea27b3e7c
To allow other singletons to use new environment variables, move env.go
to the end of the source list, so that its singleton is run last.
Change-Id: Ic92308f72fc0c92a0fedbc5546e337940e32eaab
For configuration like TARGET_GLOBAL_CFLAGS, it would be good to have
Make and Soong use the same values. This change adds an interface for
packages like cc to verify the current value of make variables match the
Soong equivalents, and once they're satisfied with the results, remove
the make definition and use the value from Soong.
The cc implementation exports a few variables currently that I expect to
match between both implementations in all cases. It also checks
{CLANG,}{HOST,TARGET}_GLOBAL_{C,LD}FLAGS, which should be mostly similar
between make and soong now.
Bug: 23566674
Change-Id: Idc8582ef31ace11a8baefcf525c3683f08a573aa
Add a new feature to cc modules that adds the cflags and libraries
necessary to implement AddressSanitizer, ThreadSanitizer, and
UndefinedBehaviorSanitizer.
Change-Id: Ibe5ffadc7ece56080a2521f2c7c00da9ef712584
Refactor all of cc in order to use composition instead of inheritance.
All cc module types exported by cc are now *cc.Module objects, with
compilation, linking, and installing steps delegated to different
objects in order to form the full module type. Additional features that
modify dependencies and flags can be inserted in a features object list,
and custom module types can be created by adding a Customizer object
that can modify properties.
Change-Id: Ie1283d14920f7856f6947b0530606b2f4d58fab0
This centralizes verification and common operations, like converting the
path to a source file to the path for a built object.
It also embeds the configuration knowledge into the path, so that we can
remove "${SrcDir}/path" from the ninja file. When SrcDir is '.', that
leads to paths like './path' instead of just 'path' like make is doing,
causing differences in compiled binaries.
Change-Id: Ib4e8910a6e867ce1b7b420d927c04f1142a7589e
This defines another mutator between HostOrDevice and Arch that will
expand host modules into a module for each host type
(Darwin/Linux/Windows) that is currently being built.
Change-Id: I4c8ac6b616c229f6bd45ad8a35902652fb6a4fff
androidbp is gone, replaced with combining kati ninja output for
makefiles with blueprint ninja output for soong.
Change-Id: I00b9e28877abf2ceb2223d3ccf0637bc1b7571bb
Add a cc_defaults module, which will prepend any matching properties to
modules that reference it with a "defaults" property.
Change-Id: I5908dd98d204f71b29ad95a4ab85403aa1621ca2
EarlyMutators are identical to BottomUpMutators, except they run before
DynamicDependencies. DynamicDependencies can be replaced with a
BottomUpMutator. Replace both EarlyMutators and DynamicDependencies
with BottomUpMutators, which allows setting the order between all
mutators through registration order.
Change-Id: Id1305d798d3d2da592061c89d7c10a71780b71a3
Blueprint has a generic AppendProperties/AppendMatchingProperties now,
use it, and replace all bool properties that might be modified by a
mutator with *bool, which provides the correct replace-if-set semantics
for append.
Also remove uses of ContainsProperty except when explicitly checking if
a property was set in a blueprints file.
Change-Id: If523af61d6b4630e79504d7fc2840f36e98571cc
Instead of using this hardcoded list, use the blueprint plugin
infrastructure to find them at build time and load them.
Bug: 23567252
Change-Id: Iae87e8c9bf020f4beb42c418b9a6bf8b0fe8b601
Allow modules to vary their properties based on product variables.
For now, DEVICE_USES_LOGD, DEVICE_USES_JEMALLOC, and DEVICE_USES_DLMALLOC,
and BOARD_MALLOC_ALIGNMENT are supported.
Product variables can provide a value (only bool and int supported for
now), and if any of the product variable properties contains a "%d"
then Sprintf will be called with the property value as the format
and the product variable value convert to an int as the only argument.
For example:
product_variables: {
dlmalloc_alignment: {
cflags: ["-DMALLOC_ALIGNMENT=%d"],
},
},
will cause -DMALLOC_ALIGNMENT=16 to be added to any top level
properties called "cflags".
Change-Id: I74882a6ab4914d3e222f8d06cfac371b7b829ae5
Fabien Sanglard