Commit Graph

150 Commits

Author SHA1 Message Date
Colin Cross 2296f5b3c4 Use indent to preformat genrule property documentation
Test: manual inspection of soong_build.html
Change-Id: Ibb5086de5497eb096ee7f68a5fd1ccaba630aa89
2017-10-18 05:44:54 +00:00
Colin Cross 54190b3b13 Add test for java generated sources
Bug: 67364649
Test: TestGeneratedSources
Change-Id: I5400e4b013dc47c04b9d32787e50c5281033484d
2017-10-09 15:36:06 -07:00
Treehugger Robot 944d6b4c01 Merge "Allow exporting filegroups to make" 2017-09-19 20:17:50 +00:00
Dan Willemsen 3e5bdf29ba Add cc_genrule
cc_genrule is the same as a normal genrule, but can depend on other cc
modules (like cc_object).

Test: mmma external/minijail
Change-Id: I8df87665c7bdc76ce89c92755c054f967a818e57
2017-09-18 23:33:25 -07:00
Colin Cross d91d7aca7a Allow exporting filegroups to make
Allow sharing lists of files between make and soong by allowing
filegroups to export them as a make variable.

Test: m -j checkbuild
Change-Id: I27ae2f6d180bf01d69a628dbe59edcdba93da015
2017-09-18 13:54:56 -07:00
Dan Willemsen 8eded0ac86 Support using blueprint_go_binary as tools in genrules
Test: m
Test: Add genrule using go tool; m
Change-Id: I9a85348b6cf41f2cdb7684f787553c07de220d67
2017-09-14 22:59:41 +00:00
Dan Willemsen d6ba0d592c Use dependency tags for genrules
So that we don't get confused when using :<module> in srcs to depend on
a module that could also be a HostBinTool.

Test: m -j
Change-Id: Ia3b1c26826e70f84c6dc5ff78c95dd11d76901b6
2017-09-14 22:59:33 +00:00
Colin Cross 36242850fd Refactor factories
Change module factories from returning a blueprint.Module and a list
of property structs to returning an android.Module, which holds the
list of property structs.

Test: build.ninja identical except for Factory: comment lines
Change-Id: Ica1d823f009db812c518f271a386fbff39c9766f
2017-06-30 21:08:36 +00:00
Jeff Gaston 193f2fb092 Make absolute OUT_DIR work with sbox
Test: make OUT_DIR=/tmp/abspath
Test: make OUT_DIR=relative-out
Bug: 35562758

Change-Id: I688d5f6117b194440c5d01c1040033b5671187c2
2017-06-12 16:36:17 -07:00
Jeff Gaston efc1b412f1 Have Soong try to enforce that genrules declare all their outputs.
This causes Soong to put the outputs of each genrule into a temporary
location and copy the declared outputs back to the output directory.
This gets the process closer to having an actual sandbox.

Bug: 35562758
Test: make

Change-Id: I8048fbf1a3899a86fb99d71b60669b6633b07b3e
2017-06-09 17:57:18 +00:00
Colin Cross 67a5c132c5 Prettify soong ninja build descriptions
Descriptions currently look like:
[  0% 4/29328] cc out-soong/.intermediates/external/clang/lib/Sema/libclangSema/android_arm_armv7-a-neon_denver_static_core/obj/external/clang/lib/Sema/SemaCodeComplete.o

This is not very helpful - most of the characters are used to show the
output path, which contains useful information like target architecture,
but also contains most of the path to the source files twice, and less
useful information like the exact variant name used by soong.
Make the descriptions look like:
[  0% 3/29329] //external/clang/lib/Sema:libclangSema clang++ SemaTemplateInstantiate.cpp

This is //path/to/module:modulename tool relative/path/to/source/file

Test: builds, looks pretty
Change-Id: I3087aa7d4eb1860ef6239d77407b8b35445616d7
2017-05-10 11:04:16 -07:00
Dan Willemsen 014de6a033 Fix genrule documentation
Change-Id: Ieffb2bd7d739527db129ff300d8aec70d82c278a
2017-05-09 23:09:17 +00:00
Colin Cross 54c5dd5238 Exit early on genrule command parsing failure
cmd may be empty if android.Expand fails, and continuing results
in a panic.  Return immediately.

Test: m -j checkbuild
Change-Id: Ibf4d57ce048db9f7f16ce118ed9e9ecea5a6551c
2017-04-19 14:27:53 -07:00
Colin Cross 795c377e14 Use a minimal set of mutators, module types, and singletons for tests
Calling android.NewContext() in tests results in a context that
contains all the mutators, module types, and singletons, which
causes unexpected interactions in unit tests.  Create an empty
context instead, and add in only the necessary mutators, module
types, and singletons.

Bug: 36366816
Test: soong tests
Change-Id: Ic61262c37e3436b3ad4ccaca18b737021c304be6
2017-03-16 16:50:10 -07:00
Colin Cross 77de5a4b9d Merge "Support data properties for test binaries" 2017-02-06 21:17:12 +00:00
Colin Cross d1b8d760b5 Allow non-tool dependencies in genrules
The genrule tool dependency handling was rejecting any dependencies
that were not tools.  This caused a failure when ExtractSourceDeps
added a source file dependency on a filegroup module.  Remove the
unnecessary check in the tool handling.

Test: builds
Bug: 35002681
Change-Id: Ibd8dff306be3ef55aac2e12a28ddc187be2234d9
2017-02-04 12:45:03 -08:00
Colin Cross faeb7aa135 Support data properties for test binaries
Allow tests to specify a data property that lists files or filegroup
modules that will be packaged alongside the test.  Also add a path
property to filegroup modules to allow shifting the path of the
packaged files, and add ExpandSourcesSubDir to expand the filegroup
sources while including a shifted relative path in the Paths objects.

Test: soong tests, manually adding data to a module
Change-Id: I52a48942660e12755d313ef13279313361b4fc35
2017-02-02 16:23:30 -08:00
Colin Cross 708c424c57 Allow genrules to reference generated sources
Call ExtractSourceDeps on genrule srcs properties.

Bug: 34283327
Test: no change to build.ninja
Change-Id: I934a418f32d4c94400728579543ca5c910bf7dfc
2017-01-13 18:06:58 -08:00
Colin Cross 068e0feace Support filegroups
filegroup is a module that contains a list of files, and can be used
to export files across package boundaries.  filegroups (and genrules)
can be referenced from srcs properties of other modules using the
syntax ":module".

Test: m -j
Change-Id: I3d6fc4819c0b4225b474e0ad42f0d947f55a5961
2016-12-17 16:34:03 +00:00
Colin Cross 5ed99c6472 genrule: let Android.bp file specify exported header dirs
Instead of exporting the generated sources dir as headers, let
the Android.bp file specify subdirectories as exported.

Test: m -j checkbuild
Change-Id: I18dd900d63ce7485c8fbfcc39dc77abad6f733d7
2016-11-22 15:41:09 -08:00
Colin Cross 33bfb0a36a genrule: support deps files
If a genrule sets deps_file: true, use a file in the gen directory
as a GCC-style deps file.

Test: m -j libLLVMObject
Change-Id: Id410165847e4eaea1853a392512e38787c431523
2016-11-22 15:41:09 -08:00
Colin Cross 6f080dff81 Support genrules with multiple tools
To allow genrules with more than one tool, rename the tool property to
tools and make it an array, replace $tool with $(location <label>),
and use $() for other variables for consistency.

Also remove the host bin directory from the genrule path, and the
$srcDir variable, using either of them would have caused dependency
issues.

Bug: 31948427
Test: compare build.ninja
Change-Id: Icf6d3bce2bea00fec1363fd65c0bdf96d09281bf
(cherry picked from commit de6bd86d24)
2016-11-07 10:53:49 -08:00
Dan Willemsen 21ec49068f Add subdir to GenPath
We were emulating this for proto files, standardize it and make the
other generators use it as well.

Test: Compare out/soong/build.ninja before/after change
Test: mmma -j system/tools/hidl
Change-Id: I1888c7b981749060a398387bbb9b481270bf6d75
2016-11-02 21:19:11 -07:00
Colin Cross 1e676bef94 Control mutator order
Register mutators inside lambdas that are called in a defined order to
correctly order mutators before and after the arch and deps mutators.

Test: build.ninja identical
Change-Id: Iefe2a3515aee8570e76a6e76925db4cda0e9e822
2016-10-12 15:33:33 -07:00
Colin Cross 798bfce9d0 Move registration into android package
Mutator registration is tightly coupled with the android package, move
all registration from the soong package to the android package.

Test: build.ninja identical
Change-Id: Ie183d0b52cc7431c9e05b231934d189208ef1efe
2016-10-12 14:30:44 -07:00
Dan Willemsen 9c8681f4f9 Support multiple outputs for genrule
The underlying code to expose outputs to other modules has suppported
this already for gensrcs (which runs the command multiple times to
create multiple outputs). This allows a genrule to run a command once,
and output multiple files.

Bug: 31742855
Test: Add multi-output genrule; Inspect build.ninja
Change-Id: I72054cc39c0d0e3e44feebba72005e5af1a1709a
2016-09-28 18:31:04 -07:00
Dan Willemsen 3f4539b035 Parse genrule's cmd property
Instead of just passing this to ninja, use os.Expand to parse the
variable substitutions. While we're here, define a new "genDir" variable
that is the root of this module's generated directory.

Bug: 31742855
Test: Ensure invalid variables cause error
Test: Inspect commands in build.ninja
Change-Id: Iff6d0499ab57367669e73df52ab7c647358da13b
2016-09-28 17:38:57 -07:00
Colin Cross e8a67a7c58 Make BottomUpMutators parallel
Append .Parallel() to all of the RegisterBottomUpMutator calls to tell
Blueprint it can run them in parallel.

Test: identical build.ninja, passes race detector
Change-Id: I969a0689522d4cba7c8ff51e2aa00fe2fd338a89
2016-08-11 13:20:16 -07:00
Colin Cross a1ad8d1889 Simplify arch target handling
Soong's multi-architecture building has grown complex, with the
combination of HostOrDevice+HostType+Arch necessary to determine how to
build a variant of a module, and three separate mutators to split each
into its variations.

Combine HostOrDevice+HostType into Os, which will be Linux, Darwin,
Windows, or Android.  Store Os+Arch as a single Target.

Change-Id: I92f2e2dac53617d595a35cc285d2bd348baa0fbd
2016-06-02 19:09:32 -07:00
Colin Cross b9db480385 Revert "Simplify arch target handling"
This reverts commit 6713fb26cbcadf525cd75e47d7d0cbc23d282b3e.

Change-Id: Ic473cea2563b0b37dc08b0bc5d3a0ac8c4b6afe6
2016-06-03 01:50:47 +00:00
Colin Cross 54c7112c43 Simplify arch target handling
Soong's multi-architecture building has grown complex, with the
combination of HostOrDevice+HostType+Arch necessary to determine how to
build a variant of a module, and three separate mutators to split each
into its variations.

Combine HostOrDevice+HostType into Os, which will be Linux, Darwin,
Windows, or Android.  Store Os+Arch as a single Target.

Change-Id: Iae677eff61a851b65a7192a47f2dc17c1abb4160
2016-06-02 17:37:02 -07:00
Colin Cross 635c3b0157 Rename common to android
Rename the "common" package to "android", because common is too
generic.  Also removes all android.Android naming stutter.

Ran:
gomvpkg -from 'android/soong/common' -to 'android/soong/android'
gorename -from '"android/soong/android".AndroidModuleContext' -to 'ModuleContext'
gorename -from '"android/soong/android".AndroidBaseContext' -to 'BaseContext'
gorename -from '"android/soong/android".AndroidModuleBase' -to 'ModuleBase'
gorename -from '"android/soong/android".AndroidBottomUpMutatorContext' -to 'BottomUpMutatorContext'
gorename -from '"android/soong/android".AndroidTopDownMutatorContext' -to 'TopDownMutatorContext'
gorename -from '"android/soong/android".AndroidModule' -to 'Module'

Change-Id: I3b23590b8ce7c8a1ea1139411d84a53163288da7
2016-05-18 15:37:25 -07:00
Dan Willemsen f7f3d69cf5 Add genrule tool_file to better handle deps
This will allow modules to specify a local script file that will be
expanded to the proper path in $tool, and will automatically be added to
the dependencies.

Change-Id: Icea4046b1fe2fd7c543aa33913d1a53201b15353
2016-04-21 13:20:01 -07:00
Dan Willemsen b40aab6955 Fix generated sources and headers
Add two cc properties: `generated_sources` and `generated_headers`,
instead of just adding genrule modules to `deps`. Label these with the
dep tagging mechanism, add the generated_headers paths to the include
paths, and add dependencies to generated headers for each compile.

Add dependencies so that regular sources can include generated yacc
headers, not just other generated lex/yacc files. Static/shared specific
sources still don't have dependencies to or from regular sources though.

Switch from an implicit dependency on generated files to an orderonly
dependency, since the depfile will take care of the necessary implicit
dependencies for incremental builds.

Change-Id: I436675acb1f57329d98b81c56dcb4384201a68ea
2016-04-20 15:54:19 -07:00
Colin Cross c99deeb961 Use blueprint DependencyTags
Blueprint now requres DependencyTags to be passed to AddDependency
calls.  Use the tags to avoid setting member variables on modules, which
will be lost after the mutators have been called.

Change-Id: I8c1d9ed1db85a300e14394b911a516d361ba9f75
2016-04-12 15:24:02 -07:00
Colin Cross 1604ecf5cc Fix gofmt
Change-Id: Ib5e4db599a6dd858b3a1b1cdb75017b01eac4911
2015-12-17 17:25:10 -08:00
Dan Willemsen 34cc69e4bf Use `Path` instead of string for file paths
This centralizes verification and common operations, like converting the
path to a source file to the path for a built object.

It also embeds the configuration knowledge into the path, so that we can
remove "${SrcDir}/path" from the ninja file. When SrcDir is '.', that
leads to paths like './path' instead of just 'path' like make is doing,
causing differences in compiled binaries.

Change-Id: Ib4e8910a6e867ce1b7b420d927c04f1142a7589e
2015-12-09 14:29:12 -08:00
Dan Willemsen 490fd49557 Support cross-compiling Windows binaries on Linux
This defines another mutator between HostOrDevice and Arch that will
expand host modules into a module for each host type
(Darwin/Linux/Windows) that is currently being built.

Change-Id: I4c8ac6b616c229f6bd45ad8a35902652fb6a4fff
2015-11-30 15:28:31 -08:00
Colin Cross 6362e27848 Remove EarlyMutators and DynamicDependencies
EarlyMutators are identical to BottomUpMutators, except they run before
DynamicDependencies.  DynamicDependencies can be replaced with a
BottomUpMutator.  Replace both EarlyMutators and DynamicDependencies
with BottomUpMutators, which allows setting the order between all
mutators through registration order.

Change-Id: Id1305d798d3d2da592061c89d7c10a71780b71a3
2015-11-03 15:46:08 -08:00
Colin Cross 463a90e587 use init functions to register module types, etc.
Instead of putting all the blueprint registrations in soong_build,
put them all in init() functions.  This puts the registration next
to the implementation.

Change-Id: Ide1a749518f5e9d1367a18ab3bb1d91da3310c76
2015-07-09 17:57:18 -07:00
Dan Willemsen 2ef08f4458 Add exclude_* and remove arch_subtract / "-file"
To align with the current make build system, add exclude_srcs and
exclude_java_resource_dirs. These replace the functionality of
arch_subtract and glob exclusions that use "-file" to exclude a file.

Change-Id: I91c23d5e3c9409f2d9f7921f950153a03a68ad61
2015-07-01 20:59:39 +00:00
Colin Cross 7d5136f033 Improve property comments for docs
Improve the comments associated with properties to work better with
Blueprint's auto-documenting feature.

Make all properties structs into named types so that thet types can be
found using reflection and cross-referenced to the source code to
auto-extract docs.

Remove the leading <property>: text from properties, the documentation
will include the lowercased name of the property.

Add filter tags to the nested arch properties.

Change-Id: I4ef5db86358886fe61456c24eb2dbe6f7b876115
2015-06-29 10:36:02 -07:00
Colin Cross 8f101b45fc Allow common.Glob to be called by singletons
Make common.Glob take an interface that is the intersection of
blueprint.ModuleContext and blueprint.SingletonContext used by
Glob and GlobRule, allowing it to be called on either.  Also
move ExpandSources and Glob into AndroidModuleContext.

Change-Id: I8d1a3160c5dd201033afdb37210e82d8065b137d
2015-06-17 15:36:48 -07:00
Colin Cross f9964d5a8d Use far dependency for genrule tool
A far dependency will not inherit any of the variations from the current
module, allowing a device genrule to depend on a host tool to generate
the sources.

Change-Id: I7f622ddd31a81d605f55d5946a109ea53e864084
2015-05-12 20:46:49 +00:00
Colin Cross d350ecd102 Add support for genrule
Add genrule, which uses a host executable (possibly built by the
build system) to generate a single source file.  This differs slightly
from gensrcs, which takes a list of sources and translates them through
a host executable to individual source files.

Change-Id: I94bda62c4c53fb3f3817def190e6a7561508d297
2015-04-29 14:58:16 -07:00
Colin Cross fce532760f Support subtracting sources from globs
Support -file or -path/glob in file lists that contain globs to
subtract files from the resulting glob.  Also move source file
prefixing and handling into a common function.

Change-Id: Ib6d74ce22f53cae7348c4ba35b779976d90359a6
2015-04-08 15:19:30 -07:00
Colin Cross 1332b0035c Move config into common and provide helper
Using ctx.Config().(Config) everywhere is a mouthful, and it is
inefficient to do the type assertion.  Put the Config interface into
the context, and provide an AConfig() to return the Config already
converted to the right type.

Change-Id: I301a1fd7d2a005580aabca7866a37c5d42ad8c69
2015-04-08 15:19:24 -07:00
Colin Cross 97ba073833 Export cc types for art to inherit from
Art needs a custom module type in order to perform complicated
build logic like depending on environment variables and varying
cflags based on cpu variant.  Export enough of the types and
functions from cc for art_cc_library to inherit from cc_library.

While I'm touching every line, also rename the New* methods
to *Factory.

Change-Id: I7123aa47019c4ced7a1ab57c394225bc7844b5ea
2015-03-25 11:33:06 -07:00
Colin Cross 70b4059e3f Update import paths for changes to blueprint
Blueprint has been modified to include a canonical import path,
update soong to match.

Change-Id: If29d31afbf3bf2e6364961a66456fa5f8f738455
2015-03-23 12:57:34 -07:00
Colin Cross 5049f02e60 Add gensrcs module type
gensrcs allows sources to be generated by a specified command.

Change-Id: I725086fcdcd72bfe6c07fb8903e7b520678a247f
2015-03-19 10:51:41 -07:00