Commit Graph

77 Commits

Author SHA1 Message Date
Colin Cross ef35448ac5 Allow cc_genrule srcs to vary by architecture
Add arch_variant to the genrule srcs property so that cc_genrule
modules can vary the srcs and outs by architecture.

Test: cc/genrule_test.go
Change-Id: Idb17013e137d7ca21ca4cfc3e4c1fa7d89977043
2018-10-23 18:40:37 +00:00
Colin Cross 08f15ab466 genrule: add $(location) for inputs and outputs
There was no way to select a single source file from a genrule that
has multiple source files.  Make Soong's genrule closer to Bazel
by supporting $(location) for input and outputs.  Change the
label used for tools referenced through filegroups to the name
of the module instead of the name of the file, which means it
matches the string used in the tools property.

Also support :module format in the tools property in preparation
for deprecating the tool_files property and putting both files
and modules in the tools property.

Bug: 117354232
Test: genrule_test.go
Change-Id: Id31a4ac4ff7064076a576c1d8ffeb7c19fc6b9a4
Merged-In: Id31a4ac4ff7064076a576c1d8ffeb7c19fc6b9a4
(cherry picked from commit 098d22b5f6)
2018-10-08 16:05:48 -07:00
Colin Cross 2a07692643 Add tests for genrule command expansion
Add tests for expanding variables in a genrule cmd property.

Test: genrule_test.go
Change-Id: I8288b8616d518bb5f24a892c4e59f68d95055d0a
2018-10-05 21:29:14 +00:00
Brandon Lee 5d45c6f6f8 Collect modules' info to create IDE project file.
- Register a singleton and implement GenerateBuildActions func in java/jdeps.go.
- Declare a interface and a struct to collect info in android/module.go.
- Implement IDEInfo for Library & Import module in java/jdeps.go.
- Implement IDEInfo for Genrule module in genrule/genrule.go.
- Implement IDEInfo for fileGroup module in android/filegroup.go.
- Test codes for jdeps.go in java/jdeps_test.go.

Bug: 111044346

Test: export SOONG_COLLECT_JAVA_DEPS=1;mmm packages/apps/Settings
      out/soong/module_bp_java_deps.json will be generated

Change-Id: If61da77b4d7614c2c5da438b6af4c725ceccc5c3
2018-09-18 17:44:10 +00:00
Dan Willemsen 59339a29e1 Fix `go vet` issues
Test: go vet ./...
Change-Id: Ifb936ccc5e2b5a2c3fcbbbcb54f680e2973ea1b3
2018-07-22 21:18:45 -07:00
Colin Cross 85a2e89a14 Fix error reporting when parsing genrule cmd
Report errors inside the android.Expand lambda and don't return an
error so that processing can continue and multiple errors can be
reported.  Check for errors (for example incorrect syntax of variables)
immediately after Expand returns.  Avoids a misleading error message:
specified depfile=true but did not include a reference to '${depfile}' in cmd

Bug: 111219250
Test: m checkbuild
Change-Id: Id9a16c1609f5fd9345bfa1a2191261cff72fd382
2018-07-09 19:01:36 +00:00
Colin Cross 2d24c1b654 Remove references to SourceDepTag in module types
SourceDepTag is going to become a set of tags of the same type
instead of a single tag, remove references to it outside the
android module.

Bug: 80144045
Test: soong tests
Change-Id: I00b2ea5040e4fc95dfbfdd79e21579853c478fcb
2018-06-06 17:05:20 +00:00
Pirama Arumuga Nainar 955dc4999e Fix few issues with filegroups
Bug: http://b/64121881
Bug: http://b/78188880

- Allow filegroup's properties to be extended by a LoadHook
- Support a filegroup (':module') in a prebuilt's 'Srcs' property to
export files from a different path as the prebuilt's sources.

This change also includes a refactoring that moves genrule/filegroup.go
to android/filegroup.go so that FileGroupFactory is visible in
prebuilt_test.go.

Test: Test
https://android-review.googlesource.com/c/platform/development/+/469159
in clang-tools branch on Linux, Darwin.  Test regular build in
aosp/master.

Change-Id: I3ff6215ab2e62955f039fd1086c31f1bd50ebcf6
2018-04-18 18:56:33 +00:00
Nan Zhang e42777a05a Fix the flaky build breakage for droiddoc target.
ExpandSourcesSubDir was calling SourceFileProducer.Srcs(), and then
doing:
moduleSrcs = append(moduleSrcs[:j], moduleSrcs[j+1:]...)
This modifies the backing store of the slice, which may affect the
original data stored in the SourceFileProducer.  Make all Srcs
implementations return slice that points to a copy of the backing
array.

Test: m out/target/common/obj/PACKAGING/checkpublicapi-current-timestamp
Bug: b/76179848 b/76397326
Change-Id: I2432ce196984814daafc5aa9a2746e81de74494c
2018-03-28 05:12:15 +00:00
Dan Willemsen 9da9d49ede Only depend on a single file for generated headers
While the rule may really need all of the generated header files to
exist, only one of them (per genrule task) needs to be in the dependency
list, since the rest are essentially aliases.

This brings an AOSP aosp_arm-userdebug out/soong/build.ninja file from
372MB to 156MB, with equivalent functionality. The Android-aosp_arm.mk
file is reduced from 11MB to 6.5MB.

Bug: 73745773
Test: diff out/soong/build.ninja
Test: diff out/soong/Android-aosp_arm.mk
Test: rm -rf out; m
Change-Id: If17377666292cc20957417fc4c3cd52f98971d0c
2018-02-22 02:37:01 +00:00
Colin Cross baccf5b984 Use __SBOX_OUT_DIR__ in sbox output file list
The path to the output directory may be arbitrarily long, use
__SBOX_OUT_DIR__ in the list of output files passed to sbox
to avoid expanding it multiple times in the command line.

Fixes:
ninja: fatal: posix_spawn: Argument list too long
09:40:14 ninja failed with: exit status 1
when building libchrome with a long OUT or OUT_DIR_COMMON_BASE.

Bug: 73726635
Test: m checkbuild
Change-Id: I59024b2164287c8e531711afd9273b692ce9c28a
2018-02-21 14:55:34 -08:00
Nan Zhang 27e284d2b0 Revert "Revert "Support filegroup in exclude_srcs""
This reverts commit 606e9de344.

Reason for revert: <try to fix the broken build yesterday>

Change-Id: I2963b9af63c7c7398159e5e9a1e448266e1c81d5
Test: unittest
2018-02-15 14:38:40 -08:00
Tobias Thierer 606e9de344 Revert "Support filegroup in exclude_srcs"
This reverts commit f36a3d9b6d.

Reason for revert: Broke several builds. I'm acting build cop, reverting.

Bug: 70351683

Change-Id: I775ada4e9cb6473519d51420b41b818af163da44
2018-02-09 09:46:11 +00:00
Nan Zhang f36a3d9b6d Support filegroup in exclude_srcs
Test: add unit-test, m -j32
Bug: b/70351683
Change-Id: Iff83c56d45dd668d9df6131c7df2e23e5c73a21b
2018-02-07 10:18:41 -08:00
Jeff Gaston 31603067b0 Escape genrule commands
Bug: 70387174
Test: Put this text into an Android.bp:

    genrule {
        name: "test_genrule",
        tool_files: ["foo"],
        out: ["foo.c"],
        cmd: "for i in a b c; do echo $$i; done > $(out)",
    }

    cc_library {
        name: "libtest_genrule",
        srcs: [":test_genrule"],
    }

  and then run `m -j libtest_genrule`. Although the library
  shouldn't compile, check that it produces a foo.c that has
  "a\nb\n\c\n" and not "\n\n\n".

Change-Id: I139106479439c9b3a95f1a2ecc23e73570d7bd59
2017-12-11 23:55:03 +00:00
Colin Cross c331599a88 Allow java binary wrapper files to reference filegroups
Also allow commands with no tools for the case when standard
shell utilities are used to munge an input file.

Test: m checkbuild
Change-Id: Ie061e90cafe1a0a0db004a89e9a17fb48709cb03
2017-12-11 23:54:35 +00:00
Colin Cross ac87c992be Allow globs in tool_files
tool_files can be used to add dependencies on files used by tools,
so let it support globs.

Test: m checkbuild
Change-Id: I1c1cb098190e1bb1c81292f6eb0c4ed0e240c1e1
2017-12-08 18:23:57 -08:00
Colin Cross 6510f91a1c Replace ModuleContext.AConfig() with Config()
AConfig() now duplicates Config().  Replace the uses of AConfig()
with Config().  Leave AConfig() for now until code in other
projects is cleaned up.

Test: m checkbuild
Change-Id: Ic88be643049d21dba45dbd1a65588ed94bf43bdc
2017-11-30 00:44:18 +00:00
Colin Cross 35143d0466 Fix genrules depending on Go tools
genrules lost the ability to depend on Go tools after
I05e945f38915d49cd3c0ab72a86576949bc7eff2 which converted
VisitDirectDeps from blueprint Modules to android Modules.
Add VisitDirectDepsBlueprint to visit all modules including
blueprint Modules, and use it in genrule.  Also add a check
for disabled modules that was being handled by
VisitDirectDeps.

Test: m checkbuild
Change-Id: I65724283166c63596d071e598c08fed87ef32896
2017-11-17 11:22:08 -08:00
Nan Zhang a5e7cb4eab Change remaining properties to *string, *bool in Soong.
Test: m -j checkbuild
Bug: b/68853585
Change-Id: I0fd10ff31e90c1941e80cfbf25e40e9988f1e202
2017-11-09 22:42:32 -08:00
Nan Zhang ea568a4a24 Change bool, and string properties to *bool, and *string for java,
python, and genrule.

Test: m -j checkbuild
Bug: b/68853585
Change-Id: Ic9a8083818e920dc399a4b00841e2aa496f70faa
2017-11-08 21:25:40 -08:00
Jeff Gaston 437d23c3ad Consolidate gensrcs implementation into one task each
in preparation for each task to delete its output dir
before running.

Bug: 38205169
Test: rm out -rf && m -j

Change-Id: Ifcd42073d4fb74f532adc0c095555b4a542feb99
2017-11-08 12:38:00 -08:00
Jeff Gaston 5acec2b0e0 Have genrule use $genDir as sbox's --output-root
rather than passing buildDir .
This will make it reasonable for sbox to delete its --output-root

Bug: 38205169
Test: m -j
Change-Id: Ic3a82a8df040117870c251c068a88545143257cf
2017-11-08 12:37:14 -08:00
Jeff Gaston 02a684b8b2 Make $(depfile) work with sbox
Most notably, the sandbox depfile path should be passed into the tool.

Bug: 68336760
Test: m -j checkbuild

Change-Id: I22f944a3f57d613fda26de0ea777a915cafcd020
2017-10-30 17:49:29 -07:00
Colin Cross d11fcda940 Convert Visit*Deps from blueprint.Module to android.Module
Also adds checks that the dependencies are android.Modules and
are not disabled.

Test: m checkbuild
Change-Id: I05e945f38915d49cd3c0ab72a86576949bc7eff2
2017-10-24 13:01:03 -07:00
Colin Cross ae88703df5 Move ModuleContext.ModuleBuild to ModuleContext.Build
Now that android.ModuleContext does not include blueprint.ModuleContext
we can rename android.ModuleContext.ModuleBuild to
android.ModuleContext.Build without colliding with
blueprint.ModuleContext.Build.  Leave ModuleBuild as a wrapper around
Build for now to avoid having to update all the users outside
build/soong simultaneously.

Test: m checkbuild
Change-Id: I18eb8cc04faf002049a11d9aac97e9732ff5d638
2017-10-24 10:59:00 -07:00
Colin Cross 15e86d938b Support multiple outputs in genrules with depfile: true
Ninja doesn't support depfiles on a rule with multiple outputs.
Use a single output and put all the rest as implicit outputs.

Bug: 68057449
Test: java_test.go
Change-Id: Ia544493b1b3b51b185c865149d8f3e0eb3c57ee2
2017-10-20 20:44:50 -07:00
Colin Cross 2296f5b3c4 Use indent to preformat genrule property documentation
Test: manual inspection of soong_build.html
Change-Id: Ibb5086de5497eb096ee7f68a5fd1ccaba630aa89
2017-10-18 05:44:54 +00:00
Colin Cross 54190b3b13 Add test for java generated sources
Bug: 67364649
Test: TestGeneratedSources
Change-Id: I5400e4b013dc47c04b9d32787e50c5281033484d
2017-10-09 15:36:06 -07:00
Treehugger Robot 944d6b4c01 Merge "Allow exporting filegroups to make" 2017-09-19 20:17:50 +00:00
Dan Willemsen 3e5bdf29ba Add cc_genrule
cc_genrule is the same as a normal genrule, but can depend on other cc
modules (like cc_object).

Test: mmma external/minijail
Change-Id: I8df87665c7bdc76ce89c92755c054f967a818e57
2017-09-18 23:33:25 -07:00
Colin Cross d91d7aca7a Allow exporting filegroups to make
Allow sharing lists of files between make and soong by allowing
filegroups to export them as a make variable.

Test: m -j checkbuild
Change-Id: I27ae2f6d180bf01d69a628dbe59edcdba93da015
2017-09-18 13:54:56 -07:00
Dan Willemsen 8eded0ac86 Support using blueprint_go_binary as tools in genrules
Test: m
Test: Add genrule using go tool; m
Change-Id: I9a85348b6cf41f2cdb7684f787553c07de220d67
2017-09-14 22:59:41 +00:00
Dan Willemsen d6ba0d592c Use dependency tags for genrules
So that we don't get confused when using :<module> in srcs to depend on
a module that could also be a HostBinTool.

Test: m -j
Change-Id: Ia3b1c26826e70f84c6dc5ff78c95dd11d76901b6
2017-09-14 22:59:33 +00:00
Colin Cross 36242850fd Refactor factories
Change module factories from returning a blueprint.Module and a list
of property structs to returning an android.Module, which holds the
list of property structs.

Test: build.ninja identical except for Factory: comment lines
Change-Id: Ica1d823f009db812c518f271a386fbff39c9766f
2017-06-30 21:08:36 +00:00
Jeff Gaston 193f2fb092 Make absolute OUT_DIR work with sbox
Test: make OUT_DIR=/tmp/abspath
Test: make OUT_DIR=relative-out
Bug: 35562758

Change-Id: I688d5f6117b194440c5d01c1040033b5671187c2
2017-06-12 16:36:17 -07:00
Jeff Gaston efc1b412f1 Have Soong try to enforce that genrules declare all their outputs.
This causes Soong to put the outputs of each genrule into a temporary
location and copy the declared outputs back to the output directory.
This gets the process closer to having an actual sandbox.

Bug: 35562758
Test: make

Change-Id: I8048fbf1a3899a86fb99d71b60669b6633b07b3e
2017-06-09 17:57:18 +00:00
Colin Cross 67a5c132c5 Prettify soong ninja build descriptions
Descriptions currently look like:
[  0% 4/29328] cc out-soong/.intermediates/external/clang/lib/Sema/libclangSema/android_arm_armv7-a-neon_denver_static_core/obj/external/clang/lib/Sema/SemaCodeComplete.o

This is not very helpful - most of the characters are used to show the
output path, which contains useful information like target architecture,
but also contains most of the path to the source files twice, and less
useful information like the exact variant name used by soong.
Make the descriptions look like:
[  0% 3/29329] //external/clang/lib/Sema:libclangSema clang++ SemaTemplateInstantiate.cpp

This is //path/to/module:modulename tool relative/path/to/source/file

Test: builds, looks pretty
Change-Id: I3087aa7d4eb1860ef6239d77407b8b35445616d7
2017-05-10 11:04:16 -07:00
Dan Willemsen 014de6a033 Fix genrule documentation
Change-Id: Ieffb2bd7d739527db129ff300d8aec70d82c278a
2017-05-09 23:09:17 +00:00
Colin Cross 54c5dd5238 Exit early on genrule command parsing failure
cmd may be empty if android.Expand fails, and continuing results
in a panic.  Return immediately.

Test: m -j checkbuild
Change-Id: Ibf4d57ce048db9f7f16ce118ed9e9ecea5a6551c
2017-04-19 14:27:53 -07:00
Colin Cross 795c377e14 Use a minimal set of mutators, module types, and singletons for tests
Calling android.NewContext() in tests results in a context that
contains all the mutators, module types, and singletons, which
causes unexpected interactions in unit tests.  Create an empty
context instead, and add in only the necessary mutators, module
types, and singletons.

Bug: 36366816
Test: soong tests
Change-Id: Ic61262c37e3436b3ad4ccaca18b737021c304be6
2017-03-16 16:50:10 -07:00
Colin Cross 77de5a4b9d Merge "Support data properties for test binaries" 2017-02-06 21:17:12 +00:00
Colin Cross d1b8d760b5 Allow non-tool dependencies in genrules
The genrule tool dependency handling was rejecting any dependencies
that were not tools.  This caused a failure when ExtractSourceDeps
added a source file dependency on a filegroup module.  Remove the
unnecessary check in the tool handling.

Test: builds
Bug: 35002681
Change-Id: Ibd8dff306be3ef55aac2e12a28ddc187be2234d9
2017-02-04 12:45:03 -08:00
Colin Cross faeb7aa135 Support data properties for test binaries
Allow tests to specify a data property that lists files or filegroup
modules that will be packaged alongside the test.  Also add a path
property to filegroup modules to allow shifting the path of the
packaged files, and add ExpandSourcesSubDir to expand the filegroup
sources while including a shifted relative path in the Paths objects.

Test: soong tests, manually adding data to a module
Change-Id: I52a48942660e12755d313ef13279313361b4fc35
2017-02-02 16:23:30 -08:00
Colin Cross 708c424c57 Allow genrules to reference generated sources
Call ExtractSourceDeps on genrule srcs properties.

Bug: 34283327
Test: no change to build.ninja
Change-Id: I934a418f32d4c94400728579543ca5c910bf7dfc
2017-01-13 18:06:58 -08:00
Colin Cross 068e0feace Support filegroups
filegroup is a module that contains a list of files, and can be used
to export files across package boundaries.  filegroups (and genrules)
can be referenced from srcs properties of other modules using the
syntax ":module".

Test: m -j
Change-Id: I3d6fc4819c0b4225b474e0ad42f0d947f55a5961
2016-12-17 16:34:03 +00:00
Colin Cross 5ed99c6472 genrule: let Android.bp file specify exported header dirs
Instead of exporting the generated sources dir as headers, let
the Android.bp file specify subdirectories as exported.

Test: m -j checkbuild
Change-Id: I18dd900d63ce7485c8fbfcc39dc77abad6f733d7
2016-11-22 15:41:09 -08:00
Colin Cross 33bfb0a36a genrule: support deps files
If a genrule sets deps_file: true, use a file in the gen directory
as a GCC-style deps file.

Test: m -j libLLVMObject
Change-Id: Id410165847e4eaea1853a392512e38787c431523
2016-11-22 15:41:09 -08:00
Colin Cross 6f080dff81 Support genrules with multiple tools
To allow genrules with more than one tool, rename the tool property to
tools and make it an array, replace $tool with $(location <label>),
and use $() for other variables for consistency.

Also remove the host bin directory from the genrule path, and the
$srcDir variable, using either of them would have caused dependency
issues.

Bug: 31948427
Test: compare build.ninja
Change-Id: Icf6d3bce2bea00fec1363fd65c0bdf96d09281bf
(cherry picked from commit de6bd86d24)
2016-11-07 10:53:49 -08:00
Dan Willemsen 21ec49068f Add subdir to GenPath
We were emulating this for proto files, standardize it and make the
other generators use it as well.

Test: Compare out/soong/build.ninja before/after change
Test: mmma -j system/tools/hidl
Change-Id: I1888c7b981749060a398387bbb9b481270bf6d75
2016-11-02 21:19:11 -07:00