Commit Graph

27 Commits

Author SHA1 Message Date
Colin Cross 77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Colin Cross 121292911a Store ninja file deps from PackageVarContext in the config
Store ninja file deps from ExistentPathForSource on a
PackageVarContext in the config instead of the PackageContext, as
the PackageContext may be shared between multiple tests running
in parallel.

Test: all soong tests
Change-Id: Ib1809a4dd4a82696e0fe48a87eac21a44684ecb5
2020-11-17 10:50:19 -08:00
Ramy Medhat 1dcc27e47c Refactor RBE support for Javac/R8/D8 to use the remoteexec package.
Test: build with and without RBE_JAVAC/RBE_R8/RBE_D8 set.
Change-Id: I1607f8cfb4d2c3cbb3672152bbeb561d9968bc60
2020-04-28 11:37:20 -04:00
Ramy Medhat 8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Colin Cross 8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Colin Cross 988414c2cf Sandbox soong_build by changing to root directory
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.

Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
2020-01-11 01:11:46 +00:00
Colin Cross 47e4f9e1e8 Revert "Sandbox soong_build by changing to root directory"
This reverts commit 05c25ccb4a.

Reason for revert: broke absolute OUT_DIR
Bug: 146437378

Change-Id: I523ed79d40e1c1ef040212ba794a7a084abea75d
2020-01-10 18:51:04 +00:00
Colin Cross 05c25ccb4a Sandbox soong_build by changing to root directory
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
2020-01-09 14:19:46 -08:00
Martin Stjernholm 7260d066ea Move/dedupe some host path functions in package_ctx.go.
These functions don't need a PackageContext object.

Test: m
Bug: 145934348
Change-Id: Ia1da2c76cbad292f9ca79617199b2b5b6b265568
2019-12-09 22:04:21 +00:00
Ramy Medhat dd0418a4d4 Run non-RBE supported actions in the local pool when USE_RBE is set.
Bug: 143938974
Test: ran CTS build at -j500 successfully.
Change-Id: I55074bd67308cd716972e24fb56a20bc393d5d9d
2019-11-05 22:57:35 +00:00
Colin Cross 2e2dbc250a Use localPool consistently for UseGoma() == true
Remove the distinction between pctx.StaticRule and
pctx.AndroidStaticRule so that all of the local rules correctly
get assigned to the localPool.  Also put Module and Singleton
rules into the localPool.

Test: compare out/soong/build.ninja
Change-Id: Id2bb38eff3c7209340fe55bc9006f00bd3661d81
2019-10-01 14:11:20 -07:00
Colin Cross fe4bc36f87 Allow '$' in some paths
The icu resource directories contain filenames that have '$'
characters.

Allow paths returned by the Glob functions to contain '$', on the
assumption that real paths on disk are unlikely to contain strings
that are valid ninja variables.  Fix the Build rules to escape any
paths that are passed as Path arguments.  Fix the resource rules to
manually escape the paths that are passed as strings.

Test: m checkbuild
Change-Id: Ie631bc6d96259e592adb280491a365c0df7ed0e2
2018-09-13 18:27:50 +00:00
Dan Willemsen 54daaf0371 Switch PackageContext functions to take a ctx
So that the Path and similar functions can be used directly, without
manually adding something like configErrorWrapper (it just uses it all
the time now).

Test: out/soong/build.ninja is identical
Change-Id: I8cb524b09a84d0b8357d7c3501c71270af411e17
2018-03-12 15:48:26 -07:00
Colin Cross 32f3898f0b Remove unused intermediates parameter from ExistentPathForSource
Test: m checkbuild
Change-Id: Id2c0a5039c2ec3b3795385c135ffec022ccd691e
2018-02-22 14:43:36 -08:00
Colin Cross 59014396ba Make SingletonContext.Rule take a blueprint.RuleParams
Make the currently-unused SingletonContext.Rule match
ModuleContext.Rule and take a blueprint.RuleParams instead
of an android.RuleParams, and delete android.RuleParams.

Test: m checkbuild
Change-Id: I018685a3dc13f60825b5ba383ef365df2bc98dfc
2017-12-12 11:12:32 -08:00
Colin Cross 5ab4e6d817 Allow building framework.jar and framework-res.apk
Update app support enough to build framework-res.apk, link
framework.jar against its generated files, and export it to
make.

Bug: 69917341
Test: m checkbuild tests docs
Change-Id: I7db29cd1f5fabb22e844483ecc7c38abfedbbe0a
2017-11-29 20:47:42 -08:00
Colin Cross aabf67968c Make *Context.Config return a Config instead of a interface{}
In Soong, a Config() method will always return a Config.  Make
ModuleContext, SingletonContext, TopDownMutatorContext and
BottomUpMutatorContext's Config() methods explictly return
a Config to avoid having to type-assert everywhere.  Overriding
the Config method requires duplicating the list of methods in
blueprint.BaseModuleContext and blueprint.BottomUpMutatorContext,
following the same pattern used by the other *Contexts.

Config() obsoletes the AConfig() method used in some places, which
will be cleaned up in the next patch.

Test: m checkbuild
Change-Id: Ibe21efde933959811d52443496967ab8ce71215e
2017-11-30 00:44:11 +00:00
Colin Cross 0875c52de7 Wrap PackageContext and SingletonContext
Wrap blueprint.PackageContext so that the *Func methods can provide
an android.Config instead of an interface{}.  The modified signatures
means that every method in ModuleContext and SingletonContext
that takes a blueprint.PackageContext now needs to be wrapped to
take an android.PackageContext.

SingletonContext wasn't previously wrapped at all, but as long
as it is, wrap everything like ModuleContext does.  This requires
updating every Singleton to use the android-specific methods.

Test: builds, all Soong tests pass
Change-Id: I4f22085ebca7def6c5cde49e8210b59d994ba625
2017-11-29 05:04:30 +00:00
Colin Cross cf31fcfc71 Make AndroidRuleFunc callback take a Config
Convert the interface{} to android.Config before calling
the callback.

Test: m checkbuild
Change-Id: Iab1173d3c0bbeff662afcbc1d6ce7c982b6b7720
2017-11-20 12:23:00 -08:00
Nan Zhang 9a36418089 Use prebuilt turbine jar for unbundled javac builds.
Test: m -j
Change-Id: If2ab85f8591fe7ab275a1d4fefdd80d871f2c36d
2017-10-25 11:11:37 -07:00
Alan Leung 1d476fcbeb Add D8 support
Bug: 67754178
Test: m -j32 checkbuild && USE_D8=true m -j32 checkbuild

Change-Id: If63afc10ceb5e753bbb7f195bb8a895eaef10775
2017-10-19 16:27:42 -07:00
Colin Cross c6bbef326f Add error-prone support
Add support for compiling java sources with the error-prone tool.

Test: m -j checkbuild
Change-Id: Ieb4ee0e05f8f34a52ed7bcf1c7cbacf1c9c4d0b5
2017-08-28 11:12:38 -07:00
Colin Cross 6416271a1f Bring java support closer to current version of make
Make the javac arguments match what is used by make, and export them
back to make.  A future change will switch make to use the the
exported ones.

This makes a dx.jar compiled with soong have identical class files
as one compiled with make.

Test: manual
Change-Id: Ia5196f1f42bc564e99de22e32e72fd2930e9fbae
2017-08-11 15:24:11 -07:00
Jeff Gaston 734e3809ad Clarify paths.go somewhat
Bug: 35562758
Test: make

Change-Id: I4cf05ffdd0962186f490467829e3576b4e269c47
2017-05-15 11:31:48 -07:00
Colin Cross 294941bee9 Update soong to use pathtools.FileSystem
Update soong to follow changes in
https://github.com/google/blueprint/pull/141

Test: soong tests
Change-Id: I49a9b83cac7590dc75b26b31136b8707c188bc4a
2017-02-02 16:19:56 -08:00
Colin Cross 9d45bb78c5 Enable goma in soong
When the UseGoma flag is set, put all rules except the C compilation
rule in an externally defined local_pool, which will have been created
by kati.  The gomacc wrapper will already be in the CC_WRAPPER
environment variable.

Bug: 31142427
Change-Id: I699d4edff2e302eee398dad8692ceb14721a628c
2016-08-31 09:48:08 -07:00
Colin Cross 635c3b0157 Rename common to android
Rename the "common" package to "android", because common is too
generic.  Also removes all android.Android naming stutter.

Ran:
gomvpkg -from 'android/soong/common' -to 'android/soong/android'
gorename -from '"android/soong/android".AndroidModuleContext' -to 'ModuleContext'
gorename -from '"android/soong/android".AndroidBaseContext' -to 'BaseContext'
gorename -from '"android/soong/android".AndroidModuleBase' -to 'ModuleBase'
gorename -from '"android/soong/android".AndroidBottomUpMutatorContext' -to 'BottomUpMutatorContext'
gorename -from '"android/soong/android".AndroidTopDownMutatorContext' -to 'TopDownMutatorContext'
gorename -from '"android/soong/android".AndroidModule' -to 'Module'

Change-Id: I3b23590b8ce7c8a1ea1139411d84a53163288da7
2016-05-18 15:37:25 -07:00