Commit Graph

21 Commits

Author SHA1 Message Date
Colin Cross 77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Colin Cross 1c217fdc96 Fix WriteFileRule escaping
WriteFileRule shouldn't force the caller to ninja escape the input,
and should shell escape spaces.

Bug: 182612695
Test: manual
Change-Id: Ide2f1ed92783eef7883279238de209d992d8f735
2021-03-16 16:53:23 -07:00
Colin Cross 00d93b143f Make common dependencies of lint use restat
Use restat for the api_versions.xml and annotations.zip dependencies
of lint so that frameworks/base changes don't always result in
rerunning lint on every module.

Bug: 181681346
Test: m lint-check
Change-Id: Ic6a540b41cf79b21441311a8baefe528a3d90d8b
2021-03-04 10:00:09 -08:00
Colin Cross c20dc8533e Add dependency to list of asset files
We had a dependency on each file in the asset directories, but that
wouldn't cause aapt2 to run if a file was removed.  Add a dependency
on a file that contains the list of files in the asset directories.

Fixes: 172867096
Test: m CarrierConfig && rm packages/apps/CarrierConfig/assets/carrier_config_no_sim.xml && m CarrierConfig
Change-Id: I35f3b85355fa890a3e95eaa6458a21466b6930e4
2020-12-11 19:54:16 +00:00
Hans Månsson d3f2bd79e8 WriteFileRule: Chunk long content and merge them to result
sbox.textproto files are created when handling genrule. The command
for the genrule and output files are written to this file in the
following format:

commands <
  command: "...."
  copy_after: <
    from: "out/asm-generic/auxvec.h"
    to: "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/auxvec.h"
  >
  copy_after: <
    from: "out/asm-generic/bitsperlong.h"
     to:
     "out/soong/.intermediates/kernel/msm-4.19/qti_generate_kernel_headers_arm/gen/asm-generic/bitsperlong.h"
  >
  ....
>

This file grow by one copy_after entry for each output file.

When generating kenrnel headers where the number of output files are
~1000 we run into problems as the contents of sbox.textproto files
are written to disk by generating a shell script using the following
template: /bin/bash -c 'echo -e "$$0" > $out' $content
If $content is very long as in the case of generating kernel headers we
run into the issue where the command line is so long that the shell
script return E2BIG.

Fix this issue by chuking contents into smaller files and then merge
them as a final step.

Test: Build
Issue: 174444967
Change-Id: I1a74023b4222d19672e4df7edb19810a9cf2136f
2020-12-02 15:58:48 +01:00
Colin Cross cf371cc1f7 Replace android.WriteFile rule with android.WriteFileRule
The android.WriteFile rule takes careful escaping to produce the
right contents.  Wrap it in an android.WriteFileRule that handles
the escaping.

Test: compare all android.WriteFile outputs
Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
2020-11-14 16:26:00 -08:00
Jingwen Chen ce679d29ec Add symlink_outputs support to Soong.
This CL adds symlink_outputs to various locations in Soong that creates
actions that creates symlink outputs, and explicitly mark them as such.

Test: m
Bug: 160568334
Change-Id: I322751bada52a9f49011c74731d84761586e03e7
2020-10-19 01:26:50 -04:00
Dan Willemsen 751ae87ed2 Reland "Add prebuilt_build_tool to allow genrules to use prebuilt tools"
Includes fixes for out dirs at absolute paths, and symlinks to absolute
paths. Also squashes support for exporting make vars, which previously
happened in a later patch (which got moved first, since it's mostly
infrastructure changes).

Original message:

cc_prebuilt_binary doesn't work well for host tools because they'll
often be using their own versions of shared libraries that may not be
compatible with what we build.

So add a module type that allows genrules to use one of these prebuilts
as a tool. Like other prebuilts, we'll use the source module if we have
it, or the prebuilt otherwise.

It supports adding extra dependencies for shared libraries or other data
files that are necessary to run the tool. Any genrules using the tool
will be rerun if any of the dependencies change.

Bug: 128690776
Test: treehugger
Test: build one-true-awk using genrule with bison
Test: with OUT_DIR=$PWD/out
Test: with OUT_DIR pointing to an absolute symlink
Change-Id: I662296cb55503b6a10ec5e5465b138e9edeb71f3
2020-08-06 15:56:38 -07:00
David Su bec7f53e4f Revert "Add prebuilt_build_tool to allow genrules to use prebuil..."
Revert submission 1366377-prebuilt_build_tool_make

Reason for revert: breaks build
Reverted Changes:
I20bf062bb:Export prebuilt tools to Make
I4bb526492:Move some prebuilt build tool configs to Soong
I195b68813:Support per-module MakeVars
Ibcb257e7b:Fix dependency loop with flex
I6150f0f39:Switch cc's use of bison and flex to prebuilt_buil...
I6939451b8:Reland "Use genrules to build a consistent awk."
Idee60640f:Add prebuilt_build_tool modules for genrule use
I00893172b:Rename bison to bison_bin
I82c26be1c:Add prebuilt_build_tool to allow genrules to use p...

Change-Id: I6b3acf306d355f3e0463564c9ebe9482fa0f609e
2020-07-24 21:25:13 +00:00
Dan Willemsen 13af8140bc Add prebuilt_build_tool to allow genrules to use prebuilt tools
cc_prebuilt_binary doesn't work well for host tools because they'll
often be using their own versions of shared libraries that may not be
compatible with what we build.

So add a module type that allows genrules to use one of these prebuilts
as a tool. Like other prebuilts, we'll use the source module if we have
it, or the prebuilt otherwise.

It supports adding extra dependencies for shared libraries or other data
files that are necessary to run the tool. Any genrules using the tool
will be rerun if any of the dependencies change.

Bug: 128690776
Test: treehugger (builds one-true-awk with bison)
Change-Id: I82c26be1c3c9fe6cab42892ddea339c301c0b316
2020-07-17 19:34:12 -07:00
Ramy Medhat 944839a88a Fix RuleBuilder remoteable actions running in the local pool.
This CL fixes a bug with RuleBuilder's handling of remoteable actions.
It adds a new type of pool to identify remoteable rules by the android
module context. The pool is then set to nil to actually run actions at
NINJA_REMOTE_NUM_JOBS parallelism.

Test: built aosp crosshatch userdebug
Change-Id: I29452f6fc7a161b94189731e3e3cc1f34907b80c
2020-03-31 22:18:40 -04:00
Colin Cross 8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Colin Cross cc0ce80ed5 Fix package path of android/soong/android pctx
android/soong/common was renamed to android/soong/android long
ago, but the pctx package path was still "android/soong/common".
This required all users of rules defined in android/soong/android
to import "android/soong/android" and then
pctx.Import("android/soong/common").

Test: m checkbuild
Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
2019-04-04 03:59:34 +00:00
Colin Cross 5c51792926 Make binaries executable
Split InstallFileName into InstallExecutable that does chmod +x
after copying the file.  Also remove InstallFile and rename
InstallFileName to InstallFile.

Test: m -j checkbuild
Change-Id: Id41ad4eafe521f6cd5d8cc250b7747ecb3da8dfc
2017-08-31 13:48:52 -07:00
Josh Gao ae15271392 Delete output files before copying to them.
Avoid ETXTBSY when copying over a file that's being executed.

Test: manual
Test: m -j
Change-Id: I0427b5018e67e12f685bce95028831b5661eee2c
2017-07-26 15:38:40 -07:00
Nan Zhang 270051119a Fix the issue "\n" didn't get interpreted properly.
Added params to WriteFile rule that allow users to turn on the
specific 'echo' options, such as "enable the interpretation of backslash
escapes".

Bug: b/38261000
Test: tested with real data files.
Change-Id: I5145056b5c2522a3a72b9436055ec95d2fbd94c0
2017-05-16 15:10:29 -07:00
Dan Albert 30c9d6e771 Generate file to map codenames to API levels.
The NDK stub library generator needs to know what unreleased API
levels exist and what the ordering is between them. This singleton
will later be expanded to generate api-level.h to cut down on
duplication.

Test: make out/soong/api_levels.json && cat out/soong/api_levels.json
Bug: None
Change-Id: I53126dd2cacb67c331c44f7d7c77c98b176b93cd
2017-03-31 12:27:42 -07:00
Dan Albert c6345fb7ec Add a "license" property to ndk_headers.
This field points to the license file for the headers being shipped.

Test: make ndk && less $SOONG_OUT/ndk/NOTICE
Bug: None
Change-Id: I386f4e6f6d9776f422ffc09b8dab69e1911b08a4
2016-10-20 13:30:07 -07:00
Colin Cross 9d45bb78c5 Enable goma in soong
When the UseGoma flag is set, put all rules except the C compilation
rule in an externally defined local_pool, which will have been created
by kati.  The gomacc wrapper will already be in the CC_WRAPPER
environment variable.

Bug: 31142427
Change-Id: I699d4edff2e302eee398dad8692ceb14721a628c
2016-08-31 09:48:08 -07:00
Dan Albert 5d723ab19a Add a timestamp touch rule.
Test: Works for the NDK changes I'm working on.
Change-Id: I2731ef159122d6d44a6494a6c975a687bb49ff56
2016-07-18 23:31:27 -07:00
Colin Cross 635c3b0157 Rename common to android
Rename the "common" package to "android", because common is too
generic.  Also removes all android.Android naming stutter.

Ran:
gomvpkg -from 'android/soong/common' -to 'android/soong/android'
gorename -from '"android/soong/android".AndroidModuleContext' -to 'ModuleContext'
gorename -from '"android/soong/android".AndroidBaseContext' -to 'BaseContext'
gorename -from '"android/soong/android".AndroidModuleBase' -to 'ModuleBase'
gorename -from '"android/soong/android".AndroidBottomUpMutatorContext' -to 'BottomUpMutatorContext'
gorename -from '"android/soong/android".AndroidTopDownMutatorContext' -to 'TopDownMutatorContext'
gorename -from '"android/soong/android".AndroidModule' -to 'Module'

Change-Id: I3b23590b8ce7c8a1ea1139411d84a53163288da7
2016-05-18 15:37:25 -07:00