Commit Graph

262 Commits

Author SHA1 Message Date
Kiyoung Kim 62abd12ba5 Handle new filetype 'linker_config'
Handle new filetype 'linker_config' which is configuration for
linkerconfig in json type and convert into protobuf in build time.

Bug: 169634881
Test: Build succeeded and cuttlefish boot succeeded
Change-Id: I56555fc738e6d6600d15a191a24f79a2ee747f52
2020-10-15 23:09:35 +09:00
Artur Satayev 4e1f2bd0d8 Track allowed transitive deps in any updatable module.
Instead of tracking per module and per module variant, track allowed
list of dependecies for all modules combined. This avoids issues with
different products and different downstream branches having different
build graphs.

To compare allowed_deps.txt vs head, run:
:; m -j out/soong/apex/depsinfo/new-allowed-deps.txt.check

To update source allowed_deps.txt, run:
:; build/soong/scripts/update-apex-allowed-deps.sh

Bug: 149622332
Test: m
Change-Id: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
Merged-In: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
(cherry picked from commit 453555083b)
(cherry picked from commit e5207cd9a6)
2020-09-30 21:17:42 +00:00
Martin Stjernholm 2829f6c09c Propagate settings for building in an unbundled tree.
Test: build/soong/scripts/build-aml-prebuilts.sh art-module-sdk
  on master-art
Change-Id: Iaff7f590d67fdd6f533f2831475deb7b92090ea2
2020-08-11 01:41:20 +01:00
Nicolas Geoffray b0df23f201 Add platform-mainline-test-exports in list of modules to build.
Needed for ART testing.

Bug: 142935992
Test: build-mainline-modules.sh

Change-Id: I842feff85f88ecc2c7db1e84e803d9e8149923af
2020-08-05 15:56:21 +01:00
Nicolas Geoffray f23054f668 Fix the lib directory being used for fetching libraries.
On 64bit archs, it needs to be lib64.

Test: scripts/build-mainline-modules.sh
Bug: 142935992
Change-Id: I049cfc39704c34c32f79edc7cc49d5f7f29f0ac2
2020-07-31 16:10:23 +01:00
Nicolas Geoffray 5c5058ffa7 Remove platform host-exports.
It is not defined anymore.

Test: scripts/build-mainline-modules.sh
Bug: 142935992
Change-Id: I43dad2e5fc52bc19996fe8cf5aaa5c7188110029
2020-07-30 16:22:17 +01:00
satayev 53bbc085e4 Revert "Track allowed transitive deps in any updatable module."
Revert submission 1312796-apex-allowed-deps

Reason for revert: b/161974327
Reverted Changes:
I52a4be72e:Add a check for apex/allowed_deps.txt to droidcore...
I56771ba3f:Track allowed transitive deps in any updatable mod...

Change-Id: I3be0a1b0dd824dafeadb485daca8e58b81a3ec5c
2020-07-23 15:15:54 +00:00
Artur Satayev 453555083b Track allowed transitive deps in any updatable module.
Instead of tracking per module and per module variant, track allowed
list of dependecies for all modules combined. This avoids issues with
different products and different downstream branches having different
build graphs.

To compare allowed_deps.txt vs head, run:
:; m -j out/soong/apex/depsinfo/new-allowed-deps.txt.check

To update source allowed_deps.txt, run:
:; build/soong/scripts/update-apex-allowed-deps.sh

Bug: 149622332
Test: m
Change-Id: I56771ba3fea748de8e9c58c80758670572f7af53
Merged-In: Ic518fbd9ebfe1b46aaf9a58df731780a7e5a676b
2020-07-20 12:22:26 +01:00
Ulyana Trafimovich 8fab5bcce3 Merge changes from topic "uses-libs-5"
* changes:
  Rewrite construct_context.sh in Python.
  Refactor class loader context generation.
2020-07-02 09:06:27 +00:00
Ulya Trafimovich 5f364b63b8 Rewrite construct_context.sh in Python.
This allows to reuse SDK version comparison routine from other Python
scripts. With the addition of non-numeric versions comparison has become
more complex, and the deleted shell script did it incorrectly: it used
comparisons like `[[ "S" -lt 28 ]]` which results in "true" as strings
are converted to zero in numeric context. This resulted in adding legacy
libraries to class loader context of apps targeting recent SDK versions.
The error was masked because currently there is only one non-AOSP app
that uses the script (GoogleDialer), and it targets numeric SDK version.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I26e752e29b5453fd3e29d5dbfbe4d9df9c0a55b7
2020-07-01 09:44:23 +01:00
Nicolas Geoffray ba7887fd53 Add host exports for platform libraries used by ART host testing.
Test: build-mainline-modules.sh
Bug: 142935992
Change-Id: I81247b89b78426463faeb6b00212cd8901319b19
2020-06-30 15:02:51 +01:00
Treehugger Robot e9eacaa061 Merge "Move gen-kotlin-build-file.sh to python" 2020-06-29 21:12:45 +00:00
Thiébaud Weksteen 5e291c238f Fix bpfmt issues and add bpfmt to preupload checks
Test: m nothing
Change-Id: I10526a33685335bdd26cc7692815133379d4e0f6
2020-06-29 09:14:08 +02:00
Colin Cross 9b1aa0cb86 Move gen-kotlin-build-file.sh to python
Kotlin common multiplatform sources support will require more
complexity in gen-kotlin-build-file.sh, move it to python instead.

Test: m checkbuild
Change-Id: I02312160ad781877f1fec971168331c0dcecf136
2020-06-26 22:41:34 -07:00
Nicolas Geoffray eb2676ea71 Merge "Add platform libraries required for ART chroot testing." 2020-06-26 19:10:30 +00:00
Nicolas Geoffray 792ea7359a Add platform libraries required for ART chroot testing.
Those libraries are not in an APEX, and are provided as stubs
in the platform-mainline-sdk. Therefore compile them explicitly and make
them available in the dist directory.

Test: scripts/build-mainline-modules.sh
Bug: 142935992
Change-Id: I919d8c580e907ce9a7b54366653d0c63ab9d00c7
2020-06-26 15:17:47 +01:00
Ulyana Trafimovich 3d88ad68be Merge "Move construct_context.sh from Make to Soong. Update OWNERS list." 2020-06-26 08:46:58 +00:00
Ulya Trafimovich 5cb30c5820 Move construct_context.sh from Make to Soong. Update OWNERS list.
This script belongs with other manifest-related scripts, and the future
plan is to rewrite it in Python and share common functionality with
other scripts.

Test: lunch aosp_cf_x86_phone-userdebug && m
Bug: 132357300
Change-Id: I4b90129e5023ca1a2c818fc160c34b0b7da943ef
2020-06-25 14:46:35 +01:00
Treehugger Robot 6f61fa75c6 Merge "Minor tweaks to usage message." 2020-06-24 20:00:31 +00:00
Thiébaud Weksteen e6ec845501 Merge "Add standard rustfmt.toml" 2020-06-24 17:22:46 +00:00
Bob Badour 8970b45c8f Minor tweaks to usage message.
Fix typo in tab separator example.
Add missing usage message to usage error.

Test: manually

Change-Id: I1d3a01fcb7caffeb39d6e47ddfe665059b2860f5
2020-06-24 08:18:15 -07:00
Colin Cross c31efeb25c Set root dir in lint project.xml
Set the root dir in the project.xml for lint so that the paths in the
lint report are relative to the top of the tree instead of relative
to the project.xml.

Bug: 153485543
Test: run lint
Change-Id: Ie163a4dadd976e708f798855de73e58084931a91
2020-06-23 10:27:05 -07:00
Thiébaud Weksteen b6db6de842 Add standard rustfmt.toml
This configuration is to be used for all internal Rust crates.

Test: rustfmt --print-config current --config-path build/soong/scripts/ .
Bug: 157238653
Change-Id: Ic49027dd1530f503e2c1f781453cdee95a983252
2020-06-22 13:34:59 +02:00
Nicolas Geoffray b758a37876 Merge "Add ICU SDK and module_exports to the list." 2020-06-19 07:22:33 +00:00
Nicolas Geoffray 7505c50e49 Add ICU SDK and module_exports to the list.
Test: m
Bug: 157734636
Change-Id: If0c967f75578471286f3760f2f15e576752e162d
2020-06-18 15:33:18 +01:00
Colin Cross 014489c1e6 Add support for running Android lint on java and android modules.
Add a rule that runs Android lint on each java and android module
and produces reports in xml, html and text formats.

Bug: 153485543
Test: m out/soong/.intermediates/packages/apps/Settings/Settings-core/android_common/lint-report.html
Change-Id: I5a530975b73ba767fef45b257d4f9ec901a19fcb
2020-06-16 15:44:16 -07:00
Martin Stjernholm 68b0c7a6e5 Include runtime-module-host-exports in the SDK build.
Test: build/soong/scripts/build-mainline-modules.sh
Bug: 152255951
Change-Id: I87a154a31409c83ccdd4637210f6163dfa2d7738
2020-06-11 15:48:13 +01:00
Paul Duffin 63d8febd35 Ensure package check is run for java_library in APEX
Package checks were not being run for java libraries that were in an
APEX and not on the platform. This change fixes that and updates the
script to report all failing classes to make it easier to update the
list of packages.

Test: m java
Bug: 157633658
Change-Id: I28044e08d3a40e9f3464bb2158ef6a28d57264d1
2020-05-28 15:06:35 +01:00
Martin Stjernholm 6ee5f754fe Include runtime-module-sdk in the SDK build.
Test: build/soong/scripts/build-mainline-modules.sh
Bug: 152255951
Change-Id: I39f9024205a198ef8ce19e2c70e3b9dced54add1
2020-05-14 17:54:20 +01:00
Martin Stjernholm 691503ea0b Reduce the number of Soong invocations, and various other minor script fixes.
-  We can pass all targets at once to build-aml-prebuilts.sh.
-  Less noise from build-mainline-modules.sh.
-  Default to the same out directory in both build scripts.
-  Addressed post-submit comments on https://r.android.com/1170907.
-  Various minor cleanups.

Test: build/soong/scripts/build-mainline-modules.sh
Change-Id: Id21ef80c2334462836e217032fc5bf63b7cd04e1
2020-05-06 22:51:15 +01:00
Nicolas Geoffray 7cad0f860b Use FORCE_BUILD_LLVM_COMPONENTS.
And remove the allow missing dependencies flag.

Test: build-mainline-modules.sh
Change-Id: I6ba8892ef66eeb42c7bd3abc989a80ad4b6a1cf9
2020-04-28 15:54:59 +01:00
Nicolas Geoffray 71233e4732 Add linux_bionic cross compilation for mainline SDKs.
ART requires linux bionic prebuilts.

Bug: 142935992
Test: build-mainline-modules.sh
Change-Id: If0f97e58d2afccbc3f5d0b2e7dabac51330cc5c7
2020-04-25 16:25:21 +01:00
Nicolas Geoffray 0a2e297206 Add conscrypt-module-host-exports to the list.
Test: build-mainline-modules.sh
Change-Id: I9ca14cc37a69b1172f52412297977dd03c6ca1a0
2020-04-23 09:35:45 +01:00
Nicolas Geoffray 11a8d879b6 Merge "Support secondary arch for host module exports." 2020-04-02 08:12:45 +00:00
Nicolas Geoffray 32e89ddb60 Merge "Update owners of build-aml-prebuilts.sh." 2020-03-31 09:08:35 +00:00
Nicolas Geoffray 90934f7177 Update owners of build-aml-prebuilts.sh.
Test: N/A
Change-Id: Ie5be75f90be00db5adeaa40662e26a92c66d1b05
2020-03-30 14:23:13 +01:00
Nicolas Geoffray abdc8c5501 Support secondary arch for host module exports.
Bug: 142935992
Test: build-aml-prebuilts.sh generate x86 prebuilts

Change-Id: Iec4d8aa59a3ad55583c788f8ec40dd0637e7af33
2020-03-27 14:57:37 +00:00
Bob Badour 0d3d8e40c1 More fix for -e
Not sure whether sort, tr or xargs returning non-zero status, but
one of them sometimes does.

Test: manually

Change-Id: I8f2b862de9d6e22dd90068252e393533ec992dfd
2020-03-24 13:42:43 -07:00
Bob Badour 455b3cb38f Fix bug caused by set -e
Empty results cause grep/egrep to return a non-zero status.

Bug: 145347092
Bug: 141258651
Bug: 69058154
Bug: 143285996

Test: manual run for notice files

Change-Id: I25d54ae307b97a07a98e99c54f283437718fe657
2020-03-18 17:51:48 -07:00
Bob Badour 02ad5e4de6 Scripts to evaluate dependency transitive closures.
Bug: 145347092
Bug: 141258651
Bug: 69058154
Bug: 143285996

Test: run manually

Change-Id: I29d674f97ccff19ff61f675022dce052f14e0dd8
2020-03-10 21:53:39 +00:00
Nicolas Geoffray 90bc3cd304 Add a build script to compile modules and SDKs in AOSP.
Test: build-mainline-modules.sh
Bug: 142935992
Change-Id: I80a9bf8ce03d77705ba677f95b03aeb2ac2451d5
2020-03-06 14:35:48 +00:00
Treehugger Robot 5d95387398 Merge "Make a soong-only copy of generate-notice-files.py" 2020-03-04 21:56:04 +00:00
Treehugger Robot 49b445cace Merge "Update build_broken_logs script" 2020-03-04 00:21:51 +00:00
Bob Badour 3911e6a1c4 Make a soong-only copy of generate-notice-files.py
This is an unchanged copy of build/make/tools/generate-notice-files.py

After this change, the make implementation of generate-notice-files.py
will change to take a required parameter for $(PRODUCT_OUT) and will
only include notices for files built under $(PRODUCT_OUT).

Because soong has the ability to walk dependency trees, the soong
version will likely change in the future to use that ability causing
both versions to move in different directions.

After those changes are complete, we can look into factoring out any
remaining shared logic.

Test: run manually and system image notices checked for changes

Change-Id: Id139a66503457615548b46e7996349ca0817e831
2020-03-03 13:50:53 -08:00
Martin Stjernholm 37fa32ce94 Output json without trailing whitespaces but with trailing newlines.
Makes them more editor friendly, primarily when copied into
system/linkerconfig/testdata/root.

Test: m
Test: No diff between apex_manifest_full.json in out/
  and in linkerconfig/testdata/root/
Bug: 140599044
Change-Id: I6dacdd4baa59e6d70586c4a6343b9b8fd1b574f6
2020-02-24 22:20:49 +00:00
Baligh Uddin 3be82c00fe Create manifest_utils library target.
This utility can also be used for parsing and
managing the Android Manifest for apex under
system/apex/apexer.

Test:
python manifest_fixer_test.py
python manifest_check_test.py

BUG: 148198056
Change-Id: Iffe465b5f36b2a3fbf81414d6f19e241c5f5648f
2020-02-20 15:58:00 +00:00
Baligh Uddin 5b16dfb39a Allow for setting a logging_parent for an Android App.
Unit test: go test ./... -test.v -run TestOverrideAndroidApp
Unit test: python manifest_fixer_test.py

BUG: 148198056
Change-Id: Ib5ff235d2a93e88b86aec1c0b16327ea938a094d
2020-02-18 09:37:56 -08:00
Sasha Smundak 9dd454b3ac Script to set up android build directory
Test: manual
Change-Id: I93aa3aa9745330ec21385ce12cc586aeaa803e12
2020-01-30 19:18:54 -08:00
Dan Willemsen 9dfaa06e45 Update build_broken_logs script
This can now discover which make module types have been deprecated and
suggest what needs to happen with the BUILD_BROKEN_USES_* flags.

Bug: 130720555
Test: go run .../build_broken_logs.go *  (see header comment)
Change-Id: Iffaa32c0b0079e99b926162257a7c15c530c15a1
2020-01-29 10:50:41 -08:00
Jaewoong Jung 3998234d8c Overwrite test-file-name in test config.
If the install apk name is different than the module name, use
test_config_fixer to update the test-file-name value in the config.

Test: app_test.go
Fixes: 147375216
Change-Id: I2141eeebbb3552995400b45634712306673fd812
2020-01-17 06:08:12 -08:00
Paul Duffin 749f98fb64 java_sdk_library: Allow api dir to be specified
For modules that provide API surfaces in addition to the standard
current, test and system it is useful to be able to specify the
directory containing the api's .txt files to make it easy to create
multiple API surfaces from within the same Android.bp file. e.g. This
is useful for conscrypt, icu and libcore to manage their intra core
and core platform APIs.

Bug: 145998881
Test: m checkbuild
Change-Id: I753631d9b6993fbf30019fef5c052a9429e519de
2020-01-02 15:38:42 +00:00
Paul Duffin 37e0b77e3b java_sdk_library: Avoid generating system and test .txt files
If the library does not provide system and test APIs then do not
generate/require corresponding .txt files.

Bug: 145998881
Test: m checkbuild
Change-Id: I21cfdb0b63fd575e8c8c63ea2b436e0c4aa8f3fc
2020-01-02 15:38:18 +00:00
Martin Stjernholm c1ecc43aec Add script for building all target arch's needed in AML (Android Mainline)
prebuilts.

This runs Soong in skip-make mode, using normal in-make mode only to query
platform versions.

The same ${OUT_DIR} cannot be used for both skip-make and in-make builds,
because Soong generates a smaller build.ninja file in in-make builds where
many build targets are expected to be provided by the mk files. Thus this
script avoids using ${OUT_DIR} if it's an in-make build, defaulting instead
to out-aml/.

The script is based on build-ndk-prebuilts.sh, but uses a separate Soong
variable Aml_abis to enable the appropriate target architectures for
Mainline modules. Aml_abis is very similar to Ndk_abis, except "armeabi-v7a"
is used instead of "armeabi", which is necessary to match prebuilt
dependencies, e.g. for LLVM.

Test: build/soong/scripts/build-aml-prebuilts.sh libart libdexfile_external
  (verify that libraries for arm, arm64, x86, x86_64 are built)
Test: build/soong/scripts/build-aml-prebuilts.sh \
  out-aml/soong/.intermediates/external/conscrypt/conscrypt-module-sdk/android_common/conscrypt-module-sdk-current.zip
  (verify that the zip file contains libconscrypt_jni.so's for all four arches)
Test: build/soong/scripts/build-aml-prebuilts.sh com.android.art.{release,debug,testing,host}
  (verify that the build completes)
Test: Two identical build/soong/scripts/build-aml-prebuilts.sh runs after each other
  (verify that the 2nd run completes both Soong and ninja steps quickly without any building)
Change-Id: I35712f9f8f0b1cbb77107314c5927c6720e6c3bf
2019-12-20 11:53:21 +00:00
Jaewoong Jung e5cd4e10a0 Optionally overwrite package value in test config.
If package_name is set for an android_test module, package names in its
AndroidTest.xml config need to be updated too.

Test: test_config_fixer_test.py
Fixes: 145011263
Change-Id: I52f17ef1d1902364b9bcfec4f6e246bd655dbf09
2019-12-05 11:17:09 -08:00
Roland Levillain 8ac05575aa Fix some typos in Soong.
Test: m
Change-Id: I032b6ce2a61491b88416db9943b42e6af4783459
2019-11-13 10:46:49 +00:00
Yi Kong c49c393f73 Repack libgcc.a to only include required objects
Previous solution by using objcopy uses a quirky behaviour of the GNU
objcopy and there is no equivalent option in llvm-objcopy.

Instead of removing symbols, extract and repack libgcc to only include
required objects.

Bug: 142585047
Test: presubmit
Change-Id: I58af74c18838f797e481da38c3265f0624fddf99
2019-10-24 16:34:54 -07:00
Yi Kong 049ae6ae3e Revert "Create libgcc_stripped with llvm-objcopy"
We rely on a weird, GNU specific behaviour of objcopy:

$ aarch64-linux-android-objcopy -w libgcc.a stripped.a --strip-unneeded-symbol=*
$ llvm-nm --defined-only stripped.a | grep multi
0000000000000000 T __multi3

$ aarch64-linux-android-objcopy -w libgcc.a stripped.a --strip-unneeded-symbol=* --keep-symbol=FOO
$ llvm-nm --defined-only stripped.a | grep multi

llvm-objcopy has a different, more consistent behaviour that does not
mark exported symbols as unneeded when "--keep-symbol" flag exists.
However this behaviour will leave most symbols not deleted in the
libgcc_stripped library.

Revert the change while I'm implementing an alternative solution.

Test: presubmit
Bug: 142585047
Change-Id: I175e811f8a1f6afc6558267fc54e159ad5e12acf
2019-10-15 16:43:12 -07:00
Yi Kong eb63f76721 Move strip.sh to use llvm-nm instead of GNU nm
Also fix an issue with deps file introduced in ab5e5142. Now correctly
depends on llvm-readelf insteaf of GNU readelf.

Test: presubmit
Change-Id: Id287e2844626c07917348a62820f8b5edbb22c3d
2019-09-14 01:39:53 -07:00
Yi Kong ab5e514208 Remove use_gnu_strip option
This option is no longer used anywhere.

Test: presubmit
Change-Id: I1a5f2d14e24ecf7c15a437054b8a95ddaf57da2f
2019-09-13 12:18:47 -07:00
Treehugger Robot f56e89f160 Merge "strip.sh: Fix argument to llvm-strip" 2019-09-09 22:06:56 +00:00
Greg Kaiser e106d8fd8c strip.sh: Fix argument to llvm-strip
We had a typo in the original checkin.  We need two hypens before
"remove-section" for llvm-strip, just like we have for strip.

Bug: 140721128
Test: Clean build of Go device
Change-Id: Ifb83dc4e6ad8f8070d9ced678f9147d6a040aee7
2019-09-09 09:55:07 -07:00
Yi Kong a0119f8754 Create libgcc_stripped with llvm-objcopy
llvm-objcopy now implements '--regex' option, which allows it to replace
GNU objcopy for creating libgcc_stripped archieve.

Test: build
Bug: 131631155
Change-Id: Ie1733ac8e9b3765f7652a562406dcedb4735cc1a
2019-09-03 13:44:54 -07:00
Inseob Kim 4288274f14 Implement sysprop_library API stability check
sysprop_library now checks the API stability itself, cutting dependency
on java_sdk_library. Under the directory {module_dir}/api,
{module_name}-current.txt and {module_name}-latest.txt hold API
signatures.

When sysprop_library is built, or a user run "m {module_name}-check-api"
command, API check is performed. First, current.txt must have exactly
same signature with built sysprop_library module. Second, current.txt
must be compatible with latest.txt.

Build system emits a handy error message to generate/update those API
files, in case of missing or mismatching. Also, a script file for
freezing API files is introduced.

Bug: 131637873
Test: 1) m && boot blueline
Test: 2) m {sysprop_library} performs API check
Test: 3) manual test for check-api, freezing api
Change-Id: I9d25f5dc64299e666527ca8e23d7233966901c4e
Merged-In: I9d25f5dc64299e666527ca8e23d7233966901c4e
Merged-In: Ib7ad4f17e82c90da5ef3f80e2ab88c0b53112c60
(cherry picked from commit 093f0eb133)
2019-08-23 06:28:34 +00:00
Jooyung Han 04329f131a Add jsonmodify tool
This tool is used to modify apex_manifest.json when building apex
module.

Here's the usage:

usage: jsonmodify [-h] [-o output] [-v path value] [-s path value]
                     [-r path] [-a path [value ...]]
                     [input]

positional arguments:
  input                 JSON file

optional arguments:
  -h, --help            show this help message and exit
  -o output, --out output
                        write result to a file. If omitted, print to stdout
  -v path value, --value path value
                        set value of the key specified by path. If path
                        doesn't exist, creates new one.
  -s path value, --replace path value
                        replace value of the key specified by path. If path
                        doesn't exist, no op.
  -r path, --remove path
                        remove the key specified by path. If path doesn't
                        exist, no op.
  -a path [value ...], --append_list path [value ...]
                        append values to the list specified by path. If path
                        doesn't exist, creates new list for it.

Bug: 138695532
Test: m jsonmodify
Test: echo {} | jsonmodify -v name hello -a list.nested a b c
{
  "name": "hello",
  "list": {
    "nested": [
      "a",
      "b",
      "c"
    ]
  }
}

Change-Id: I2cd043c614b3ad2306a0c27ccee302633c6d2525
2019-08-02 00:08:05 +09:00
Vic Yang 6cd1be8993 Add sort_bss_symbols_by_size property for shared libs
If sort_bss_symbols_by_size is true, a shared library is built twice.
The first build generates an unsorted output file, which is used to
generate the symbol ordering file.  The output of the second build is
a shared library with its bss symbols sorted by their size.

With this, the only user of symbol_ordering_file, libc, is migrated to
use the new property, so we remove symbol_ordering_file support as well.

Bug: 135754984
Test: Build and check the resulting libc.so has its bss symbols sorted.
Change-Id: I5c892b44d82eb99cbc070cfa2c680be3087f3364
2019-07-29 13:25:55 -07:00
Yi Kong 93e6783410 Use double dashes before llvm-strip arguments
Newer llvm-strip no longer accepts single dash for multi-character
arguments.

Test: TreeHugger
Bug: 131328001
Change-Id: I4ca862d480860c00e7f8f644f79e300cb523dad0
2019-07-19 16:41:51 +00:00
Colin Cross e5cae33f5d Allow jar wrapper to take quoted arguments
Make the jar wrapper script correctly handle quoted arguments with
spaces in them.  Also allow JVM arguments in the form -J-XX, not just
-JXX.

Test: m checkbuild
Change-Id: Iec5105bc390f2a12c6a4cda7f76d37585c39f520
2019-07-16 11:12:04 -07:00
Tom Cherry 0dd3ab5387 system-clang-format: Add Standard: Cpp11
clang-format tries to automatically detect the Cpp standard used,
particularly when determining to put spaces in template definitions:

Cpp03: std::vector<std::pair<bool, bool> >
vs
Cpp11: std::vector<std::pair<bool, bool>>

It doesn't always get this correct as seen in
https://android-review.googlesource.com/c/platform/system/core/+/973463/8
but since we know that all code within Android is C++17, we can safely
assume it should be formatted with the newer standard.

Test: clang-format doesn't break the above change.
Change-Id: I7d1d709690b7bca6da3863cc9a58e53eaec751b0
2019-07-10 15:04:04 -07:00
Jaewoong Jung 96eac767df Merge "Touch up manifest if there's no source code." 2019-06-06 13:52:37 +00:00
Tom Cherry c077eefc1e system-clang-format: Add IncludeBlocks: Preserve
As of [1], the Google format, which we default to, uses IncludeBlocks:
Regroup, which sorts all includes with <> and without .h into the top
group reserved for libc headers, e.g.:

+#include <android-base/properties.h>
+#include <android-base/strings.h>
 #include <ctype.h>
 #include <stdlib.h>
 #include <sys/system_properties.h>

 #include <iostream>

-#include <android-base/properties.h>
-#include <android-base/strings.h>

This is accurate to the Google C++ style guide, which states that
other library headers should be included with "", however we have a
strong tendency to use <> for these headers for Android, so we must
revert this change.

Note, clang-format allows regexes for creating the blocks as well,
however, short of including all possible libc headers, I don't think
there's a way to differentiate between <sys/wait.h> and
<other_lib/header.h>.

[1]:
62e3198c4f (diff-b689ebff2e51e675b346a88289e74542)

Test: clang-format doesn't do the above.
Change-Id: Ifef605ffd6b4c8a36512377644e64ddd26b68a21
2019-06-05 15:30:14 -07:00
Jaewoong Jung c27ab6678b Touch up manifest if there's no source code.
The new package manager behavior requires packages without source code
to have an application element with hasCode attribute set to false in
their manifest. With this change, Soong can now automatically insert one
for codeless apps.

Test: app_test.go, manifest_fixer_test.py
Fixes: 124375490
Change-Id: Ied89a8d07c63805ab910859a4f7c45fc1c60bb73
2019-06-03 10:51:42 -07:00
Treehugger Robot 20272e1480 Merge "Remove BUILD_BROKEN_ANDROIDMK_EXPORTS" 2019-05-29 01:17:01 +00:00
Colin Cross ff3f43857f Merge changes If15abf79,Iaae177ef
* changes:
  Add manifest_check tool
  Move manifest_fixer to a python_binary_host module
2019-05-28 16:26:05 +00:00
Dan Willemsen edff034366 Remove BUILD_BROKEN_ANDROIDMK_EXPORTS
Bug: 73959648
Test: treehugger
Change-Id: I932cc3acc6345f8b1ad8e395235489ae8275e2db
2019-05-25 17:13:55 -07:00
Dan Willemsen 7101d44442 Remove BUILD_BROKEN_ENG_DEBUG_TAGS
All users have been removed.

Test: treehugger
Change-Id: I35e55f3ce79c8e68585be120b5dec810563c7a83
Merged-In: I35e55f3ce79c8e68585be120b5dec810563c7a83
2019-05-25 22:48:12 +00:00
Colin Cross 7211910fd0 Add manifest_check tool
Add a tool that can check that the <uses-library> tags in an
AndroidManifest.xml file match a list provided by the build.

Bug: 132357300
Test: manifest_check_test
Change-Id: If15abf792282bef677469595e80f19923b87ab62
2019-05-24 13:43:27 -07:00
Colin Cross 4af387c20e Move manifest_fixer to a python_binary_host module
In preparation for making it use multiple files so that some
implementation can be shared with a new tool.

Also make manifest_fixer_test a python_test_host, and add it
to TEST_MAPPING.

Test: m checkbuild
Change-Id: Iaae177efcb978187b126d90359b82cdfd3176857
2019-05-24 13:43:19 -07:00
Christopher Ferris b43fe7a838 Add an option to preserve symbols and debug_frame.
New strip option named keep_symbols_and_debug_frame, that will keep the
symbols and the .debug_frame. This is meant for use by libc.so only on
arm32. Other libraries might want to use it to keep better unwinding
information on device.

Bug: 132992102

Test: Built libc.so with this option and verified that it contains
Test: the .debug_frame section.
Change-Id: I823a28199dec8316e8b26fe31ff9f17e6b11d406
2019-05-17 17:52:18 -07:00
Jaewoong Jung e4948c79d3 Fix manifest_fixer_test.py
Test: manifest_fixer_test.py
Change-Id: I18067070154578e2effad8c70bb07a414cbc6dc0
2019-05-14 08:07:27 -07:00
Colin Cross 6c6e6cd64d Don't hide *.kotlin_module in turbine dependencies
The jars containing the merged header classes from transitive
static dependencies were stripping all of META-INF/.  Kotlin
puts metadata in META-INF that is used to map the package to
the static class that contains the package-level functions.

This also exposed that the metadata in META-INF is always
called "name.kotlin_module", so a library that contains
kotlin files with a dependency that also contains kotlin files
would shade the metadata file.  Use a unique name instead.

Fixes: 131709692
Test: m checkbuild
Change-Id: I5cd276e563206e37c4c0d90fe9f346e9396f88c0
2019-05-08 14:30:12 -07:00
Yi Kong acee27cd72 Strip libgcc to only keep fallback symbols
We use libgcc as fallback for symbols not present in libclang_rt
builtins, however we didn't know what exact symbols were being used,
some may not be intended to fallback.

Create libgcc_stripped, which only contains unwind symbols from libgcc.

Bug: 29275768
Test: bionic-unit-tests
Change-Id: I5b349fa6138e51663bf3b67109b880b4356da8e8
2019-04-30 10:07:00 -07:00
Dan Willemsen 2fbb7571bf Obsolete BUILD_BROKEN_DUP_COPY_HEADERS
There are no remaining users.

Test: treehugger
Change-Id: If8719480f73a8fa009d9733479390d2abd4021b1
2019-04-19 11:22:27 -07:00
Dan Willemsen 6097746769 Obsolete BUILD_BROKEN_PHONY_TARGETS
There are no remaining users.

Also mark BUILD_BROKEN_ANDROIDMK_EXPORTS and BUILD_BROKEN_ENG_DEBUG_TAGS
as deprecated in scripts/build_broken_logs.go

Test: treehugger
Change-Id: If7892bef1b9001f12a99565f886b395cf1985e70
2019-04-19 11:16:52 -07:00
Colin Cross 9dc186f120 Merge "Add robolectric-shadows to setup_go_workspace_for_soong.sh" 2019-04-10 20:28:56 +00:00
Dan Willemsen 25e6f09c06 Add BUILD_BROKEN_USES_NETWORK
Some people apparently still talk to the network during their build.
Allow this temporarily with a BUILD_BROKEN_USES_NETWORK check.

Bug: 129992021
Test: attempt to talk to the network during the build with and without
      this flag.
Change-Id: Ifb967c656aa24c4599e7232d0f1b5a303b5bac52
2019-04-09 10:23:49 -07:00
Vladimir Marko 0975ee0de3 Check package restrictions for Java libs.
Test: m checkbuild; inspect verbose log.
Test: Manual - compile with unmet restrictions.
Bug: 122937705
Change-Id: I9360ae8b6d9ce016b7827be5e8ffc6eb521809b7
2019-04-08 14:02:14 +01:00
Colin Cross 5e5b2c7f1c Add robolectric-shadows to setup_go_workspace_for_soong.sh
Bug: 122331577
Test: setup_go_workspace_for_soong.sh
Change-Id: I173174927f9d107c4a196968b568ace6473394c7
2019-04-08 05:46:07 +00:00
George Burgess IV c4624c0189 Remove our clang-tidy wrapper
Looks like the reason for it existing has been fixed. It should probably
just be removed.

Bug: None
Test: WITH_TIDY=1 m

Change-Id: I770b2fec4ac44f265ff31731c9c0bd4da14d5b0f
2019-04-04 17:58:00 -07:00
Patrice Arruda b4e93fbdc2 Soong: Add unbind option to setup_go_workspace_for_soong.sh script
The setup_go_workspace_for_soong now allows the reverse operation
of unbinding the directories of the previous operation. Also,
refactored the script to easily add more directories to bind/unbind
in the future, catch failures on running in subshell commands and
a cleaner way to find the repo top directory.

Bug: b/129407866
Test: Manually tested by running the script on binding and unbinding
the directories. Tested both darwin and linux OS.

Change-Id: I7c93230aeab819ab5747e990e95aa26077071d9e
2019-03-29 15:57:47 -07:00
Treehugger Robot 21c81326ff Merge "Notice support for APEX" 2019-03-21 06:04:14 +00:00
Vic Yang efd249e62a Add support for no-vendor-variant VNDK
When no-vendor-variant VNDK is enabled, the vendor variant of VNDK
libraries are not installed.  Since not all VNDK libraries will be
ready for this, we keep a list of library names in cc/vndk.go to
indicate which libraries must have their vendor variants always
installed regardless of whether no-vendor-variant VNDK is enabled.

Also add --remove-build-id option to the strip script to facilitate
the check of functional identity of the two variants.

Bug: 119423884
Test: Add a dummy VNDK library and build with
      TARGET_VNDK_USE_CORE_VARIANT := true, with the corresponding
      build/make change.

Change-Id: Ieb1589488690e1cef1e310669a8b47a8b8759dac
2019-03-21 04:29:24 +00:00
Jiyong Park 52818fcde8 Notice support for APEX
Notice file for an APEX is created by merging notice files for the
modules included in it (plus the notice file for the APEX itself if
specified).

Notice files having the same content are not duplicated; it is emitted
only once.

Bug: 128701495
Test: m (apex_test is amended)
Test: m and inspect $(PRODUCT_OUT)/obj/NOTICE.txt to check there are
license entries for /system/apex/*.apex files

Change-Id: I169d91038291a6c71615de97cf5b03174afab5d4
2019-03-21 08:05:50 +09:00
Inseob Kim 8098faad9f Refine error messages in case of missing api files
Some non-interpretable error messages has been shown in case of missing
current api files. Instead of that messy errors, This clearly tells
users what is the problem and how to fix it.

Bug: 126259114
Test: 1) try to build a sysprop_library and a java_sdk_library without
creating txt files under api/, and see it fails.
Test: 2) Copy and paste the emitted command, and try to build again.

Change-Id: I986781cc710475647c2b996b0f1c51a6f50c0ba2
2019-03-20 15:33:06 +09:00
Colin Cross e4246abd7f Make manifest and APK agree on uncompressed native libs
Only put uncompressed native libs in an APK if the min_sdk_version
supports it (>= 23, Marshmallow), and set
android:extractNativeLibs="false" in the AndroidManifest.xml so
that the platform won't extract them anyways.

Bug: 117618214
Test: m checkbuild
Change-Id: I760017e48bf3c6b618aabde0982df45995765d48
2019-02-08 15:24:47 +00:00
Dan Willemsen 01f0a05361 Add BUILD_BROKEN_ENG_DEBUG_TAGS to soong.log
And add a helper script that can help parse the output from
build_test.bash

Test: check for BUILD_BROKEN_ENG_DEBUG_TAGS in soong.log
Test: go run ../build/soong/scripts/build_broken_logs.go *
Change-Id: Idd0fc8b59770dcdbe44eeba262558708a9497f96
2019-02-05 15:15:16 -08:00
Yi Kong b5c34d7f40 Use llvm-{strip,objcopy} by default
... except for Darwin Mach-O, as it is not supported by llvm-strip.

Test: m checkbuild
Bug: 119221035
Change-Id: I021637b6dd3530bff1f563f2ec7c2168e1083b7e
2019-02-04 18:20:28 +08:00
Pirama Arumuga Nainar 03b58e2210 Switch to clang-r349610
Bug: http://b/122481018

The '-keep' option of llvm-strip has changed to '-keep-section'.  Update
scripts/strip.sh accordingly.

Test: Tested on various targets in AOSP and internal branch.  Googlers
      see http://go/clang-r349610-testing
Change-Id: Ibfc97ecdcdc305c97a4169cead1e1e3cddda9c4c
2019-01-31 16:32:34 +00:00
Treehugger Robot 05f9055505 Merge "Rename preferCodeIntegrity to useEmbeddedDex" 2019-01-30 05:07:22 +00:00
Tom Cherry eca62454e8 Revert "system-clang-format: DontAlign"
"AlignAfterOpenBracket: DontAlign" is explicitly wrong and was not
common before.

Wrong:

  int fs_mgr_do_mount(Fstab* fstab, const char* n_name, char* n_blk_device, char* tmp_mount_point,
          bool needs_checkpoint) {

Right:

  int fs_mgr_do_mount(Fstab* fstab, const char* n_name, char* n_blk_device, char* tmp_mount_point,
                      bool needs_checkpoint) {


This reverts commit a4319f7800.
Test: clang-format works correctly.
Bug: 122835115

Change-Id: Icd988070f208e1963bb68651c63fa369cb1e2782
2019-01-30 00:29:23 +00:00
Victor Hsieh d181c8ba76 Rename preferCodeIntegrity to useEmbeddedDex
Test: build and run testing app
Bug: 112037137
Change-Id: Ia82c2c3ba7eb32117a4be078ac31ee2ba510f9eb
2019-01-29 13:09:10 -08:00
Steven Moreland a4319f7800 system-clang-format: DontAlign
AlignAfterOpenBracket: DontAlign

The main consideration is this:

if (foo && ....
    foo) {
    // code

->

if (foo && ....
        foo) {
    // code

Change-Id: Ife0073c8544b9554df2eb54e10dc2e7611631299
Fixes: 122835115
Test: manual
2019-01-28 18:21:15 -08:00
Steven Moreland 1aa839c275 Move system-clang-format to a central location.
So that system/core clang-format can be used in more places.

Bug: 122835115
Test: manual
Change-Id: I233a5003c4229126f37a89e90eac4a75dc5ad104
2019-01-22 13:27:45 -08:00
Victor Hsieh a2c16c1aa0 manifest_fixer: rename to --prefer-code-integrity
During code review, the name change was suggested.

Test: local CTS passed
Bug: 112037137
Change-Id: I7eb25210afb45c7477b0d606574048a15c9c721d
2019-01-03 09:51:11 -08:00
Dan Albert 23d37e09e9 Allow NDK APIs to be marked as drafts.
Draft APIs are available to the platform and to CTS to allow
developers to iterate on an API, but hidden from the NDK artifacts to
avoid releasing the API until it is ready.

Test: Mark binder_ndk headers and library as drafts, make checkbuild,
      build-ndk-prebuilts.sh, verify missing from NDK artifact.
Bug: http://b/120091134
Change-Id: I8685e92bdaaea581e17fe98e7a2bfb9388f9f132
2018-11-28 09:52:12 -08:00
Victor Hsieh f9919c7d52 manifest_fixer: make --prefer-integrity an enabler
The last change should have changed this from taking a boolean to a
simple switch.

Test: build a package with LOCAL_PREFER_INTEGRITY=true
Bug: 112037137
Change-Id: I5ce3b1455404b5c8e781496d26c0431db52a170f
2018-11-19 19:30:16 +00:00
Dan Willemsen 7481487d24 build-ndk-prebuilts: Remove unused device config
These are all overwritten by setting Ndk_abis.

Bug: 73545680
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I1c922f5cd1611ed055653d449709fec69dca410b
2018-10-30 21:11:42 -07:00
Colin Cross 395f2cfa89 Move Ndk_abis from soong.config to soong.variables
soong.config is not cleared between builds, which can cause
problems when switching between an ndk build and a platform build.

Bug: 118398924
Test: OUT_DIR=out_ndk DIST_DIR=dist build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: Ifc03a0c25f46625375cdda5723e4a09d7d8050d5
2018-10-24 23:20:10 +00:00
Victor Hsieh ce7818ed6e Add --prefer-integrity option to manifest_fixer.py
If provided (--prefer-integrity=true/false), the script will set the
value in the the manifest.  The script will fail if the value mismatches
the original in the manifest, if any.

Test: scripts/manifest_fixer_test.py
Test: aapt dumps the attribute and observe
Bug: 112037137
Change-Id: I2b333a7c0747dbcbed4d419f1c9ed46d4a4c98e9
2018-10-24 10:32:10 -07:00
Colin Cross 3adb2ed734 Remove strip.sh intermediates
The intermediate files generated by strip.sh --keep-symbols can be
very large, and are not useful after the build.  Remove them once
the final output file has been generated.

Saves ~6GB on an aosp_sailfish-userdebug build.

Test: m checkbuild
Change-Id: I0413f16ac5f423bc1b010cc9b8538f19bdea561e
2018-10-18 18:07:44 +00:00
Jiyong Park fa17afef53 android.test.{base|mock} are added as optional libraries.
The two java shared libraries do not exist in the pre-P devices.
Therefore, marking the dependencies to the libraries as optional so that
an app is still installable to the old devices.

This is safe because the classes in the java shared libs exist in other
shared libraries (bootclasspath or android.test.runner).

Bug: 113800422
Test: run aapt dump badging on the built GtsBackupHostTestCases

uses-library tags for android.test.base|mock are with
android:required=false, while the same tag for other libs are without
the attribute.

Change-Id: I266c8efb9686e8975be37f31aea0dc22f82370c3
2018-10-16 18:05:49 +09:00
Paul Duffin d616b6719f Make automatically added <uses-library> entries optional
Test: run aapt dump badging on build apk
Bug: 113800422
Change-Id: I8cdfc3bea0c23e93f44b5707414a7a5a1e3af1c0
2018-10-11 14:32:50 +01:00
Dan Willemsen feea4dff66 Stop using GCC in toolchain_library
Instead, hardcode the ~dozen paths into build/soong/Android.bp, which
will unblock removing more GCC support.

Bug: 114286031
Test: m
Change-Id: I2508432e00b1469141f01e667f3c6a2fe30cd805
2018-10-07 23:50:34 -07:00
Colin Cross 4b176069c3 Don't set targetSdkVersion to '1' for libraries
Setting targetSdkVersion to '1' causes ManifestMerger to add
implicit permissions when merging to a higher targetSdkVersion.
It should really be unset, but ManifestMerger treats unset
targetSdkVersion as 'Q' if minSdkVersion is 'Q' (but not if
minSdkVersion is '28').  Set it to something low so that it
will be overriden by the main manifest, but high enough that
it doesn't cause implicit permissions grants.

Bug: 115415671
Bug: 117122200
Test: m checkbuild
Change-Id: I1d2d031a21314f6b55d8ea1cc7c4c8e3ecae7f06
2018-10-02 12:39:38 -07:00
Stephen Hines c91ab9eb6d Switch to clang-r339409.
This also changes over llvm-strip to use the -o flag, rather than
relying on nonstandard (and no longer functional) positional arguments.

Bug: http://b/111998531
Test: Multiple new builds. For googlers - http://go/clang-r339409-testing
Change-Id: I8c82d06304af24a001e6552e36e2f540d8e364ce
2018-09-26 21:15:08 +00:00
Colin Cross 8e877af5f4 Hide warning on static executables in strip.sh
Stripping static executables prints a warning:
bash: prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/arm-linux-androideabi-nm: No such file or directory

The message is harmless, just hide it.

Also turn on pipefail so that the script will exit if the command
returns an error.

Bug: 24409581
Test: m checkbuild
Change-Id: I9647c55c3509e4573e6d4f2f6d88119bdf31e9fb
2018-09-19 23:23:51 +00:00
Colin Cross b496cfd9d2 Enable toc support for Darwin and Windows
Bug: 113936524
Test: m checkbuild
Change-Id: I0f2030ad75daae2cbe44e8cbedad329d33df55f7
2018-09-10 23:12:11 -07:00
Colin Cross 7b59e7b2d0 Make manifest_fixer.py keep targetSdkVersion high with min_sdk_version
targetSdkVersion should stay as "current" when min_sdk_version
is set but sdk_version is not.

Bug: 112438448
Test: atest CtsTelecomTestCases
Change-Id: I11dc27eccd31200d1ce27c7e332106bb825651b4
2018-09-10 14:41:56 -07:00
Colin Cross 496d66d425 Fix manifest_fixer_test.py
Fix tests after I8fcf0c5f452707565ba1808f6fe552ffed055c47.

Test: manifest_fixer_test.py
Change-Id: Ie55e28b537b960a5e0f4785eda2c0f619b89275c
2018-09-10 14:41:56 -07:00
Colin Cross 0abcbe6a6d Fix inverted logic for objcopy vs. llvm-objcopy with --keep-symbols
Bug: 113936524
Test: m checkbuild
Change-Id: I958c4277342ed6885714a01bd9f35183ed73eda7
2018-09-07 17:36:40 -07:00
Colin Cross 02b04bb299 Allow strip.sh --keep-mini-debug-info to work on non-elf files
strip.sh --keep-mini-debug-info matches the default for Make, and
is currently used on lots of files that are not elf files.  For
now, just make the behavior match Make and skip files when
strip --strip-all fails.

Bug: 113936524
Test: m checkbuild
Change-Id: I3648c914c0fd7cc9b68aa93fd3cb0b77504d42f6
2018-09-05 16:30:16 -07:00
Colin Cross f843290eb8 Fix strip.sh --keep-mini-debug-info
strip.sh --keep-mini-debug-info had extra $$ escaping from when it
was copied from make.

Bug: 113936524
Test: m checkbuild
Change-Id: I47dec958152584ca94c6149b11a06e64be2f22f9
2018-09-05 16:30:16 -07:00
David Brazdil 9b106574fa Merge "Support setting android:usesNonSdkApi in manifest_fixer.py" 2018-08-31 09:59:39 +00:00
Dan Willemsen e97e68a05a Clean up darwin cc toolchain
Remove the ability for Soong to build 32-bit darwin code. We've already
disabled this in Make, this is just removing the unused bits in Soong
and simplifing the toolchain config.

Test: m host
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I968c5d98bdf162297d639f7988918dadb7ba6e07
2018-08-28 20:08:50 -07:00
David Brazdil d5b74996be Support setting android:usesNonSdkApi in manifest_fixer.py
Add new command line flag to manifest_fixer.py which will add
'android:usesNonSdkApi="true"' attribute to the <application> tag.

Bug: 113315999
Test: build/soong/scripts/manifest_fixer.py
Change-Id: If030c90a4ced3f5c5176727c579a87d0ecab6cf8
2018-08-28 12:41:01 +01:00
Chih-Hung Hsieh a7aa958e6e Add clang-tidy.sh to filter out troublesome flags.
* Some flags are accepted by clang but not clang-tidy.
  They could cause the diagnostic-unused-command-line-argument warning.
  Flag -flto messed up the -I flags.
* Add clang-diagnostic-unused-command-line-argument to
  default clang-tidy checks.
* Move CLANG_TIDY_UNKNOWN_CFLAGS to build/make/core/clang/tidy.mk.

Bug: 111850071
Bug: 111885396
Test: build with WITH_TIDY=1
Change-Id: Iabeeb27715acf83ef6aafe3e77206b9a01a0d889
2018-08-22 09:21:45 -07:00
Colin Cross 1b6a3cfea4 Set targetSdkVersion in manifest_fixer
If targetSdkVersion is not set in the manifest, set it to the
value it was implicitly using before changing minSdkVersion.
Requires passing --library to manifest_fixer.py to distinguish
between apps, where the implicit value was set by aapt2 to
current, or libraries where the implicit value was 1.
Fixes cases where the manifest does not specify targetSdkVersion
and was inheriting the minSdkVersion value until manifest merger
started merging a lower targetSdkVersion value from a library.

Bug: 111347801
Test: manifest_fixer_test.py
Change-Id: I8fcf0c5f452707565ba1808f6fe552ffed055c47
2018-07-26 11:09:24 -07:00
Dan Willemsen b82471ad6d Add a unified status reporting UI
This adds a new status package that merges the running of "actions"
(ninja calls them edges) of multiple tools into one view of the current
state, and gives that to a number of different outputs.

For inputs:

Kati's output parser has been rewritten (and moved) to map onto the
StartAction/FinishAction API. A byproduct of this is that the build
servers should be able to extract errors from Kati better, since they
look like the errors that Ninja used to write.

Ninja is no longer directly connected to the terminal, but its output is
read via the protobuf frontend API, so it's just another tool whose
output becomes merged together.

multiproduct_kati loses its custom status routines, and uses the common
one instead.

For outputs:

The primary output is the ui/terminal.Status type, which along with
ui/terminal.Writer now controls everything about the terminal output.
Today, this doesn't really change any behaviors, but having all terminal
output going through here allows a more complicated (multi-line / full
window) status display in the future.

The tracer acts as an output of the status package, tracing all the
action start / finish events. This replaces reading the .ninja_log file,
so it now properly handles multiple output files from a single action.

A new rotated log file (out/error.log, or out/dist/logs/error.log) just
contains a description of all of the errors that happened during the
current build.

Another new compressed and rotated log file (out/verbose.log.gz, or
out/dist/logs/verbose.log.gz) contains the full verbose (showcommands)
log of every execution run by the build. Since this is now written on
every build, the showcommands argument is now ignored -- if you want to
get the commands run, look at the log file after the build.

Test: m
Test: <built-in tests>
Test: NINJA_ARGS="-t list" m
Test: check the build.trace.gz
Test: check the new log files
Change-Id: If1d8994890d43ef68f65aa10ddd8e6e06dc7013a
2018-07-12 14:15:31 -07:00
Peter Kalauskas 39b0deb0d7 Fix error in build when OUT_DIR_COMMON_BASE is used
Without this change, gen-kotlin-build-file.sh will generate
kotlinc-build.xml file with path that look like:

/path/to/aosp-top//path/to/aosp-out/soong/.intermediates/...

Also fixed a couple lint errors.

Test: Set OUT_DIR_COMMON_BASE and was able to build
Bug: 111309264
Change-Id: I86173571667deca79ee0d9a7324715c9a0e4cd97
2018-07-10 12:42:22 -07:00
Jiyong Park c08f46fdfc add --uses-library option to manifest_fixer.py
`--uses-library foo` adds <uses-library android:name="foo"
android:required="true"/> element to the manifest.

When there is already a uses-library element with the same
name, then the hand-written element is respected.

Test: m -j
Test: build/soong/scripts/manifest_fixer_test.py

Change-Id: Ia5e989d3c3c3fba7cc9015a6d16dac70464282a8
2018-06-20 12:03:09 +09:00
Colin Cross 8bb10e8f8a Add a script to inject values into manifests
Add a script that can inject a <uses-sdk minSdkVersion=""> into
AndroidManifest.xml files.  This will help with merging
LOCAL_STATIC_ANDROID_LIBRARIES, because ManifestMerger treats
a missing minSdkVersion as minSdkVersion=1 and throws errors
if libraries use a larger minSdkVersion.  It will also help
with cases where an app has a manifest that specifies an old
minSdkVersion, but the build system is compiling the app in
a way that is not compatibile with old devices, for example
using a newer dex format.

Bug: 110167203
Test: m java
Test: build/soong/scripts/manifest_fixer_test.py
Change-Id: I528d71a225feb86464c530e11b223babb0ea9edf
2018-06-14 14:54:27 -07:00
Chih-Hung Hsieh 30485c920c Use llvm-{objcopy,strip} when clang lld is used.
* Pass LLVM_{OBJCOPY,STRIP} to makefile rules.
* strip.sh is used only in soong-only mode.
  It will use llvm-strip and  and llvm-objcopy
  when --use-llvm-strip is given.
  Keep flags of strip.sh in alphabetic order.
* Tested build/soong/scripts/build-ndk-prebuilts.sh
  by adding "UseClangLld": true, to soong.variables file.

Bug: 80093681
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I612267304eea434c7a33cc086b27b577d5f64094
2018-06-07 16:04:14 -07:00
Dan Willemsen fdb20d05bc Fix mac sdk build
Use the host platform "strip", and also expose "nm" and "otool" used for
toc.

Test: `m PRODUCT-sdk-sdk` on Mac
Change-Id: I7a6d127f479d3e563385ead46bc3a4b49b99776b
2018-06-03 16:19:56 -07:00
Elliott Hughes 314fddbfc6 Flounder's been dead so long it smells.
(And confuses people when it shows up in NDK build failures in 2018.)

Bug: N/A
Test: builds
Change-Id: Ie055b17d1c4bd7c8b73f10adcc7f7b784f9e9c91
2018-05-10 13:55:30 -07:00
Ryan Prichard afefacf196 Fix mini_debug_info stripping when no symbols should be kept.
If there are no symbols to keep (i.e. all function symbols are also in
the dynsym table), then the "${outfile}.keep_symbols" file has size 0.
When objcopy parses a --keep-symbols file, it has a special case for
0-sized files where it silently fails (exits with status 1, no error
message). On the other hand, objcopy is happy with a file containing no
symbols, as long as there is some whitespace or a comment. Avoid the
special case by appending a newline to keep_symbols.

Bug: b/62815515
Bug: b/77242617
Test: manual
Change-Id: I90fd3258426176dc18aa5a8c19389c55fe6329c7
2018-03-29 01:39:13 +00:00
Colin Cross 436b76564d Replace extract-srcjars.sh with zipsync tool
extract_srcjars.sh uses zipinfo and unzip, which fail with an
error on an empty zip file.  Instead of trying to hack around
this (which is hard to make guarantees for since they are
non-hermetic host tools), replace them with a go tool to unzip
a set of zip files into a directory.

Bug: 73885582
Test: m checkbuild
Change-Id: I151fed347ed5196726e36866ffc27bc831799afb
2018-03-15 16:42:43 -07:00
Dan Willemsen 8fec83a8b9 Use xz prebuilt
Instead of whichever xz happens to be on the PATH.

Bug: 36130900
Test: prebuilts/build-tools/build-prebuilts.sh
Change-Id: Ic9b66062ca7947f0eca9d839c0864da492d4ef71
2018-03-09 10:53:26 -08:00
Przemyslaw Szczepaniak 4b5fe9d1b4 Add rsp and srcjar support to kotlinc build rule
Rsp files are supported through helper script (gen-kotlin-build-file.sh)
that generates the kotlinc module/build xml file.

Since rsp files are supported, I've added ExtractSrcJarsCmd step
to handle srcjars extraction.

Minor reorderings to make sure that TransformKotlinToClasses
recives only .java and .kt files when called from Module.compile.

Bug: 73281388
Test: make -j hidl-doc
Change-Id: I5a40b914569018dc529903a7f2864a5aeae838e5
2018-02-14 20:47:17 +00:00
Stephen Hines 6ea0f81c3f Remove unnecessary DeviceUsesClang.
This build function is unnecessary now that GCC is completely
unsupported for Android platform builds. It is similar to
USE_CLANG_PLATFORM_BUILD from the build/make side of things.

Bug: http://b/64032869
Test: Builds
Change-Id: Iddf5f91cc997c337c77a644265cb8dc4e5a915b4
2018-01-11 14:29:41 -08:00
Treehugger Robot 7c25d88e6b Merge "Have diff_build_graphs.sh print the number of lines in the diff" 2017-12-05 02:04:29 +00:00
Dan Willemsen 932ecc1a97 Make setup_go_workspace_for_soong work on Mac
Test: build/soong/scripts/setup_go_workspace_for_soong.sh on Mac and Linux
Change-Id: Idfcad3ace6f3ff4b68a438b8f1e0d94ff35573e0
2017-12-04 14:58:00 -08:00
Jeff Gaston 0c573d383d Have diff_build_graphs.sh print the number of lines in the diff
Bug: b/69133815
Test: build/soong/diff_build_graphs.sh \
      'build/soong:work^' 'build/soong:work'

Change-Id: Iaeb66ba53e1b3d7ddd4b4551b3ac29b12ce227c5
2017-11-29 12:54:43 -08:00
Colin Cross d00350c61b Add license headers to all go and shell files
Test: none
Change-Id: I75c443e05f2b1e17fcb6823182717d2e6f5df7c4
2017-11-17 23:05:26 +00:00
Colin Cross 6dfde48ba1 Merge changes from topics "soong_logtags", "droiddoc_srcjars"
* changes:
  Fix genrules depending on Go tools
  Use logtags without merged file
  Export extract-srcjars.sh to Make
  Remove partial javastream proto support
  Add support for .srcjar files from genrules and srcs
  Fix java AIDL properties to match C/C++
  Fix proto flags in java
2017-11-17 21:32:27 +00:00
Colin Cross af05017b75 Add support for .srcjar files from genrules and srcs
Allow srcs to contain .srcjar files, which will be extracted just
before javac.  Also allow genrules and generated sources to directly
return .srcjar files.

Test: m checkbuild
Change-Id: Ie4cf60ecb9d2ec63a4c2275221544203b1383597
2017-11-17 11:22:08 -08:00
Jeff Gaston ef633e0959 Implement diff_build_graphs.sh
It runs multiproduct_kati against the pre-changed and post-changed
code and confirms that the ninja files remain unchanged

Test: ./diff_build_graphs.sh 'build/blueprint:work^' 'build/blueprint:work'
Bug: 67478260

Change-Id: Ib43f54428b7c5abbe7706f464ee16dc8d0ef6f76
2017-11-15 16:07:14 -08:00
Colin Cross d7903535ea Use relative paths in copygcclib.sh deps files
Strip $PWD/ off the beginning of the path printed by gcc.

Bug: 68719465
Test: m checkbuild
Test: prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-${TARGET_PRODUCT}.ninja -t deps | grep -E '^[[:space:]]*/' | sort | uniq
2017-11-06 12:56:34 -08:00
Nan Zhang 3c807db06e Add default jar wrapper when wrapper property is not specified
Bug: b/68779881
Test: manually copied the jar-wrapper.sh to out/soong/host/linux-x86/framework
Change-Id: Idee1e7e64c6e3c89c89a8cd9c107a38533356b6c
2017-11-03 15:19:15 -07:00
Colin Cross 8eadbf0aaf Fix source jars
Source jars were not working as designed because javac will only
compile files from the -sourcepath if there are references to them
starting from files on the command line.  Switch to extracting
the source jars into a directory and passing a list of the files
to javac.

Test: m checkbuild
Change-Id: I9f7d824f8538d081b2f5ad64ae3cbfd0e96213af
2017-10-26 01:00:46 +00:00
Colin Cross 1369cdb280 Initial support for converting jars to java9 system modules
Adds a java_system_modules module type that (when
EXPERIMENTAL_USE_OPENJDK9 is set to true) converts a list of
java library modules and prebuilt jars into system modules,
and plumbs the system modules through to the javac command
line.

Also exports the location of the system modules to make
variables, as well as the name of the default system module.

Test: TestClasspath in java_test.go, runs automatically as part of the build
Bug: 63986449
Change-Id: I27bd5d2010092422a27b69c91568e49010e02f40
2017-10-16 15:00:02 -07:00
Dan Albert 6807ba2148 Fix ndk prebuilts script.
This wasn't actually including the quotes, so it was a no-op.

Test: build/soong/scripts/build-ndk-prebuilts.sh
Bug: http://b/65113115
Change-Id: I43fda02f58b9661fc78eef9055063347baaf84b3
2017-08-30 13:55:35 -07:00