Commit Graph

187 Commits

Author SHA1 Message Date
Colin Cross 1b6a3cfea4 Set targetSdkVersion in manifest_fixer
If targetSdkVersion is not set in the manifest, set it to the
value it was implicitly using before changing minSdkVersion.
Requires passing --library to manifest_fixer.py to distinguish
between apps, where the implicit value was set by aapt2 to
current, or libraries where the implicit value was 1.
Fixes cases where the manifest does not specify targetSdkVersion
and was inheriting the minSdkVersion value until manifest merger
started merging a lower targetSdkVersion value from a library.

Bug: 111347801
Test: manifest_fixer_test.py
Change-Id: I8fcf0c5f452707565ba1808f6fe552ffed055c47
2018-07-26 11:09:24 -07:00
Dan Willemsen b82471ad6d Add a unified status reporting UI
This adds a new status package that merges the running of "actions"
(ninja calls them edges) of multiple tools into one view of the current
state, and gives that to a number of different outputs.

For inputs:

Kati's output parser has been rewritten (and moved) to map onto the
StartAction/FinishAction API. A byproduct of this is that the build
servers should be able to extract errors from Kati better, since they
look like the errors that Ninja used to write.

Ninja is no longer directly connected to the terminal, but its output is
read via the protobuf frontend API, so it's just another tool whose
output becomes merged together.

multiproduct_kati loses its custom status routines, and uses the common
one instead.

For outputs:

The primary output is the ui/terminal.Status type, which along with
ui/terminal.Writer now controls everything about the terminal output.
Today, this doesn't really change any behaviors, but having all terminal
output going through here allows a more complicated (multi-line / full
window) status display in the future.

The tracer acts as an output of the status package, tracing all the
action start / finish events. This replaces reading the .ninja_log file,
so it now properly handles multiple output files from a single action.

A new rotated log file (out/error.log, or out/dist/logs/error.log) just
contains a description of all of the errors that happened during the
current build.

Another new compressed and rotated log file (out/verbose.log.gz, or
out/dist/logs/verbose.log.gz) contains the full verbose (showcommands)
log of every execution run by the build. Since this is now written on
every build, the showcommands argument is now ignored -- if you want to
get the commands run, look at the log file after the build.

Test: m
Test: <built-in tests>
Test: NINJA_ARGS="-t list" m
Test: check the build.trace.gz
Test: check the new log files
Change-Id: If1d8994890d43ef68f65aa10ddd8e6e06dc7013a
2018-07-12 14:15:31 -07:00
Peter Kalauskas 39b0deb0d7 Fix error in build when OUT_DIR_COMMON_BASE is used
Without this change, gen-kotlin-build-file.sh will generate
kotlinc-build.xml file with path that look like:

/path/to/aosp-top//path/to/aosp-out/soong/.intermediates/...

Also fixed a couple lint errors.

Test: Set OUT_DIR_COMMON_BASE and was able to build
Bug: 111309264
Change-Id: I86173571667deca79ee0d9a7324715c9a0e4cd97
2018-07-10 12:42:22 -07:00
Jiyong Park c08f46fdfc add --uses-library option to manifest_fixer.py
`--uses-library foo` adds <uses-library android:name="foo"
android:required="true"/> element to the manifest.

When there is already a uses-library element with the same
name, then the hand-written element is respected.

Test: m -j
Test: build/soong/scripts/manifest_fixer_test.py

Change-Id: Ia5e989d3c3c3fba7cc9015a6d16dac70464282a8
2018-06-20 12:03:09 +09:00
Colin Cross 8bb10e8f8a Add a script to inject values into manifests
Add a script that can inject a <uses-sdk minSdkVersion=""> into
AndroidManifest.xml files.  This will help with merging
LOCAL_STATIC_ANDROID_LIBRARIES, because ManifestMerger treats
a missing minSdkVersion as minSdkVersion=1 and throws errors
if libraries use a larger minSdkVersion.  It will also help
with cases where an app has a manifest that specifies an old
minSdkVersion, but the build system is compiling the app in
a way that is not compatibile with old devices, for example
using a newer dex format.

Bug: 110167203
Test: m java
Test: build/soong/scripts/manifest_fixer_test.py
Change-Id: I528d71a225feb86464c530e11b223babb0ea9edf
2018-06-14 14:54:27 -07:00
Chih-Hung Hsieh 30485c920c Use llvm-{objcopy,strip} when clang lld is used.
* Pass LLVM_{OBJCOPY,STRIP} to makefile rules.
* strip.sh is used only in soong-only mode.
  It will use llvm-strip and  and llvm-objcopy
  when --use-llvm-strip is given.
  Keep flags of strip.sh in alphabetic order.
* Tested build/soong/scripts/build-ndk-prebuilts.sh
  by adding "UseClangLld": true, to soong.variables file.

Bug: 80093681
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: I612267304eea434c7a33cc086b27b577d5f64094
2018-06-07 16:04:14 -07:00
Dan Willemsen fdb20d05bc Fix mac sdk build
Use the host platform "strip", and also expose "nm" and "otool" used for
toc.

Test: `m PRODUCT-sdk-sdk` on Mac
Change-Id: I7a6d127f479d3e563385ead46bc3a4b49b99776b
2018-06-03 16:19:56 -07:00
Elliott Hughes 314fddbfc6 Flounder's been dead so long it smells.
(And confuses people when it shows up in NDK build failures in 2018.)

Bug: N/A
Test: builds
Change-Id: Ie055b17d1c4bd7c8b73f10adcc7f7b784f9e9c91
2018-05-10 13:55:30 -07:00
Ryan Prichard afefacf196 Fix mini_debug_info stripping when no symbols should be kept.
If there are no symbols to keep (i.e. all function symbols are also in
the dynsym table), then the "${outfile}.keep_symbols" file has size 0.
When objcopy parses a --keep-symbols file, it has a special case for
0-sized files where it silently fails (exits with status 1, no error
message). On the other hand, objcopy is happy with a file containing no
symbols, as long as there is some whitespace or a comment. Avoid the
special case by appending a newline to keep_symbols.

Bug: b/62815515
Bug: b/77242617
Test: manual
Change-Id: I90fd3258426176dc18aa5a8c19389c55fe6329c7
2018-03-29 01:39:13 +00:00
Colin Cross 436b76564d Replace extract-srcjars.sh with zipsync tool
extract_srcjars.sh uses zipinfo and unzip, which fail with an
error on an empty zip file.  Instead of trying to hack around
this (which is hard to make guarantees for since they are
non-hermetic host tools), replace them with a go tool to unzip
a set of zip files into a directory.

Bug: 73885582
Test: m checkbuild
Change-Id: I151fed347ed5196726e36866ffc27bc831799afb
2018-03-15 16:42:43 -07:00
Dan Willemsen 8fec83a8b9 Use xz prebuilt
Instead of whichever xz happens to be on the PATH.

Bug: 36130900
Test: prebuilts/build-tools/build-prebuilts.sh
Change-Id: Ic9b66062ca7947f0eca9d839c0864da492d4ef71
2018-03-09 10:53:26 -08:00
Przemyslaw Szczepaniak 4b5fe9d1b4 Add rsp and srcjar support to kotlinc build rule
Rsp files are supported through helper script (gen-kotlin-build-file.sh)
that generates the kotlinc module/build xml file.

Since rsp files are supported, I've added ExtractSrcJarsCmd step
to handle srcjars extraction.

Minor reorderings to make sure that TransformKotlinToClasses
recives only .java and .kt files when called from Module.compile.

Bug: 73281388
Test: make -j hidl-doc
Change-Id: I5a40b914569018dc529903a7f2864a5aeae838e5
2018-02-14 20:47:17 +00:00
Stephen Hines 6ea0f81c3f Remove unnecessary DeviceUsesClang.
This build function is unnecessary now that GCC is completely
unsupported for Android platform builds. It is similar to
USE_CLANG_PLATFORM_BUILD from the build/make side of things.

Bug: http://b/64032869
Test: Builds
Change-Id: Iddf5f91cc997c337c77a644265cb8dc4e5a915b4
2018-01-11 14:29:41 -08:00
Treehugger Robot 7c25d88e6b Merge "Have diff_build_graphs.sh print the number of lines in the diff" 2017-12-05 02:04:29 +00:00
Dan Willemsen 932ecc1a97 Make setup_go_workspace_for_soong work on Mac
Test: build/soong/scripts/setup_go_workspace_for_soong.sh on Mac and Linux
Change-Id: Idfcad3ace6f3ff4b68a438b8f1e0d94ff35573e0
2017-12-04 14:58:00 -08:00
Jeff Gaston 0c573d383d Have diff_build_graphs.sh print the number of lines in the diff
Bug: b/69133815
Test: build/soong/diff_build_graphs.sh \
      'build/soong:work^' 'build/soong:work'

Change-Id: Iaeb66ba53e1b3d7ddd4b4551b3ac29b12ce227c5
2017-11-29 12:54:43 -08:00
Colin Cross d00350c61b Add license headers to all go and shell files
Test: none
Change-Id: I75c443e05f2b1e17fcb6823182717d2e6f5df7c4
2017-11-17 23:05:26 +00:00
Colin Cross 6dfde48ba1 Merge changes from topics "soong_logtags", "droiddoc_srcjars"
* changes:
  Fix genrules depending on Go tools
  Use logtags without merged file
  Export extract-srcjars.sh to Make
  Remove partial javastream proto support
  Add support for .srcjar files from genrules and srcs
  Fix java AIDL properties to match C/C++
  Fix proto flags in java
2017-11-17 21:32:27 +00:00
Colin Cross af05017b75 Add support for .srcjar files from genrules and srcs
Allow srcs to contain .srcjar files, which will be extracted just
before javac.  Also allow genrules and generated sources to directly
return .srcjar files.

Test: m checkbuild
Change-Id: Ie4cf60ecb9d2ec63a4c2275221544203b1383597
2017-11-17 11:22:08 -08:00
Jeff Gaston ef633e0959 Implement diff_build_graphs.sh
It runs multiproduct_kati against the pre-changed and post-changed
code and confirms that the ninja files remain unchanged

Test: ./diff_build_graphs.sh 'build/blueprint:work^' 'build/blueprint:work'
Bug: 67478260

Change-Id: Ib43f54428b7c5abbe7706f464ee16dc8d0ef6f76
2017-11-15 16:07:14 -08:00
Colin Cross d7903535ea Use relative paths in copygcclib.sh deps files
Strip $PWD/ off the beginning of the path printed by gcc.

Bug: 68719465
Test: m checkbuild
Test: prebuilts/build-tools/linux-x86/bin/ninja -f out/combined-${TARGET_PRODUCT}.ninja -t deps | grep -E '^[[:space:]]*/' | sort | uniq
2017-11-06 12:56:34 -08:00
Nan Zhang 3c807db06e Add default jar wrapper when wrapper property is not specified
Bug: b/68779881
Test: manually copied the jar-wrapper.sh to out/soong/host/linux-x86/framework
Change-Id: Idee1e7e64c6e3c89c89a8cd9c107a38533356b6c
2017-11-03 15:19:15 -07:00
Colin Cross 8eadbf0aaf Fix source jars
Source jars were not working as designed because javac will only
compile files from the -sourcepath if there are references to them
starting from files on the command line.  Switch to extracting
the source jars into a directory and passing a list of the files
to javac.

Test: m checkbuild
Change-Id: I9f7d824f8538d081b2f5ad64ae3cbfd0e96213af
2017-10-26 01:00:46 +00:00
Colin Cross 1369cdb280 Initial support for converting jars to java9 system modules
Adds a java_system_modules module type that (when
EXPERIMENTAL_USE_OPENJDK9 is set to true) converts a list of
java library modules and prebuilt jars into system modules,
and plumbs the system modules through to the javac command
line.

Also exports the location of the system modules to make
variables, as well as the name of the default system module.

Test: TestClasspath in java_test.go, runs automatically as part of the build
Bug: 63986449
Change-Id: I27bd5d2010092422a27b69c91568e49010e02f40
2017-10-16 15:00:02 -07:00
Dan Albert 6807ba2148 Fix ndk prebuilts script.
This wasn't actually including the quotes, so it was a no-op.

Test: build/soong/scripts/build-ndk-prebuilts.sh
Bug: http://b/65113115
Change-Id: I43fda02f58b9661fc78eef9055063347baaf84b3
2017-08-30 13:55:35 -07:00
Dan Willemsen 99a75cd2a5 Obsolete bootstrap.bash and ./soong wrappers
We can call directly into the blueprint bootstrap.bash using values that
soong_ui has already calculated.

Instead of calling into blueprint.bash, build minibp with microfactory,
and directly run ninja. This allows us to get individual tracing data
from each component.

Test: m -j blueprint_tools
Test: m clean; m -j blueprint_tools
Change-Id: I2239943c9a8a3ad6e1a40fa0dc914421f4b5202c
2017-08-18 10:13:22 -07:00
Colin Cross b852a58aa6 Convert soong java from soong_zip to jar
soong_zip produces jar files that are not compatible with java's
ZipInputStream.  Switch to jar using ugly sed scripts to munge
file lists into the alternating -C and file arguments required
by jar.

Bug: 64536066
Test: m -j checkbuild
Test: build/soong/scripts/jar_args.sh --test
Change-Id: Ifcc4bdab25e7d02342720eb246c673ff9a58bddb
2017-08-14 10:07:53 -07:00
Dan Willemsen e0879fc3ae Add --skip-make to replace Soong's blueprint wrapper with soong_ui
This way we only have one way to start a build, which always has logging
/ tracing / etc, even if we don't need Kati.

There's two ways to use this:

As a direct replacement for mkdir out; cd out; ../bootstrap.bash;
./soong -- as long as --skip-make is always passed, we'll never run
Kati, and Soong will run outside of it's "make" mode. This preserves
most of the speed, and allows full user control over the Soong
configuration.

A (experimental, dangerous) way to temporarily bypass the product
variable and kati steps of a build. As long as a user is sure that
nothing has changed from the last build, and they know exactly which
Ninja targets they want to build (which may not be the same as the
arguments normally passed to 'm'), this can lead to shorter build
startup times.

Test: rm -rf out; m --skip-make libc
Test: rm -rf out; m libc; m --skip-make libc
Test: rm -rf out; mkdir out; cd out; ../bootstrap.bash; ./soong libc
Test: build/soong/scripts/build-ndk-prebuilts.sh
Change-Id: Ic0f91167b5779dba3f248a379fbaac67a75a946e
2017-08-08 22:49:28 -07:00
Dan Albert 59034511c5 Pass Platform_version_all_codenames to NDK build.
We're still only generating the released versions and "current" when
we build with build-ndk-prebuilts.sh. Pass
Platform_version_all_codenames from make to generate the in-progress
platform versions as well.

Also move this over to soong.variables, since that's the appropriate
place for them. The Platform_sdk_version we were passing wasn't
actually being used before.

Test: OUT_DIR=ndk-out DIST_DIR=ndk-dist \
          build/soong/scripts/build-ndk-prebuilts.sh
Bug: None

Change-Id: Iccb709f5c12309754073ac8f0d4bc365019dfcd3
2017-08-03 14:33:26 -07:00
Dan Willemsen 91f9b54767 Switch to blueprint's microfactory
Bug: 63720725
Test: m -j nothing
Test: mkdir o; ../bootstrap.bash; ./soong
Change-Id: Id1f8312678f36afd08b49f8511fec3e6e5ba8de2
2017-07-24 15:29:14 -07:00
Colin Cross 1b59409256 Make mini-debug-info stripping pass on already stripped binaries
Prebuilts may already be stripped.  Instead of requiring every stripped
prebuilt to be annotated with LOCAL_STRIP_MODULE := false, just make
the default (mini-debug-info) pass through already-stripped modules
without failing.

Bug: 36793128
Test: manual
Change-Id: I9aec008a867eb7df0fbba82b6dd36605dcbf6b9f
2017-04-13 16:19:34 -07:00
Jeff Gaston a739f24cb2 Add script to make a Go-style workspace for Soong
Test: ./scripts/setup_go_workspace_for_soong.sh

Change-Id: If091175d3b69976d3e1488304c4274296cc8afb4
2017-02-07 18:24:10 -08:00
Colin Cross 9fe6ea834f Pass PREBUILT_SDK_VERSION to soong ndk build
Extract PREBUILT_SDK_VERSION from the make build system and pass
it to soong when doing ndk builds.

Test: untested
Change-Id: I573900dfb83aa8665c2bc349b7823e64497754d3
2016-11-09 13:01:41 -08:00
Dan Albert 65780b2c24 Add a script to build the NDK prebuilts.
Test: OUT_DIR=ndk-out DIST_DIR=ndk-dist \
          ./build/soong/scripts/build-ndk-prebuilts.sh
Bug: None

Change-Id: I3c930d2ec4d73196e355b90766eb14f0f0361460
2016-10-19 14:35:15 -07:00
Colin Cross 26c34ede29 Add support for toc optimization in soong
Skip relinking against shared libraries whose interface hasn't changed.

Test: mmma -j frameworks/native/libs/gui
Test: touch frameworks/native/libs/gui/BufferItem.cpp
Test: mmma -j frameworks/native/libs/gui, see nothing relinks past libgui
Bug: 26014946
Change-Id: I4d4b8da6a35c682341ae51869f5c72b51e192053
2016-09-30 21:05:59 -07:00
Colin Cross 665dce9320 Support stripping shared libraries and binaries
Strip all shared libraries and binaries by default.  Use a shell script
to wrap the long sequences of commands needed by some strip variants.

Change-Id: I465bf7cc48330913e60e24762fd55fa2a7731c26
2016-05-03 14:03:38 -07:00
Colin Cross 1474741435 Move shell and python scripts to scripts/ directory
Change-Id: Icdff44a54d14ddfc2266d99cf0578a8105716918
2016-04-29 13:59:56 -07:00