Commit Graph

730 Commits

Author SHA1 Message Date
Pirama Arumuga Nainar ada83ec0a6 Add PGO support to soong
Bug: http://b/63768402
Bug: http://b/65598278

Add support for the 'pgo' property to specify how a module is processed
under PGO.  A sample property is below:

pgo: {
    instrumentation: true, // could be "sampling: true" when supported
    profile_file: "pgo_simple.profdata",
    benchmarks: ["pgo_simple"],
}

1. Runtime profiles can be gathered using "sampling" or
"instrumentation".  Sampling is not supported initially.

2. If 'toolchain/pgo-profiles' project is found,
'toolchain/pgo-profiles/${profile_file}' is passed to the compiler and
linker when building this module.

3. If ANDROID_PGO_INSTRUMENT environment variable is set, and includes a
benchmark in the 'benchmarks' list, appropriate flags (for e.g.
-fprofile-generate for instrumentation) are passed to the compiler and
linker when building this module.

Test: Add example modules that specify the pgo property and verify
appropriate flags and dependencies in the Ninja file.  Some
tests/examples are in https://android-review.googlesource.com/474805

Change-Id: I6242e0c904497a115e367dea6927ba1c4b906355
2017-09-21 12:37:19 -07:00
Dan Willemsen c77a0b3f9c Extract the linker and embed it into host bionic binaries
The linux kernel requires that the ELF interpreter (runtime linker)
that's referenced by PT_INTERP be either an absolute path, or a relative
path from the current working directory. We'd prefer a relative path
from the binary, similarly to how we handle looking up shared libraries,
but that's not supported.

Instead, extract the load sections from the runtime linker ELF binary
and embed them into each host bionic binary, omitting the PT_INTERP
declaration. The kernel will treat it as a static binary, and we'll use
a special entry point (linker_wrapper) to fix up the arguments passed by
the kernel before jumping to the embedded linker. From the linker's
point of view, it looks like the kernel loaded the linker like normal.

Bug: 31559095
Test: Enable host bionic,
      out/soong/host/linux_bionic-x86/nativetest64/libdemangle_test/libdemangle_test
Change-Id: I8d0aea9790b5e86fcc3ea6e2d00cfa33907e2853
2017-09-20 15:34:33 -07:00
Dan Willemsen efb1dd93bf Add prefix_symbols support to cc_object
Test: add cc_object with prefix_symbols, use objdump to check
Change-Id: I15eb47e6888eb1c4837734a44a7ad86205a48c02
2017-09-20 13:16:13 -07:00
Dan Willemsen 38394b962f Fix ToolchainClangCflags for host bionic
Move these flags from the normal flags to the toolchain flags, since we
want them to always apply.

Test: Add #ifdef __ANDROID__ to crtbegin.c, check to see if it's set for host bionic
Change-Id: I382dd229e3e4a889bc916f6f8e1248c7debb8fad
2017-09-20 13:16:13 -07:00
Dan Willemsen 6f91fbff76 Don't add a rpath to the linker
It doesn't need one, and doesn't handle it well either (it tries
allocating memory before the memory allocators are initialized).

Test: out/soong/host/linux_bionic-x86/bin/linker64
Change-Id: Ic4f75e8914093f13f28d53cf771f518f2b4e5d2e
2017-09-20 13:16:13 -07:00
Dan Willemsen 3e5bdf29ba Add cc_genrule
cc_genrule is the same as a normal genrule, but can depend on other cc
modules (like cc_object).

Test: mmma external/minijail
Change-Id: I8df87665c7bdc76ce89c92755c054f967a818e57
2017-09-18 23:33:25 -07:00
Dan Willemsen 0c16293821 Allow cc_object to use libc headers under the VNDK
Bionic headers aren't implicit when building with the VNDK, they're
exported from libc. So add libc to the shared libraries of a cc_object
if it wants the standard flags.

Removes a duplicate No_default_compiler_flags

Test: Add cc_object with vendor_available, build on aosp_arm64_ab (VNDK-enabled)
Change-Id: I9bfcd1e1a65de4edc597bd348b79d625ecae4f5e
2017-09-18 16:51:35 -07:00
Dan Willemsen 15d54d5551 Make test_suites arch_variant
In most cases, we don't care about arm vs x86 when selecting whether a
module is in a test suite or not, but we do care about whether we need
to add the host or device module(s) to the suite. So enable the arch
mutator.

Test: Add android-specific test_suites property, look in out/soong/Android.mk
Change-Id: I1e258d6e97e18c8d20297568cd8ee8d4161d5023
2017-09-18 16:51:35 -07:00
Dan Willemsen d6ba0d592c Use dependency tags for genrules
So that we don't get confused when using :<module> in srcs to depend on
a module that could also be a HostBinTool.

Test: m -j
Change-Id: Ia3b1c26826e70f84c6dc5ff78c95dd11d76901b6
2017-09-14 22:59:33 +00:00
Yi Kong 2f7a367e7c Merge "Add support for thin LTO" 2017-09-13 05:07:36 +00:00
Jayant Chowdhary a4fce191ba Dump abi for vendor variants of VNDK libraries.
Currently, abi is dumped for platform variants of system libraries.
Dump them for vendor variants since they are the ones which need to be abi
stable on security updates. This also ties abi dumping to BOARD_VNDK_VERSION.

Test: For libfoo:
	1) Added a source file with a dummy function for the core
	   variant.
	2) Excluded the source file from the vendor variant.
	3) BOARD_VNDK_VERSION=current mm -j64 produces libfoo.so.lsdump with no
	   dummy_function since the source file was not included in the vendor
	   variant.

Test: Inspected build.ninja and confirmed that all of a library's abi
dump dependencies (.sdump files are dependencies of the final .lsdump files) are
from vendor variants.

Change-Id: Ie0bf91fcd81606c131845d9872261166b5db72aa
2017-09-11 17:36:46 -07:00
Colin Cross 38f794ee49 Refactor proto in preparation for java proto support
Test: m -j checkbuild
Change-Id: Idf00ea0bacb2777458f9af2c7eb47e1e1854eeba
2017-09-11 12:41:58 -07:00
Dan Willemsen b18cf7a5a0 Add windows libraries used by mdnsresponder
Test: Check for presence in prebuilts/gcc/linux-x86/host/x86_64-w64-mingw32-4.8
Change-Id: Iaa0b56ca8ff94ebcd925238f211d9bd5d52a4a0c
2017-09-09 13:06:25 -07:00
Dan Willemsen 4f1c3d4116 Add support for Windows Message Compiler
We've only got one user of this currently, but since it needs the
location of the toolchain, and different flags based on 32 vs 64-bit,
it's easier just to add support than to get this into a genrule.

Test: Convert external/mdnsresponder to Soong, build
Change-Id: I6c19a82bc14f6ab556f6cc5f37164862ba5f9083
2017-09-09 13:06:25 -07:00
Treehugger Robot ace7a6ba43 Merge "Make -ldl -lrt -lpthread implicit for host builds" 2017-09-09 01:32:23 +00:00
Colin Cross c5fdbb842a Make -ldl -lrt -lpthread implicit for host builds
Instead of requiring every host module to specify -ldl -lrt -lpthread
and then break the mac build because -lrt doesn't exist, make them
implicit to match the behavior of modules built for the device.

Test: m -j checkbuild
Change-Id: I63c26adef71e71d314b9c9caa31c1aeb1d8f6651
2017-09-08 16:39:05 -07:00
Colin Cross 28690e9c74 Move benchmarks to /data/benchmarktest[64] to match make
Port Ife0f43062e36ffc856f41683ddc4d91a7787333e to soong.

Test: m -j checkbuild
Change-Id: I66a0a53061b547e30aedd16ee617723197069171
2017-09-08 16:32:16 -07:00
Nan Zhang d4e641b6e9 <Hermetic> Replace Soong Python bootstrap process with embedded
launcher.

For Python2, we bundle embedded launcher as bootstrapper within every
.par file. This feature is only enabled for linux_x86_64 for now. We
provide a user flag: hermetic_enabled within bp file. By default, Pyhon2
still use classic bootstrapping way to construct .par file and relys on
host interpreter. Once embedded_launcher is enabled, launcher will be
used to bootstrap .par file and execute user program.

For Python3, the launcher will be ready soon, and for now it still relys
on classic bootstrapping.

Test: Real example is used to test.

Bug: b/63018041

Change-Id: I28deba413d8ad3af407595e46f77d663e79a3705
2017-09-05 17:26:57 -07:00
Yi Kong 244bf079f9 Add support for thin LTO
ThinLTO achieves comparable performance to full LTO while taking much
less time for compilation.

Test: Build hwui with "thin" and "full" LTO
Change-Id: If3400b82af0d5e0226410c8b740999cdad746a59
2017-09-01 14:21:44 +08:00
Colin Cross 5c51792926 Make binaries executable
Split InstallFileName into InstallExecutable that does chmod +x
after copying the file.  Also remove InstallFile and rename
InstallFileName to InstallFile.

Test: m -j checkbuild
Change-Id: Id41ad4eafe521f6cd5d8cc250b7747ecb3da8dfc
2017-08-31 13:48:52 -07:00
Treehugger Robot 25393ebcc4 Merge "Remove more duplicated flags and dependencies" 2017-08-29 22:38:57 +00:00
Dan Willemsen fe92c968b9 Remove more duplicated flags and dependencies
All the stats below are for a hikey960-eng build on AOSP master.

Number of order-only inputs in out/soong/build.ninja:
Before: 2847162
After:   606508

Size of:                       Before  After
out/soong/build.ninja          572MB   233MB  -59%
out/soong/Android-hikey960.mk   15MB     9MB  -40%
out/build-hikey960.ninja       367MB   358MB   -2%

Ninja time in `m nothing`:
Before: 6.9s
After:  5.0s

Soong generation time:
Before: 11.6s
After:   6.6s

Test: m nothing
Test: wrote script that counted inputs, only duplicates were removed
Test: treehugger
Change-Id: I6b7c5ef5b1395014b7bf5fd0b8112d42bee127bf
2017-08-29 13:39:46 -07:00
dimitry feda20b947 Support version_script for cc_binaries
Bug: http://b/65151367
Test: make
Change-Id: Ib9b2da45e3c748eeafa792dfd0447b756855f6ef
2017-08-29 15:00:01 +02:00
Jayant Chowdhary ec5b3aa567 Merge "Allow apis from exported shared library headers as well, while dumping abi." 2017-08-25 16:50:32 +00:00
Isaac Chen cd3d542187 Merge "Revert "Revert "Build support for 32-bit armv8-a""" 2017-08-25 01:55:56 +00:00
Jayant Chowdhary af6eb71ad2 Allow apis from exported shared library headers as well, while dumping abi.
header-abi-diff has switched to recursive diffing of types and can
diff more accurately if types included in re-exported shared library
headers are included as well.

Bug: 62060883

Test: make -j64, android.hardware.light@2.0.so.lsdump contains structs
from libcutils as well (libcutils' headers are re-exported by
android.hardware.light@2.0)

Change-Id: I814819e4f7258b4b380350fe8ad0ccf8dbd5ce5c
2017-08-24 16:48:25 -07:00
Zach Riggle 06bbd89c16 Enhance coverage options to include those needed by Honggfuzz for coverage-driven fuzzing
Test: make m
Bug: 64903541
Change-Id: Id5f10b6d7c122444f875134883f7c0a95d71e12d
2017-08-24 19:25:05 +00:00
Treehugger Robot 2e78e5e37f Merge "Add dependency support for yasm" 2017-08-24 18:02:18 +00:00
Isaac Chen a2a582413e Revert "Revert "Build support for 32-bit armv8-a""
This reverts commit 80b27e4e20.

Fixed CTS build issue by adding -march=armv7-a in cts/tests/tests/os/jni/Android.mk since the library built here needs/uses instruction (swp) obsolete in ARMv8.

Bug: 64964151
Test: lunch aosp_arm64-userdebug; make -j cts
      lunch aosp_x86-userdebug; make -j cts
      lunch aosp_sailfish-userdebug; make -j cts
      lunch aosp_bullhead-userdebug; make -j cts

Change-Id: I6ccafce043e101c769ad370a81ffa0eb62719b5e
2017-08-23 10:57:17 +00:00
Dan Willemsen 1d3e545a97 Add dependency support for yasm
Bug: 37719672
Test: lunch aosp_x86-eng; m native
Test: m out/soong/.intermediates/external/libvpx/libvpx/android_x86_static_core/obj/external/libvpx/libvpx/vp8/common/x86/copy_sse2.o
Test: NINJA_ARGS="-t deps out/soong/.intermediates/external/libvpx/libvpx/android_x87_static_core/obj/external/libvpx/libvpx/vp8/common/x86/copy_sse2.o" m
Change-Id: I01763b5870f994f7f0c6ac9d10f112f73addd7c7
2017-08-22 21:02:08 -07:00
Colin Cross ef88ae2369 Make libdl.so be loaded after libc.so
Make sure that libdl is always after libc on the command line.
Simplifies the logic to always support system_shared_libs for
sdk and vndk builds.

For backwards compatibility without updating lots of Android.bp
files, allow libdl to be listed in shared_libs as long as it
is also in system_shared_libs or libc is not in system_shared_libs.

Remove all the places that libdl is added as a dependency, since
it will always be present unless explicitly removed now.

Bug: 62815515
Test: m -j checkbuild
Change-Id: I0233178ffea87a2f0b82190746022476304a68e2
2017-08-23 12:54:54 +09:00
Tobias Thierer 9dffe1d807 Merge "Revert "Build support for 32-bit armv8-a"" 2017-08-22 10:30:29 +00:00
Tobias Thierer 80b27e4e20 Revert "Build support for 32-bit armv8-a"
This reverts commit 9c02066b13.

Reason for revert: Broke the build for some targets (including marlin and angler).

$ make cts
build/core/combo/TARGET_linux-arm.mk:43: kryo is armv8-a.
build/core/combo/TARGET_linux-arm.mk:45: TARGET_2ND_ARCH_VARIANT, armv7-a-neon, ignored! Use armv8-a instead.
[...]
cts/tests/tests/os/jni/android_os_cts_CpuInstructions.cpp:88:20: error: instruction requires: armv7 or earlier
    asm volatile ( "swp r0, r0, [%0]" : "+r"(ptr) : : "r0" );
                   ^
<inline asm>:1:2: note: instantiated into assembly here
        swp r0, r0, [r1]
        ^

Change-Id: I3fa7ca74d88aedf6574855e988d35e341f6e5067
2017-08-21 15:03:26 +00:00
Isaac Chen 1c508b6c53 Merge "Build support for 32-bit armv8-a" 2017-08-19 06:10:56 +00:00
Isaac Chen 9c02066b13 Build support for 32-bit armv8-a
Add armv8-a as a valid/supported 32-bit arm architecture variant. If
some known armv8-a core, like cortex-a53, cortex-a73, etc. is specified
as TARGET(_2ND)_CPU_VARIANT, the associated TARGET(_2ND)_ARCH_VARIANT
will be treated as if armv8-a is specified.

Bug: 62895439
Test: "bionic-unit-tests-static --gtest_filter=*strlen*" on Nexus 4
      (armv7-krait), emulator (armv7), and sailfish (armv8-kryo).
      The test binary for the first 2 is built with armv7-a as its
      TARGET_ARCH_VARIANT; The test binary for the last is built with
      armv8-a as its TARGET_2ND_ARCH_VARIANT.
      TARGET(_2ND)_CPU_VARIANTs of both binaries are set to "generic".

Change-Id: I53bee2974346cf485b2da54cf7aad010b0425910
2017-08-18 19:46:38 +08:00
Treehugger Robot fd697f4256 Merge "Expose the minimum supported SDK version to make." 2017-08-18 02:34:36 +00:00
Dan Albert f5415d7cc6 Expose the minimum supported SDK version to make.
Right now this is hard coded in two places (here and in
core/binary.mk). Keep it in one place so it's easier to change.

Test: make checkbuild
Bug: None
Change-Id: I2a2c784d4c667b326f871e6144db92753c16c85f
2017-08-17 16:54:03 -07:00
Colin Cross 766efbc9e9 Keep vndk library lists sorted
soong_build was writing a different make_vars-*.mk on every run
because the lists of vndk libraries were not sorted, causing
kati to regenerate the ninja file.

Test: m -j nothing && rm out/soong/build.ninja && m -j nothing
Change-Id: If1ef51bad602d61a337ee6f62d4758d4123cb00d
2017-08-17 14:57:12 -07:00
Stephen Crane f5b9b95f01 Work around LLVM bug where TLS not emulated in LTO mode
In LTO mode the backend constructs a target exclusively from command line flags,
without opportunity to accept code gen arguments from Clang. Clang does not
currently pass the -emulated-tls parameter for Android, even though it defaults
to emulated TLS for Android in non-LTO builds. Until this is fixed upstream, we
can explicitly specify that the LTO output for Android should use emulated TLS.

Test: build libandroid with LTO, verified that it uses __emutls_get_address
rather than __tls_get_addr.

Change-Id: I18acac41aac885fc6635fbd55f96ba7c845eb5e7
2017-08-15 13:18:11 -07:00
Stephen Crane ba090d16c2 Add LTO support to soong
Enabling the lto property for a module builds that module and all static
dependencies with LTO.

LTO (link-time optimization) allows the compiler to optimize and
generate code for the entire module at link time, rather than
per-compilation unit. LTO is required for Clang CFI and other
whole-program optimization techniques. LTO also allows cross-compilation
unit optimizations that should result in faster and smaller code, at the
expense of additional compilation time.

Test: make -j12 libc with lto: true for libc

Change-Id: Ib8baefedf60e02701d44673a7c473e0845730101
2017-08-15 12:54:12 -07:00
Stephen Crane b3c2154f2a Pass correct emulation string to gold
Gold recognizes a different set of emulation strings to specify targets
than the bfd linker accepts. Clang only passes bfd emulations to the
linker, so we need to override these with the versions accepted by gold
when using gold.

Gold normally picks up the correct target from the first ELF input file
if it cannot parse the emulation parameter. However when using LTO, all
objects may be bitcode files, which causes gold to rely on the passed
--oformat or -m parameter to determine the proper target. If gold cannot
parse the emulation string passed by clang, it fails to link.

Test: build LTO version of libc

Change-Id: I38e78bb912fd3cc5fb7b4a762284f50ddd4f3998
2017-08-15 12:54:12 -07:00
Jiyong Park a622de8498 cflags for vendor variants
cflags that are only applied when building the vendor variant of a lib
is now supported. This will be particularily useful when existing macro
should be turned off for the vendor variant:

cflags: ["-DENABLE_FEATURE_X"],

target: {
    vendor: {
        cflags: ["-UENABLE_FEATURE_X"],
    },
},

Without this, the library author has to undef the macro inside the
source code using __ANDROID_VNDK__, which is not desired especially for
external projects.

Bug: 64514237
Test: BOARD_VNDK_VERSION=current m -j libsqlite libsqlite.vendor
Merged-In: Ib484b5b12e14ab776276a30ed2763dffadc9d642
Change-Id: Ib484b5b12e14ab776276a30ed2763dffadc9d642
(cherry picked from commit 1e7495d973)
2017-08-14 11:46:42 +09:00
Jiyong Park 7447228e64 a shared lib header is not exported to vendors if it is in exclude_shared_libs
Since a lib in exclude_shared_libs are not used for the vendor variant,
it should also be removed from the export_shared_lib_headers.

Bug: 62471389
Test: BOARD_VNDK_VERSION=current m -j libxml2.vendor successful
(libxml2 has libicuuc in both exported_shared_lib_headers and
exclude_shared_libs for the vendor variant)

Merged-In: Ic9ddbd7be3c76df563e428ba2409a03bd2e1e113
Change-Id: Ic9ddbd7be3c76df563e428ba2409a03bd2e1e113
2017-08-14 11:40:54 +09:00
Treehugger Robot b8245ea839 Merge changes Id9717132,I3e3b7251,I021c7971,I8a117a86,Ia5196f1f, ...
* changes:
  Allow AndroidMkData.Custom handlers to extend normal values
  Remove error from AndroidMkDataProvider.AndroidMk
  Remove error from AndroidMkData.Custom
  Remove error from AndroidMkData.Extra
  Bring java support closer to current version of make
  Rename java_prebuilt_library to java_import
2017-08-12 01:11:59 +00:00
Colin Cross 0f86d186b1 Allow AndroidMkData.Custom handlers to extend normal values
Pass accumulated AndroidMkData to AndroidMkData.Custom handlers
and expose WriteAndroidMkData so that Custom handlers can write
out the normal make variables and then add their own.

Test: No change to out/soong/Android-aosp_sailfish.mk
Change-Id: Id9717132bbd6c5cf3af8596f3eaa9bbb05d98e40
2017-08-11 15:24:11 -07:00
Colin Cross a18e9cfa29 Remove error from AndroidMkDataProvider.AndroidMk
It's never anything except nil, and it unnecessarily complicates
the implementations.

Test: m -j checkbuild
Change-Id: I3e3b7251f32ffa84dbdfd0448faf248c306ca808
2017-08-11 15:24:11 -07:00
Colin Cross 91825d2279 Remove error from AndroidMkData.Custom
It's never anything except nil, and it unnecessarily complicates
the implementations.

Test: m -j checkbuild
Change-Id: I021c7971ede3e11bbe08cf1601f4690ed4d1a036
2017-08-11 15:24:11 -07:00
Colin Cross 27a4b05441 Remove error from AndroidMkData.Extra
It's never anything except nil, and it unnecessarily complicates
the implementations.

Test: m -j checkbuild
Change-Id: I8a117a86aa39aeb07d9d8d0686ef869c52784f19
2017-08-11 15:24:11 -07:00
Colin Cross 74d73e2bfb Rename java_prebuilt_library to java_import
And make it work like bazel's java_import, using a "jars" property
instead of "srcs", and allowing multiple jars to be listed.

Test: soong tests
Change-Id: Ida2ace6412bd77b4feb423646000a1401004e0ea
2017-08-11 15:24:11 -07:00
Colin Cross 6774e2893a Use --hash-style=both for NDK binaries
Old devices don't support gnu hashes, use --hash-style=both when
the sdk_version property is set.

Test: m -j checkbuild
Bug: 64608507
Change-Id: Iec5229a08fc192f1c8872466ee4bf866adf50443
2017-08-11 22:21:39 +00:00