Commit Graph

38 Commits

Author SHA1 Message Date
Pirama Arumuga Nainar 19a5aae35e [cc/pgo] Mark pgo.sampling property as an arch-variant
Bug: http://b/189438896

This allows disabling sampling PGO for some arches.

Test: Build this topic.
Change-Id: Ib6ddca6685c2e53092afd440ad95ec42d3a9538b
2021-06-16 22:01:41 -07:00
Pirama Arumuga Nainar bedca74471 [pgo] Do not explicitly link profile runtime
Bug: http://b/181740505

Explicitly adding dependency to profile runtime causes apex dependency
errors.  We're already passing -fprofile-instr-generate to the linker so
depending on the profile runtimes is unnecessary.

Test: pgo-coral-config1 on internal release branch and run tests on
      APCT.
Change-Id: Ie54785726c7ac044e5a5a2299073d02d45d5e0a7
2021-03-23 18:07:30 -07:00
Jaewoong Jung 18aefc1977 Remove unnecessary snake case variables.
Test: m nothing + TreeHugger
Change-Id: I99f7162944daa6c57c6ae4763261e108bb5cb6b1
2020-12-22 12:38:35 -08:00
Yi Kong 110cd5fe7f Always generate profiling debug info
The additional debug info is used for sampling PGO. This does not affect
the size of final stripped binaries.

Also removed `-gmlt` flag in pgo.go, since it's actually a lower level
of debug option than the default (-g).

Test: build, verify final binaries are identical size
Bug: 79161490
Change-Id: Ie4cf4998695f78a50a7ac7a7a0853c625cbd8880
2020-11-03 18:13:21 +00:00
Pirama Arumuga Nainar 1150fd7c68 Propagate PGO instr. flags to dependencies of a static lib
Bug: http://b/119560349
Bug: http://b/168925229

To build with PGO instrumentation, the link step also needs
-fprofile-generate flag.  Omitting this will result in a linker error.
This surfaces if module A depends on module B but PGO is used only for
module B.

Previous uses of PGO were avoiding or working around the issue but to
accommadate recent changes, this CL propagates the PGO linker flag to
module A.

Test: pgo-coral-config1 in release branch, TH
Change-Id: I12f6243152d4f5ea5526fbce03c25dba232ddab7
2020-09-22 11:49:25 -07:00
Pirama Arumuga Nainar fe1da75972 Revert "Revert "[pgo] Return updated Flags after adding flags for PGO/AFDO""
This reverts commit 988ff8d149.

Reason for revert: RBE can now handle PGO profiles during remote linking.

Bug: http://b/162702246

Test: m ANDROID_PGO_INSTRUMENT=hwui hwuimacro and check hwuimacro has
      PGO instrumentation.

Change-Id: I724b53dc086923beeb6f4b8903136545d18f4ece
2020-09-02 17:44:06 +00:00
Bowgo Tsai 988ff8d149 Revert "[pgo] Return updated Flags after adding flags for PGO/AFDO"
This reverts commit 5010a97388.

Reason for revert:  b/166728338 - build failures on master branch

Bug: 162702246
Bug: 166728338
Change-Id: Ie9e51e7c5533d220ad3e160439cb38ff88f068ac
Test: TreeHugger
2020-08-28 08:08:36 +00:00
Pirama Arumuga Nainar 5010a97388 [pgo] Return updated Flags after adding flags for PGO/AFDO
Bug: http://b/162702246

Test: m ANDROID_PGO_INSTRUMENT=hwui hwuimacro and check hwuimacro has
      PGO instrumentation.
Change-Id: I8044702e0aed4fa7bb9ac71608a83a95f319abd9
2020-08-27 16:28:45 -07:00
Yi Kong a575ff38a8 [pgo] Allow profile gather and use flags to coexist for sampling PGO
Bug: 79161490
Test: presubmit
Change-Id: I90df9c04dbe2a423c06e9a966fe9bcaed0a84a65
2020-07-22 02:01:49 +08:00
Yi Kong 84803c50a6 [pgo] Sampling PGO does not require benchmark property
Test: build with Sampling PGO
Bug: 79161490
Change-Id: I70bb8dc2e4492d6a3713ca16fcb3f1216615f8ac
2020-07-22 01:36:20 +08:00
Pirama Arumuga Nainar 9d544a83bf [cc/pgo] Fix incorrect project name
Bug: http://b/159920645

Look for internal profiles at the correct project.

Test: N/A
Change-Id: I6a64adbb043da7e419f99c3182d2ec29240ad8f3
2020-06-29 09:25:51 -07:00
Yi Kong ceb5b76c91 Add platform-wide sampling PGO option
This causes the compiler to emit some additional debug infomation that
will be used for sampling PGO. These debug infomation will get stripped
so it only affects intermediate files.

Test: build
Bug: 79161490
Change-Id: Ie4d1d5ffbd311ba6e268cb94a618f5272be246ef
2020-03-24 07:04:54 -07:00
Pirama Arumuga Nainar 807d49bc43 Disable PGO profile use in Clang-based coverage builds
Bug: http://b/143977934

Clang errors that -fprofile-use and -fprofile-instr-generate are
incompatible.

Test: m CLANG_COVERAGE=true COVERAGE_PATHS="*" on cf_x86_phone target
Change-Id: I584b615639fcef86668b7fa34b7511b4820d9548
2020-02-12 15:12:07 -08:00
Yi Kong b6ec66a173 Add -fprofile-sample-accurate flag for sampling PGO
This flag indicates that the sample profile data matches the source
and can be fully trusted. The compiler will make aggressive
assumption that functions without any samples are cold functions,
and will optimize for size for them.

Test: binary size reduces to match instr PGO
Bug: 79161490
Change-Id: I53d6d05be70c39e5eb28b2f5b0549d9eb6b5cc62
2020-01-31 12:36:38 +08:00
Yi Kong 61a1b988fd Add -fdebug-info-for-profiling flag to emit more debug info for sampling pgo
Test: m
Bug: 79161490
Change-Id: I25cfbf78b27d653286dc7f956743f8855b852b35
2020-01-30 21:15:12 +08:00
Yi Kong 92474e5b8f Add "-no-warn-sample-unuse=true" to work around AutoFDO issue
Autofdo generates profile for an instruction even if there is no debug
information associated with it or no debug information associated with
the function. A bogus offset will be produced in the profile. Add the
flag to suppress Clang from generating error for such cases.

Test: build with ETM profile
Bug: 147844018
Bug: 79161490
Change-Id: I37da1ba3a4962072ccdf01f79fbf2c2b4c77b56b
2020-01-16 17:09:19 -08:00
Yi Kong 0b9932164b Support sampling PGO profiles
Bug: 79161490
Test: build and flash with ETM generated sampling profile
Change-Id: I82bf61ebfc9f1a1c74a370f0421c46862dd470db
2020-01-02 16:21:03 -08:00
Colin Cross 4af21ed26f Split local and global cflags
Native compiler flags are currently applied in approximately:
global cflags
local cflags
local include dirs
global include dirs
global conlyflags
local conlyflags
global cppflags
local cppflags

This means that a flag that is enabled in the global cppflags
cannot be disabled in the local cflags, and an Android.bp author
must know to disable it in the local cppflags.  A better order
would be:
global cflags
global conlyflags
global cppflags
local cflags
local conlyflags
local cppflags
local include dirs
global include dirs

We are mixing both the global and local cflags into a single
variable, and similar for conlyflags and cppflags, which
prevents reordering them.  This CL prepares to reorder them
by splitting the global and local cflags into separate variables.

Bug: 143713277
Test: m native
Change-Id: Ic55a8c3516c331dc5f2af9d00e59ceca9d3e6c15
2019-11-07 15:27:58 -08:00
Pirama Arumuga Nainar 3a25405acc Remove redundant -fexperimental-new-pass-manager
This is now turned on by default (cc/config/clang.go).

Test: N/A
Change-Id: I61bcf2049448b4200ca303ebcad555cc24a43c18
2019-06-14 09:54:23 -07:00
Yi Kong 69c1ed91d8 Turn on new pass manager for PGO projects
With new pass manager, dex2oat benchmark showed 1.61% performance
improvement though binary size saw an increase of 3.37%.

For Googlers, detailed benchmark results are in go/android-pgo-npm.

Test: m checkbuild
Bug: 118894695
Change-Id: I1687886acac7f61e67265a9c73997fb241ad31e6
2019-03-21 23:43:42 +00:00
Colin Cross 571cccfcbc Prepare for a type-safe OnceKey
Add an opaque OnceKey type and use it for all calls to Once in
build/soong.  A future patch will convert the arguments to
Once* to OnceKey once users outside build/soong have been updated.

Test: onceper_test.go
Change-Id: Ifcb338e6e603e804e507203c9508d30ffb2df966
2019-02-06 01:52:41 +00:00
Yi Kong ca610d2be4 Take Enable_profile_use into account when checking if PGO is enabled
Test: m
Change-Id: I3c589043af145450d0d97ef354a74845e9fae483
2018-05-07 15:16:36 -07:00
Pirama Arumuga Nainar 8aed42c798 Use a versioned profile file if available
Bug: http://b/74395273

Update profile-search logic to look for profile files named
<profile_file>.<arbitrary-version> in the profileProject and use that if
available.  This works around an issue where ccache serves stale cache
entries when the profile file has changed.

Test: touch toolchain/pgo-profiles/art/dex2oat.profdata.foo and check it
gets used.

Change-Id: I3de095c263b952cede25af7d0b63dc7280914d76
2018-03-09 16:05:54 +00:00
Colin Cross 32f3898f0b Remove unused intermediates parameter from ExistentPathForSource
Test: m checkbuild
Change-Id: Id2c0a5039c2ec3b3795385c135ffec022ccd691e
2018-02-22 14:43:36 -08:00
Yi Kong 7e53c57ed6 Disable inlining and loop unrolling in LTO without PGO profile
Such optimisations may significantly increase the binary size when
compiler heuristics are off. Disabling these helps cut down the
binary sizes with negligible decrease in performance, but allows us to
be more comfortable enabling LTO across various projects.

Test: m
Test: dex2oat, hwui, skia benchmark
Bug: 62839002
Change-Id: Id63e8dd295df2972f76ae4e29ee367080fff8429
2018-02-14 21:21:23 +08:00
Pirama Arumuga Nainar 49540800f0 Search for PGO profiles in PGO_ADDITIONAL_PROFILE_DIRS
This variable can be set in BoardConfig.mk to specify a list of
additional paths that contain PGO profiles.  These directories are
searched after the predefined paths in soong/cc/pgo.go while finding
PGO profiles.

Test: Set this variable in a BoardConfig and verify that such profiles
are found and that these paths are searched after the predefined paths
in soong/cc/pgo.go.

Change-Id: I0bb9523de614d0f23aba8d51c887d8fc8f41c993
2018-02-13 09:41:13 -08:00
Pirama Arumuga Nainar 28316d4e3b Store missing profile files into a Make variable
Bug: http://b/72642679

Store missing profile files and the modules that refer to them in the
SOONG_MODULES_MISSING_PGO_PROFILE_FILE variable passed to Make.  The
contents of this variable will be written to
$DIST_DIR/pgo_profile_file_missing.txt as part of the 'dist' target.

Test: 'm dist' and verify creation of pgo_profile_file_missing.txt.
Change-Id: I237cd0398be418be2b7db6fa65ece5ef347ecbc1
2018-01-29 10:02:27 -08:00
Pirama Arumuga Nainar e236b5acfa Support instrumenting all PGO-enabled modules
Bug: http://b/63768402

If ANDROID_PGO_INSTRUMENT has "all" or "ALL", all PGO-enabled modules
are built for profile generation.

Test: Build with and without 'all' in ANDROID_PGO_INSTRUMENT
Change-Id: I3b1a9b562775e80a4ab3965100341d9e8e4ffde9
2018-01-22 20:26:16 -08:00
Pirama Arumuga Nainar 64946fec54 Update profile_file discovery rules
Bug: http://b/63768402

Profile files are searched in multiple directories in a pre-define list,
instead of just one.

PGO profile_file enforcement is also removed - profile_file dependency
and -fprofile-use are added iff the profile_file is present.

Test: Build https://android-review.googlesource.com/c/474805 with
.profdata files in vendor/google_data/pgo-profiles and
toolchain/pgo-profiles and verify expected behavior.

Change-Id: I2604713a8a7a682bd5c6bde569dc32335be5eca9
Merged-In: I2604713a8a7a682bd5c6bde569dc32335be5eca9
2018-01-18 10:26:34 -08:00
Pirama Arumuga Nainar 690ed55363 Add cflags sub-property to 'pgo'
This (possibly arch-specific) property adds extra flags while building
for profiling.  For example, dex2oat requires setting
'-Wno-frame-larger-than' when instrumentation is enabled.

Test: Build 'pgo' module with the new property and inspect build.ninja
Change-Id: Ia36422a400b0e0b87cbb33b1c8e565569475a56d
2017-12-13 17:48:33 -08:00
Colin Cross 6510f91a1c Replace ModuleContext.AConfig() with Config()
AConfig() now duplicates Config().  Replace the uses of AConfig()
with Config().  Leave AConfig() for now until code in other
projects is cleaned up.

Test: m checkbuild
Change-Id: Ic88be643049d21dba45dbd1a65588ed94bf43bdc
2017-11-30 00:44:18 +00:00
Pirama Arumuga Nainar 6aeed8b439 Add enable_profile_use property
Bug: http://b/65598278

This property defaults to 'true' and setting it to false skips profile
use.  This escape hatch lets us disable PGO for a module without
completely removing the 'pgo' property.  Additionally, this also helps
selectively disabling PGO for some architectures, if desired.

Test: Test that -fprofile-use is not added for a test module if
'enable_profile_use: false' is set.

Change-Id: Ifcf1a48c194bc86efd88a529cc2d66a47b7ab080
2017-10-16 13:36:29 -07:00
Pirama Arumuga Nainar 0fdfc459cf Do not use profiles if ANDROID_PGO_NO_PROFILE_USE is set
Bug: http://b/65598278

Non-PGO builds (with this environment variable set) can be used to
measure the performance difference induced by PGO.

Test: Build PGO modules with ANDROID_PGO_PROFILE_USE set.
Change-Id: Ib23bad5208ac7f54894c7768d7532f53b6b91179
2017-10-10 15:41:25 -07:00
Pirama Arumuga Nainar 3f5bb9c89b Refactor methods to be on PgoProperties struct
These methods only need the 'PgoProperties' struct.  Avoid the extra
indirection by directly using this struct.

Test: Build modules with PGO property.
Change-Id: I1923ebde4b0d546810de8e696514d218b3a4f54b
2017-10-10 10:58:37 -07:00
Pirama Arumuga Nainar 6fc8d9126c Check if "instrumentation" and "sampling" are set together
Setting these simultaneously is an error.

Test: Check error generation in a test module

Change-Id: I2ee127fa1b7157fa5f4489d2541ff023c64f9ba5
2017-10-05 10:33:57 -07:00
Pirama Arumuga Nainar 49b53d5c7c Explicitly link the profile runtime during PGO
Bug: http://b/65598278

The profile runtime depends on libgcc for some symbols (only under some
circumstances - armv5, ndk r14, static executables).  Since Android
build passes -nostdlib and adds libgcc manually, the profile runtime
gets passed to the linker later than libgcc.

Instead, explicitly add the profile runtime to the linker command (and
pass one other flag added by the clang driver to the link).

Test: Build a library with profile instrumentation that otherwise fails
instrumented build.

Change-Id: I24b34cebd2c3bb6a540f8f4c465ace1be4eb90f3
Signed-off-by: Pirama Arumuga Nainar <pirama@google.com>
2017-10-04 22:25:19 -07:00
Pirama Arumuga Nainar f4c0baf317 Pass -Wno-backend-plugin when using profiles
Bug: http://b/63768402

Wno-backend-plugin warnings are generated by IR-based PGO when sources
evolve and the profiles become stale.  Disable these warnings when
profiles are passed to the compiler.

Test: Build a module with a PGO property when profiles are used.
Change-Id: I462bbf55aafd8770a90c2c23462bf71607ac9108
2017-10-03 14:35:18 -07:00
Pirama Arumuga Nainar ada83ec0a6 Add PGO support to soong
Bug: http://b/63768402
Bug: http://b/65598278

Add support for the 'pgo' property to specify how a module is processed
under PGO.  A sample property is below:

pgo: {
    instrumentation: true, // could be "sampling: true" when supported
    profile_file: "pgo_simple.profdata",
    benchmarks: ["pgo_simple"],
}

1. Runtime profiles can be gathered using "sampling" or
"instrumentation".  Sampling is not supported initially.

2. If 'toolchain/pgo-profiles' project is found,
'toolchain/pgo-profiles/${profile_file}' is passed to the compiler and
linker when building this module.

3. If ANDROID_PGO_INSTRUMENT environment variable is set, and includes a
benchmark in the 'benchmarks' list, appropriate flags (for e.g.
-fprofile-generate for instrumentation) are passed to the compiler and
linker when building this module.

Test: Add example modules that specify the pgo property and verify
appropriate flags and dependencies in the Ninja file.  Some
tests/examples are in https://android-review.googlesource.com/474805

Change-Id: I6242e0c904497a115e367dea6927ba1c4b906355
2017-09-21 12:37:19 -07:00