Commit Graph

13182 Commits

Author SHA1 Message Date
Kousik Kumar e78212a55c Merge "Check UseRBE is set before replacing any template with the RE version." am: 3e0b9c031c am: 4e2977ee81
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/15747538

Change-Id: Ifa33cdad01dac8e81724d97f6373b745ff775ca9
2021-09-07 14:32:59 +00:00
Kousik Kumar 4e2977ee81 Merge "Check UseRBE is set before replacing any template with the RE version." am: 3e0b9c031c
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1419252

Test:
1. Applied https://googleplex-android-review.git.corp.google.com/c/platform/build/soong/+/15723183, confirmed that the build fails on rvc-dev (by running `USE_RBE=false m `)
2. Subsequently applied this patch and tested to confirm that the failure is fixed (by running `USE_RBE=false m out/soong/.intermediates/vendor/qcom/sm7250/proprietary/commonsys/qcrilOemHook/qcrilmsgtunnel/android_common/combined/qcrilmsgtunnel.jar` and `USE_RBE=true m out/soong/.intermediates/vendor/qcom/sm7250/proprietary/commonsys/qcrilOemHook/qcrilmsgtunnel/android_common/combined/qcrilmsgtunnel.jar`).

Bug: b/198045383
Merged-In: I6df58b4e700f0d231367af2710672d731d20a736
Change-Id: I52b7a3810fdf3314ae107f4b8330e9d6f62c6302
(cherry picked from commit f15ed4fe61)
2021-09-02 19:15:35 +00:00
Kousik Kumar 233c1be0bd Merge "Load RBE related env vars from config files" am: b3a5d18470 am: 9f6fa868c5
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/15454953

Change-Id: Ie1e7b2bdb392c33a286d7f52e0fee4ccf560b1b7
2021-08-05 18:08:31 +00:00
Kousik Kumar 9f6fa868c5 Merge "Load RBE related env vars from config files" am: b3a5d18470
Original change: https://android-review.googlesource.com/c/platform/build/soong/+/1773209

Bug: b/194679562
Change-Id: Id84167a3ae6ab15c8b74b72b4cc2c179e0da5d0c
Merged-In: I416e8da75f84aa2b53995f525cf50501488dc972
(cherry picked from commit a7ab7c9394ccfc80e163b5e405792a3975eaf2f3)
2021-08-03 13:22:44 -04:00
Kousik Kumar 1f3a7f6ccf Print the output from RBE bootstrap shutdown am: 91a46d0ef4
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/14371203

Change-Id: Ie450c8f1b27f21246b5f17ae6a56b918b7a3a78a
2021-04-30 04:49:24 +00:00
Kousik Kumar 91a46d0ef4 Print the output from RBE bootstrap shutdown
The output from bootstrap would contain the a one-line summary of RBE
download / upload stats.

Bug: b/183007125
Change-Id: I90082a8433504e40bbf92992c3c8d1d656c49429
Merged-In: I90082a8433504e40bbf92992c3c8d1d656c49429
2021-04-29 20:49:13 -04:00
Bob Badour 64a932b9c7 [automerger skipped] Minimal license feature. am: 459beab694 -s ours am: b5d58b3eaa -s ours
am skip reason: skip tag Change-Id Ic78672cd25da13aa9078a48b06446f66780c792d with SHA-1 58d85b87bc is already in history

Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/13928045

Change-Id: I4393e98e96b34b0e4604e0c03a7cc2b16e7a5c2e
2021-03-19 20:54:22 +00:00
Bob Badour b5d58b3eaa [automerger skipped] Minimal license feature. am: 459beab694 -s ours
am skip reason: skip tag Change-Id Ic78672cd25da13aa9078a48b06446f66780c792d with SHA-1 58d85b87bc is already in history

Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/13928045

Change-Id: If527f71987f23aee63aa0a1209572479672398c5
2021-03-19 20:20:01 +00:00
Bob Badour 6b4ddeb66c Minimal license feature. am: 58d85b87bc
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/13929165

Change-Id: I49453a723ea46a2bda5d21655e569c212564c076
2021-03-19 19:35:42 +00:00
Bob Badour 459beab694 Minimal license feature.
Keeps license declarations from breaking the build without implementing
the full feature.

Test: m all

Bug: 181768532

Change-Id: Ic78672cd25da13aa9078a48b06446f66780c792d
Merged-in: Ic78672cd25da13aa9078a48b06446f66780c792d
Merged-in: If3fbc157c6ffed8e3ee4bdd9aa56fabe2c24444b
Merged-in: Ie7e1695ecb8cd943124426a0cad2c0d1db2b46ed
2021-03-19 01:28:12 -07:00
Bob Badour 58d85b87bc Minimal license feature.
Keeps license declarations from breaking the build without implementing
the full feature.

Test: m all

Bug: 181768532

Change-Id: Ic78672cd25da13aa9078a48b06446f66780c792d
Merged-in: Ic78672cd25da13aa9078a48b06446f66780c792d
Merged-in: If3fbc157c6ffed8e3ee4bdd9aa56fabe2c24444b
Merged-in: Ie7e1695ecb8cd943124426a0cad2c0d1db2b46ed
2021-03-19 00:42:52 -07:00
Patrice Arruda 43743ee2ce [DO NOT MERGE] Capture the build command that executed the build system. am: 807a8f9142
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12969335

Change-Id: Ib496613df81acb9e17f1ca4b717583fab39643cc
2020-11-02 20:44:03 +00:00
Patrice Arruda 807a8f9142 [DO NOT MERGE] Capture the build command that executed the build system.
We need to capture the user build command to understand what additional
arguments is passed in to Soong. This data is useful to know which
targets are being executed and have statistics on top build targets
are specified.

Bug: b/169453712
Test: ran "m blueprint_tools", "m nothing", "lunch 1" and checked
      in soong_metrics proto.
Change-Id: I7ab94dcf037211f1abffe6afd012b03ffbf99756
Merged-In: I7ab94dcf037211f1abffe6afd012b03ffbf99756
2020-11-02 09:50:47 -08:00
Kousik Kumar 04f6efa716 Deprecate both USE_GOMA and FORCE_USE_GOMA flags am: 6d0034f49f
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12886656

Change-Id: If52cdcd09678cc154257b22b869f6ebabd8557f0
2020-10-21 03:53:09 +00:00
Kousik Kumar 6d0034f49f Deprecate both USE_GOMA and FORCE_USE_GOMA flags
Test:
m USE_GOMA=true GOMA_DIR=$(goma_ctl goma_dir) toybox
m FORCE_USE_GOMA=true GOMA_DIR=$(goma_ctl goma_dir) toybox
both result in error message being printed.

Bug: b/171325288
Change-Id: I94ccaf217dba71aca990d88d205bad669a49a2f5
Merged-In: I94ccaf217dba71aca990d88d205bad669a49a2f5
2020-10-20 19:51:14 +00:00
Patrice Arruda e32b384719 Collect the hostname of the machine as a metric. am: af7b75b8db
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12849657

Change-Id: I21d6d894ce261bb9181dc6863c41ff67c5af904a
2020-10-15 22:49:36 +00:00
Patrice Arruda af7b75b8db Collect the hostname of the machine as a metric.
The hostname is collected during a build to determine where the user
is build from (from office workstation, cloudtop, etc...). This is a
useful metrics for RBE related builds.

Bug: b/169395325
Test: m nothing, ran printproto to verify that soong_metrics has the
      hostname.
Change-Id: I3e5d64f948ae43c5dda60de1d4a52b187e36dff8
Merged-In: I3e5d64f948ae43c5dda60de1d4a52b187e36dff8
Merged-In: Ib6898b49c16a830d814efe74581bde463caedf76
2020-10-15 17:39:49 +00:00
Patrice Arruda 87deed46bf Collect FORCE_USE_GOMA environment variable as a metric. am: 3fc4c98bd9
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12722544

Change-Id: I97b6cc360664bb8a2c2f959a632efd99b9d9021a
2020-09-29 22:48:52 +00:00
Kousik Kumar 64d43056ac Deprecate USE_GOMA and replace with FORCE_USE_GOMA am: dbe59cdd81
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12721330

Change-Id: I6ff581a4b66298a34453282c5709102fbc691d7d
2020-09-29 22:48:49 +00:00
Patrice Arruda 3fc4c98bd9 Collect FORCE_USE_GOMA environment variable as a metric.
FORCE_USE_GOMA is needed to be collected as a metric for the
deprecation of GOMA on Android builds.

Bug: b/169423400
Test: m nothing, ran printproto command
Change-Id: I12300389f1f27239d8f21875d1b6e4ad069d95d7
Merged-In: I12300389f1f27239d8f21875d1b6e4ad069d95d7
2020-09-29 16:40:54 +00:00
Kousik Kumar dbe59cdd81 Deprecate USE_GOMA and replace with FORCE_USE_GOMA
Bug: b/163361527
Test: Ran builds with USE_GOMA / FORCE_USE_GOMA flags
Change-Id: I67b229fbf52e913b0c4f5e7932021132ddc0598f
Merged-In: I67b229fbf52e913b0c4f5e7932021132ddc0598f
2020-09-29 16:40:38 +00:00
TreeHugger Robot 4d5943fa65 Merge "[DO NOT MERGE] Add Goma deprecation PSA to soong." into qt-dev am: 3f4bdf5015
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12614869

Change-Id: I37b43e7591eeedc1e78da38362b6b90bfc98118f
2020-09-16 23:38:56 +00:00
Ramy Medhat db6b17e7ac [DO NOT MERGE] Add Goma deprecation PSA to soong. am: ee50d5b5b0
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12614869

Change-Id: I9503e5c8bbd0412973439f57ffe22f470201d092
2020-09-16 23:38:53 +00:00
TreeHugger Robot c3e751fdab Merge "Start collecting build configuration metrics." into qt-dev am: b4435196a3
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12618631

Change-Id: Ifa3de692aea1bb7a9434d9660dbc845f7f56e3f5
2020-09-16 23:00:36 +00:00
TreeHugger Robot 3f4bdf5015 Merge "[DO NOT MERGE] Add Goma deprecation PSA to soong." into qt-dev 2020-09-16 22:28:05 +00:00
TreeHugger Robot b4435196a3 Merge "Start collecting build configuration metrics." into qt-dev 2020-09-16 22:25:54 +00:00
TreeHugger Robot 793f74ffeb Merge "Add eakammer and patricearruda to OWNERS" into qt-dev am: 0a129a52cf
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12617609

Change-Id: I6e16f05db1d0b0963b69af04758894e26608d2c0
2020-09-16 21:05:26 +00:00
TreeHugger Robot 0a129a52cf Merge "Add eakammer and patricearruda to OWNERS" into qt-dev 2020-09-16 20:06:23 +00:00
Patrice Arruda 5eae8cdca1 Start collecting build configuration metrics.
This is an initial CL that collects the environment variables
USE_GOMA and USE_RBE. This is a needed metric to determine how
many users are currently using GOMA so we can help them migrate
to RBE.

Bug: b/163598127
Test: m nothing and checked soong_metrics
Change-Id: I39536eabad01aafbb3bd7182515d72230b36f658
Merged-In: I39536eabad01aafbb3bd7182515d72230b36f658
2020-09-16 19:44:42 +00:00
Ramy Medhat e4e319e8d4 [DO NOT MERGE] Make the default exec strategy for javac/r8/d8 remote local fallback. am: 320c041797
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595318

Change-Id: I8d2efe13bd6ff2bacc45606ef8dcc5d3ea24dae9
2020-09-16 19:17:38 +00:00
Ramy Medhat 2a3f05258b [DO NOT MERGE] Add a default value for the RBE_log_dir variable. am: 8229916e1f
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595317

Change-Id: I6b392c782a248305d7d4e8380d7695015b548c9e
2020-09-16 19:17:36 +00:00
Ramy Medhat 989f2d167f [DO NOT MERGE] Do not generate a random socket address for RBE if NOSTART_RBE is set. am: 00ff4cc1ef
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595316

Change-Id: I998695e40231538fdcaf485ba2e2dccad0480606
2020-09-16 19:17:33 +00:00
Ramy Medhat 6f14c848ca [DO NOT MERGE] Add reasonable defaults to RBE configuration parameters. am: 19c39230e9
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595315

Change-Id: I684817629c64a6cb8a953ce10a7e98548343e1ee
2020-09-16 19:17:31 +00:00
Patrice Arruda 7abebafe4e [DO NOT MERGE] Add newline to each generated bash script in rbe_test.go. am: b258d9367c
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12598126

Change-Id: I67d348cbdc93248976c1d22b1554e0c365e7644d
2020-09-16 19:17:27 +00:00
Ramy Medhat ee50d5b5b0 [DO NOT MERGE] Add Goma deprecation PSA to soong.
Test: build with Goma
Bug: b/166273085
Change-Id: Ie7e146285afd40bc0a1dc17a0b898960f57a118b
Merged-In: Ie7e146285afd40bc0a1dc17a0b898960f57a118b
2020-09-16 17:53:46 +00:00
Colin Cross 486337ea5f Add eakammer and patricearruda to OWNERS
Bug: b/168709033
Test: N/A
Change-Id: I4209fc5149dcce6bf75aef130afc69c4600c7f3b
Merged-In: I4209fc5149dcce6bf75aef130afc69c4600c7f3b
2020-09-16 17:17:44 +00:00
Patrice Arruda 13ea2a5da1 [DO NOT MERGE] Add RBE metrics dump in Soong UI. am: fbc4e3f9b6
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595314

Change-Id: I7086330bf6e2637cd3cc3ff494027aae7a30818e
2020-09-16 09:23:58 +00:00
Ramy Medhat 8743dfac7d [DO NOT MERGE] Generate RBE socket address randomly for each invocation of soong. am: 498a732edb
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595309

Change-Id: I474582214c26fab393c5b1cd48a932bdfa98fe62
2020-09-16 09:23:57 +00:00
Ramy Medhat 72510b31dd [DO NOT MERGE] Refactor RBE support for Javac/R8/D8 to use the remoteexec package. am: fc5a03c85c
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595308

Change-Id: I19e875e0ad156334e6368760999c446bf3dc7afe
2020-09-16 09:23:56 +00:00
Ramy Medhat 5f90e8d236 [DO NOT MERGE] Add support for remotely executing header-abi-dumper actions. am: afa3c5f2ed
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595307

Change-Id: I4d964589e83a3fa710fe3d375315f89149e09b87
2020-09-16 09:23:54 +00:00
Ramy Medhat 3fd34b853e [DO NOT MERGE] Allow remote execution of link actions. am: c726ab533c
Original change: https://googleplex-android-review.googlesource.com/c/platform/build/soong/+/12595306

Change-Id: I50619ab98b836bdecda1be03b44b8b5a940a9f19
2020-09-16 09:23:52 +00:00
Ramy Medhat 320c041797 [DO NOT MERGE] Make the default exec strategy for javac/r8/d8 remote local fallback.
Bug: b/166182389
Test: presubmits
Change-Id: Ia67c2a1e13638db1fb9c93755ce72159767d46b4
Merged-In: Ia67c2a1e13638db1fb9c93755ce72159767d46b4
2020-09-15 18:10:24 +00:00
Ramy Medhat 8229916e1f [DO NOT MERGE] Add a default value for the RBE_log_dir variable.
Bug: b/166182389
Test: built simple target.
Change-Id: Id93dd80119037b594e5f8c370f4ac2fbc12a5144
Merged-In: Id93dd80119037b594e5f8c370f4ac2fbc12a5144
2020-09-15 18:10:24 +00:00
Ramy Medhat 00ff4cc1ef [DO NOT MERGE] Do not generate a random socket address for RBE if NOSTART_RBE is set.
Bug: b/166182389
Test: A build with NOSTART_RBE=1
Change-Id: I5872632442c754b597901b7729eed597ee45859f
Merged-In: I5872632442c754b597901b7729eed597ee45859f
2020-09-15 18:10:24 +00:00
Ramy Medhat 19c39230e9 [DO NOT MERGE] Add reasonable defaults to RBE configuration parameters.
Bug: b/166182389
Test: simple one action build
Change-Id: Ic66ad2b89866a67008950035bc3b559dae4e3a3e
Merged-In: Ic66ad2b89866a67008950035bc3b559dae4e3a3e
2020-09-15 18:10:24 +00:00
Patrice Arruda b258d9367c [DO NOT MERGE] Add newline to each generated bash script in rbe_test.go.
"exec format error" showed up on Mac when running the rbe_test.go.
The generated bash scripts did not have a newline at the end of
the file.

Bug: b/162600461
Fixes: b/162600461

Test: "m blueprint_tools" on Android Mac builder.
Change-Id: I425fa5f428733b6569c3fcdaa312023f9761d020
Merged-In: I425fa5f428733b6569c3fcdaa312023f9761d020
2020-09-15 18:10:13 +00:00
Patrice Arruda fbc4e3f9b6 [DO NOT MERGE] Add RBE metrics dump in Soong UI.
From aosp/1329396, the RBE metrics protobuf file is part of the
metrics uploading process. The RBE metrics protobuf file is
generated by running the bootstrap shutdown command. A new function
named DumpRBEMetrics was written in order to generate the RBE metrics
protobuf file before sending to the uploading process.

Bug: b/140638454
Test: * Unit test cases
      * Ran RBE build on my local host and verified the
        metrics protobuf file is created.
      * Ran non-RBE build after RBE build and verified that
        the previous metrics protobuf file was deleted.

Change-Id: I4b8068905cb67c4b8c2d94793917b98974fed707
Merged-In: I4b8068905cb67c4b8c2d94793917b98974fed707
2020-09-15 13:05:11 +00:00
Ramy Medhat 498a732edb [DO NOT MERGE] Generate RBE socket address randomly for each invocation of soong.
Test: started soong multiple times and saw different socket files in
out/soong/.temp

Bug: b/166182389
Change-Id: Ib0087dd3867bdbf584c6868196070840cff291aa
Merged-In: Ib0087dd3867bdbf584c6868196070840cff291aa
2020-09-15 10:22:12 +00:00
Ramy Medhat fc5a03c85c [DO NOT MERGE] Refactor RBE support for Javac/R8/D8 to use the remoteexec package.
Bug: b/166182389
Test: build with and without RBE_JAVAC/RBE_R8/RBE_D8 set.
Change-Id: I1607f8cfb4d2c3cbb3672152bbeb561d9968bc60
Merged-In: I1607f8cfb4d2c3cbb3672152bbeb561d9968bc60
2020-09-15 10:16:51 +00:00
Ramy Medhat afa3c5f2ed [DO NOT MERGE] Add support for remotely executing header-abi-dumper actions.
Bug: b/166182389
Test: built with and without RBE_ABI_DUMPER set.
Change-Id: Ie752c07325453076d191813eaa50da03e3a0c2d8
Merged-In: Ie752c07325453076d191813eaa50da03e3a0c2d8
2020-09-15 09:57:12 +00:00