Commit Graph

306 Commits

Author SHA1 Message Date
Dan Willemsen ce41e943be Remove support for ONE_SHOT_MAKEFILE
Test: treehugger
Change-Id: Icdbe62bcb7bc4717228c5d974962b939d8eafee6
2019-07-31 18:11:27 -07:00
Elliott Hughes 521602194b Merge "Switch to toybox egrep(1) and grep(1)." 2019-07-31 16:55:43 +00:00
Dan Willemsen a2a8ecb330 Remove reference to PRODUCT-*
The code still needs to exist, but for buildspec.mk files, not the
PRODUCT-* arguments.

Test: none
Change-Id: I60c6044ab2d1627af7e6e2e8831b77d9db12aa55
2019-07-29 15:14:11 -07:00
Kousik Kumar 61afb56e84 Add dist_dir to goma trigger
This is done so that prototype tooling can write logs inside the DIST_DIR.

Test: I wrote out all the environment variables being passed to
goma_ctl.py and the output is https://paste.googleplex.com/6663469954760704. It contains both DIST_DIR and other variables.

Change-Id: Ia5c70e58aa1b87db9777c59cf3e5df1c9e20f08c
2019-07-23 16:16:15 -04:00
Elliott Hughes 7ceddbae8e Merge "Switch to prebuilt bzip2." 2019-07-23 19:28:02 +00:00
Elliott Hughes 2ee8b33edc Switch to prebuilt bzip2.
Test: treehugger
Change-Id: I4953ff6f9ead5ad79519ac7fa71b557807f25851
2019-07-23 15:08:58 +00:00
Ramy Medhat 2b78fda705 Merge "Add USE_RBE support to soong." 2019-07-23 11:49:10 +00:00
Treehugger Robot 1ecfa940d1 Merge "Reland "Switch to toybox tar."" 2019-07-23 02:08:40 +00:00
Ramy Medhat bbf2567cc1 Add USE_RBE support to soong.
Test: Built aosp_arm-user with and without USE_RBE. USE_RBE uses
a proxy script in place of rewrapper.

Change-Id: I5bf008a940513872d70b5b215bd6209f759826ae
2019-07-17 18:46:47 +00:00
Patrice Arruda 19bd53e39c soong_ui: Do not clean the OutDir path multiple times.
If OUT_DIR has been specified through the command line arguments,
it is sanitized first and NewConfig sets the OUT_DIR in the build
environment. When calling OutDir from Config instance, the out
was being sanitized again which is unnecessary.

Bug: b/118730755
Test: Ran unit test cases, "lunch" followed by "m" command.
Change-Id: Ieb4ffa1026c802c76c7369e8cdb3923a4c70f4b3
2019-07-16 11:22:22 -07:00
Patrice Arruda 0dcf27f639 soong_ui: Construct the proper MODULES-IN-<dir name> for m* build commands.
Previously, any of the build commands (mma, mmma, etc...) would traverse
up the directory path until an Android build file has been found. That
directory path would eventually become the MODULES-IN-<dir name> build
target. The directory where the m* command was executed may contain a
build file in a sub directory. For that case, the build target is the
MODULES-IN-<dir name> where dir is the directory of the executed m*
command since the target has been constructed by the core makefile.

Bug: b/118730755
Test: Wrote and ran unit test cases. Ran "lunch", "m". Went to a
      directory in AOSP that had build files in sub directories and
      executed mma. Went to a directory that had only build files in
      parent directories and executed mma.

Change-Id: I9458cbc4bf0fc25bfb0848e96e92d261442d5cb0
2019-07-16 11:19:05 -07:00
Elliott Hughes a6d1d373ae Reland "Switch to toybox tar."
This reverts commit afeb6cb817.

Change-Id: I67733254fd7edad51ac3fbb18c929b109e86d32e
2019-07-12 15:57:33 +00:00
Patrice Arruda 9450d0b226 soong_ui: Do not find a build file if targets are specified.
For mmma and mmm, the findBuildFile function in config.go is invoked every time
for specified directories and directories with targets. For directories with
targets, an Android build file must exist in the directory where mmma and mmm
was invoked. There is no need to invoke findBuildFile function as a
simple check of the build file exists in the specified directory.

This is also refactoring the code for b/118730755

Bug: b/118730755
Test: Executed unit test cases through Intellij and executed mmma
command: "mmma external/protobuf:aprotoc external/bzip2".

Change-Id: I5428e3a3c36f77ff740617564b7853705521f29f
2019-07-11 14:38:19 -07:00
Elliott Hughes c5a34551e1 Switch to toybox egrep(1) and grep(1).
Test: treehugger
Change-Id: I9053f45d591a8cb53eb36e258ecb97b591d806a1
2019-07-09 08:51:30 -07:00
Justin Yun d5f6c8261a Rename product_services to system_ext
Bug: 134359158
Test: build and check if system_ext.img is created
Change-Id: Ice3229baf97a94c24a6eea22e7d4571941d7f843
2019-07-09 08:57:05 +00:00
Treehugger Robot f67e1bee4a Merge changes Icfc893c8,I40f03fc0
* changes:
  Allow debugging with SOONG_DELVE=<listen addr>
  Print ninja stdout live during the build
2019-07-04 03:34:22 +00:00
Colin Cross aa812d122c Allow debugging with SOONG_DELVE=<listen addr>
Allow running Soong in a headless delve debugger by passing
SOONG_DELVE=<listen addr> in the environment.

Bug: 80165685
Test: SOONG_DELVE=:1234 m nothing
Change-Id: Icfc893c8a8354a9bbc99112d9c83259cb41906d1
2019-07-03 16:33:31 -07:00
Colin Cross 7b97ecd1f5 Print ninja stdout live during the build
When running ninja stream stdout to the status writer.  This improves
the ninja -d explain behavior, and will also allow Soong to be put
into the ninja console pool letting it print timely output to the
console.

Bug:  80165685
Test: NINJA_ARGS="-d explain" m
Change-Id: I40f03fc01d837ad91d400eae378007a7ce807705
2019-07-03 16:33:10 -07:00
Patrice Arruda baba9a9be6 soong_ui: Build under a path that contains a symbolic link.
The build directory passed in to soong_ui can contain a
symbolic link. soong_ui was not evaluating the build
directory to retrieve the true path, hence failing to
execute the internal soong.

Fixes: b/135995632
Test: Unit test case and executed the scenario in the bug
      description.

Change-Id: I5779c6aa3f3183810437dbe2b2d4e40acbafb205
2019-07-03 13:03:23 -07:00
Treehugger Robot ba9a51979d Merge "Revert "Switch to toybox tar."" 2019-06-25 05:24:44 +00:00
Elliott Hughes afeb6cb817 Revert "Switch to toybox tar."
This reverts commit 3b584b7861.

Reason for revert: missing --mode.

Change-Id: I0cde94c38905e26c79c31a03fd0177d7d57daa32
2019-06-25 01:40:42 +00:00
Elliott Hughes 9b74dfe96c Merge "Switch to toybox tar." 2019-06-24 21:24:53 +00:00
Treehugger Robot 45cdb71290 Merge "Soong: Add BUILD_MODULES build action in soong_ui." 2019-06-21 03:11:34 +00:00
Patrice Arruda 3928206ecc Soong: Add BUILD_MODULES build action in soong_ui.
There was one case that did not work:
  1) Create dir in root source tree
  2) Run "m libc"

The build would fail as it was using the mma logic. A separate
one named BUILD_MODULES was added to allow building specific
modules in any directories using the "m" command.

Bug: b/130049705
Test: unit test cases, ran "m libc" in a tmp directory inside
      of the source tree.

Change-Id: I8d23e685a673a311001ee8edd89bd73b662392dd
2019-06-20 16:35:12 -07:00
Elliott Hughes 63e70530d3 Merge "Allow gzcat as an alias for gzip's -dc mode." 2019-06-20 21:03:11 +00:00
Elliott Hughes e671f5a397 Allow gzcat as an alias for gzip's -dc mode.
This gets us closer to toybox tar (and toybox gzip/gzcat).

Test: treehugger
Change-Id: If934270439b258eae55cdd3e97ac0e83b1c79c70
2019-06-19 14:02:02 -07:00
Dan Willemsen c69d715d72 Use prebuilt m4 instead of system m4
Bug: 117561006
Test: treehugger
Change-Id: I588d867d95cc52963ce764018a9c09b26973c8c3
2019-06-19 12:50:32 -07:00
Treehugger Robot eec8d3aee3 Merge "Soong: Fix the package name in metrics.proto file" 2019-06-17 23:42:33 +00:00
Elliott Hughes 3b584b7861 Switch to toybox tar.
Test: treehugger
Change-Id: I4c561186c14cc44782a9e5c83d4d9c002addb3f6
2019-06-17 16:39:35 -07:00
Elliott Hughes 2d10e3f54e Merge "Revert "Revert "Switch to toybox timeout.""" 2019-06-17 23:35:16 +00:00
Elliott Hughes 0e868d0391 Revert "Revert "Switch to toybox timeout.""
This reverts commit 50e91c1a7a.

Reason for revert: SIGRTMIN+2 support added.

Change-Id: I3cb251a102e427335315c4931e23a69b95253ee7
2019-06-17 16:45:15 +00:00
Patrice Arruda 0cc5b21516 Soong: Fix the package name in metrics.proto file
The metrics.proto needs to be imported to Google3 in order to unmarshal
the build metrics data from bigstore. Cleaned up the enum names to use
the Camel naming convention and renamed the build_metrics to
soong_metrics to be more specific to soong and to allow quering
soong_metrics files from bigstore.

Bug: b/135280521
Test: lunch andf m -j. Checked if soong_metrics was generated correctly
      by using printproto command.

Change-Id: I998c8d05db592e94a653d6ca32250b80df3c9b21
2019-06-14 15:27:46 -07:00
Treehugger Robot 49cd4954a0 Merge "soong_ui: Add build actions commands in soong_ui." 2019-06-13 23:14:56 +00:00
Patrice Arruda 1384822a63 soong_ui: Add build actions commands in soong_ui.
Add the following build actions {BUILD_MODULES_IN_A_DIRECTORY,
BUILD_MODULES_IN_DIRECTORIES} in soong_ui config so the bash code version of
build commands (m, mm, mma, mmm, mmma) in build/make/envsetup.sh can be deprecated.
This is to allow up to date bug fixes on the build commands.

Bug: b/130049705
Test: Unit test cases
Change-Id: I772db1d4e9c1da5273374d1994eb5e8f17cd52f2
2019-06-13 13:15:12 -07:00
Colin Cross 097ed2a37c Remove terminal.Writer
terminal.Writer is now just a wrapper around stdio.Stdout() without
any useful functionality.  Replace it with stdio.Stdout() as an
io.Writer.

Test: ui/terminal/status_test.go
Change-Id: I5bc5476afdca950b505642f0135a3af9d37fbe24
2019-06-12 21:30:56 -07:00
Elliott Hughes bc31b9190a Switch to toybox date.
Test: treehugger
Change-Id: I558870092c29946419b9e15c33a6ee8ee7c6088f
2019-06-11 10:09:08 -07:00
Lokesh Gidra 50e91c1a7a Revert "Switch to toybox timeout."
This reverts commit e9f471635f.

Reason for revert: Multiple art-test failures

Change-Id: I550864c23485a438889ab73c7e4a877cd730cabb
2019-06-08 00:46:09 +00:00
Elliott Hughes c1bfeed81b Switch to toybox sed (again).
The pathological case has been fixed, so let's see if it sticks this
time.

Test: treehugger
Change-Id: I043d823b05dd8796764e008c5e16e3563b80cec8
2019-06-07 13:10:04 -07:00
Elliott Hughes e9f471635f Switch to toybox timeout.
Test: treehugger
Change-Id: Ieea84deac5d0ccef0f3d4949aa1ba870d406e482
2019-06-06 21:41:48 -07:00
Dan Willemsen edff034366 Remove BUILD_BROKEN_ANDROIDMK_EXPORTS
Bug: 73959648
Test: treehugger
Change-Id: I932cc3acc6345f8b1ad8e395235489ae8275e2db
2019-05-25 17:13:55 -07:00
Dan Willemsen 7101d44442 Remove BUILD_BROKEN_ENG_DEBUG_TAGS
All users have been removed.

Test: treehugger
Change-Id: I35e55f3ce79c8e68585be120b5dec810563c7a83
Merged-In: I35e55f3ce79c8e68585be120b5dec810563c7a83
2019-05-25 22:48:12 +00:00
Amin Shaikh de730781d9 Make soong lock file sync time configurable..
- Add SOONG_LOCK_TIMEOUT environmental variable for users to set a
duration (e.g. 1h), which defaults to the current 10 seconds.
- Instead of logging once per second waiting to acquire the lock,
only log once before the first wait

Fixes: 131854539
Test: make; open another tab and make
Change-Id: I0a02a1b071964f74473a5f55034f916526e1b309
2019-05-24 17:10:48 -04:00
Elliott Hughes 18e0d5e0f0 Go back to the host sed until we have a faster prebuilt.
Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I120fe1200216afdb492bd1353848d12a4be1f496
2019-05-09 13:59:01 -07:00
Dan Willemsen 7e52fa9f33 Don't dump BUILD_BROKEN_USES_BUILD_[HOST_]SHARED_TEST_LIBRARY
These have been errors for years, and are now marked as obsolete.

Test: treehugger
Change-Id: I8055ab689ce5085df763378ad40d9b9aaa7ca85b
2019-05-08 12:38:49 -07:00
Treehugger Robot 6ff39230e8 Merge "Support install-clean and data-clean" 2019-05-03 23:38:40 +00:00
Colin Cross 806fd94bed Support install-clean and data-clean
Allow install-clean as an alias for installclean and data-clean as
an alias for dataclean.

Test: m install-clean
Change-Id: I9c97d60572a524ad68caecd26d52bfb987468075
2019-05-03 13:38:57 -07:00
Elliott Hughes b0e7320fbc Merge "Allow the host expr." 2019-05-03 17:24:00 +00:00
Elliott Hughes 3a653f419b Allow the host expr.
(Kernel builds require GNU unary + extension.)

Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I404db44d2ad0503d571d6f554a20445ab36aa7de
2019-05-01 12:52:25 -07:00
Elliott Hughes 6b3a0ddf0b xz: force use of the prebuilt.
Also comment why we can't yet do the same for bzip2.

Test: treehugger
Change-Id: I34d5bc0076f5790fcb01c185ef6d08824a4f6142
2019-04-30 22:20:43 -07:00
Elliott Hughes 3d87302a6d Revert "bzip2/xz: use the prebuilts."
This reverts commit e114b3f0a9.

Build broke as follows:

out/soong/host/linux-x86/bin/sbox --sandbox-path out/soong/.temp --output-root out/soong/.intermediates/system/update_engine/ue_unittest_disk_imgs/gen -c 'tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C __SBOX_OUT_DIR__/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img' __SBOX_OUT_DIR__/gen/disk_ext2_1k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k_empty.img __SBOX_OUT_DIR__/gen/disk_ext2_unittest.img 
"bzip2" is not allowed to be used. See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools for more information. 
tar: Child returned status 1 
tar: Error is not recoverable: exiting now 
sbox command (tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C out/soong/.temp/sbox849645882/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img) failed with err "exit status 2" 

Change-Id: I3fe7100886ea3b748f5a6b5fc6d2811391bb72a8
2019-04-30 21:55:23 +00:00