Commit Graph

280 Commits

Author SHA1 Message Date
Elliott Hughes 18e0d5e0f0 Go back to the host sed until we have a faster prebuilt.
Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I120fe1200216afdb492bd1353848d12a4be1f496
2019-05-09 13:59:01 -07:00
Dan Willemsen 7e52fa9f33 Don't dump BUILD_BROKEN_USES_BUILD_[HOST_]SHARED_TEST_LIBRARY
These have been errors for years, and are now marked as obsolete.

Test: treehugger
Change-Id: I8055ab689ce5085df763378ad40d9b9aaa7ca85b
2019-05-08 12:38:49 -07:00
Treehugger Robot 6ff39230e8 Merge "Support install-clean and data-clean" 2019-05-03 23:38:40 +00:00
Colin Cross 806fd94bed Support install-clean and data-clean
Allow install-clean as an alias for installclean and data-clean as
an alias for dataclean.

Test: m install-clean
Change-Id: I9c97d60572a524ad68caecd26d52bfb987468075
2019-05-03 13:38:57 -07:00
Elliott Hughes b0e7320fbc Merge "Allow the host expr." 2019-05-03 17:24:00 +00:00
Elliott Hughes 3a653f419b Allow the host expr.
(Kernel builds require GNU unary + extension.)

Bug: https://issuetracker.google.com/131747477
Test: treehugger
Change-Id: I404db44d2ad0503d571d6f554a20445ab36aa7de
2019-05-01 12:52:25 -07:00
Elliott Hughes 6b3a0ddf0b xz: force use of the prebuilt.
Also comment why we can't yet do the same for bzip2.

Test: treehugger
Change-Id: I34d5bc0076f5790fcb01c185ef6d08824a4f6142
2019-04-30 22:20:43 -07:00
Elliott Hughes 3d87302a6d Revert "bzip2/xz: use the prebuilts."
This reverts commit e114b3f0a9.

Build broke as follows:

out/soong/host/linux-x86/bin/sbox --sandbox-path out/soong/.temp --output-root out/soong/.intermediates/system/update_engine/ue_unittest_disk_imgs/gen -c 'tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C __SBOX_OUT_DIR__/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img' __SBOX_OUT_DIR__/gen/disk_ext2_1k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k.img __SBOX_OUT_DIR__/gen/disk_ext2_4k_empty.img __SBOX_OUT_DIR__/gen/disk_ext2_unittest.img 
"bzip2" is not allowed to be used. See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools for more information. 
tar: Child returned status 1 
tar: Error is not recoverable: exiting now 
sbox command (tar -jxf system/update_engine/sample_images/sample_images.tar.bz2 -C out/soong/.temp/sbox849645882/gen disk_ext2_1k.img disk_ext2_4k.img disk_ext2_4k_empty.img disk_ext2_unittest.img) failed with err "exit status 2" 

Change-Id: I3fe7100886ea3b748f5a6b5fc6d2811391bb72a8
2019-04-30 21:55:23 +00:00
Elliott Hughes e114b3f0a9 bzip2/xz: use the prebuilts.
We added the prebuilts a while ago, but forgot to clean this up.

Test: treehugger
Change-Id: I8920f0c9d77951332a663750102ceeabcb565970
2019-04-29 16:23:08 -07:00
Dan Willemsen 2fbb7571bf Obsolete BUILD_BROKEN_DUP_COPY_HEADERS
There are no remaining users.

Test: treehugger
Change-Id: If8719480f73a8fa009d9733479390d2abd4021b1
2019-04-19 11:22:27 -07:00
Dan Willemsen 6097746769 Obsolete BUILD_BROKEN_PHONY_TARGETS
There are no remaining users.

Also mark BUILD_BROKEN_ANDROIDMK_EXPORTS and BUILD_BROKEN_ENG_DEBUG_TAGS
as deprecated in scripts/build_broken_logs.go

Test: treehugger
Change-Id: If7892bef1b9001f12a99565f886b395cf1985e70
2019-04-19 11:16:52 -07:00
Dan Willemsen 368e55671b Dump make deprecation status to logs
So that the output of build_test can be parsed to understand which
devices need changes.

Test: read out/soong.log
Change-Id: I40b25527ef72e3f646c0cfe62114b452fa3f27f6
2019-04-17 14:44:33 -07:00
Dan Willemsen 25e6f09c06 Add BUILD_BROKEN_USES_NETWORK
Some people apparently still talk to the network during their build.
Allow this temporarily with a BUILD_BROKEN_USES_NETWORK check.

Bug: 129992021
Test: attempt to talk to the network during the build with and without
      this flag.
Change-Id: Ifb967c656aa24c4599e7232d0f1b5a303b5bac52
2019-04-09 10:23:49 -07:00
Elliott Hughes c2dbadc363 Add toybox seq.
Bug: https://issuetracker.google.com/129992021
Test: builds
Change-Id: I63d5b3fd1c71b771e995638094ee4310435105ee
2019-04-08 09:19:45 -07:00
Colin Cross 1aeb049a54 Panic if logging is attempted after Close
Attempt to catch places where logs are truncated by panicing if
logging is attempted after Close.

Test: m nothing
Change-Id: If670f20d08832ed65b63af5589b548e9815f2f0d
2019-03-21 21:17:04 -07:00
Colin Cross b98d3bcf4e Wait for ninja proto processing to finish when exiting
Wait for the ninja proto processing goroutine to notice the fifo
has closed and exit before continuing.

Test: m nothing
Change-Id: I8cf5f3b8bf6a91496c6d2bbbd3e811eb7f0c9d21
2019-03-21 21:17:04 -07:00
Steven Moreland 0002028c09 dumpvars += BOARD_VNDK_VERSION
Dump variable in log to track remaining targets which don't set it.

Bug: 122954981
Test: $ grep VNDK out/soong.log
2019/03/07 09:22:04.235604 build/soong/ui/build/dumpvars.go:109: BOARD_VNDK_VERSION current

Change-Id: I34146c752a4bf0d799374bb93cc0d75b0f1b048c
2019-03-07 09:28:35 -08:00
Dan Willemsen 0f0214699e Use python / python2.7 prebuilts
Bug: 117811537
Test: treehugger
Change-Id: I35b335edcfa202a45f55f588a643c62c2ced0f26
2019-02-18 19:34:00 +00:00
Elliott Hughes 47de2a2f52 Switch sed(1) to toybox.
Test: treehugger
Change-Id: I575ae9db8d94cbec811d0029691df98373f8fe35
2019-02-15 15:49:12 -08:00
Dan Willemsen 733547d522 Use prebuilt awk on Darwin too
Test: build on mac
Change-Id: I25d21c0df0b9023d6c43b11dde60e6066f107576
2019-02-14 20:11:26 -08:00
Dan Willemsen 9121973130 Stop calling our host $PATH prebuilts toybox
There's more than just toybox in this group now, so let's rename our
variables to something closer to the desired behavior, rather than the
first user.

Test: treehugger
Change-Id: I76d4407792061c8110b194cfe73f1ddc84dbc22f
2019-02-14 20:07:02 -08:00
Yoshisato Yanagisawa 02eaccafbd More verbose message on goma_ctl.py ensure_start failure.
For ease of investigating the issue with error reports, let me make
the error message more verbose.

Test: make goma_ctl.py raise Exception and run.
Test: I confirmed verbose error message shown.
Change-Id: I04f9a5ca4808746994c41ddce8a5b22a1975fcc2
Signed-off-by: Yoshisato Yanagisawa <yyanagisawa@google.com>
Bug: 124218940
2019-02-13 10:49:10 +09:00
Elliott Hughes a0f23ad233 Merge "Switch to one-true-awk." 2019-02-13 00:56:35 +00:00
Sasha Smundak c0c9ef9964 Provide an option to reduce build's verbosity.
If ANDROID_QUIET_BUILD environment variable is set to '1', do not show 20
lines of the environment variables. In addition, for the failing step show
only the step name and output, omitting the command proper (the verbose log
as well as error.log still contains it).
If build succeeds, the output of the build itself consists of a single
line:
```
> m androidmk
[100% NN/NN] <last command>
```

When it fails, the output does not contain sometimes very long command
line:
```
> m androidmk
[ 97% NN/MM] test androidmk
FAILED: <step>
--- FAIL: TestEndToEnd (0.01s)
    androidmk_test.go:1025: failed testcase 'prebuilt_etc_TARGET_OUT_ETC'
        input:
        include $(CLEAR_VARS)
        LOCAL_MODULE := etc.test1
        LOCAL_MODULE_CLASS := ETC
        LOCAL_MODULE_PATH := $(TARGET_OUT_ETC)/foo/bar
        include $(BUILD_PREBUILT)

        expected:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",
        }

        got:
        prebuilt_etc {
            name: "etc.test1",
            filename: "foo/bar",

        }

FAIL
17:50:53 ninja failed with: exit status 1
```
[The related change in build/make/envsetup.sh suppresses timing display
when the same variable is set.]

Change-Id: I4d3c72457de031ff58a324c2fe98f4c1d10f8239
Test: treehugger
2019-02-07 12:21:45 -08:00
Elliott Hughes cc74d59a80 Switch to one-true-awk.
Test: treehugger
Change-Id: I297d21f5a463c4d3070335c7533972397014f171
2019-02-06 18:06:48 +00:00
Dan Willemsen 01f0a05361 Add BUILD_BROKEN_ENG_DEBUG_TAGS to soong.log
And add a helper script that can help parse the output from
build_test.bash

Test: check for BUILD_BROKEN_ENG_DEBUG_TAGS in soong.log
Test: go run ../build/soong/scripts/build_broken_logs.go *
Change-Id: Idd0fc8b59770dcdbe44eeba262558708a9497f96
2019-02-05 15:15:16 -08:00
Dan Willemsen dcac078c91 Turn on more warnings during CleanSpec processing
We don't want these to be warnings, may as well be errors instead of
turning off the warnings.

Bug: 123583617
Test: treehugger
Change-Id: I6ca518c9647e712426952cb88bdb044d933b23f2
2019-02-04 20:58:25 -08:00
Treehugger Robot de53d15b0f Merge "Increase max line length for KatiReader bufio.Scanner" 2019-01-31 01:46:07 +00:00
Colin Cross 75ec1ec13f Increase max line length for KatiReader bufio.Scanner
bufio.Scanner defaults to a relatively small max line length of
64kB.  Kati output may have long lines, use a max length of 2MB
instead.

Fixes: 123590367
Test: m checkbuild
Change-Id: Ia869c5c2526621244f0065736e0d5fd9a088df06
2019-01-30 23:19:23 +00:00
Dan Willemsen ef661b7f18 Fix runtime panics being suppressed
fatalLog was matching runtime errors, and essentially hiding them.

Test: m blueprint_tools
Change-Id: Ib48e7e142fc096998bc14b21fb717392adcff0ec
2019-01-28 20:00:01 -08:00
Treehugger Robot ee04139154 Merge "Add --top_level_phony to catch more real-to-phony problems" 2019-01-26 05:08:15 +00:00
Elliott Hughes 48d716c367 Switch cp(1) to toybox.
Test: treehugger
Change-Id: I908e1d6aa2aa7016432ac7ff4abc56e8163bad8a
2019-01-25 22:29:58 +00:00
Dan Willemsen cc62890866 Add --top_level_phony to catch more real-to-phony problems
As more phony rules move either to the packaging kati run, or soong, the
existing real-to-phony check failed to notice that they were phony
rules.

So I added --top_level_phony to Make that assumes that all leaf nodes
(source files, or nodes generated by another ninja file generator) that
do not have a '/' in them are phony targets.

The existing phony-looks-real check in Kati should prevent these from
being real generated files, and our source tree shouldn't have any
source files used at the root level.

Test: build_test on downstream branches
Change-Id: I77cff84b536c6cd6402307a9aa5a9c273e72c71f
2019-01-24 15:53:06 -08:00
Elliott Hughes d5fdd3a134 Switch hostname(1) to toybox.
Test: treehugger
Change-Id: I52908a14da86ed8f162150bb48f72e141e6f060c
2019-01-24 15:18:06 -08:00
Elliott Hughes d2c476cd2d Revert "Switch to toybox hostname(1)."
This reverts commit b115b81095.

Reason for revert: http://b/123123255

Change-Id: I0f2bfebf1189fdc71e093538a8b228dd0e0b6946
2019-01-19 18:55:54 +00:00
Elliott Hughes 44f0d4e1fc Switch to toybox which(1).
Test: treehugger
Change-Id: Ic14692b90c2b1246c1c8f72d00d6e0a5223aaf70
2019-01-19 17:43:06 +00:00
Elliott Hughes db96fc8744 Switch to toybox pgrep(1), pkill(1), and ps(1).
Test: treehugger
Change-Id: Ic357b317e0233a626c89c38df152c3361ee8ea82
2019-01-19 17:41:53 +00:00
Elliott Hughes b115b81095 Switch to toybox hostname(1).
Test: treehugger
Change-Id: Ibf89209befc8564b308c4b26bf08cef721938099
2019-01-18 15:11:22 -08:00
Elliott Hughes 46e68636f8 Switch to toybox wc(1).
Test: treehugger
Change-Id: I14d22406c1a462d4075a8538e549a8d0f5b88e8d
2019-01-18 08:28:54 -08:00
Elliott Hughes c97a91183d Switch to toybox md5sum/sha1sum/sha256sum/sha512sum.
Test: treehugger
Change-Id: Icc8c3d5765d73f191e47adfe0a9d68727d075e50
2019-01-17 12:37:40 -08:00
Elliott Hughes 3aa6533f39 Merge "Switch to toybox mv(1)." 2019-01-17 18:32:08 +00:00
Dan Willemsen 3a4dbd651f Fix nested minijail0 execution
We've got an internal testcase that uses minijail0 inside an Android.mk
rule. That was failing since we turned on the linux sandbox, as /proc
was mounted read-only, which prevented setting up the uid/gid mappings
for a child namespace.

Fixes: 122985455
Test: treehugger & forrest of breaking build
Change-Id: Ia77a91a7f4eeeb8a24e84075d8272287f5087587
2019-01-16 23:04:16 -08:00
Elliott Hughes 6064792393 Switch to toybox mv(1).
Test: treehugger
Change-Id: If4e53cc8d088c7b79399ae8e10601c9b22bd118d
2019-01-16 21:54:30 +00:00
Elliott Hughes c0cf454a63 Switch to toybox rm(1).
Test: treehugger
Change-Id: Ib31182ca7ce827396476699ac9d1c636cdd324fc
2019-01-16 17:44:29 +00:00
Dan Willemsen 24024eafee Effectively disable network access during the build
This starts a new network namespace without any connections to the
outside.

Bug: 122270019
Test: USE_GOMA=true m libc
Test: treehugger
Test: add rule to use /usr/bin/wget, fails after this change
Change-Id: Iba262025ce0e4e3bef5c34c817cc678d6c61403b
2019-01-15 16:58:27 -08:00
Dan Willemsen 63663c6bc9 Implement linux sandboxing with nsjail
This really only initializes the sandbox, it does not attempt to change
the view of the filesystem, nor does it turn off networking.

Bug: 122270019
Test: m
Test: trigger nsjail check failure; lunch; m; cat out/soong.log
Test: USE_GOMA=true m libc
Change-Id: Ib291072dcee8247c7a15f5b6831295ead6e4fc22
2019-01-15 13:47:31 -08:00
Elliott Hughes 713ae39f83 Move to toybox sort(1).
Test: treehugger
Change-Id: Ide649bbe9a73ab0e51d9bf4242d5c3c9172a85d6
2019-01-14 21:56:58 +00:00
Elliott Hughes 61456d1515 Switch to toybox readlink(1).
Test: treehugger
Change-Id: Ie1f6be5c2f273809a2691e506fc34c6840c942dd
2019-01-14 20:23:10 +00:00
Elliott Hughes 6712c0eb87 Switch to toybox xargs(1).
Test: treehugger
Change-Id: I8db68577498c1f6c0e1fdff951bc8972184b2754
2019-01-14 17:25:29 +00:00
Elliott Hughes f1f016eec1 Move mktemp(1) to toybox.
Test: clean build
Change-Id: I7b02c7ee8dafb504c90d3af63698a09627556947
2019-01-12 17:51:43 +00:00