Commit Graph

357 Commits

Author SHA1 Message Date
Paul Duffin 3e2db5c10b Make copyBootJarsToPredefinedLocations simpler and less fragile
Previously, copyBootJarsToPredefinedLocations relied on all its
parameters having the same length and the same order. That made it
quite fragile as changes to one of the parameters without corresponding
changes to the other would cause failures. It also combined the
retrieval of the boot dex jars from the modules, handling of missing
boot dex jar files and the generation of the rules to copy the files.

This change separates the retrieval of boot dex jars and handling of
missing files from the copying of those files while at the same time
making the function less fragile by replacing the three ordered
parameters with two maps that shared common keys.

Bug: 179354495
Test: m nothing
Merged-In: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
Change-Id: Idbcd24a7e8af89f7895a20aeddc58502dcbaad03
(cherry picked from commit 5f148ca7cf2e9a9478922577b7ed70b1caacb55e)
2021-06-16 03:06:35 +01:00
satayev 95bfbb169f Populate individual classpath_fragments' classpaths.proto configs.
To avoid duplicates on *CLASSPATH environ variables at runtime, remove
split entries from platform-*classpath, i.e. all updatable jars that
have their own classpath fragments should not appear in the
platform-*classpath's classpaths.proto config.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: Id2759ab8e106cc183e695bf3509a6ab60ab0ef2a
2021-05-20 19:07:29 +01:00
satayev 3db3547d1f Generate empty classpaths.proto for bootclasspath_fragment.go.
- Adds all required details for bootclasspath_fragment to implement
  classpath_fragment.
- Keeps the actual boot jars in platform-bootclasspath to begin with.
- Makes sure to put the file in apex/etc/classpath on device. Note that
  for platform versions of classpath fragment AndroidMkEntries perform
  the installation, while for APEXes it must be plumbed via apex.go.

Bug: 180105615
Test: m && launch_cvd; atest CtsClasspathsTestCases
Change-Id: I6101ebdf5b8bcbe95c0b7ce21f3f67a2685aef50
2021-05-11 10:34:47 +01:00
Treehugger Robot 7548b1ad05 Merge "Remove draft API support." 2021-04-15 22:58:15 +00:00
Treehugger Robot d820c37425 Merge "Fix evaluation order of (Cfi|Memtag) exclude paths." 2021-04-15 03:49:38 +00:00
Dan Albert d05ba00415 Remove draft API support.
This is unused and doesn't work with our current API review process
anyway (un-drafting an API won't be flagged for API council review).

Test: treehugger
Bug: None
Change-Id: I6d8fcc9885b82dac5ada7772d9e3fb9101524ece
2021-04-13 15:55:47 -07:00
Evgenii Stepanov 779b64ec5d Fix evaluation order of (Cfi|Memtag) exclude paths.
Before this change, exclude paths disabled sanitization of targets that
would otherwise be enabled by SanitizeDevice product variable (aka
SANITIZE_TARGET).

With this change, in addition to the above logic, exclude path disables
sanitization of targets that would otherwise be enabled by the
corresponding include path.

Effectively, this change disables sanitization of targets that are
covered by *both* include and exclude paths.

Test: build/soong/cc/cc_test.go
Bug: b/184976817
Change-Id: I96f1e2a808d88b352b92b7490fa1df5c010314c3
2021-04-13 10:29:15 -07:00
Lukacs T. Berki 53b2f36bd6 Delete android/env.go .
Properly plumb the available environment to the configuration instead of
going through the global variable originalEnv.

Test: Presubmit.
Change-Id: Ia1ea753d5e72c02a0dcaa4c0b43bd6e24fc47cec
2021-04-13 08:10:01 +02:00
Hridya Valsaraju 5a5c7d58df Add a variable that indicates build-time debugfs restrictions
This patch adds a variable to indicate to check if the target has the
flag PRODUCT_SET_DEBUGFS_RESTRICTIONS enabled.

Bug: 184381659
Test: make with/without PRODUCT_SET_DEBUGFS_RESTRICTIONS set
Change-Id: Ic14a3759b155ed4e16048d62fcd990f4ba7c906a
2021-04-02 16:45:24 -07:00
Jingwen Chen 95c6eb3959 Merge "bp2build: add allowlist for package-level conversions." 2021-03-24 21:38:07 +00:00
Jingwen Chen 12b4c2706d bp2build: add allowlist for package-level conversions.
This CL adds the support for specifying lists of directories in
build/soong/android/bazel.go, which are then written into
out/soong/bp2build/MANIFEST. Using this configuration,
modules/directories can either default to bp2build_available: true or
false, while still retaining the ability to opt-in or out at the module level.

It also ensures that ConvertWithBp2Build returns true iff the module
type has a registered bp2build converter.

Test: go tests
Test: demo.sh full
Test: TreeHugger presubmits for bp2build and mixed builds.

Change-Id: I0e0f6f4b1b2ec045f2f1c338f7084defc5d23a55
2021-03-24 02:27:19 -04:00
Inseob Kim 7b85eeb41e Add WithDexpreopt soong variable
Test: m selinux_policy on sc-arc
Change-Id: I9e8ba5746e4648a04fe4d8fbe5d3403ba06b5914
2021-03-23 20:52:24 +09:00
Inseob Kim 0c4eec85fa Fix default value of BoardSepolicyVers
PlatformSepolicyVersion and BoardSepolicyVers haven't assigned correctly
so far. Below is the reason why it hasn't been discovered yet.

DeviceConfig.PlatformSepolicyVersion() was added to support mixed
sepolicy build (setting BOARD_SEPOLICY_VERS and building vendor sepolicy
with old plat policy files). Soong compares PlatformSepolicyVersion()
and BoardSepolicyVers(), and used old vendor sources if both are
different. Back then, the only place where such logic played a role was
selinux contexts. Test codes were running as intended: after setting
BOARD_SEPOLICY_VERS the context files were built against old policies.

But there were two mistakes:

1) PlatformSepolicyVersion() was not added to soong_config.mk, so it was
always empty.
2) BOARD_SEPOLICY_VERS was set to default in system/sepolicy/Android.mk,
which was processed after soong_config. So if BOARD_SEPOLICY_VERS was
not set in BoardConfig.mk, BoardSepolicyVers() was empty, not
PLATFORM_SEPOLICY_VERSION.

And there were no issues as Soong only checked equality.

To fix the issue correctly, this commit adds the variable
PlatformSepolicyVersion, and then handles BoardSepolicyVers correctly by
returning PlatformSepolicyVersion if it's empty.

Test: set BOARD_SEPOLICY_VERS and see vendor_file_contexts changes
Change-Id: Ie55827f47d2903e9739eccacc3874d354d5c9f2d
2021-03-22 22:34:56 +09:00
Colin Cross d2db7953a0 Merge changes Iab4e09d9,Icf2f24dd,I15be5ef1,Ic0db9619
* changes:
  Run lint actions in sbox
  Support sbox-in-RBE
  Move android package on top of remotexec
  Support sandboxing inputs in RuleBuilder
2021-03-19 17:25:14 +00:00
Inseob Kim fc8e9c7f00 Merge "Add sepolicy related variables" 2021-03-19 00:07:58 +00:00
Colin Cross 77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Treehugger Robot d19b824e7f Merge "Check for frameworks/base/Android.bp instead of frameworks/base" 2021-03-18 20:56:54 +00:00
Lukács T. Berki 734b215d66 Merge "Compile soong_build for debugging if needed." 2021-03-18 08:58:16 +00:00
Paul Duffin 3bf545fa6b Merge "Convert android/arch_test.go to test fixtures" 2021-03-18 08:55:29 +00:00
Paul Duffin 738a6d5bc4 Convert android/arch_test.go to test fixtures
Bug: 182885307
Test: m nothing
Change-Id: I4c9c133a1560731869fa5e8fdd81ecc4f54f4821
2021-03-17 16:18:10 +00:00
Lukacs T. Berki 720b3964fe Fix writing soong.variables .
ConfiguredJarList had a marshaler but no unmarshaler.

Bug: 182965747
Test: Presubmits.
Change-Id: Id03669f4a0a3d389063a4e4b11af6d6be63dbba3
2021-03-17 16:35:13 +01:00
Lukacs T. Berki 5f6cb1d6d3 Compile soong_build for debugging if needed.
That is, when SOONG_DELVE= is set.

Test: "m nothing; SOONG_DELVE=port m nothing; m nothing"
Change-Id: I16bd4e54e81b93202573935df134d413feea9bb2
2021-03-17 15:03:14 +01:00
Inseob Kim 67e5add194 Add sepolicy related variables
Bug: 33691272
Test: try building with SELINUX_IGNORE_NEVERALLOWS := true
Change-Id: Id2aa450740d412af7ac66f096fa2121957964cac
2021-03-17 18:39:03 +09:00
Lukacs T. Berki d1e3f1ff08 Embed minibp into soong_ui.
This requires linking Blueprint into soong_ui. It lets us avoid the
complicated dance of Ninja files and shell scripts: now the information
as to how soong_build is built is passed directly to Blueprint using a
struct that contains all the information the command line arguments used
to contain.

The ability to run Blueprint from the command line is kept (for now).

Some variables in bootstrap/command.go needed public accessor functions
because soong_build reads them. This will be disentangled by moving the
flag parsing to soong_build.

The presence of the flag definitions in Blueprint means that soong_ui
now also accepts them. This is not a problem in practice because they
are ignored and because soong_ui itself is hidden behind a few layers of
shell scripts.

Test: Presubmits + the new bootstrap_test.sh .
Change-Id: I6dca478f356f56a8aee1e457d71439272351390b
2021-03-17 08:35:52 +01:00
Lukács T. Berki 645610e791 Merge "Remove global variables from Blueprint." 2021-03-17 07:08:24 +00:00
Colin Cross 5a756a63e4 Check for frameworks/base/Android.bp instead of frameworks/base
Calling ExistentPathForSource from a PackageVarContext has to fall
back to a less optimal glob dependency implementation that adds the
file to out/soong/build.ninja.d.  Checking for "frameworks/base"
adds the directory to build.ninja.d, resulting in any writes to that
directory potentially causing a regeneration.  Check for
"frameworks/base/Android.bp", which will already cause a regeneration
if its timestamp changes and therefore not increase regenerations.

Bug: 158342713
Test: m nothing && touch frameworks/base/foo && m nothing
Change-Id: Iac7ac0445dbaa365c90e9a012f1f32020afc6c08
2021-03-16 23:55:50 +00:00
Lukacs T. Berki 89e9a16ebd Remove global variables from Blueprint.
The end goal of this exercise is to remove all mutable global state so
that multiple Blueprint instances can be run in the same process.

Test: Presubmit.
Change-Id: Idb34b0920f2c7d92efb9328ce8a78b7306f89571
2021-03-16 08:40:51 +01:00
Justin DeMartino 383bfb392f Allow extending of the list of directories to be scanned for VSDK
Bug: 180925851
Bug: 181564692
Test: m nothing, manually
Change-Id: Ifff95db4c9ec362322fecca08f7fd1a7b60755c0
2021-03-15 23:14:47 +00:00
Paul Duffin 4e1916011c Merge "Convert fuchsia tests to use test fixtures" 2021-03-11 14:07:01 +00:00
Lukacs T. Berki 21d5c7aaa9 Hard-wire the path to xcrun as /usr/bin/xcrun.
This lets us not leak $PATH to soong_build. It was only needed for
Darwin tests anyway.

Test: Will ask jingwen@ who has a Mac.
Change-Id: I4647e41275b323fe6283580f8f92718c6229f23e
2021-03-11 08:52:39 +01:00
Paul Duffin ecdac8a6d6 Convert fuchsia tests to use test fixtures
Bug: 181070625
Test: m nothing
Change-Id: Ic2b86f7c603ff1c4b48884cb58908db536d98f37
2021-03-11 07:26:11 +00:00
Lukacs T. Berki 8158356421 Print a correct error message when a non-hermetic tool is not found.
The old error message was wrong because this message is emitted
specifically when a non-hermetic tool is not found.

Test: Manual ("m nothing")
Change-Id: I857c1c2bb059aff894e8bd3dd0c09b030c82035d
2021-03-10 10:43:13 +01:00
Lukacs T. Berki deba72161c Make tests work again on Darwin.
aosp/1609473 broke this by making originalEnv empty unless
InitEnvironment() is called, which doesn't happen in tests.

Bug: 181835672
Test: "m nothing" on Linux and Darwin.
Change-Id: I33f9cce65a9151e620e1dddafb62d4c599ed2cac
2021-03-04 10:54:50 +01:00
Paul Duffin 358161232c Add test fixture support
Adds the test fixture support and converts a few tests to exercise the
code and show how it works.

Bug: 181070625
Test: m nothing
Change-Id: I0a2b40fff93b6041f9aa8c4ef0aba91da1bc8bf3
2021-02-26 23:01:50 +00:00
Treehugger Robot a61a042fa6 Merge "Limit the number of the Java source files in a single compilation unit" 2021-02-22 19:54:00 +00:00
Treehugger Robot f0a869847a Merge "Remove EnforceRROExemptedTargets" 2021-02-20 01:47:43 +00:00
Jeongik Cha cee5ba973d Remove EnforceRROExemptedTargets
There is no more target relying on EnforceRROExemptedTargets

Bug: 150820813
Test: m
Change-Id: If50d22c0e4f99e8c50d6a30cb94a0c3a5646b6fa
2021-02-19 12:16:12 +09:00
Pedro Loureiro 8a75058d3f Merge "Add support for lint baseline files" 2021-02-18 15:53:13 +00:00
Pedro Loureiro 5d190cc24e Add support for lint baseline files
Test: m droid
Test: go test ^TestJavaLint # (from soong/build/java)

Change-Id: I249a0a0597b0bf8495460ed283b476ad2eb36edc
2021-02-18 11:15:30 +00:00
Sasha Smundak b0addafd93 Limit the number of the Java source files in a single compilation unit
KYTHE_JAVA_SOURCE_BATCH_SIZE environment variable controls this setting.
The limit is 1000 if this variable is not set.

Fixes: 179932118
Test: run prebuilts/build-tools/build-prebuilts.sh, use it to build kzips, check
Change-Id: I9ad57dfd1d2c2dce5cff755b1bd61cf933420bd3
2021-02-16 15:14:52 -08:00
Inseob Kim 0cac7b4a5b Add BUILD_BROKEN_VENDOR_PROPERTY_NAMESPACE to BoardConfig
Bug: 175526482
Test: m vendor_property_contexts after making violations
Change-Id: I489312f07e068c3c0e0b7bd6e6c49698707aa9a8
2021-02-16 20:35:17 +09:00
Inseob Kim 60c32f0102 Add PRODUCT_SHIPPING_API_LEVEL to device config
Bug: 175526482
Test: m selinux_policy
Change-Id: I040897f4bdb918aa4b1acb3539a7253640e670a3
2021-02-16 20:35:15 +09:00
Jose Galmes 4c6895e5a3 Implement directed recovery snapshot.
Bug: 177451657
Test: m -j nothing
Change-Id: I0b84c34fb2e9e9340d9f9e7405428cd2cc4d43b5
2021-02-09 08:25:34 -08:00
Paul Duffin 9c3ac96f1f Detect empty apex in ConfiguredJarList
Previously, ConfiguredJarList would accept an empty apex name,
e.g. ":jar" which makes no sense as every apex has to have a non-empty
name. This change makes an empty apex invalid.

In order to improve the test coverage of the TestConfiguredJarList test
this change also changes the implementation of
CreateTestConfiguredJarList([]string) to marshal the supplied strings
into a json list and then unmarshal into a ConfiguredJarList which more
closely matches how it is used at runtime.

Bug: 178361284
Test: m nothing
Change-Id: I7dfec6b4cc1923aa99746e976da0393922ef0791
2021-02-04 10:07:41 +00:00
Bill Peckham 1c8e5e817e Merge changes from topic "prebuilt-hiddenapi"
* changes:
  Supply LOCAL_SOONG_DEX_JAR for java_import with compile_dex.
  Enable an apex to include a java_import.
  Enable prebuilt hiddenapi CSV files.
2021-01-20 17:43:48 +00:00
Bill Peckham bae4749de2 Enable prebuilt hiddenapi CSV files.
By enabling these hiddenapi CSV files to be prebuilt, it
becomes possible to create a split build that supports
the hiddenapi encode dex step, but doesn't contain all
of the java sources needed to generate the CSV files.

Bug: 175048716
Test: m nothing
Test: new TestHiddenAPISingletonWithPrebuiltCsvFile
Test: local build without prebuilt hiddenapi
Test: local build with prebuilt hiddenapi
Change-Id: I805ea3ec05860d2df211a3985ec025bf36f0d775
2021-01-19 10:07:45 -08:00
Inseob Kim 7cf1465d2e Implement directed vendor snapshot
Vendors can now generate only needed modules by setting the following
Makefile variables:

- DIRECTED_VENDOR_SNAPSHOT: set to true
- VENDOR_SNAPSHOT_MODULES: list of snapshot candidates

e.g.

DIRECTED_VENDOR_SNAPSHOT := true
VENDOR_SNAPSHOT_MODULES := toybox_vendor sh_vendor libbase libcutils ...

Bug: 157967325
Test: m dist vendor-snapshot after setting those in BoardConfig.mk
Change-Id: I6515a43d9358d94483d7c7fa2b066f9dd457f6aa
2021-01-18 20:08:09 +09:00
Evgenii Stepanov 4beaa0c964 Include/exclude lists for memtag_heap sanitizer.
Bug: b/135772972
Test: cc_test.go / TestSanitizeMemtagHeap
Change-Id: I263b23647f1874ae3024101dce1b07091c1c9403
2021-01-11 17:50:07 -08:00
Jose "Pepe" Galmes 7ba6f1cca7 Merge "Support for recovery snapshot." 2021-01-06 21:17:54 +00:00
Inseob Kim 4a6d53d6cd Merge "Support building mixed versions of sepolicy" 2021-01-06 05:07:05 +00:00