Commit Graph

1776 Commits

Author SHA1 Message Date
Ulya Trafimovich b28cc3758c Forbid generating boot image files for jars in updatable modules.
This is to guard against the potential situation when someone adds
updatable modules to the list of boot jars by mistake.

Test: aosp_walleye-userdebug builds.

Test: Manually break the checks and observe the errors:
  - move updatable module 'conscrypt' from
    PRODUCT_UPDATABLE_BOOT_JARS to ART_APEX_JARS:
      internal error: module 'conscrypt' from updatable apex 'com.android.conscrypt' is not allowed in the ART boot image

  - add updatable module 'conscrypt' to ART_APEX_JARS
    (but do not remove it from PRODUCT_UPDATABLE_BOOT_JARS):
      error: A jar in PRODUCT_UPDATABLE_BOOT_JARS must not be in PRODUCT_BOOT_JARS, but conscrypt is.

  - move updatable module 'framework-tethering' from
    PRODUCT_UPDATABLE_BOOT_JARS to PRODUCT_BOOT_JARS:
      internal error: module 'framework-tethering' from updatable apex 'com.android.tethering' is not allowed in the framework boot image

  - add non-updatable (in AOSP) module 'android.net.ipsec.ike'
    to PRODUCT_BOOT_JARS:
      internal error: failed to find a dex jar path for module 'com.android.ipsec.ike', note that some jars may be filtered out by module constraints

Bug: 147579140

Change-Id: I25ca2f52530fcfa1f9823b2cfa3485db9c0d0db1
2020-04-07 17:09:59 +01:00
Paul Duffin 923e8a5e9e Extract DepIsInSameApex and RequiredSdks interfaces
The DepIsInSameApex() and RequiredSdks() methods were defined in a few
places to avoid having to depend on the whole ApexModule/SdkAware
interfaces directly. However, that has a couple of issues:
1) It duplicates functionality making it difficult to change, changes
   to the definitions outside the main interfaces do not cause compile
   time failures, instead they result in a runtime change in behavior
   which can be difficult to debug.
2) IDE navigation (specifically in Intellij) does not detect that the
   duplicate definitions can resolve to the definitions in the main
   interface.

This change extracts the methods into their own interfaces and reuses
those interfaces instead of duplicating the methods to fix both of
these issues.

Bug: 152878661
Test: m nothing
Change-Id: I0cfdf342a14eb0bfb82b1bd17e0633d81c7facfb
2020-04-07 15:26:06 +01:00
Colin Cross f8e80229fe Revert "Add sdk mutator for native modules"
Revert submission 1242911-sdk_version_variant

Reason for revert: b/153394225
Reverted Changes:
Ife99745fb:Use libnativewindow for platform variant of libagq...
I1bae84c43:Use libnativewindow for platform variant of androi...
I6e6021ed3:Use stl to depend on libc++
Ife99745fb:Use libnativewindow for platform variant of libRSS...
I2c9f439b9:Fix static dependency on libprotobuf-cpp-lite-ndk
Iff2aff9cf:Set sdk_version for cc_genrules used by modules wi...
I7d72934aa:Add sdk mutator for native modules
Ief378a007:Use sdk variant of Soong modules when LOCAL_SDK_VE...

Bug: 149591340
Change-Id: I798fa902c779469c6382b6699351e5d12bf14785
Fixes: 153394225
2020-04-07 04:21:21 +00:00
Treehugger Robot dd22225b3a Merge "Add product_variables.eng.optimize.enabled" 2020-04-06 04:09:54 +00:00
Ramy Medhat ff6231920d Merge "Fix RuleBuilder remoteable actions running in the local pool." 2020-04-03 20:28:41 +00:00
Treehugger Robot 4066e5e882 Merge "soong_config: bool_variables shortcut" 2020-04-03 15:43:42 +00:00
Sasha Smundak 116ec92e0e Make the names of 'soong_config_module_type_import' modules unique.
blueprint.writeAllModuleActions attemts to sort modules by name, and
sorting them assumes that each module's name is unique. Although
soong_config_module_type_import modules will not generate anything,
their names should be unique, too for that reason.

Fixes: 150421585
Test: m nothing
Change-Id: I6782e42c129dc3b0fc7649ce97d5f94effc63fa7
2020-04-02 20:26:22 +00:00
Jooyung Han 7556839772 Enforce apex.min_sdk_version for bundled builds
Previously, when Q-targeting apexes are bundled-built, they are built
against the latest stubs.

It was because unwinder is linked dynamically in R and APIs are provided
by libc while Q apexes should run on Q where libc doesn't provide those
APIs. To make Q apexes run on Q device, libc++ should be linked with
static unwinder. But, because libc++ with static unwinder may cause problem
on HWASAN build, Q apexes were built against the latest stubs for bundled
build.

However, Q apexes should be built against Q stubs.

Now, only for HWASAN builds, Q apexes are built against the latest stubs
(and native modules are not linked with static unwinder).

Bug: 151912436
Test: TARGET_SANITIZE=hwaddress m
      => Q apexes(media, resolv, ..) are linked with the latest stubs
      m
      => Q apexes are linked with Q stubs,
         and Q apexes' libc++ is linked with static unwinder
Merged-In: If32f1b547e6d93e3955c7521eec8aef5851f908c
Change-Id: If32f1b547e6d93e3955c7521eec8aef5851f908c
(cherry picked from commit 7406660685)

Exempt-From-Owner-Approval: cp from internal
Change-Id: If32f1b547e6d93e3955c7521eec8aef5851f908c
2020-04-02 03:34:30 +00:00
Colin Cross 82e192c3ae Add sdk mutator for native modules
Compiling native modules against the NDK disables platform features
like ASAN.  For anything shipped on the system image there is no
reason to compile against the NDK.  Add a new mutator to Soong that
creates a platform and an SDK variant for modules that set
sdk_version, and ignore sdk_version for the platform variant.  The
SDK variant will be used for embedding in APKs that may be installed
on older platforms.  Apexes use their own variants that enforce
backwards compatibility.

Test: sdk_test.go
Test: TestJNIPackaging
Bug: 149591340
Change-Id: I7d72934aaee2e1326cc0ba5f29f51f14feec4521
2020-04-01 16:09:05 -07:00
Felka Chang f13642ef24 Add product_variables.eng.optimize.enabled
The default build is to enable optimization for all of module.
In order to have the feasibility to the owners of the modules,
to create product_variables.eng.optimize.enabled let the owners
to have the choice to decide whether the build enable the optimization
or not.

Test: atest -m # build soong
Test: # follow Developing for Soong & setup GOPATH
      # to run the new tests in variables_test.go
    cd $GOPATH; \
    go test android/soong/android
Test: #1. change frameworks/base/packages/SystemUI/Android.bp to change
          product_variables.eng.optimize.enabled = false;
      #2. source build/setupenv.sh;
      #3. lunch aosp_x86-eng; make -j16 SystemUI
      #4. lunch aosp_x86-userdebug; make -j16 SystemUI
      #5. aosp_x86-eng should be bigger than aosp_x86-userdebug
Bug: 130259652

Change-Id: Ie62c35458ca91d66ec65bda58d315fb971139849
2020-04-01 12:23:12 +00:00
Ramy Medhat 944839a88a Fix RuleBuilder remoteable actions running in the local pool.
This CL fixes a bug with RuleBuilder's handling of remoteable actions.
It adds a new type of pool to identify remoteable rules by the android
module context. The pool is then set to nil to actually run actions at
NINJA_REMOTE_NUM_JOBS parallelism.

Test: built aosp crosshatch userdebug
Change-Id: I29452f6fc7a161b94189731e3e3cc1f34907b80c
2020-03-31 22:18:40 -04:00
Dan Shi 7182c45fe7 Merge "Rename VTS to VTS10" 2020-03-31 17:32:34 +00:00
Bill Peckham fff3f8a3f8 Include partition tag in apexkeys.txt.
The partition tag helps merge_target_files.py determine
which apexkeys.txt to select from the framework partial
target files and which to select from the vendor partial
target files. The partition tag is the pysical partition
name, for example, a system_ext module on a device where
system_ext is at system/system_ext has a system
partition tag.

Bug: 138942268
Change-Id: Ia07887b34f1aa77dae94ef23610dfef83c1a5849
2020-03-30 22:33:50 +00:00
Dan Shi 984c129c84 Rename VTS to VTS10
Bug: 151896491
Test: m -j vts10
Change-Id: I7b70b13401678d70e4b46f9e7eb9d844bcfa315e
2020-03-28 16:42:17 -07:00
Dan Willemsen b0935db8c3 soong config: add value_variable substitution
There are some cases that aren't handled with the existing variable
types for booleans or known lists of strings. Similarly to our
product_variables that uses %s / %d for things like
PLATFORM_SDK_VERSION, allow vendors to define their own config variables
to be substituted into properties.

For example, some of the makefiles that I've attempted to convert had
the option to pass in version numbers from the board, or the default
display size:

  -DDISPLAY_VERSION=550
  -DDISP_H=1080

These examples happen to be integers, but since our configuration
language (make) doesn't support numbers, %s works just as well.

This change will allow the above to be represented using:

  soong_config_module_type {
      name: "acme_cc_defaults",
      module_type: "cc_defaults",
      config_namespace: "acme",
      value_variables: [
          "DISPLAY_VERSION",
	  "DISP_H",
      ],
      properties: ["cflags"],
  }

  acme_cc_defaults {
      name: "my_defaults",
      soong_config_variables: {
          DISPLAY_VERSION: {
	      cflags: ["-DDISPLAY_VERSION=%s"],
	  },
	  DISP_H: {
	      cflags: ["-DDISP_H=%s"],
	  }
      },
  }

Test: built-in tests
Change-Id: I18f35746b5cc39c304a136980249e886d38c6df6
2020-03-24 07:15:16 -07:00
Yi Kong ceb5b76c91 Add platform-wide sampling PGO option
This causes the compiler to emit some additional debug infomation that
will be used for sampling PGO. These debug infomation will get stripped
so it only affects intermediate files.

Test: build
Bug: 79161490
Change-Id: Ie4d1d5ffbd311ba6e268cb94a618f5272be246ef
2020-03-24 07:04:54 -07:00
Dan Willemsen 2b8b89cfa2 soong_config: bool_variables shortcut
Using a lot of boolean variables can become very verbose without adding
really any new information:

      variables: ["a", "b", "c"],
  }

  soong_config_bool_variable {
      name: "a",
  }

  soong_config_bool_variable {
      name: "b",
  }

  soong_config_bool_variable {
      name: "c",
  }

Now turns into:

      bool_variables: ["a", "b", "c"],
  }

Test: built-in tests
Change-Id: If5455a38433431c7ecbce1e5b32cfbb47f42602a
2020-03-23 22:42:32 -07:00
Paul Duffin 495ffb9ff9 Remove old SdkMemberType API for creating snapshot modules
Migrates system modules and droid stubs over to use the new API for
creating the snapshot modules and removes the old API.

Test: m nothing
Change-Id: Ia825767f1f7ee77f68cfe00f53e09e6f6bfa027f
2020-03-23 08:48:03 +00:00
Paul Duffin a551a1c2f9 Improve consistency of handling java snapshot properties
Previously, java snapshot properties (java_library and java_test)
relied on the properties not being optimized when there was a single os
type and instead being added directly to the common os type properties.
However, that means that the behavior is inconsistent for other member
types depending on whether there was one os type or not.

This change updates the java sdk member handling to support
optimization. This involved:
1) Adding AidlIncludeDirs field to librarySdkMemberProperties to
   specify the aidl include dirs instead of extracting that from the
   library field.

2) Renaming jarToExport to JarToExport (in both
   library/testSdkMemberProperties)to allow it to be optimized.

3) Adding MemberType() and Name() methods to SdkMemberPropertiesContext
   to avoid having to store the former in the properties struct and
   retrieve the latter from the library/test fields.

4) Removing the now unused library/test fields from the properties
   structures.

5) Separating the processing of the jar/test config in
   AddToPropertySet(...) as they may be optimized separately.

6) Ditto for the jar/aidl include dirs.

7) While doing this work I noticed that although the contents of the
   aidl include dirs are copied into the snapshot the java_import does
   not make use of them. Raised bug 151933053 and added TODO to track
   that work.

Bug: 142935992
Test: m nothing
Change-Id: Iba9799e111ca5672b2133568163d8c49837ba9cd
2020-03-23 08:48:03 +00:00
Paul Duffin 3a4eb50829 Make new module creation API more flexible
Previously passing additional information to the implementations of
AddPrebuiltModule() or the SdkMemberProperties interface would have
required making changes to the API. This change added an
SdkMemberContext object into which additional information can easily
be added without requiring changes to existing implementations.

The BuildSnapshot() method was not modified because it is deprecated
and will be removed in a follow up change.

It also switches the API from passing variants as android.SdkAware to
android.Module. That is for a couple of reasons:
1) SdkAware is designed for managing the relationship between the
   module and the SDK, not for generating the output snapshot. As such
   there is nothing in SdkAware that is needed for generating the
   output snapshot.
2) Accepting android.Module instead makes it easier to use the
   underlying code for generating the snapshot module as well as the
   individual member modules.

This is in preparation for a number of improvements and bug fixes in
both the snapshot creation code and implementations to address found
while trying to built the platform against ART prebuilts.

Bug: 151937654
Test: m nothing
Change-Id: Iac10f1200c0f283aa35402167eec8f9aeb65a38e
2020-03-23 08:47:38 +00:00
Paul Duffin 9b76c0b65a Add support for using cc_library in sdk/module_exports
Added link type support in module creation code as cc_prebuilt_library
has to provide both shared and static libraries.

Had to add some calls to FirstUniquePaths() around the include dirs
settings in library_sdk_member as otherwise the shared variant contains
duplicate include dirs (presumably because it gets one set from its
dependency on the static variant and one set of its own). That
difference in include dirs causes problems in the generated
cc_prebuilt_library.

Test: m nothing
Bug: 142918168
Change-Id: Ie7f23fc2341c83c7814cc98e3970df4f5d4c8423
2020-03-20 16:55:24 +00:00
Martin Stjernholm 71da478f51 Merge changes I2ab8f6aa,I53d58100
* changes:
  Add SDK member support for cc_object.
  Add cc_prebuilt_object.
2020-03-20 16:53:51 +00:00
Martin Stjernholm cd07bce437 Add SDK member support for cc_object.
Test: m nothing
Test: Add
    sdk {
        name: "runtime-module-sdk",
        native_shared_libs: [
            "libc",
            "libdl",
            "libm",
            "ld-android",
        ],
        native_objects: [
            "crtbegin_dynamic",
            "crtbegin_static",
            "crtend_android",
        ],
    }
  to bionic/apex/Android.bp. Then:
    build/soong/scripts/build-aml-prebuilts.sh runtime-module-sdk
  Take the generated runtime-module-sdk-current.zip and unzip into a
  master-art tree without bionic/, edit the generated Android.bp to
  extend cc_prebuilt_* modules with:
    nocrt: true,
    stl: "none",
    system_shared_libs: [],
    apex_available: ["//apex_available:anyapex"],
    recovery_available: true,
    vendor_available: true,
    ramdisk_available: true,
  Then "m com.android.art.debug". This passes Soong but fails in the
  build step because more members are required.
Bug: 148934017
Change-Id: I2ab8f6aadb1440b325697cae4a8ed761c62d15d2
2020-03-20 00:16:14 +00:00
Treehugger Robot 4726b9c8a0 Merge "Apply EnforceRROExemptedTargets in Soong" 2020-03-19 01:47:54 +00:00
Paul Duffin 175ef303a3 Merge "Refactor prebuilt to use srcs supplier function" 2020-03-17 09:55:58 +00:00
Elliott Hughes cc868de8a7 Merge "Remove unused mips workarounds." 2020-03-16 21:06:13 +00:00
Paul Duffin dcb4bd6fb4 Refactor prebuilt to use srcs supplier function
Previously, prebuilt had two ways to provide the src value, as a
pointer to an []string and using reflection. That meant that any code
that needed to check the src to use had to understand the two ways of
accessing it.

This change refactors the code to abstract the origin of the src values
being a supplier function. That insulates the callers from having to
know the details and makes it easy to add new suppliers in future.

Test: m nothing
Bug: 142918168
Change-Id: I5cc7517c46cce96924c9f34ba1b046b468f7684a
2020-03-16 19:27:11 +00:00
Paul Duffin 2c73260636 Merge "Remove SdkMemberType.FinalizeModule" 2020-03-16 11:04:26 +00:00
Jooyung Han 5417f775e5 apex: Deprecate legacy_android10_support prop
min_sdk_version = 29 implies that the module should support Android10.

Bug: 150431944
Test: m
Change-Id: Iad90a239898f59456900ae7816b90379b1b43406
2020-03-14 16:03:24 +09:00
Paul Duffin 0174d8d2c9 Remove SdkMemberType.FinalizeModule
This was only being used to set the "stl" property for cc library sdk
member type and so that functionality was moved to AddPrebuiltModule()
and FinalizeModule was removed.

Required a few test changes to move the property to the correct
position in the generated module.

Bug: 142918168
Test: m nothing
Change-Id: If6400189833d4ff3285e7a7adf63a9b509e2a03b
2020-03-13 17:25:16 +00:00
Paul Duffin 13f0271478 Copy shared_libs and system_shared_libs to module snapshot
This change ensures that the runtime dependencies between a
binary/shared library are correctly specified in the snapshot so that
the build can ensure that shared libraries are built before the targets
that use them.

It adds support for differentiating between references that are
required to refer to another sdk member (required) and those that may
refer to either an sdk member or a non-sdk member (optional). The
latter is used for shared library references as the libraries used by
an sdk member may be provided from outside the sdk. e.g. liblog is not
part of the ART module but is used by some members of the ART sdk.

Bug: 142935992
Test: m nothing
Change-Id: Ia8509ffe79b208c23beba1880fe9c8a92b732685
2020-03-13 11:14:07 +00:00
Paul Duffin b07fa51196 Support extracting common values from embedded structures
This change also added support for excluding properties from common
value extraction by using a struct tag of `sdk:"keep"` That was needed
to prevent the fields in SdkMemberPropertiesBase from having their
values cleared.

The purpose of this change is to make it easier to share functionality
across sdk member types.

Bug: 142935992
Test: m nothing
Change-Id: Ie5160a8f854056920e411801ca20721eab7c8578
2020-03-13 10:47:14 +00:00
Paul Duffin 7d74e7bea3 Copy white listed apex available settings into snapshot
Makes sure that the module snapshots do not rely on the white list
of apex available settings so that when those lists are removed it is
not necessary to update any snapshots.

Bug: 142935992
Test: m nothing
Change-Id: Iedcff7dfc2646a4da77258d16e06657dd2f411f9
2020-03-11 18:31:45 +00:00
Paul Duffin a04c107bfa Add support for multiple os types
Updates the member snapshot creation code to support multiple os types.
It basically sorts the variants by os type, then applies the code to
optimize the arch properties and then it optimizes the properties that
are common across architectures and extracts any properties that are
common across os types.

The java and cc member types needed to be modified to make the location
of the generated files within the snapshot os type dependent when there
is more than one os type. That was done by adding an OsPrefix() method
to the SdkMemberPropertiesBase which returns the os prefix to use when
there is > 1 os type and otherwise returns an empty string.

Added three tests, one for cc shared libraries, one for cc binary and
one for java header libraries.

Bug: 150451422
Test: m nothing
Change-Id: I08f5fbdd7852b06c9a9a2f1cfdc364338a3d5bac
2020-03-09 15:54:31 +00:00
Paul Duffin b3c763711a Merge "Refactor snapshot module creation" 2020-03-09 11:02:29 +00:00
Jeongik Cha b9b1327c2d Apply EnforceRROExemptedTargets in Soong
As PRODUCT_ENFORCE_RRO_EXEMPTED_TARGETS is defined in make, define it in
soong accordingly

Bug: 150820813
Test: m
Change-Id: I309482b6ad439a7602127f68f2f7ffa856b9e192
2020-03-09 16:06:19 +09:00
Elliott Hughes da3a071eaa Remove unused mips workarounds.
This was never really finished, and hasn't been supported for years.

Test: treehugger
Change-Id: I21d4c3112aa8cf0c56e59f0cc19ff8725ef714b9
2020-03-06 18:15:44 -08:00
Jooyung Han 03b5185b88 apex: choose stub according to min_sdk_version
Native modules within APEX should be linked with proper stub version
according to its min_sdk_version.

For example, when min_sdk_version is set to "29", libfoo in the apex
would be linked to libbar of version 29 from platform, even if it has
a newer version like 30.

Bug: 145796956
Test: m nothing (soong tests)
Change-Id: I4a0b2002587bc24b7deeb5d59b6eeba5e1db5b1f
2020-03-07 03:12:45 +09:00
Paul Duffin 8c3fec4c37 Enable androidmk processing in sdk testing
Previously, while sdk tests would pass the sdk code would often fail
in androidmk processing. This change makes the tests more realistic
and will catch the errors earlier.

Bug: 142935992
Test: m nothing
Change-Id: Ifd0b2d7cf24e941c919f6b6e0beb2403a67d4308
2020-03-06 09:45:54 +00:00
Paul Duffin 88f2fbe92e Refactor snapshot module creation
Generalize the processing of arch specific properties to reduce
duplication in snapshot module creation and simplify addition of
support for handling multiple os types.

Supporting multiple os types with the current method for building
snapshot modules would require every affected module type to add
support for it. Rather than duplicate multiple os type handling code
across those module types this work generalizes the process cc modules
use for handling arch types as it can be used as a basis for handling
multiple os types. Migrating module types over to this new process
will insulate them from having to handle multiple os types.
OB
SdkMemberType changes:
* BuildSnapshot is deprecated in favour of the new AddPrebuiltModule()
  method.
* Additional methods, CreateVariantPropertiesStruct() and
  FinalizeModule() are added.
* A new interface SdkMemberProperties, is defined that handles
  extracting information from the variant (prior to common value
  optimization) and adding properties to a property set.

The sdk module type uses these new methods and types to delegate the
member type specific processing to the relevant member types while
handling the behavior that is common across all members types, e.g.
extracting common values across multiple architectures. A future change
will leverage this processing to add support for multiple os types.

This change also refactors the cc module processing to use the new
process.

Bug: 150451422
Test: m nothing
Change-Id: If6ab2498407b17f50391d062cd9afc01b5e01af4
2020-03-05 13:00:26 +00:00
Nicolas Geoffray 8f20563514 Merge "Add a nice install paths for module SDKs and exports." 2020-03-05 09:25:33 +00:00
Paul Duffin befa4b91d7 Copy apex_available properties to snapshot modules
Bug: 142935992
Test: m nothing
Change-Id: I1cdaae5b4e13a89dc46541dacd34ef5a44735b6a
2020-03-05 08:29:02 +00:00
Treehugger Robot 5d95387398 Merge "Make a soong-only copy of generate-notice-files.py" 2020-03-04 21:56:04 +00:00
Nicolas Geoffray 1228e9c0ba Add a nice install paths for module SDKs and exports.
Following how NDK also creates its own install path.

Bug: 142935992
Test: cc_sdk_test.go, java_sdk_test.go
Change-Id: I98a3656903f37f6d7c90e6cf609431b2461a6161
2020-03-04 14:20:46 +00:00
Justin Yun 0ecf0b223f Enable vndk-ext for product variants
To build vndk-ext for product variants use `vndk.extends` property
with `product_specific: true` as for the vndk-ext for vendor
variants. For example:
cc_library {
    name: "libvndk_ext_product",
    product_specific: true,
    vndk: {
        enabled: true,
        extends: "libvndk",
    },
}

It will install the vndk-ext libs for product variants in
product/lib[64]/vndk/

Test: m nothing
Bug: 147778025
Change-Id: If1ee5be93c579abad302f44f18e6316f27e70019
2020-03-04 01:22:16 +00:00
Bob Badour 3911e6a1c4 Make a soong-only copy of generate-notice-files.py
This is an unchanged copy of build/make/tools/generate-notice-files.py

After this change, the make implementation of generate-notice-files.py
will change to take a required parameter for $(PRODUCT_OUT) and will
only include notices for files built under $(PRODUCT_OUT).

Because soong has the ability to walk dependency trees, the soong
version will likely change in the future to use that ability causing
both versions to move in different directions.

After those changes are complete, we can look into factoring out any
remaining shared logic.

Test: run manually and system image notices checked for changes

Change-Id: Id139a66503457615548b46e7996349ca0817e831
2020-03-03 13:50:53 -08:00
Colin Cross 1a0f984d43 Merge "Optimize FirstUniqueStrings and FirstUniquePaths" 2020-03-03 20:48:35 +00:00
Colin Cross 27027c765e Optimize FirstUniqueStrings and FirstUniquePaths
FirstUniquePaths is called on some long lists where the O(n^2)
behavior is problematic.  Use a map-based implementation for
longer lists.

Test: TestFirstUniqueStrings
Change-Id: I7181aba869e5ccc0f99c2fa7b8f03839f06e4307
2020-03-02 15:00:15 -08:00
Paul Duffin ca7f0efc2d Improve documentation of CompileMultiTargets and related properties
Test: m nothing
Change-Id: I1e3f0bf7e2a1303d2519d8405a228e9234235514
2020-03-02 19:32:40 +00:00
Paul Duffin 1356d8c0f3 Add CommonOS variant for sdk
Adds a CommonOS variant for sdk that depends on the os specific
variants and is used to generate a single sdk for multiple OsTypes,
e.g. host linux and android.

At the minute the member types only support a single OsType but the
basic mechanism for managing the CommonOS variant and collating the
variants across all of them is there.

The only visible effect of this change is that the location of the
generated snapshot is changed, it is no longer os specific and instead
is in the same location irrespective of which os it is built for.

A lot of tests needed to be changed to specify "common_os" as the
variant type instead of the specific os type. As that is the same across
all tests it is hard coded in CheckSnapshot method.

Test: m nothing
Bug: 150451422
Change-Id: If36be39b06d6910453649f7c288c2d34f688b2f4
2020-03-02 19:31:26 +00:00
Colin Cross 9403e4b704 Merge "Collect metrics from inside soong_build" 2020-02-28 23:23:29 +00:00
Treehugger Robot f14cd86e23 Merge changes from topic "soong_build_number_file"
* changes:
  Add an order-only dependency on the build number file
  Add support for order-only dependencies to RuleBuilder
2020-02-28 20:15:52 +00:00
Colin Cross 2a2e0db2a2 Add an order-only dependency on the build number file
Remote execution and other tools can be confused by references to
build_number.txt without a dependency.  Add an order-only dependency,
which maintains the current behavior.

Test: BUILD_NUMBER=1 && m aapt && aapt version # shows 1
      BUILD_NUMBER=2 && m aapt && aapt version # shows 1
      rm out/soong/.intermediates/frameworks/base/tools/aapt/aapt/linux_glibc_x86_64/aapt
      BUILD_NUMBER=2 && m aapt && aapt version # shows 2
Change-Id: Icfa98d6840b1dc2e273ba29c33011635d1cf93b1
2020-02-28 18:16:56 +00:00
Colin Cross da71eda20d Add support for order-only dependencies to RuleBuilder
Test: TestRuleBuilder
Change-Id: I1609a790dd4d0a03c8308b6e552622fe33fa2499
2020-02-28 18:16:13 +00:00
Treehugger Robot 2642bc6d82 Merge changes I9170c7e2,I058201b2,Icf37bb3d
* changes:
  Make filesToInstall return InstallPaths and add it to Module
  Add pathForInstall and InstallPaths
  Add InstallForceOS, fix testcases for host
2020-02-27 15:28:49 +00:00
Bob Badour 40017be1c3 Merge "Unless overridden include LICENSE files in notices." 2020-02-26 20:37:22 +00:00
Colin Cross b72c909899 Collect metrics from inside soong_build
Collect the number of modules and variants and some basic statistics
on memory usage inside soong_build by writing out a proto that is
read back in by soong_ui.

Test: examine soong.log
Change-Id: I6926876377a4f6229cf41fdbf166ae03c885ea55
2020-02-25 20:50:00 +00:00
Jooyung Han cd87c69de4 Fix forwarding a vararg parameter
vararg parameter is passed without unpacking, which prints
not-so-useful value.

Bug: N/A
Test: m
Change-Id: I03f011557d10c6e2bc06813491cd8df2cce7c6ea
2020-02-26 02:05:18 +09:00
Colin Cross 897266ed5c Make filesToInstall return InstallPaths and add it to Module
Test: m checkbuild
Change-Id: I9170c7e22f000a2d0343e74a96079b24e8ad66f5
2020-02-24 15:51:27 +00:00
Colin Cross 609c49a389 Add pathForInstall and InstallPaths
pathForInstall will allow code in the android package to get paths
to the install directory.

InstallPaths provides helpers for handling lists of install paths.

Test: m checkbuild
Change-Id: I058201b205bc8bc67f3c1f62952aaaf758c47fd8
2020-02-24 15:51:18 +00:00
Bob Badour a75b057e17 Unless overridden include LICENSE files in notices.
As a second step to removing the go/android3p instructions to copy or
to link NOTICE to LICENSE, include LICENSE files in the notices, which
will allow deleting all of the copied/linked NOTICE files.

The change causes a few additions to the system image notice files.

Test: manually built and compared before and after notices
Change-Id: Ia7bc58e2eba7bed5e63934881b5298201a93bc3e
2020-02-21 20:40:43 +00:00
Treehugger Robot 53fe467c67 Merge changes from topic "soong_tests_ndk"
* changes:
  Make apex use cc.GatherRequiredDepsForTests
  Move NDK test modules into cc/testing.go
2020-02-20 20:39:16 +00:00
Paul Duffin 44885e29d6 Simplify visibility rules that include //visibility:public
While it is invalid to mix //visibility:public with other rules in the
visibility property in a .bp file tt was possible, by overriding
defaults, to have //visibility:public mixed in with other rules in the
effective visibility rules. That caused problems when those effective
rules were used in an sdk snapshot.

This change replaces any set of rules that include //visibility:public
with just the //visibility:public rule. That simplifies those rules,
making them cheaper to process and ensures that the effective rules are
valid in the visibility property.

Adding test support required some refactoring of the
effectiveVisibilityRules(BaseModuleContext, ...) and underlying methods
to take a Config instead of BaseModuleContext as the tests do not have
access to BaseModuleContext.

Bug: 142935992
Test: m nothing - new tests failed without change, work with it
      Add dex2oat to art-module-host-exports, build it and check the
      generated Android.bp file in the snapshot to ensure the
	  visibility property for the dex2oat prebuilt does not mix
	  //visibility:public with other rules.
Change-Id: I08e7f0dcb40838d426fe88fedf69eae27b77473c
2020-02-20 11:41:17 +00:00
Colin Cross f28329de73 Move NDK test modules into cc/testing.go
Move the NDK modules defined in app_test.go into cc/testing.go in
preparation for using sdk_version: "current" in more tests.

Test: all soong tests
Change-Id: I80bc31225fb4562fce42781219cefbbd6affae06
2020-02-19 18:59:15 -08:00
Dan Shi 24f3c99af8 Merge "Allow vts config to be packaged in other test suites" 2020-02-19 21:05:55 +00:00
Colin Cross 6e3594003b Add InstallForceOS, fix testcases for host
Robolectric tests compile against device modules but are installed
and run as host modules.  Allow a module to override its install
OS.

Test: TestPathForModuleInstall
Change-Id: Icf37bb3d4cc1222a9b079602c6a5fdb8b51c86ed
2020-02-19 20:41:22 +00:00
Peter Collingbourne d8b80bc683 Merge "Statically link the unwinder into binaries packaged into an unbundled APEX with legacy_android10_support: true." 2020-02-18 18:12:37 +00:00
Jaewoong Jung 36b6aec26a Merge "Simple refactoring of prefix related functions." 2020-02-18 15:08:41 +00:00
Jaewoong Jung 3aff5787e2 Simple refactoring of prefix related functions.
Rename the two prefix-in-list funcs so that their usages are clearer.
Also find and replace all the code that essentially does either.

This introduces additional loops in some places, but I think the added
readability and simplicity outweighs the performance degradation, which
should be negligible anyway.

Test: m nothing
Test: TreeHugger
Change-Id: I37e2276ca0c815105ed0031f23c0b1264b480e4f
2020-02-14 14:33:49 -08:00
Dan Willemsen b13a9487a0 Add apex name to description
Otherwise we see multiple concurrent builds of different actions with
the same description:

    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc

Is now:

    0:09 //external/fmtlib:fmtlib clang++ src/format.cc
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.resolv]
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.adbd]
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.art.debug]
    0:09 //external/fmtlib:fmtlib clang++ src/format.cc [com.android.media]

Test: m  <watch output>
Change-Id: Ib8fccc144e8fe5fc33cafe8536538b892bd42765
2020-02-14 12:33:52 -08:00
Ivan Lozano d0bf3dbea8 Merge "Remove execute-only support." 2020-02-14 14:42:49 +00:00
Peter Collingbourne dc4f986b2a Statically link the unwinder into binaries packaged into an unbundled APEX with legacy_android10_support: true.
Bug: 149075752
Test: tapas com.android.conscrypt com.android.tethering arm64
Test: m out/target/product/generic_arm64/{,symbols/}apex/com.android.{tethering,conscrypt}/lib64/libc++.so
Test: Verified that unwinder was dynamically linked to tethering's
Test: libc++ and statically linked to conscrypt's.
Test: lunch flame-userdebug && m
Test: Verified that unwinder was dynamically linked to /system/lib64/libc++.so
Change-Id: I98eed7cb4316962b19b5c12e150c224c25d0e91d
2020-02-13 22:29:44 -08:00
Dan Shi 403c9d3e7a Allow vts config to be packaged in other test suites
Bug: 149249068
Test: m vts-core
Change-Id: I582ceec53cf7747f5754613cf774871c84776cf5
2020-02-13 16:06:49 -08:00
Treehugger Robot 680afd7bc7 Merge "apexDepsMutator is a top-down mutator" 2020-02-13 22:29:23 +00:00
Ivan Lozano 8107b85189 Remove execute-only support.
With kernel support being removed for execute-only memory layouts,
remove support for XOM until there's kernel support again.

This allows us to gain the code-size improvements from
"-z noseparate-code".

Bug: 147300048
Test: Compiled binaries no longer marked XOM by default.
Change-Id: Ie84bdd2ca025e8b14867008291bf9ce143b5e999
2020-02-13 13:50:34 -05:00
Ulyana Trafimovich cc9a5edd4f Merge "Make PRODUCT_BOOT_JARS and PRODUCT_UPDATABLE_BOOT_JARS disjoint." 2020-02-13 13:36:25 +00:00
Ulya Trafimovich 4cdada2d1f Make PRODUCT_BOOT_JARS and PRODUCT_UPDATABLE_BOOT_JARS disjoint.
This is to unify boot jars with system server jars:
PRODUCT_SYSTEM_SERVER_JARS and PRODUCT_UPDATABLE_SYSTEM_SERVER_JARS
do not overlap.

Test: aosp_walleye-userdebug boots

Exempt-From-Owner-Approval: cherry-pick.
Bug: 146363577
Change-Id: If001049a46f8e9d38921dabeedbd61e971ebfe73
Merged-In: If001049a46f8e9d38921dabeedbd61e971ebfe73
(cherry picked from commit ef4358e536)
2020-02-13 13:35:39 +00:00
Treehugger Robot f048446f92 Merge "Add vendor snapshot modules" 2020-02-13 11:01:10 +00:00
Treehugger Robot 6a20b179d2 Merge "Improve documentation of the enabled property" 2020-02-12 21:08:32 +00:00
Treehugger Robot 6eb0da1530 Merge "Update the doc about apex_available" 2020-02-12 20:58:24 +00:00
Jiyong Park 9a1e14ef6c Update the doc about apex_available
The default has changed.

Exempt-From-Owner-Approval: documentation change

Bug: 128708192
Bug: 149393721
Test: n/a
Change-Id: I91f36fc1fb5f2302deac8ee23bb847077b1281c9
2020-02-12 20:58:08 +00:00
Paul Duffin 54d9bb70d0 Improve documentation of the enabled property
Makes it clear that disabling a module is something that should be
avoided if possible.

Test: m soong_docs
Change-Id: Ief11e705f049edfd53b2af66bdf753f24d32aa71
2020-02-12 10:28:40 +00:00
Jiyong Park f760cae41b apexDepsMutator is a top-down mutator
apex { name: ["myapex"], native_shared_libs: ["libX", "libY"] }
cc_library { name: "libX", shared_libs: ["libY"] }
cc_library { name: "libY", shared_libs: ["libZ"], stubs: {...} }

apexDepsMutator was a bottom up mutator and it uses WalkDeps to traverse
the dependency tree rooted at myapex in a depth-first order. While
traversing the tree, if calls BuildForApex for a module that will be
part of the APEX.

libY is visited twice. Once via libX and once via myapex. If the visit
from libX was before the visit from myapex (since this is a depth-first
traversing), BuildForApex is not called for libY and its dependency
libZ, because libY provides a stub. And then when libY is again visited
via myapex, BuildForApex is correctly called for the module, but not for
its dependencies libZ because the paths from libY to libZ was already
visited.

As a result, the apex variant of libY has a dependency to the non-apex
variant of libZ.

Fixing the problem by changing the mutator a top-down one.

Bug: 148645937
Test: m
Change-Id: Ib2cb28852087c63a568b3fd036504e9261cf0782
2020-02-12 17:01:26 +09:00
Steven Moreland 226adf28ae Merge "neverallow: update for different HIDL build." 2020-02-11 23:10:34 +00:00
Steven Moreland 51ce4f6f25 neverallow: update for different HIDL build.
libhidlbase is now being built without intermediate static libraries.
This requires a different neverallow exception for the
'enforce_vintf_manifest' variable.

Bug: N/A
Test: N/A
Change-Id: I5b7f7fc8eb4ac37d658e60a491c39baa5ac137f8
2020-02-10 17:21:32 -08:00
Treehugger Robot 705cd4600d Merge "Implement vendor snapshot" 2020-02-10 23:40:50 +00:00
Paul Duffin 7b81f5e9d7 Add java_system_modules to sdk/module_exports
Adds an SdkMemberType implementation for java_system_modules. It
specifies that java_system_modules can be used with sdk as well as
module_exports, and also that the libs property should be included
as transitive members in the sdk.

It also adds support for treating appropriate tagged properties in
the snapshot prebuilts module as references to sdk members so that
they are correctly transformed when creating the versioned modules.

Bug: 142940300
Test: m nothing
Change-Id: Ic10b5a6d5b92b6018334fe876f06feaf79cc55e9
2020-02-07 14:03:03 +00:00
Paul Duffin f4ae4f1390 Add support for transitive sdk members
Allow an sdk member type to treat some of its dependencies as being
members of the sdk.

Needed for the java_system_modules type whose libs property are an
implementation detail of the system module and so should not be
explicitly listed in the sdk module but still have to be included in
the sdk snapshot.

Bug: 142940300
Test: m nothing
Change-Id: I90f37dae269ef64a6fe9debd0bbaf29a64dd74d8
2020-02-07 14:03:03 +00:00
Treehugger Robot 7300095182 Merge "java_sdk_library - Use prebuilt/prefer for unbundled app builds" 2020-02-07 05:28:35 +00:00
Colin Cross df51b061cd Merge changes I38fb22b2,I281bdefe,Ieaaa590c
* changes:
  Add product_variables.native_coverage.src
  Fix product variables in defaults modules
  Fix product variable zero value check
2020-02-07 03:35:04 +00:00
Colin Cross 2b10ba0ee2 Add product_variables.native_coverage.src
Test: m checkbuild
Fixes: 148088129
Change-Id: I38fb22b28de1176ed880708733f7e7f76bee2e50
2020-02-06 17:46:26 -08:00
Colin Cross eabaedd520 Fix product variables in defaults modules
Product variables structs are generated at runtime to contain only
the properties that apply to the current module.  Defaults modules
always contained all product variable properties. Defaults modules
apply their properties to the target module using
proptools.PrependProperties, which prepends structs that have
matching types.  Filtered property structs had a different type
and were dropped.

Even after adding filtering to the defaults product variable
properties, defaults modules may contain more property structs
than the target module they are applied to, so the product
variables struct for the defaults module could contain more
fields than the product variables struct for the target module.
Use proptools.PrependMatchingProperties when applying defaults
of product variables instead, which will apply matching properties
across types.

Test: defaults_test.go
Test: variable_test.go
Change-Id: I281bdefef92053457a3b7b65383493a4e7d999df
2020-02-06 17:43:29 -08:00
Colin Cross 6961a491a5 Fix product variable zero value check
The zero value check was being done by using reflect.DeepEqual on a
field from the default product variables, but this results in
comparison against a random type when the product variables struct
for the module has been filtered down.  Luckily this will always
fail false, which just removed and optimization but left the
behavior correct.

Use reflect.IsZero instead, which is both faster and correct.

Test: variable_test.go
Change-Id: Ieaaa590c2788ca39230e6695397e8ba8d1c6c103
2020-02-06 17:41:19 -08:00
Jiyong Park 1fd192302c Merge changes from topic "apex_available"
* changes:
  shared_lib dependency from a static lib crosses the APEX boundary
  apex_available tracks static dependencies
2020-02-06 22:56:08 +00:00
Colin Cross 3beeb1ebb4 Add test for capitalized Soong config variable
Soong config variables come from Make where they might be capitalized,
but Blueprint didn't handle capitalized variables well.  Add test
coverage for capitalized Soong config variables.

Bug: 148865218
Test: soong_config_module_test.go
Change-Id: I1e434e392d5ee660a221a0d3f959811c35e65865
2020-02-06 19:36:05 +00:00
Colin Cross 39e545cc06 Add dependency on Soong config module definition file
Add a dependency on the Soong config module definition file in case
it wasn't called Android.bp.

Fixes: 148866376
Test: m checkbuild
Change-Id: Ib441881bcee52fd1dc3a8d1c5ae4f5f0b7efe3ce
2020-02-06 19:35:49 +00:00
Inseob Kim eec88e1de8 Add vendor snapshot modules
This implements four modules (static/shared/header libraries, and
binaries) for vendor snapshot. These modules will override source
modules if BOARD_VNDK_VERSION != current.

Bug: 65377115
Test: 1) VNDK_SNAPSHOT_BUILD_ARTIFACTS=true m dist vndk vendor-snapshot
Test: 2) install snapshot under source tree
Test: 3) set BOARD_VNDK_VERSION and boot cuttlefish
Change-Id: I24ddb4c3aa6abeab60bbfd31bcbd8753e2592dc5
2020-02-07 02:59:24 +09:00
Inseob Kim 8471cdaced Implement vendor snapshot
Vendor snapshot can be captured with "m dist vendor-snapshot". With
vendor snapshot and vndk snapshot, older version of /vendor and newer
version of /system will be able to be built together by setting
BOARD_VNDK_VERSION to past vendor's version.

Only vendor modules under AOSP are to be captured. In detail, modules
under following directories are ignored:
- device/
- vendor/
- hardware/, except for interfaces/, libhardware/, libhardware_legacy/,
  and ril/

Test modules (cc_test, etc.) and sanitized modules are also ignored.

Bug: 65377115
Test: m dist vendor-snapshot
Change-Id: If7a2f6de7f36deee936930c0ccf7c47c4a0cebf6
2020-02-07 02:58:29 +09:00
Jiyong Park 0f80c1848a apex_available tracks static dependencies
This change fixes a bug that apex_available is not enforced for static
dependencies. For example, a module with 'apex_available:
["//apex_available:platform"]' was able to be statically linked to any
APEX. This was happening because the check was done on the modules that
are actually installed to an APEX. Static dependencies of the modules
were not counted as they are not installed to the APEX as files.

Fixing this bug by doing the check by traversing the tree in the method
checkApexAvailability.

This change includes a few number of related changes:

1) DepIsInSameApex implementation for cc.Module was changed as well.
Previuosly, it returned false only when the dependency is actually a
stub variant of a lib. Now, it returns false when the dependency has one
or more stub variants. To understand why, we need to recall that when
there is a dependency to a lib having stubs, we actually create two
dependencies: to the non-stub variant and to the stub variant during the
DepsMutator phase. And later in the build action generation phase, we
choose one of them depending on the context. Also recall that an APEX
variant is created only when DepIsInSameApex returns true. Given these,
with the previous implementatin of DepIsInSameApex, we did create apex
variants of the non-stub variant of the dependency, while not creating
the apex variant for the stub variant. This is not right; we needlessly
created the apex variant. The extra apex variant has caused no harm so
far, but since the apex_available check became more correct, it actually
breaks the build. To fix the issue, we stop creating the APEX variant
both for non-stub and stub variants.

2) platform variant is created regardless of the apex_available value.
This is required for the case when a library X that provides stub is in
an APEX A and is configured to be available only for A. In that case,
libs in other APEX can't use the stub library since the stub library is
mutated only for apex A. By creating the platform variant for the stub
library, it can be used from outside as the default dependency variation
is set to the platform variant when creating the APEX variations.

3) The ApexAvailableWhitelist is added with the dependencies that were
revealed with this change.

Exempt-From-Owner-Approval: cherry-pick from internal

Bug: 147671264
Test: m

Merged-In: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
(cherry picked from commit fa89944c79)
Change-Id: Iaedc05494085ff4e8af227a6392bdd0c338b8e6e
2020-02-06 14:45:08 +09:00
Jaewoong Jung 387ad5c576 Merge "Add rules to handle asset resources." 2020-02-05 22:24:39 +00:00
Treehugger Robot e89f7354f8 Merge "Minor cleanup in soong_config_modules documentation." 2020-02-05 18:41:53 +00:00
Paul Duffin 50061511d4 java_sdk_library - Use prebuilt/prefer for unbundled app builds
Previously, the java_sdk_library had special support for disabling the
stubs library when the build was configured to use prebuilts for sdks,
e.g. in an unbundled app build. That caused the prebuilt version of the
stubs library to be used instead. The disabling was done irrespective
of whether a prebuilt was available which prevents java_sdk_library
from being used in situations when prebuilts are not available, e.g.
when they have not been created yet.

This change moves the logic into tha java_sdk_library_import and
leverages the existing prebuilt/prefer mechanism to ensure that the
prebuilt version is used when required.

* Adds a ForcePrefer() method to Prebuilt to allow a module to forcibly
  set the value of the prefer property.
* Sets prefer true on the java_sdk_library_import and the stubs modules
  it creates when the prebuilt version should be used.
* Refactors PrebuiltJars for use by both java_sdk_library and
  java_sdk_library_import as they both need to provide access to
  prebuilts for previously released versions of the library.
* Removes disabling logic from java_sdk_library.

This will probably require some additional java_sdk_library_import
modules to be added to prebuilts/sdk/current/Android.bp.

Bug: 148080325
Test: m droid && TARGET_BUILD_APPS=Camera2 m
Change-Id: I0b5f751e82a2179a967ae64ca03dc9b9e7665c16
2020-02-05 08:09:57 +00:00
Kyriakos Ispoglou 08cc141860 Merge "Add support for LINE_COVERAGE (1/2)" 2020-02-04 22:37:41 +00:00
Bill Peckham c93258bfec Minor cleanup in soong_config_modules documentation.
Test: build
Change-Id: Ia5a43a024203ca4b714926bcc89f7ec12523b8ad
2020-02-04 13:17:24 -08:00
Treehugger Robot dcbc329713 Merge "Update NDK ABIs config away from armv5." 2020-02-03 20:28:41 +00:00
Kyriakos Ispoglou db03c22e54 Add support for LINE_COVERAGE (1/2)
This is the 1st part of the CL. It adds support to enable compilation
of line coverage instrumented binaries. For more details please refer
to the design doc: go/android-line-coverage-doc

Bug: b/147604881
Test: Code works successfully on Taimen devices
Change-Id: I07745c1438b611041ed032dd4b7788cb50130845
2020-01-31 14:47:12 -08:00
Paul Duffin 67edce7adb Merge "Make sdkMemberDependencyTag usable outside sdk package" 2020-01-31 10:11:16 +00:00
Treehugger Robot 86ef38cfbf Merge "Add native_coverage to product variables" 2020-01-31 03:12:22 +00:00
Dan Albert 6bba644205 Update NDK ABIs config away from armv5.
Test: treehugger
Bug: None
Change-Id: I0e4909e8a394814edbcc3e9f80d3a6e4a273127b
2020-01-30 16:26:41 -08:00
Paul Duffin f8539922d4 Make sdkMemberDependencyTag usable outside sdk package
Moves the struct to android/sdk.go and abstracts it behind a factory
method and interface. That allows it to be used outside the sdk
package.

This change is in preparation for adding support for module types that
have transitive sdk members.

Bug: 142940300
Test: m nothing
Change-Id: I71e5e0adf839b28a3a0952f82637637887f02688
2020-01-30 11:45:47 +00:00
Paul Duffin 5b511a200e Allow an arbitrary tag to be associated with a bp property
This is preparation for enhancing the versioning tranformer to support
applying per property transformations. Specifically, to allow
properties to reference other libraries within the sdk.

Bug: 142940300
Test: m nothing
Change-Id: I99cdff4b407763ed395ff358d8110a63c6cf5589
2020-01-30 11:45:47 +00:00
Colin Cross 041ac6f2ce Merge "Allocate OutputPath.String in PathForOutput" 2020-01-30 05:06:35 +00:00
Colin Cross d63c9a7fc1 Allocate OutputPath.String in PathForOutput
In an AOSP aosp_blueline-userdebub build, OutputPath.String was
allocating 802MB of strings in filepath.Join to prepend the
out dir to each path.  Allocate the joined string in PathForOutput
instead, which results in ~57MB of new allocations in PathForOutput
but no allocations in OutputPath.String.

Test: all soong tests
Change-Id: Id452e0c46a2aeda71bfac11a227bb6edb8e3523d
2020-01-29 16:56:37 -08:00
Christopher Ferris c0ebc81e3d Merge "Add whole_static_libs to non-svelte configs." 2020-01-29 17:22:01 +00:00
Colin Cross 43e789d667 Follow input changes to proptools.CloneEmptyProperties
Test: m checkbuild
Change-Id: I1fd53d03722d134009f7ed663f05bd6dc5980dd1
2020-01-28 12:17:06 -08:00
Christopher Ferris c71193ae53 Add whole_static_libs to non-svelte configs.
This is to support enabling scudo only for non-svelte configs.

Also, add exclude_static_libs to allow removing the jemalloc libs.

Bug: 137795072

Test: Verified that a svelte and non-svelte config can use this method
Test: to properly choose between scudo and jemalloc.

Change-Id: Iec6bfe159f8491138e93dde1d225a8c874c7ce31
2020-01-28 12:02:46 -08:00
Ramy Medhat 5e0dbe4e3e Merge "Add support for experimentally enabling RBE support on specific rules." 2020-01-28 19:49:05 +00:00
Ramy Medhat 8ea054a81e Add support for experimentally enabling RBE support on specific rules.
This CL adds RBE support to javac, r8, and d8 rules which is only
enabled if respective environment variables are set.

Test: an aosp_crosshatch build with and without the new variables.
Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-28 12:42:47 -05:00
Colin Cross d9a121ba2a Add native_coverage to product variables
Allow native_coverage to adjust sources in product_variables.

Fixes: 148088129
Test: m checkbuild
Change-Id: I9c9d491cda92d69726a0d598408de2060241365b
2020-01-27 16:09:57 -08:00
Treehugger Robot 057dd35669 Merge "Abstract sdk_version string using sdkSpec type" 2020-01-28 00:07:37 +00:00
Colin Cross b7ec426e01 Merge "Simplify vendor conditionals" 2020-01-27 19:03:23 +00:00
Jiyong Park 6a927c4e6a Abstract sdk_version string using sdkSpec type
The value format that sdk_version (and min_sdk_version, etc.) can have
has consistently evolved and is quite complicated. Furthermore, with the
Mainline module effort, we are expected to have more sdk_versions like
'module-app-current', 'module-lib-current', etc.

The goal of this change is to abstract the various sdk versions, which
are currently represented in string and is parsed in various places,
into a type called sdkSpec, so that adding new sdk veresions becomes
easier than before.

The sdk_version string is now parsed in only one place 'SdkSpecFrom', in
which it is converted into the sdkSpec struct. The struct type provides
several methods that again converts sdkSpec into context-specific
information such as the effective version number, etc.

Bug: 146757305
Bug: 147879031
Test: m
Change-Id: I252f3706544f00ea71c61c23460f07561dd28ab0
2020-01-25 21:56:43 +09:00
Treehugger Robot 6cc0515e82 Merge "Create a highmem pool and put metalava into it" 2020-01-25 01:09:43 +00:00
Colin Cross 9d34f35815 Simplify vendor conditionals
Support vendor conditionals with no Go code.

Test: TestSoongConfigModule
Change-Id: I42546e7f17324921ada80f4d8e1cd399830f8dfc
2020-01-24 16:43:40 -08:00
Yifan Hong a8c82cc218 Merge changes from topics "board_use_recovery_as_root_soong", "ramdisk"
* changes:
  libatomic / libgcc_stripped: ramdisk_available
  Add target.ramdisk
  Ramdisk modules install to correct location
  Add ramdisk image.
2020-01-24 22:39:41 +00:00
Colin Cross 8b8bec3b3a Create a highmem pool and put metalava into it
Create a highmem pool based on the total RAM and the number of CPUs,
with an override via the NINJA_HIGHMEM_NUM_JOBS variable.  Put
metalava into the highmem pool.

Ninja does not support nested pools, and when goma or RBE is enabled
the maximum ninja parallelism is set very high with local jobs in a
local pool.  When both the local pool and highmem pool are enabled,
the total number of local jobs will be as high as the sum of the sizes
of the two pools.  Keep the highmem pool limited to 1/16th of the
local pool when remote builds are enabled to try to minimize the
effect while still limiting highmem jobs.

Fixes: 142644983
Test: m nothing, examine pools
Test: m USE_GOMA=true nothing, examine pools
Change-Id: Id79f11f44948992960ac34ecf831dacbe21bd332
2020-01-24 11:44:23 -08:00
Colin Cross c61b38904e Merge changes from topic "filtershard"
* changes:
  Pass desired size proptools.FilterPropertyStructSharded
  Do a better job removing tags from arch structs
  Split the runtime arch property structs into 3
2020-01-23 21:26:39 +00:00
Colin Cross b1d8c99e12 Pass desired size proptools.FilterPropertyStructSharded
proptools.FilterPropertyStructSharded now takes a desired max size
of the name.

Fixes: 146234651
Test: all soong tests
Change-Id: Ifa10039e5f55a5d0f4a35172983d43f33af88e93
2020-01-22 18:30:56 -08:00
Colin Cross b4fecbfef4 Do a better job removing tags from arch structs
Remove more android struct tags from runtime created arch structs
to reduce the size of their names, which hit the 64kB limit in
runtime.StructOf if they are too long.

Bug: 146234651
Test: m checkbuild
Change-Id: I6362765275b93c8932eb0b1abbcb4be47031d9b1
2020-01-22 18:30:56 -08:00
Yifan Hong 82db735fbc Ramdisk modules install to correct location
Install to recovery/root/first_stage_ramdisk if BOARD_USES_RECOVERY_AS_BOOT,
otherwise ramdisk. In addition, append /system if not InstallInRoot().

Test: pass
Bug: 147347110
Change-Id: Ie5895ac61a3d5d85f5d958a27c7980d6c79d997e
2020-01-22 16:54:18 -08:00
Yifan Hong 1b3348d50a Add ramdisk image.
It is similar to recovery image.
Test: m nothing -j

Change-Id: I11389777c6bfb0c0d73bbb4c70091c1e70f44077
2020-01-22 16:54:18 -08:00
Oliver Nguyen b6aeb14150 Merge "Add coverage flags for Clang coverage builds." 2020-01-22 22:34:35 +00:00
Colin Cross cbbd13f9bc Split the runtime arch property structs into 3
Instead of creating a runtime generated struct type:
struct {
    Arch struct {
        Arm struct {
            Armv7-a struct {
                ...
            }
        }
        ...
    }
    Multilib struct {
        Lib32 struct {
            ...
        }
        ...
    }
    Target struct {
        Android struct {
            ...
        }
        ...
    }
}

Replace it with a top-level struct of empty interfaces:
type archPropRoot struct {
        Arch, Multilib, Target interface{}
}

And then embed nil pointers to smaller structs inside it:
struct {
    Arm struct {
        Armv7-a struct {
            ...
        }
    }
    ...
}

This reduces the maximum number of times the archecture specific
properties are embedded in a runtime generated struct from 139
to 97, which makes it easier to avoid hitting the runtime generated
struct name size limit of 64kB.

Bug: 146234651
Test: all soong tests
Change-Id: I901c434ac9a1d99c74fc2370c5e6803b3682dc73
2020-01-21 11:54:53 -08:00
Martin Stjernholm 710ec3aef2 Add FinalDeps mutator phase, where CreateVariations is disallowed.
This is for use to add dependencies between specific module variants,
without getting the dependencies potentially duplicated as more
variants are created.

Moved genrule tool dependency registration to this phase, to avoid
potential splitting of its dependencies, and for consistency with the
same kind of dependency on dex2oat in https://r.android.com/1205730.

Test: m nothing
Bug: 145934348
Change-Id: I1f9e75354e359cef5f41d14e5016b0a67d7655d7
2020-01-21 11:40:02 +00:00
Treehugger Robot dca59a516d Merge "Clarify a bit more what apex_available means." 2020-01-20 13:07:07 +00:00
Martin Stjernholm 06ca82d6ff Clarify a bit more what apex_available means.
Test: n/a - comment change only
Change-Id: I5311dca32a4e43c3765a3ea26d2b8781ce113433
2020-01-17 13:02:56 +00:00
Treehugger Robot d66b2fd3cf Merge "Fix dependencies through symlinks" 2020-01-16 23:21:25 +00:00
Jaewoong Jung 6431ca7a3a Add rules to handle asset resources.
AAPT2 ignores assets in overlay resource inputs, so separate rules are
required to extract and merge assets from dependencies.

Test: app_test.go
Test: Added assets to Settings-core for testing
Test: Confirmed Settings.apk contains them with this change.
Fixes: 146655310
Change-Id: Iab8bc61b719541dae64f0e3502bc9cb45a353687
2020-01-16 12:19:49 -08:00
Treehugger Robot 1be2d48005 Merge "Save compilation units in protobuf format." 2020-01-16 18:55:26 +00:00
Dan Willemsen 40efa1cd31 Fix dependencies through symlinks
Now that ninja uses lstat so that we can get the mtime of the symlink
itself, it's safe to use a real input here rather than an order only
input. This matches how our other install rules work for host files.
Symlinks should be quick to recreate, so I'm not copying the logic to do
order-only dependencies for the device.

Bug: 128577186
Bug: 145278193
Bug: 135515543
Test: m out/target/product/walleye/apex/com.android.art.debug/javalib/arm/boot.art
      rm out/soong/host/linux-x86/bin/dex2oatd64
      m out/target/product/walleye/apex/com.android.art.debug/javalib/arm/boot.art

Change-Id: Ief5ec752ce0d0216aeecd8e904afc59f3f08ed7d
2020-01-16 17:13:35 +00:00
Anton Hansson eec79ebe03 Default apex_available to //apex_available:platform
This means everything that goes into apexes need to be
explicitly labeled to be available for apex.

Whitelist the current offenders. This list should be
trimmed down.

Bug: 147364041
Test: m
Test: multiproduct_kati -only-soong
Exempt-From-Owner-Approval: cherry-pick from internal
Change-Id: I837299c6a15d46f8a5ba544b613776b1cc27d7b8
Merged-In: I837299c6a15d46f8a5ba544b613776b1cc27d7b8
(cherry picked from commit 93488cbb10d4882845abb732f8e53714f0982031)
2020-01-16 09:14:03 +00:00
Sasha Smundak 6c2d4f91f8 Save compilation units in protobuf format.
Bug: 146224091
Test: manual
Change-Id: I7a856bab13f54c78efa0061421c3fcb9341bc6e7
2020-01-15 12:40:03 -08:00
Paul Duffin 78ac5b962e Exclude source->prebuilt deps from visibility enforcement
When both prebuilt and source versions of a module are present in the
build an implicit dependency is added from source -> prebuilt. This
change excludes that dependency from visibility enforcement as it
provides no value and would otherwise require an sdk snapshot to
rewrite the visibility of the generated prebuilt modules which would
be complicated.

The rewriting would include:
* Detecting //visibility:public and just passing that straight through.
* Detecting //visibility:private and replacing that with the location
  of the source.
* Otherwise, adding the location of the source to the visibility
  property.

This adds a general mechanism to allow any dependency to be excluded
from visibility enforcement by simply using a tag that implements
the ExcludeFromVisibilityEnforcementTag interface.

Bug: 142940300
Test: m nothing
Change-Id: I0668ff5aa798152d17faf3aac1bb8eff8d6350c3
2020-01-15 11:17:57 +00:00
Paul Duffin c132742c96 Dedup package build components registration
Bug: 146540677
Test: m nothing
Change-Id: Iff2d7063b7f06313e9068c61a5627229463c98dd
2020-01-15 11:17:57 +00:00
Paul Duffin a80ef84652 Support registering hard coded pre arch mutators
Some pre arch mutators are hard coded into mutator.go and so could not
share code for registering those mutators between tests and runtime.
This change adds a new HardCodedPreArchMutators(RegisterMutatorFunc)
method to RegistrationContext which allows hard coded mutators to be
registered alongside other build components during init() and testing.

The method is treated as a noop on the InitRegistrationContext and
behaves just like the PreArchMutators(RegisterMutatorFunc) method on
the TestingContext.

Bug: 146540677
Test: m nothing
Change-Id: I6f8b1e2d54d9dc4e86f951ced61d1ee7b0fe4b2e
2020-01-15 11:17:57 +00:00
Treehugger Robot ae7cddd5ad Merge "Sandbox soong_build by changing to root directory" 2020-01-11 06:00:21 +00:00
Colin Cross 988414c2cf Sandbox soong_build by changing to root directory
This relands I12a0f907753fefd1997ab8b4ea2ac331234093cf along with
a fix to blueprint for absolute paths.

Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Change-Id: I24ac485677aa102eec1a2521d16820da6ee1ae77
Fixes: 146437378
Test: m checkbuild
Test: m OUT_DIR=/tmp/out nothing
2020-01-11 01:11:46 +00:00
Treehugger Robot 5fa3a1f743 Merge "Rename OverridableModuleBase.properties." 2020-01-10 23:59:34 +00:00
Jaewoong Jung b64dd002b3 Rename OverridableModuleBase.properties.
The generic name causes confusion, even to the IDE.

Test: TreeHugger
Change-Id: I0cb4959ab41b4732e7ee7a5d1d818ef771674023
2020-01-10 13:26:00 -08:00
Colin Cross 47e4f9e1e8 Revert "Sandbox soong_build by changing to root directory"
This reverts commit 05c25ccb4a.

Reason for revert: broke absolute OUT_DIR
Bug: 146437378

Change-Id: I523ed79d40e1c1ef040212ba794a7a084abea75d
2020-01-10 18:51:04 +00:00
Colin Cross 05c25ccb4a Sandbox soong_build by changing to root directory
Store the current working directory and then change to the root
directory so that all file accesses must go through helpers in
the android package that properly track dependencies.

Fixes: 146437378
Test: m checkbuild
Change-Id: I12a0f907753fefd1997ab8b4ea2ac331234093cf
2020-01-09 14:19:46 -08:00
Treehugger Robot 64d5ccef7a Merge changes from topic "carry_required"
* changes:
  APEX carries required properties of its dependencies
  *RequiredModuleNames functions are available via Module interface
2020-01-07 03:51:29 +00:00
Colin Cross 31a738bda0 Use blueprint's load hooks
Make android.AddLoadHook wrap blueprint.AddLoadHook.  Also pass
the config object to ParseBlueprintsFiles.

Test: all soong tests
Change-Id: I60c988b717d395f52498ec23ef7c9046d9861a6e
2020-01-06 10:57:26 -08:00
Colin Cross c34d2329b4 Use blueprint.EarlyModuleContext
Make android.EarlyModuleContext wrap blueprint.EarlyModuleContext

Test: m checkbuild
Change-Id: I5e05c4f3dfb177811bc244a5cc2d71f42e47613b
2020-01-06 10:57:24 -08:00
Jiyong Park 6a8cf5fab4 *RequiredModuleNames functions are available via Module interface
...so that the required module names can be known to other modules

Bug: N/A
Test: m
Change-Id: Ib032b51c4ea4fd77bdebcf09b2d1c2aec1c8eb27
2020-01-06 10:35:58 +09:00
Colin Cross 1184b647d5 Add EarlyModuleContext for LoadHookContext
Make LoadHookContext embed a new EarlyModuleContext instead of
BaseModuleContext to reduce its API surface in preparation for
moving it to run during parsing instead of mutators.

Test: m checkbuild
Change-Id: I1cd3ff3b636e7e24991a9184d7521903473e505a
2020-01-04 08:40:55 -08:00
Colin Cross 87f790c509 Delete arch hooks
They are not used anywhere.

Test: m checkbuild
Change-Id: Ice7d68b97ca894c0c4c9efc6dc55c0870a132b9c
2020-01-03 17:21:04 +00:00
Paul Duffin e602918294 Restrict SdkMemberTypes that can be used with sdk/sdk_snapshot
By default SdkMemberTypes are only supported on module_exports module
type. Support for sdk module type has to be explicitly specified.

The java_header_libs, native_shared_libs and stubs_sources are
supported on sdk. The latter is required to provide the stubs source
for an API specified in java_header_libs as they should be kept in
sync.

Bug: 146341462
Test: m nothing
Change-Id: I19b9e60792780a797458d4a9e489506602b13144
2019-12-31 15:08:59 +00:00
Jiyong Park 3ff1699683 Create APEX variation only when needed
Create an APEX variation of a module only when it is either directly
included in an APEX or the dependency to the module demands that the
module should be included. For example, a non-static dependency to a
java library no longer creates an APEX variant of the library.

Bug: 146907857
Test: m (apex_test amended)
Test: cat out/soong/build.ninja | grep "Module:" | wc -l
reduced from 47132 to 45881

Change-Id: Icd79fb4b60668d852b9f018343b934b5e42a8eda
2019-12-27 14:11:47 +09:00
Paul Duffin e2bc36f12c Merge "Generate .srcjar for prebuilt_stubs_sources" 2019-12-23 13:32:08 +00:00
Jiyong Park d061df9ef3 android_*_stubs_current modules are moved to frameworks/base
The modules were moved from developement/build to frameworks/base.
In addition, they no longer uses 'sdk_version:"none"'. Reflect that
change here.

Bug: N/A
Test: m

Exempt-From-Owner-Approval: cherry-pick from internal

Merged-In: I6f69f7cb64a6ed3c2032a116500436eabe562dca
(cherry picked from commit 12cc126760)
Change-Id: I6f69f7cb64a6ed3c2032a116500436eabe562dca
2019-12-22 17:20:27 +09:00
Treehugger Robot 727173ecca Merge "Create product variant for product partition" 2019-12-21 00:56:45 +00:00
Jaewoong Jung dcee89cad6 Merge "Make apex.overrides overridable by override_apex." 2019-12-20 18:20:50 +00:00
Martin Stjernholm d732beed90 Merge "Add script for building all target arch's needed in AML (Android Mainline) prebuilts." 2019-12-20 11:53:33 +00:00
Martin Stjernholm c1ecc43aec Add script for building all target arch's needed in AML (Android Mainline)
prebuilts.

This runs Soong in skip-make mode, using normal in-make mode only to query
platform versions.

The same ${OUT_DIR} cannot be used for both skip-make and in-make builds,
because Soong generates a smaller build.ninja file in in-make builds where
many build targets are expected to be provided by the mk files. Thus this
script avoids using ${OUT_DIR} if it's an in-make build, defaulting instead
to out-aml/.

The script is based on build-ndk-prebuilts.sh, but uses a separate Soong
variable Aml_abis to enable the appropriate target architectures for
Mainline modules. Aml_abis is very similar to Ndk_abis, except "armeabi-v7a"
is used instead of "armeabi", which is necessary to match prebuilt
dependencies, e.g. for LLVM.

Test: build/soong/scripts/build-aml-prebuilts.sh libart libdexfile_external
  (verify that libraries for arm, arm64, x86, x86_64 are built)
Test: build/soong/scripts/build-aml-prebuilts.sh \
  out-aml/soong/.intermediates/external/conscrypt/conscrypt-module-sdk/android_common/conscrypt-module-sdk-current.zip
  (verify that the zip file contains libconscrypt_jni.so's for all four arches)
Test: build/soong/scripts/build-aml-prebuilts.sh com.android.art.{release,debug,testing,host}
  (verify that the build completes)
Test: Two identical build/soong/scripts/build-aml-prebuilts.sh runs after each other
  (verify that the 2nd run completes both Soong and ninja steps quickly without any building)
Change-Id: I35712f9f8f0b1cbb77107314c5927c6720e6c3bf
2019-12-20 11:53:21 +00:00
Justin Yun 5f7f7e8a39 Create product variant for product partition
When PRODUCT_PRODUCT_VNDK_VERSION is set to 'current', product
modules are enforced to use only VNDK libs from the system partition
as BOARD_VNDK_VERSION does to vendor partition.

Modules with 'vendor_available: true' create product variant as well
as core and vendor variants. The product variant as an image variant
is used for the modules in /product or /system/product.

It must not affect the current build behavior without
PRODUCT_PRODUCT_VNDK_VERSION set.

Bug: 134099726
Bug: 138966004
Bug: 144534640
Test: build without PRODUCT_PRODUCT_VNDK_VERSION set
Change-Id: I4d3585c110d84493e45bf76d550dc240bb26137f
2019-12-20 17:56:08 +09:00
Paul Duffin 9b478b0831 Generate .srcjar for prebuilt_stubs_sources
Changes prebuilt_stubs_sources to generate a .srcjar from its input
instead of just exposing the srcs it is given. This ensures that it can
be used as a drop in replacement for a droidstubs module.

Updates the test for prebuilt_stubs_sources to be more representative
of the actual use made of it by sdk snapshot which outputs a directory
not a glob pattern. Added some documentation of the
prebuilts_stubs_sources srcs property to make it clear that it is
supposed to be a set of directories.

Extracts common code from sdk/testing.go for normalizing path/paths
for testing.

Bug: 143678475
Test: m conscrypt-module-sdk conscrypt-module-host-sdk conscrypt-module-test-sdk
      unzip those in place of external/conscrypt
	  build core-current-stubs-source which expects it to provide a .srcjar.
Change-Id: I8204a022557a9b0b45e19eac79ecba98ff95213d
2019-12-20 08:13:45 +00:00
Jaewoong Jung 7abcf8ead8 Make apex.overrides overridable by override_apex.
Test: apex_test.go
Change-Id: Id47e5e5bec45ec1ada68f9d2d806585c5141a2f9
2019-12-19 17:32:06 -08:00
Treehugger Robot 82f4f99780 Merge "Dedup prebuilt mutator registration" 2019-12-20 01:25:51 +00:00
Treehugger Robot 6b295312f6 Merge "Remove unused testWritablePath and associated methods" 2019-12-19 23:14:55 +00:00
Treehugger Robot f5f1ecb746 Merge "Extend RegistrationContext to support pre/post deps mutators" 2019-12-19 23:06:32 +00:00
Treehugger Robot b237de1914 Merge changes I7e9440a0,I7f1a4b64,If5d6fdac
* changes:
  Dedup cc prebuilts module type registration
  Detect registration of duplicate module/singleton types
  Dedup prebuilt apis module type/mutator registration
2019-12-19 22:43:02 +00:00
Colin Cross f6b213f050 Merge "Add helper functions for getting output files from modules" 2019-12-19 22:36:04 +00:00
Colin Cross 35a261de1f Merge "Make sh_binary a HostToolProvider" 2019-12-19 22:35:48 +00:00
Paul Duffin 3eb9dfe136 Remove unused testWritablePath and associated methods
Bug: 143678475
Test: m checkbuild
Change-Id: I4855f97499d3e4f3f5024fd67069b1bb2f9c349f
2019-12-19 20:38:53 +00:00
Paul Duffin 0c4979bbdf Dedup prebuilt mutator registration
Test: m checkbuild
Bug: 146540677
Change-Id: I9f8a11adf61fe6f352ec1eabc7c5bac332348ba2
2019-12-19 19:19:40 +00:00
Paul Duffin 2ccaffd1d7 Extend RegistrationContext to support pre/post deps mutators
Test: m checkbuild
Bug: 146540677
Change-Id: If5b30f1124ce5671c26ec76ca1f7b029841e20b1
2019-12-19 19:19:40 +00:00
Paul Duffin 0a2868309a Detect registration of duplicate module/singleton types
Module type and singleton type names have to be unique but duplicates
are not checked when the relevant android.Register...() func is called.
Instead they are collated in lists and then registered with the Context
later on, at which point duplicates are detected. That loses
information that can be helpful in fixing the issue.

This is not an issue when testing as the module and singleton types
are registered directly with the Context.

This change adds duplicate detection to the initRegistrationContext
Register... methods prior to calling the android.Register... methods
which should make it easier to detect duplicates registered from an
init() function.

Test: m checkbuild
Bug: 146540677
Change-Id: I7f1a4b649072867717a9829c737a44454b12266c
2019-12-19 19:19:40 +00:00
Paul Duffin a48f758d84 Dedup prebuilt apis module type/mutator registration
Test: m checkbuild
Bug: 146540677
Change-Id: If5d6fdace2574df6314fbcf6441838cd11df58ae
2019-12-19 19:19:40 +00:00
Colin Cross 5e70805690 Add helper functions for getting output files from modules
Add OutputFileForModule and OutputFilesForModules to get paths from
OutputFileProducers.

Bug: 122332855
Test: m checkbuild
Change-Id: I2e9aeeb55987742077250390cdb804ba22e79442
2019-12-19 17:52:50 +00:00
Colin Cross 7c7c11445b Make sh_binary a HostToolProvider
sh_binary modules should implement HostToolProvider so that they
can be used in genrules.  Install the script and then return
the installed path.

Also remove the unused SourceFilePath method.

Bug: 122332855
Fixes: 146496647
Test: m checkbuild
Change-Id: I191ce4eef1d50ac0b6a296b819feec6d6a18e753
2019-12-19 09:22:36 -08:00
Paul Duffin f9b1da0fcb Dedup registration code for module types and singletons
The registration of module types and singletons is duplicated between
init() functions that register them for use in the build runtime and
test context creation code that registers them for testing.

This is a proof of concept for a mechanism that will allow the code
to be shared. It defines a RegistrationContext interface that is
implemented by both the TestContext and the new initRegistrationContext
type. An instance of the the latter is available through the
InitRegistrationContext variable.

The intent is that the registration of the module types and singleton
types will be extracted from the init() function into a separate
function that takes a RegistrationContext parameter. That method is
called from init() passing in the InitRegistrationContext and from a
test passing in the TestContext. Something like this:

  func init() {
    RegisterBuildComponents(android.InitRegistrationContext)
  }

  func RegisterBuildComponents(ctx android.RegistrationContext) {
    ctx.RegisterModuleType(....)
    ....
  }

A test would do something like this:

  ctx := android.NewTestContext()
  RegisterBuildComponents(ctx)

Test: m nothing
Change-Id: I97173cabb6d6cf7ce98fdb5f73418438b1997b35
2019-12-19 10:09:53 +00:00
Colin Cross 0bfb9e8d9b Merge "Don't create native bridge variants of vendor modules" 2019-12-18 23:16:04 +00:00
Colin Cross 83bead469b Don't create native bridge variants of vendor modules
The logic in cc.Module.ImageMutatorBegin() to skip creating vendor
variants for native bridge modules stopped working when the image
mutator was moved before the arch mutator because the native
bridge variant hadn't been created yet.  Move the logic to the arch
mutator to avoid creating native bridge variants for vendor variants.

Fixes: 146486214
Test: lunch aosp_cf_x86_pasan-userdebug && m SANITIZE_TARGET=address
Change-Id: I46cff4905f929c9989d005f0ceb01388d47323c1
2019-12-18 10:45:46 -08:00
Colin Cross 98be1bb00f Move filesystem into Config
The filesystem object was available through ModuleContext.Fs(), but
gives too much access to the filesystem without enforicing correct
dependencies.  In order to support sandboxing the soong_build
process move the filesystem into the Config.  The next change will
make it private.

Bug: 146437378
Test: all Soong tests
Change-Id: I5d3ae9108f120fd335b21efd612aefa078378813
2019-12-18 08:19:10 -08:00
Treehugger Robot 572aeed6a4 Merge "Move imageMutator before archMutator" 2019-12-18 02:34:36 +00:00
Jooyung Han 70415ceb9e Merge changes Icbdf4979,I1a6f135d
* changes:
  Fix apex to override certificate
  Fix apex to follow deps of modules with stubs.
2019-12-18 01:04:53 +00:00
Colin Cross fb0c16e95a Move imageMutator before archMutator
Run the imageMutator between osMutator and archMutator so that
different arch variants can be set for the different partitions.

Bug: 142286466
Test: m checkbuild
Change-Id: I65d05714b75aa462bf9816da60fdc2deda4de593
Merged-In: I65d05714b75aa462bf9816da60fdc2deda4de593
(cherry picked from commit 9c8f687584)
2019-12-17 14:45:11 -08:00
Jooyung Han 671f1ce75e Fix apex to follow deps of modules with stubs.
In case that an apex module depends on a module with stubs directly
*and* indirectly, the build system should follow the deps graph further.

Note that WalkDeps() visits deps in DFS and it won't visit again visited
modules.

Bug: n/a
Test: m (apex_test.go amended)
Change-Id: I1a6f135dbda6d1eb641575a3fbbc9bbee0622076
2019-12-17 15:18:54 +09:00
Treehugger Robot 7cf4b5de6b Merge "hostdex:true modules are available for platform" 2019-12-17 06:02:05 +00:00
Treehugger Robot bd0624304e Merge "Use empty string for core image variant" 2019-12-16 20:54:32 +00:00
Anton Hansson f56f74be4a Fix duplicate sentence in visibility documentation
This paragraph was duplicated.

Test: treehugger
Change-Id: I6057b96e310ac0ebf8f6f2a86f93efafd189579e
2019-12-16 11:39:17 +00:00
Jiyong Park b02bb40f33 hostdex:true modules are available for platform
Java libraries with hostdex: true are available for the platform even if
it doesn't have "//apex_available:platform" in the apex_available
property. Note that the java libraries are still prevented from being
installed to the device.

Bug: 128708192
Test: m

Change-Id: I6463ebc59cf7fd861b812999d7a79c387bbb3335
2019-12-16 15:19:13 +09:00
Jiyong Park 505221f6e0 ignore apex_available for host variants
The main purpose of apex_available is to prevent a module from being
installed to the system partition (or being statically linked to
something that is installed to the system partition). It's not the goal
to prevent host tools from using the modules.

However, since both host and device variants are not mutated for
platform when the module is not available to platform, the host tools
that depends on those modules couldn't be built. To solve the problem,
the platform variation is NOT skipped for the host variants.

Bug: 128708192
Test: m

Change-Id: I1d662cd6d165581f344138d872329a15bfc43d17
2019-12-16 15:19:13 +09:00
Treehugger Robot 0558ecce51 Merge "Decouple addition of new sdk member types from sdk code" 2019-12-14 14:57:42 +00:00
Colin Cross 7113d20774 Use empty string for core image variant
Use the empty string for the core image variant so that modules
added to imageMutator do not change their build directory.

Bug: 142286466
Test: m checkbuild
Change-Id: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
Merged-In: Ida4534d9a4d6176236aaa480fed359ce27acfaa1
(cherry picked from commit 72d685ee7f45e5393be44ae4159edf083ac918de)
2019-12-13 20:44:36 -08:00
Treehugger Robot e676810de5 Merge "Install flattened/unflattend apexes for GSI" 2019-12-13 21:10:56 +00:00
Paul Duffin 255f18e584 Decouple addition of new sdk member types from sdk code
Previously, adding a new SdkMemberType would require adding a new
sdkMemberListProperty instance to the sdkMemberListProperties as well
as adding a new property into the sdkProperties struct. They are
potential sources of conflict and couple the sdk code with all the
packages that add members to it. This change switched to a
registration model that allows each package to register its sdk
member types decoupling them from the sdk code.

Adds an SdkPropertyName() method to SdkMemberType that specifies the
name of the property to use in the sdk/sdk_snapshot. Also provides
an SdkMemberTypeBase struct to be used by providers of SdkMemberType
implementations.

SdkMemberType instances are registered using the
RegisterSdkMemberType() func which sorts the registered instances
by their SdkPropertyName() to ensure the behavior is consistent and
not affected by order of registration.

When creating a new sdk module a dynamicSdkMemberTypes instance is
created that contains the following:

* A properties struct is created dynamically that contains a field for
  each registered SdkMemberType, corresponding to that type's
  SdkPropertyName().

* A list of sdkMemberListProperty instances is also created, one for
  each registered SdkMemberType.

The dynamicSdkMemberTypes instance is cached using a key that uniquely
identifies the set of registered types just in case new types are
registered after one has been created, e.g. by tests.

Bug: 142918168
Test: m checkbuild
Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
2019-12-13 20:07:33 +00:00
Paul Duffin 57b9e1da75 Improve handling of generated include dirs
The exported include dirs includes both source and generated
directories (e.g. containing aidl generated headers). The latter are
always arch specific so if they are present they make all the include
directories arch specific.

This change separates the source and generated include dirs so that
the source include dirs (which are probably not arch specific) can be
optimized separately from the arch specific generated include dirs.

The FilterPathList() func was refactored to extract the more general
FilterPathListPredicate() func.

A number of tests needed to be updated to reflect the more optimal
snapshot creation.

Bug: 142918168
Test: m checkbuild
Change-Id: Id1a23d35a45b250ae2168834f9c2a65c86a5fd77
2019-12-13 09:59:48 +00:00
Jiyong Park 0b0e1b9804 AndroidMkEntries() returns multiple AndroidMkEntries structs
AndroidMkEntries now returns multiple AndroidMkEntires so that a module
can emit multiple Make modules if needed.

Bug: 128708192
Test: m

Change-Id: I56b6f76d22943b80329951c5acb80a1b932441ad
2019-12-11 17:25:27 +09:00