Commit Graph

285 Commits

Author SHA1 Message Date
Jiyong Park 16e91a067d Fix: static dependency across an APEX is lost
This change fixes following problem:

1) a native lib having stubs is defined.
2) the lib is included in an APEX.
3) a static binary is linking the lib from outside of the APEX.
4) then, the dependency from the binary to the lib is vanishing.

This is happening because cc.depsToPaths() mistakely does not
distinguish static lib deps from shared lib deps. For shared lib deps,
it creates two dependencies (one for stubs variant and the other for
non-stubs variant) and choose the stubs variant when the lib and the
current module is not in the same APEX (i.e. dependency to the non-stubs
variant is discarded). However, since we don't have stubs variant for
static library, it ends up having no dependency to the library if the
link is static.

Fixing the issue by skipping the variant selection routine when the link
is static.

Test: m (apex_test added)
Test: build with https://android-review.googlesource.com/c/platform/bionic/+/849044
Change-Id: I21102a31cc5c0b105da2affdd035bd5cc571a6ab
2018-12-21 18:01:39 +09:00
Jiyong Park da6eb592bf Add use_vendor to APEX module
use_vendor, when set to true, brings vendor variant of the native
libraries and binaries to the APEX.

Bug: 115707625
Test: m (apex_test updated)
Change-Id: Ib4e996f8652f4ce4645a9c22f6914e2ab35edda6
2018-12-20 14:33:02 +09:00
Jiyong Park 5a8320201a Fix a nill pointer dereference when TARGET_FLATTEN_APEX
When TARGET_FLATTEN_APEX is set to true, there is no single output file
for an APEX that other modules can reference via ":module" syntax.
Return nothing in that case.

Bug: 121221006
Test: TARGET_FLATTEN_APEX=true m
Change-Id: I556b55073720b16fd30fd133af58aac229e958c0
2018-12-20 09:54:35 +09:00
Jiyong Park a88948419a Fix: Flattend APEX is breaking build
When TARGET_FLATTEN_APEX=true, individual files in an APEX are exported
to make. However, because they lack LOCAL_SOONG_CLASSES_JAR,
soong_java_prebuilt.mk get confused and the build breaks.

Fixing the bug by correctly emitting LOCAL_SOONG_CLASSES_JAR and
LOCAL_SOONG_HEADER_JAR.

Test: TARGET_FLATTEN_APEX=true m
out/target/common/obj/JAVA_LIBRARIES/com.android.conscrypt.conscrypt_intermediates/classes.jar

Change-Id: I65ce86ec825f978a43715146bbd5c1d5af513de0
2018-12-19 23:07:23 +09:00
Treehugger Robot 8cc51b5620 Merge changes Idb2b552b,I190bca35
* changes:
  When a stub is built for APEX, it is generated with --apex
  APEX-specific symbols can be tagged as # apex
2018-12-19 06:28:23 +00:00
Treehugger Robot ace2be498e Merge "Add installable property to apex module type" 2018-12-15 17:59:41 +00:00
Jiyong Park 3fd0baf651 When a stub is built for APEX, it is generated with --apex
Now, symbols that are only to be visible to across APEXes can be tagged
as # vndk. Then when generating the stubs library, the tagged symbol
is included. The symbol is NOT included in other cases; build NDK stubs,
etc.

Bug: 120638081
Test: m (apex_test updated.)
Change-Id: Idb2b552badddfc26af113cc8d4b984788f478813
2018-12-16 02:13:14 +09:00
Jiyong Park 6437995170 Remove -include <file> when compiling stubs.
Otherwise, the force included headers might cause conflicting types
error with the symbols in the generated stubs source code. e.g.

double acos(double); // in header
void acos() {} // in the generated source code

Bug: 120266448
Test: m (apex_test also updated, and passing)
Change-Id: I717df990959d1b24ec515f0f52a4e28b94e92f32
2018-12-15 23:03:47 +09:00
Jiyong Park 92c0f9ce54 Add installable property to apex module type
When set to false, the apex is built but not installed. Useful for not
polluting the system image with test APEXes when doing mma.

Bug: 120960303
Test: add 'installable: false' to apex.test under system/apex/tests/Android.bp
m; check that out/..../system/apex/apex.test.apex does not exist.

Change-Id: I383d5cdcc8aec705b6a5778dbd07233471d289d0
2018-12-15 22:58:14 +09:00
Jiyong Park 0ddfcd1188 Don't create unnecessary APEX variations
This change fixes a problem that APEX variations are created for the
modules that actually shouldn't built for any APEX. For example,
consider this case.

apex { name: "myapex", native_shared_libs: ["mylib"],}

cc_library { name: "mylib", shared_libs: ["libfoo#10"],}
cc_library { name: "libfoo",
             shared_libs: ["libbar"],
             stubs: { versions: ["10"], }, }
cc_library { name: "libbar", ...}

Before this change, both the stubs and non-stubs variations of libfoo
were mutated with apexMuator, which is incorrect for the non-stubs
varia; there is no dependency chain from the apex "myapex" to the
non-stubs variation, but to the stubs variation due to the #10 syntax.

This was happening becauses we used the name of the module to determine
whether it should be built for APEX or not. Both stubs and non-stubs
variations have the same module name "libfoo".

Fixing this issue by recording the list of APEX variations required
directly on the module. So, the stubs variation of libfoo has myapex in
its apex variations list, but the non-stubs variation doesn't, and thus
apexMutator does not pick up the non-stubs variation.

Test: m (apex_test updated and passing)
Test: cherry-pick ag/5747464 and m
Change-Id: I31e618626809a828a55fff513ef5f81f79637afa
2018-12-13 10:48:15 +09:00
Jiyong Park 74e240be03 Output of an apex module type can be included in tests
By implementing SourceFileProducer interface, output of an apex module
can be included in tests (via ':modulename' syntax in 'data' property)

Bug: 120055902
Test: replace apexd_testdata/* with :apex.test in
system/apex/apexd/Android.bp. m apex_file_test.
out/host/linux-x86/nativetest/apex_file_test/ has apex.test.apex in it.

Change-Id: I8b721a68e0edc65a3e674febeed485e06b96bc24
2018-12-12 18:33:30 +09:00
Jiyong Park 28d395a149 Fix: build error when a lib with stubs is included in an APEX
apex { name: "foo", native_shared_libs: ["mylib"] }
cc_library { name: "mylib", shared_libs: ["other_lib"],
             stubs: { versions: ["1"]}, }

This is causing build error due to missing variant for other_lib.
This is happening because the stubs variant of mylib is added to apex
foo instead of the non-stubs variant. Because stubs variant does not
have any further dependencies, other_lib is not included to the APEX and
is not built for it.

Fixing this issue by specifying the version variant when adding a lib to
the dependency of an APEX, so that non-stub variant of the lib is
depended on.

Test: m (apex_test updated)
Change-Id: I972b6dcbce11942f83a76212715ba915534ec3df
2018-12-10 13:33:41 +09:00
Jiyong Park 7c2ee713ec Fix: a module with sub_dir can't be placed in APEX
This change fixes the bug that when a module is defined with sub_dir,
then build breaks when the module is included in an APEX.

This was happening because, for example when we have a prebuilt_etc
module having sub_dir set to "foo/bar", then only etc/foo/bar is added
to the canned_fs_config file and other intermediate directories (etc,
etc/foo) are not added. e2fsdroid however expects that every directories
to be listed.

Fixing the problem by adding parent directories when adding a directory
to canned_fs_config.

Bug: 120600179
Test: m (a new test case added to apex_test)
Change-Id: If712ff65761a7e1e3216371bb2eb7acf9cb5dc9e
2018-12-07 14:29:59 +09:00
Alex Light ee25072e35 Make apex.ApexBundleFactory public
Art needs to be able to create an ApexBundle with a LoadHook to
support art testing using the HOST_PREFER_32_BIT=true hack.

Bug: 120617876
Test: HOST_PREFER_32_BIT=true m nothing
Change-Id: Ia11e61a92094dfbc013c6c53a6edff33371ed8e1
2018-12-06 14:02:16 -08:00
Alex Light 5098a612fc Add zip-apex
This adds support for an apex-like file where the payload is a zip
file instead of a FS image. This makes it easier to examine and use
without root. These can also be built for the host.

This can be controlled using the 'type' modifier in the apex build
rule. 'type' may be "image" to build an apex file called {name}.apex
with an FS image payload, "zip" to build an apex with called
{name}.zipapex with a zip-file payload or "both" to build both. By
default we will use "image".

This also adds support for setting apex rules to
"host_supported: true".

Test: cd art/build/apex; mma; examine the com.android.runtime.host.zipapex
      artifact.
Test: go test android/soong/apex -v
Bug: 120436895

Change-Id: I534d330672211ac5ccc3bd5a0c89b9fc507bf51e
2018-12-04 13:10:17 -08:00
Jiyong Park 25fc6a9cc9 Stubs variant is used when building for APEX
When a native module is built for an APEX and is depending on a native
library having stubs (i.e. stubs.versions property is set), the stubs
variant is used unless the dependent lib is directly included in the
same APEX with the depending module.

Example:

apex {
    name: "myapex",
    native_shared_libs: ["libX", "libY"],
}

cc_library {
    name: "libX",
    shared_libs: ["libY", "libZ"],
}

cc_library {
    name: "libY",
    stubs: { versions: ["1", "2"], },
}

cc_library {
    name: "libZ",
    stubs: { versions: ["1", "2"], },
}

In this case, libX is linking to the impl variant of libY (that provides
private APIs) while libY is linking to the version 2 stubs of libZ. This is
because libY is directly included in the same apex via
native_shared_libs property, but libZ isn't.

Bug: 112672359
Test: apex_test added
Change-Id: If9871b70dc74a06bd828dd4cd1aeebd2e68b837c
2018-12-04 17:46:22 +09:00
Shahar Amitai 328b077e20 Revert "Add resources.pb to the apex bundle"
This reverts commit 1ed0fc532d.

Reason for revert: resources.pb not needed, version code is taken from AndroidManifest.xml

Bug: 117295826
Test: tapas com.android.runtime.debug; m dist
$ unzip -l com.android.runtime.debug-base.zip
Archive:  out/dist/com.android.runtime.debug-base.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
 29618176  1980-01-01 09:00   apex/armeabi-v7a.img
       52  1980-01-01 09:00   root/apex_manifest.json
      464  1980-01-01 09:00   manifest/AndroidManifest.xml
---------                     -------
 29618692                     3 files

Change-Id: I677b9e70726087aa3ecc2949648976a0dfcd57dd
2018-11-27 18:12:09 +09:00
Dario Freni 4abb1dc10b Rename manifest and image files.
manifest.json -> apex_manifest.json
image.img -> apex_payload.img

Bug: 119672727
Test: m, installed on device, boots.
Change-Id: Id79c6a1ac8f5d306f534477d94b76f8ed0e7d7a7
2018-11-24 16:31:59 +00:00
Jiyong Park 1ed0fc532d Add resources.pb to the apex bundle
The file was missing.

Bug: 117295826
Test: tapas com.android.runtime.debug; m dist
$ unzip -l com.android.runtime.debug-base.zip
Archive:  com.android.runtime.debug-base.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
 29614080  1980-01-01 09:00   apex/armeabi-v7a.img
       52  1980-01-01 09:00   root/manifest.json
      379  1980-01-01 09:00   manifest/AndroidManifest.xml
       59  1980-01-01 09:00   resources.pb
---------                     -------
 29614570                     4 files

Change-Id: I916ad5ba479c952a106b055c0e71e7e6556b6101
2018-11-23 13:33:11 +09:00
Jiyong Park bfe64a1a59 Align entries in APEX at 4K boundary
Use -a option of signapk to specify the alignment to 4K. The default is
4.

Test: m apex.test; zipalign -c -v 4096 $(OUT)/system/apex/apex.test.apex
shows ok

Change-Id: I7b1a3d442b69b89838cf09451939cd23d5646bde
2018-11-22 03:09:40 +09:00
Jiyong Park 25560152c3 Remove buildspam from apexer
Turning --verbose off.

Bug: 119768180
Test: m apex.test
[ 99% 5758/5761] //system/apex:apex.test apex [common]
[100% 5761/5761] Install: out/target/product/walleye/system/apex/apex.test.apex

Change-Id: Iddebbbc6a6ad1bc8145abc7878bc310eac4d2792
2018-11-20 09:58:50 +09:00
Colin Cross a49259071a Convert apex modules to app bundle modules
Convert .apex files to base.zip files suitable for bundletool.
They will be copied to the dist directory when built with
TARGET_BUILD_APPS and dist.

Also fix up apex.go to use Build instead of the deprecated ModuleBuild,
and ctx.ModuleName() instead of module.ModuleBase.Name().

Bug: 117295826
Test: tapas com.android.runtime && m dist
Change-Id: I7919c8d3da280e87c624a6628a1715e8060852dc
2018-11-16 12:25:28 -08:00
Jiyong Park 02196574a9 Rename APEX file_contexts as <name>-file_context
It was <name>_file_context before. Changing for better readability.

Bug: 119034475
Test: m apex.test com.android.tzdata com.android.runtime
Change-Id: I0d3d3319370cdbb366d06626c8ac978ca8d875c3
2018-11-15 07:25:08 +09:00
Jiyong Park 8fd6192480 APEX can be flattened
When TARGET_FLATTEN_APEX is set to true, APEXes are flattened, which
means files in an APEX is not packaged into the mini file system image,
but instead directly copied to the system partition.

This option is for devices where kernel does not support loopback
devices or the maximum number of loopback devices is too small (though
the threshold is TBD as of now).

This CL also fixes a bug that jars having bytecode are installed instead
of those having dex.

Bug: 118485880
Test: TARGET_FLATTEN_APEX=true m apex.test; tree
out/target/product/.../system/apex/apex.test shows list of files in it.

Test; m apex.test, then a file out/target/product/.../system/apex/apex
.test.apex exists.

Change-Id: I5a3d62d392d05f2779c4925388afe4f6e460059b
2018-11-14 13:41:54 +09:00
Jiyong Park d0a65bae17 Move file_contexts file for APEXes to under /system/sepolicy
For centralized development of sepolices, file_contexts files for APEXes
are all moved to under /system/seplicy.

The meaning of the existing file_contexts has been changed; when it is
set to <value>, then /system/sepolicy/apex/<value>_file_contexts is
used. When unset, it defaults to the name of the module. The property is
not removed in order to support creating multiple versions of the same
apex (for testing purpose) built with same file_contexts file.

Bug: 119034475
Test: m apex.test com.android.tzdata com.android.runtime
Change-Id: I7d14a9e37baea9ab78d9e15e1164cce54d256f56
2018-11-13 10:59:40 +09:00
Jiyong Park fa0a37365b Fail gracefully when apek_key is not found
When the private key file can not be obtained, e.g., due to missing git
project where the apex_key module is defined, fail gracefully with an
error message instead of panicing.

Bug: 119256776
Test: modify the key property in /systme/apex/Android.bp to
apex.test.key2 and run `ALLOW_MISSING_DEPENDENCIES=true m apex.test`
The build fails with

error: system/apex/Android.bp:32:8: module "apex.test" variant "android_common_apex.test": key: private_key for "apex.test.key2" could not be found

Change-Id: I2cd5fed0f046de1f0b380fbdac5f8b82816b0e24
2018-11-09 05:55:32 +09:00
Roland Levillain 2b11f74130 Fix ext4 image's root directory permissions of APEX packages.
The generateFsConfig rule used to not give execution
permissions (0111) to the root (`/`) directory of the ext4 image
stored in an APEX package. This change add these bits, turning the
permissions from 0644 (drw-r--r--) to 0755 (drwxr-xr-x).

Previously, not having the execution permissions on the ext4 image's
root directory meant that the contents of the top-level directory of
the mounted image could not be listed.

Test: `adb shell ls /apex/<apex-package>` works
Bug: 117580281
Bug: 112672359
Bug: 113373927
Change-Id: I96f859de6022e04c0b82ccdf1db683887643e588
2018-11-02 19:22:45 +00:00
Jiyong Park c00cbd9e7b APEXs are signed with apk signer
The entire APEX (which is a zip file) is signed with the apk signer.
Certificate can be specified via the 'certificate' property just like
ordinary apps. Note: multiple additional certificates are not supported.

Bug: 115721587
Test: m apex.test
Test: jarsigner -verify -verbose -certs .../apex.test.apex shows the
certificate info

Change-Id: Ia4c898d3427779a3809fdc683b85d7661ca65137
2018-10-31 10:13:54 +09:00
Roland Levillain 54bdfda21c Use SDK prebuilt 'aapt2' in 'apexer' in source trees without "frameworks/base".
Test: system/apex/apexer/runtests.sh on a master source tree
Test: system/apex/apexer/runtests.sh on a master-art source tree
Bug: 113373927
Bug: 112458021
Change-Id: Ie23eab3f7e4961ebbe011b1fe25ce819f99767a2
2018-10-30 14:35:40 +00:00
Jiyong Park 397e55e0cf binaries and native_shared_libraires are multilib-aware props
The properties 'binaries' and 'native_shared_libraries' can be
multilib-aware, i.e, can be under multilib.type where type can be either
first, both, lib32, lib64, or prefer32.

Native modules listed in multilib.first are installed only for the first
ABI of the device. Similarily, multilib.both are for both of the ABIs,
while multilib.lib32 and multilib.lib64 are 32 and 64-bit ABI only,
respectively. multilib.prefer32 is for 32-bit only when 32-bit ABI is
available.

Another change is that the binaries property, when not within multilib,
targets only the first ABI.

Test: m apex.test on ...
1) aosp_arm64 without TARGET_PREFER_32_BIT_EXECUTABLES=true
2) aosp_arm64 with TARGET_PREFER_32_BIT_EXECUTABLES=true
3) aosp_arm
in all cases, vold, surfaceflinger and drmserver are all intalled under
./bin directory of the APEX. And native libraries are installed under
both ./lib and ./lib64 directories in the case of 1) and 2).

Change-Id: Idd7f8526a61bceca89d43c0c69ccedb471b67d31
2018-10-26 15:47:53 +09:00
Jiyong Park 678529ef29 arch-specific dependencies are correctly handled in apex
The change fixes the problem that multiple APEXs having native component
(shared lib or executable) can't be built with an error similar to this:

error: bionic/linker/Android.bp:298:1: failed to find variation "com.android.art" for module "libclang_rt.builtins-aarch64-android" needed by "ld-android"

This is happening because the dependency to the built-in library is
arch-specific but apex.go ignores it. Specifically, let's assume that
32-bit variant of libFoo depends on libX while 64-bit variant of libFoo
depends on libY. Also assume that libFoo is included in two APEXs: apex1
(which is 64-bit) and apex2 (which is 32-bit). Then apexDepsMutator
records that libFoo shall be split into apex1 and apex2, while libX will
be only split into apex1 and libY will be split into apex2.

The problem is that, during apexMutator, both 32 and 64-bit varants of
libFoo are split into apex1 and apex2. As a result, a dependency to
apex2 variant of libX and a dependency to apex1 variant of libY are
requested, which don't exist.

Fixing this issue by using module's name AND target string as the key.
So, only the 32-bit variant of libFoo is split into apex1 and 64-bit
variant of libFoo is split into apex2.

Test: have following module somewhere. `m` is successful.

  apex {
      name: "com.android.art",
      manifest: "manifest.json",
      file_contexts: "file_contexts",
      binaries: ["dex2oat"],
      key: "apex.test.key",

      target: {
          android: {
              // Prefer the 32-bit versions of these binaries.
              compile_multilib: "prefer32",
          },
      },
  }
Change-Id: I562b7be8e7c0325bd8d728dbee2ddcae608c181a
2018-10-25 00:02:18 +09:00
Jiyong Park ff1458f670 APEX can be signed with different keys
A new module type 'apex_key' is defined to specify public and private
key pair for APEXs. An APEX can refer to the module via the property
'key'. When building the APEX, the private key from the key module is
used to sign it. In addition, the public key from the key module is
automatically installed to /system/etc/security/apex.

Bug: 115721587
Test: m apex.test; m
/apex/com.android.example.apex@1 exists

Change-Id: I82666db095bd7a09f6c1b9cbea2db57ebc076cbf
2018-10-16 09:53:11 +09:00
Jiyong Park 92905d6b4b Directories and executables files in an APEX have x bit set
Files under /bin and all directories in an APEX now have x (executable)
bit set correctly.

Bug: 117580281
Test: m apex.test, push it to /data/apex and reboot.
adb shell ls -al /apex/com.android.example.apex@1 shows that the
directories have x bit set

Change-Id: I76e4188d86dc9cdf65e9f8e52be1981e25441a6e
2018-10-11 14:24:32 +09:00
Jiyong Park ab3ceb3855 Make APEX build rules consistent
Don't directly iterate over the copyManifest map to generate the copy
commands. Iterating over a map in golang isn't guaranteed to give
consistent order. This causes the apex build rules to be executed even
when there is no source file change.

Fix the issue by creating a sorted list of the key and then iterate over
the list.

Bug: 117453592
Test: m apex.test; m.apex.test   nothing is built during the second
build

Change-Id: I329a91ec0b6a34cbe745bf9a9ceb0843b63c200c
2018-10-11 14:24:32 +09:00
Jiyong Park 48ca7dc535 Add apex variants
Now, APEX-aware modules (the ones implementing android.ApexModule
interface) are created with multiple variants for each APEX that they
are included.

For example, if a module is included (either directly or indirectly -
via static linking) to two APEXs, the module is built separately for the
two APEXs (and of course separately for platform). This is a first step
to limit the symbol visibility to the modules built for APEXs; platform
private symbols and libs shouldn't be allowed for them.

In addition, the build system now tracks transitive dependencies of
the modules in APEXs. For example, if
native_shared_lib_modules:["libFoo"] then libFoo and its dependencies
are all automatically included to the APEX.

Bug: 112672359
Test: m apex.test; the built apex has additional libs (such as liblog,
libc++, ...) that are dependencies of the ones specified in Android.bp

Change-Id: Id9e3fc486dd4e7e36f8b6799dfb041868c5198d5
2018-10-11 14:24:18 +09:00