ModuleSrcPath was designed as a type that ensured that modules only
referenced sources inside the directory that contained the Android.bp
file. In practice they don't work very well, because allowing
filegroups and genrules as inputs to any module that takes a source
path means that the path might end up being to a file in another
source directory or to a generated file in the output directory.
Remove ModuleSrcPath, replacing it with SourcePath in the places
that need to explicitly refer to a path in the source tree, or
Path where it may be a source path or a generated path.
Make PathForModuleSrc return a Path instead of a SourcePath in
preparation for consolidation with ctx.ExpandSources, which will
make it possibly return paths to generated files.
Test: All soong tests
Change-Id: I973a78470ed14307eea5f6d0cc93942775a65715
Replace ExpandSourcesSubDir with ExpandSources plus
PathsWithModuleSrcSubDir, which loops over the paths and uses
Join to create paths relative to subdir on any results that
are ModuleSrcPaths.
Test: All soong tests
Change-Id: I11a7face88641e2c26ccdca0a3117d5c38ab588e
Copying p.rel doesn't work, as rel needs to match the end of the
value in path. Apply the same transformation to p.rel as p.path.
Test: paths_test.go
Change-Id: I42d676c6c4fc18d9852c1a73f25e5a791d7553d0
Add PathsForOutput to convert multiple strings into WritablePaths.
Add OutputPath.InSameDir to build a new OutputPath pointing to a
file in the same directory as an existing OutputPath.
Add WritablePathForTesting and WritablePathsForTesting that mirror
PathForTesting and PathsForTesting but return WritablePaths.
Add PathContextForTesting to return a minimal PathContext
implementation.
Test: paths_test.go
Change-Id: I9708eb164b273514a96dae0a260ef9a963fb9bcf
Add a helper method that returns a new OutputPath with the
extension replaced.
Test: paths_test.go
Change-Id: I7f93b3475031a19b53652022740488373a1b7d97
Quote the path parameter in the message to distinguish it
from the text. This makes messages more understandable.
Before: source path include does not exist
After: source path "include" does not exist
Test: build Android
Change-Id: I99dbbce3cf090682a230d05bf120549a2cc7af3c
Specifying [] for resource_dirs or asset_dirs will prevent using
the default "res" or "assets" directories.
Test: TestResourceDirs
Bug: 124035856
Change-Id: I96e38ac1319260db43950299a8b1774da68ea85e
filepath.Abs is surprisingly expensive, it calls os.Getwd every
time, which involves multiple syscalls, a lock, and and allocations.
Use IsAbs and prefix matching instead.
Test: paths_test.go
Change-Id: Ia6cf34d6bef24c694702af1e7a6ff08ffd2d822b
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules. The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.
This relands Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb and
I462182638bd57b1367b5bfb0718e975c11ae66f7, along with multiple fixes
to depsfile generation in dexpreopt_gen that caused .odex files for
modules in defined make to be missing dependencies on boot.art, and
a fix to not dexpreopt and strip tests.
Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
only expected changes to dexpreopt outputs on system_other
(.vdex files for privileged Soong modules no longer incorrectly
contain .dex contents).
Test: OUT_DIR=$PWD/out m
Test: NINJA_ARGS="-t deps out/target/product/sailfish/obj/APPS/Contacts_intermediates/dexpreopt.zip" m
Change-Id: I6bb2c971cee65d2338839753aa0d84939f335b1b
MaybeRel was failing with an error when comparing an absolute out
path to a relative source path. Make it return false instead, and
add tests.
Bug: 119412419
Test: paths_test.go
Change-Id: I462182638bd57b1367b5bfb0718e975c11ae66f7
Port the dexpreopt logic from Make to the dexpreopt package in Soong,
and use it to dexpreopt Soong modules. The same package is also
compiled into the dexpreopt_gen binary to generate dexpreopt scripts
for Make modules.
Bug: 119412419
Bug: 120273280
Test: no differences to dexpreopt outputs on aosp_sailfish system/,
only expected changes to dexpreopt outputs on system_other
(.vdex files for privileged Soong modules no longer incorrectly
contain .dex contents).
Change-Id: Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb
We're only using it to distribute files in case of failure, which isn't
well supported currently, but can be handled for now by using the
DIST_DIR environment variable during the command execution.
This was at least one cause that we'd be re-running Soong during every
build server build, as the DIST_DIR values are unique.
Test: m dist
Change-Id: Ibd5e6b6c46695350de80b745bfb6a6aa685033a0
pathtools.Glob now takes a ShouldFollowSymlinks argument. Add
pathtools.FollowSymlinks to all the calls to pathtools.Glob to
maintain current behavior.
Test: m checkbuild
Change-Id: I0215efb212be4ba4513669612d88c2baade43747
The icu resource directories contain filenames that have '$'
characters.
Allow paths returned by the Glob functions to contain '$', on the
assumption that real paths on disk are unlikely to contain strings
that are valid ninja variables. Fix the Build rules to escape any
paths that are passed as Path arguments. Fix the resource rules to
manually escape the paths that are passed as strings.
Test: m checkbuild
Change-Id: Ie631bc6d96259e592adb280491a365c0df7ed0e2
Some paths contain glob characters. For now allow paths with glob
characters as long as they are in glob results. In the future we
may need to allow escaping glob characters.
Test: m checkbuild
Test: paths_test.go
Change-Id: I1cbeea658e8fc4975ca0b6a50a8c24ac2de026c5
Make :module on a java_library provide the output file, which is
normally the implementation jar. For java_library modules with
installable: true or compile_dex: true this will be the dexjar
instead. For android_app modules this will be the apk.
Bug: 80144045
Test: no change to out/soong/build.ninja
Change-Id: I739674aee60a38bfccb859369e4414b46f293d82
This is an adaptation of Icc4f8c16bc389fe20db680849f311d02df1299c3, to
support modules that are installed on the /product-services partition.
Bug: 80741439
Test: m -j both with and without enabling the new partition
Change-Id: I72b335ad38baff5848cd3da7489343f8cf98ff16
This commit adds an option to read ABI reference dump in text file
format directly from `prebuilts/abi-dumps`. If both the text file and
the gzip format exist, an error will be emitted.
Bug: 78650426
Test: create libexif.lsdump.gz and it works as usual
Test: create libexif.lsdump (decompressed) and it works
Test: touch both libexif.lsdump.gz and libexif.lsdump and it errors
Change-Id: I420a5953fb80855cb5c07e5a4d347fb6709f0340
This commmit removes `vndkVsNdk()`, which is essentially
`!inList(ctx.baseModuleName(), llndkLibraries)`.
Test: lunch aosp_arm64_ab-userdebug && make
Change-Id: I8e2352f302df30057997944678f176f4550d3f75
This commit removes `isSourceDump` from `PathForVndkRefAbiDump()`
because the binary dump paths are not being used at all.
Test: lunch aosp_arm64_ab-userdebug && make
Change-Id: I7a6bb9053a6b052590fb1152982949fee897df8d
Recovery partition has same layout as system. In other words,
executables are installed to <recovery_root>/system/bin and libs are
installed to ../system/lib.
This is made possible because the recovery partition is now
self-contained, i.e., shell, adbd, etc. are hosted in the recovery
partition and thus we no longer has to mount the real system.img
to /system. So /system is now available to executables built for
recovery mode.
Bug: 63673171
Test: `adb reboot recovery; adb devices` shows the device ID
Test: Select 'mount /system' in the recovery mode, then `adb shell`.
$ lsof -p `pidof adbd` shows that libm.so, libc.so, etc. are loaded from
the /lib directory.
Change-Id: I6c4f903ad21fd3dd10a07c4588be85d3d678c099
`recovery: true` installs a module to the recovery partition.
`recovery_available: true` makes a module to be available to other
`recovery:true` or `recovery_available: true` modules.
These to are very similar to vendor, vendor_available properties, except
for the target partition.
Bug: 67916654
Bug: 64960723
Test: m -j, toybox_recovery is installed to the recovery/root/sbin
Change-Id: Iaebe0593de16c69fa70de251a61f4d018a251509
LOCAL_RESOURCE_DIRS puts the highest priority overlay first, but
aapt2 expects the highest priority overlay last. Soong stores the
list in aapt2 order (low to high priority), but that means when it
exports to Make as LOCAL_SOONG_RRO_DIRS, which goes to
build_rro_package.mk and then package_internal.mk, it gets reversed
again and comes out backwards.
Bug: 78032566
Test: m checkbuild
Change-Id: If72bf929fbf1d126f9051a2f21ec1eb4e3030e6e
All access to these should be going through the methods on Config /
DeviceConfig.
Bug: 76168832
Test: m blueprint_tools
Change-Id: I47512dd58fb1a1a3f25838a9b1adaed2c41af8d3
Instead of open-coding the logic of whether there is one, or where to
find it.
Test: diff out/soong/build.ninja without dist
Test: diff out/soong/build.ninja with dist specified
Change-Id: Ia3f1ef335e2d6e2175343338d04867d778a50300
Binder bitness does not always correlate directly with primary arch, as
assumed earlier. For example: it is possible to have devices with
primary arch 'arm' and use 64 bit binder.
Bug: 74362434
Test: utils/create_reference_dumps.py;
m -j vndk_package for aosp_arm64_ab, aosp_arm_ab invokes
header-abi-diff on both arm and arm64 arches.
Test: mm -j64 in system/libhwbinder on troublesome internal target.
Change-Id: Iea0a24b57cdb3033e25b6fe126c5d5d717f45b4e
binder_size_t has a different size for builds with different primary
arches.
Also maintain seperate reference dumps for different arch variants,
since different cflags may be legally specified for them (similar to
what GSI does)
Test: create reference dump for libjpeg at
prebuilts/abi-dumps/vndk/current/arm64/arm64_armv8-a/source-based/libjpeg.so.lsdump.gz
mm -j64; header-abi-diff gets invoked.
Change-Id: I55eae4d4811c9754fe8dbd1009c7929fea119eeb
GlobFiles had allowed results to be anywhere in the source tree,
restrict it to results within the current module directory.
Then use it for ExpandSources and other places where we only want files.
This fixes using '*' in cc_test's `data` property, which can only
support files.
The only thing this changes today is that java_resource_dirs and
java_resources no longer pass directories to soong_zip's -f argument.
core-libart previously added some icu directories, now it only passes
files.
Bug: 71906438
Test: only expected changes in out/soong/build.ninja
Test: add data: ["**/*"] to a cc_test, build successfully
Change-Id: Iff1bd8c005a48e431c740706d7e23f4f957d8b1d
This reapplies Id7925999a27ea75a05e9301bbf1eb9f9a6bc4652 with
additional fixes to not use PathForSource in PathForModuleSrc.
PathsForSource was handling the AllowMissingDependencies case, but
PathForSource was not. Refactor PathForSource and
ExistentPathForSource, and add logic to PathForSource to fall back
to behavior similar to ExistentPathForSource when
AllowMissingDependencies is set.
PathForModuleSrc uses PathForSource, which causes too many
globs (>50k). The AllowMissingDependencies check doesn't make
much sense for PathForModuleSrc, since we already know the
project containing the definition of the module exists, we can
expect its local source files to exist. Use pathForSource and
do an manual existence check instead.
Test: paths_test.go
Test: m ALLOW_MISSING_DEPENDENCIES=true
Change-Id: If1690a708393964d3030cb908beaf7b6897c0084
This reverts commit 94a321045a.
Reason for revert: Broke builds with ALLOW_MISSING_DEPENDENCIES=true
Change-Id: I9604887f796a79809ef8d6c741597796219dcaf3
PathsForSource was handling the AllowMissingDependencies case, but
PathForSource was not. Refactor PathForSource and
ExistentPathForSource, and add logic to PathForSource to fall back
to behavior similar to ExistentPathForSource when
AllowMissingDependencies is set.
Test: paths_test.go
Change-Id: Id7925999a27ea75a05e9301bbf1eb9f9a6bc4652
The next patch will need to more complicated custom error handling,
so make validatePath return an error and let the caller handle it.
Test: paths_test.go
Change-Id: I4fe11c3f319303d779596709f4819e828b5bdb9b
Bug: 64195575
Test: succeeded building product.img with BOARD_PRODUCTIMAGE_PARTITION_SIZE,
BOARD_PRODUCTIMAGE_FILE_SYSTEM_TYPE and PRODUCT_PRODUCT_VERITY_PARTITION.
Change-Id: Icc4f8c16bc389fe20db680849f311d02df1299c3
Added three properties (soc_specific, device_specific, and
product_specific) that shows what a module is specific to.
`soc_specific: true` means that the module is specific to an SoC
(System-On-a-Chip) and thus need to be installed to vendor partition.
This has the same meaning as the old `vendor: true` setting.
`device_specific: true` means that the module is specific to the entire
hardware configuration of a device includeing the SoC and off-chip
peripherals. These modules are installed to odm partition (or /vendor/odm
when odm partition does not exist).
`product_specific: true` means that the module is specific to the
software configuration of a product such as country, network operator,
etc. These modules are installed to oem partition (or /system/oem when
oem partition does not exist). These modules are assumed to be agnostic
to hardware, so this property can't be true when either soc_specific or
device_specific is set to true.
Bug: 68187740
Test: Build. path_tests amended.
Change-Id: I44ff055d87d53b0d2676758c506060de54cbffa0
This allows TestModule.Output("foo") to be used to find
a file called "foo" that was installed using ctx.InstallFile.
Test: soong tests
Change-Id: I04833c9ee8ac5baa6b6afd35715d1191c1622a78
AConfig() now duplicates Config(). Replace the uses of AConfig()
with Config(). Leave AConfig() for now until code in other
projects is cleaned up.
Test: m checkbuild
Change-Id: Ic88be643049d21dba45dbd1a65588ed94bf43bdc
In Soong, a Config() method will always return a Config. Make
ModuleContext, SingletonContext, TopDownMutatorContext and
BottomUpMutatorContext's Config() methods explictly return
a Config to avoid having to type-assert everywhere. Overriding
the Config method requires duplicating the list of methods in
blueprint.BaseModuleContext and blueprint.BottomUpMutatorContext,
following the same pattern used by the other *Contexts.
Config() obsoletes the AConfig() method used in some places, which
will be cleaned up in the next patch.
Test: m checkbuild
Change-Id: Ibe21efde933959811d52443496967ab8ce71215e
Use aapt2 instead of aapt to compile Android app resources.
Also generate all files into srcjars instead of individual
sources.
Test: m checkbuild
Change-Id: I5a67991a0daf0017e8159b46fcff7d5564a91468
Wrap blueprint.PackageContext so that the *Func methods can provide
an android.Config instead of an interface{}. The modified signatures
means that every method in ModuleContext and SingletonContext
that takes a blueprint.PackageContext now needs to be wrapped to
take an android.PackageContext.
SingletonContext wasn't previously wrapped at all, but as long
as it is, wrap everything like ModuleContext does. This requires
updating every Singleton to use the android-specific methods.
Test: builds, all Soong tests pass
Change-Id: I4f22085ebca7def6c5cde49e8210b59d994ba625
Add an android.DirectorySortedPaths that stores paths sorted such
that all paths in a directory including subdirectories are in a
contiguous subslice. This will allow efficient O(log(N)) finding
of all paths in a directory using a binary search on the directory
prefix.
Test: TestDirectorySortedPaths in paths_test.go
Change-Id: I5a06a89351ae06e88c06526be54a6b79075361b7
Move firstUniqueElements to android.FirstUniqueStrings,
lastUniqueElements to android.LastUniqueStrings, and lastUniquePaths
to android.LastUniquePaths.
Test: m checkbuild
Change-Id: Ieac840405126c7f8f98afb4a4ef35c01a18fe7fb
Make Rel() on ModuleOutPath and ModuleGenPath return the path
relative to the module out and module gen directories respectively,
and make TestingModule.Output() match against the full relative
path to the module.
Test: java_test.go still passes
Change-Id: Id5b2ec3fdef41d4169b943e68d032fc64a2b6f92
This uses knowledge of transitive dependencies to reorder
linker command line arguments such that if module A depends
on module B, then module A is automatically listed before
module B in the linker command line.
This should mostly remove the need for Android.bp files to
list all of their static dependencies in link order
Bug: 66260943
Test: reorder the entries of static_libs in an Android.bp and see that linking still succeeds
Change-Id: I20f851ab9f2f30031254e4f30023b6140d15d6c3
Allow java libraries to specify .kt sources, precompile them with
kotlin, and then pass them in the classpath to javac.
Bug: 65219535
Test: java_test.go
Change-Id: Ife22b6ef82ced9ec26a9e5392b2dadacbb16546f
Add a properties to allow including files as resources, including
support for filegroups. Also add a flag that causes module sources
to be included in the final jar.
Test: java_test.go TestResources
Change-Id: Ida8ee59b28df9fe66952170f46470d3a09fd5d65
target.linux_glibc will apply to host builds with glibc, which is
identical to the current target.linux. In a future change, target.linux
will change to affect all targets using the Linux kernel (android,
linux_bionic, and linux_glibc).
target.bionic will apply to all OS variants using Bionic.
Bug: 31559095
Test: Add target.linux_glibc, target.bionic sections to an Android.bp, build
Test: m host
Change-Id: I677a67c22fba148fec264132311e355283f9d88d
All the stats below are for a hikey960-eng build on AOSP master.
Number of order-only inputs in out/soong/build.ninja:
Before: 2847162
After: 606508
Size of: Before After
out/soong/build.ninja 572MB 233MB -59%
out/soong/Android-hikey960.mk 15MB 9MB -40%
out/build-hikey960.ninja 367MB 358MB -2%
Ninja time in `m nothing`:
Before: 6.9s
After: 5.0s
Soong generation time:
Before: 11.6s
After: 6.6s
Test: m nothing
Test: wrote script that counted inputs, only duplicates were removed
Test: treehugger
Change-Id: I6b7c5ef5b1395014b7bf5fd0b8112d42bee127bf
These should be install in /data/nativetest* with the rest of the tests,
but had been moved to /vendor/nativetest* accidentally. Add some tests
so that this doesn't happen again.
Bug: 63393698
Test: m -j blueprint_tools
Test: compare out/soong/Android-aosp_arm64.mk
Test: compare out/soong/build.ninja
Change-Id: Id3b08a7e3908955df18a043a02ea576dc88086c3
This CL fixes a bug with how Soong builds the install path for native
tests. Tests are installed in /data/nativetest* by default, but the
logic was wrongly redirecting sanitized versions to
/data/asan/system/nativetest*. With this fix, they are correctly
redirected to /data/asan/data/nativetest* instead.
Bug: 37942061
Test: lunch marlin_asan-userdebug && \
SANITIZE_TARGET="address" m -j70 \
# nativetests are generated in /data/asan/data/nativetest*
Change-Id: I0c132af5d443151c44219d231770049fddf79bbe
We now add export_static_lib_headers, export_generated_headers to the
filters while dumping the abi of a library using header-abi-dumper
(through -I<dir> additions to the invocation of header-abi-dumper and
header-abi-linker)
Also add support for zipped reference source based abi dumps.
Test: mm -j64 in hardware/interfaces/nfc/default/1.0 produces
android.hardware.nfc@1.0.so.lsdump with abi filtered out using generated
headers.
Test: Copied the linked abi dumps produced by mm -j64 in bionic/libc to
prebuilts/abi-dumps/ndk and gzipped them. Ran mm -j64 again in
bionic/libc and verified header-abi-diff getting invoked.
Bug: 32750600
Change-Id: I26210af908c87a6143e39fa25f50307acb68a387
header-abi-dumper: dumps abi exported by source files for Vndk.
header-abi-linker: links abi dumps produced by header-abi-dumper.
header-abi-diff: compares linked dumps.
Test: mm -j64 showcommands > make_log in bionic/libc.
This produced linked dumps in out/soong/.intermediates.
Copied these dumps to
prebuilts/abi-dumps/ndk/current/arm64/source-based/.
Changed the abi and re-ran mm -j64 showcommands > make_log
confirmed that the build reported compatibility breakge without
actually failing (advisory mode).
Change-Id: Iccad6908fe68a80f47230751671d156893b96ead
The vendor image will have more than just proprietary modules in it
under Treble, so let's stop marking open source code as proprietary just
to move it to vendor.
Bug: 36452052
Bug: 37134596
Test: compare build.ninja before/after, no changes.
Test: Set vendor: true, ensure it works.
Change-Id: I44b0ec7007d0e311bdcbd44b238b1ef2d05cc6ff
This CL moves the location of ASAN-ified libraries on disk in the
following manner:
/data/lib* --> /data/asan/system/lib*
/data/vendor/* --> /data/asan/vendor/*
There are a couple of advantages to this, including better isolation
from other components, and more transparent linker renaming and
SELinux policies.
Bug: 36574794
Bug: 36674745
Test: m -j40 && SANITIZE_TARGET="address" m -j40 and the device
boots. All sanitized libraries are correctly located in /data/asan/*.
Change-Id: I06bf459260ec451d4495a02562f640ad622f34c2
Allow tests to specify a data property that lists files or filegroup
modules that will be packaged alongside the test. Also add a path
property to filegroup modules to allow shifting the path of the
packaged files, and add ExpandSourcesSubDir to expand the filegroup
sources while including a shifted relative path in the Paths objects.
Test: soong tests, manually adding data to a module
Change-Id: I52a48942660e12755d313ef13279313361b4fc35
ModuleSrcPath contains an embedded basePath as well as a
SoucePath that contains another basePath. Remove the embedded
basePath, and make the SourcePath embedded.
Test: no change to build.ninja
Change-Id: I3cdf3477eca41ed35fac08a892aab22cbcdb2224
Move Soong's globbing-with-dependencies support into Blueprint so it can
be used for subdirs= lines in Android.bp files.
Blueprint has a slight change in behavior around subname= lines, it now
always uses the subname and doesn't fall back to Blueprints. To support
the Blueprints files in build/blueprint, use them directly with build=.
Test: build, add source file that matches glob, rebuild
Change-Id: Ifd0b0d3bc061aae0a16d6c7ca9a1cd8672656b4d
We were emulating this for proto files, standardize it and make the
other generators use it as well.
Test: Compare out/soong/build.ninja before/after change
Test: mmma -j system/tools/hidl
Change-Id: I1888c7b981749060a398387bbb9b481270bf6d75
Soong's multi-architecture building has grown complex, with the
combination of HostOrDevice+HostType+Arch necessary to determine how to
build a variant of a module, and three separate mutators to split each
into its variations.
Combine HostOrDevice+HostType into Os, which will be Linux, Darwin,
Windows, or Android. Store Os+Arch as a single Target.
Change-Id: I92f2e2dac53617d595a35cc285d2bd348baa0fbd
Soong's multi-architecture building has grown complex, with the
combination of HostOrDevice+HostType+Arch necessary to determine how to
build a variant of a module, and three separate mutators to split each
into its variations.
Combine HostOrDevice+HostType into Os, which will be Linux, Darwin,
Windows, or Android. Store Os+Arch as a single Target.
Change-Id: Iae677eff61a851b65a7192a47f2dc17c1abb4160
Colin Cross