This CL adds a simple python_binary_host to native py_binary converter
for standalone modules that don't depend on libs. It also adds support
for the conditional py2/py3 build target based on the version prop.
Test: milestone-2/demo.sh full
Test: bazel query 'kind("py_binary rule", //bionic/...)'
Test: bazel run //bionic/libc/tools:genfunctosyscallnrs
Test: bazel run //bionic/libc/tools:genseccomp
Test: go tests
Fixes: 182236395
Change-Id: Ibe5ec6cd0dc12a61b3a449a8c723d80b891fae42
This is to enable consolidating test compatibility suite handling in
Soong's AndroidMk files.
Test: go test soong tests
Test: m nothing and compare soong/Android mk files -- only diffs is that
lines matching "^LOCAL_SHARED_LIBRARIES := $" are absent
Change-Id: I4922a1c8b0231cd98ebd2bc7257956eb9c0d240c
Relands Ic22603a5c0718b5a21686672a7471f952b4d1017 with a minor
change to track libc++ dependencies for python hosts and after
a fix to an internal genrule that depended on transitively
installed java libraries (ag/13068670).
Soong currently assumes that installed files should depend on
installed files of all transitive dependencies, which results
in extra installed file dependencies through genrules, static
libs, etc.
Annotate dependency tags for dependencies for which the
installed files are necessary such as shared libraries
and JNI libraries.
This avoids extra installed files, and is also a first step
towards genrules using their own copy of tools instead of
the installed copy.
Bug: 124313442
Test: m checkbuild
Test: java.TestBinary
Test: cc.TestInstallSharedLibs
Test: deptag_test.go
Change-Id: I725871249d561428e6f67bba6a7c65b580012b72
csuite has a python module that wants to embed the outputs of java
modules. This has caused issues with mismatched variants bewteen
the arch-specific python module and the common java modules. Add
a java_data property that is similar to the data property but
uses the common arch variant.
Bug: 173977903
Test: m checkbuild
Change-Id: I0f2f0e4159650cd5d42b510d5177678e7ee91b4d
Use a local variation for python version splits. This causes
dependencies from python modules not to look for variations
that match the version split, which will simplify a python module
depending on a java module (for example for test data) because
Soong won't look for a python version variant of the java module.
Bug: 173977903
Test: go test ./python
Change-Id: Ib034140c478ffbc7467ab830d3cfa5683c31d05c
When multiple os/arch variants are supported, java_test_host could not
find a matching arch due to java having arch:common, whereas native
binaries support a specific architecture. This change adds the property
`data_native_bins` in order to support binaries with the appropriate
os/arch variants.
Test: m FirmwareDtboVerification with data_native_bins
Test: forrest
Bug: 153848038
Change-Id: I45adebff0fde2811d5ef5620c697b97b768c951f
This enables python binaries to be included in data properties with the
format ":module".
Test: m FirmwareDtboVerification with local changes
Bug: 153848038
Change-Id: I2a02dc382fcbb142d8e4fbdbb6ca461986627d25
The arch variants are hardcoded in every module type. Refactor
them out into a Target.Variations() method in preparation for
splitting the arch mutator into two, which will require using
different variations.
Test: m checkbuild
Change-Id: I28ef7cd5168095ac888fe77f04e27f9ad81978c0
Add a proto.plugin property to allow specifying a custom protoc
plugin to generate the code.
Fixes: 70706119
Test: m am StreamingProtoTest
Change-Id: I1ecdd346284b42bbcc8297019d98d2cd564eb94c
Using blueprint.Rule for protoc commands was causing code duplication
because there was no good way to run the same protoc for cc, java and
python but then run custom source packaging steps for java and python.
Move most of the code into a common function that returns a
RuleBuilder, and then let java and python add their own commands at
the end of the rule.
Bug: 70706119
Test: All Soong tests
Test: m checkbuild
Change-Id: Ic692136775d273bcc4f4de99620ab4878667c83a
Move the logic from ctx.ExpandSources into android.PathsForModuleSrc
and ctx.ExpandSource into android.PathForModuleSrc, and deprecate
them. When combined with the pathDepsMutator this will let all
properties that take source paths also take filegroups or genrule
outputs, as long as they are tagged with `android:"path"`.
Test: All soong tests
Change-Id: I01625e76b5da19240e9649bf26a014eeeafcab8f
Add `android:"path"` to all properties that take paths to source
files, and remove the calls to ExtractSource(s)Deps, the
pathsDepsMutator will add the necessary SourceDepTag dependency.
Test: All soong tests
Change-Id: I488ba1a5d680aaa50b04fc38acf693e23c6d4d6d
Mainly so that we can build a `python` prebuilt that acts like the
normal python install, though you could also build different ones with
more packages pre-installed.
Bug: 117811537
Test: move built py2-cmd into prebuilts/build-tools/path/linux-x86/python and build
Change-Id: I21215f6fd3754d89f8c65e1dfeb3f2deea23239f
VisitDepsDepthFirst is almost never correct, as it can't query
dependency tags of multiple dependencies between the same two modules.
Replace VisitDepsDepthFirst with WalkDeps in sanitize.go and
python.go. Also verify the dependency tag before continuing to
recurse to ensure that they don't recurse through genrules and into
unrelated modules.
Test: m checkbuild
Change-Id: I2f7560126f56b51a40ec39dfbdcc18b5891489f7
The problem came from Python libraries doesn't know the information that
we enabled embedded_launcher (only Python binary knows about that). And
we can't simply remove runfiles dir for Python libraries since host
Python mode need this.
Bug: b/80441699
Test: m perf_profo_flames
Change-Id: I73ffc4d7504f95a708ae7cca47bc6c15a673aa31
As I mentioned in b/79751992 #4, when launcher loads __main__ as entry point,
the module dict got set using absolute pkg path which
was runfiles.google.protobuf which caused the type checking failed
(expected: google.protobuf for type()). After then, starting with non entrypoint
python files, the importing module dict would be set with from...import...
So remove dir:runfiles/ from zip section so that during entrypoint Python file
importing the proto files using
from...import..., the type checking can be passed.
Bug: b/79751992
Test: m clean && m out/host/linux-x86/bin/perf_proto_stack;
And execute it:
usage: perf_proto_stack [-h] [--syms SYMS] [--json-out JSON_OUT]
[--print-samples] [--skip-kernel-syms]
[--print-pid-histogram] [--print-sym-histogram]
[--print-dso-histogram] [--parallel PARALLEL]
file [file ...]
perf_proto_stack: error: too few arguments
Change-Id: I95a4a9ee95ce7f90d893ea0529c384d7b944abe4
Python protobuf std libs will be wrapped in final binary/test par file.
Bug: b/70568913
Test: manually create real examples.
Change-Id: I7376ec9175f3e03d1adbd20858a7f74e826387ad
Each Python module will generate a zip file containing source & data
files. The Python binary will collect all its dependencies and use
merge_zips to merge each zip file to create a final .par file.
Test: m -j checkbuild && real examples:
Bug: b/70568913
Change-Id: I9ff232d461d33e1c06026e7dcb5b124bf02c3ce5
This reverts commit 606e9de344.
Reason for revert: <try to fix the broken build yesterday>
Change-Id: I2963b9af63c7c7398159e5e9a1e448266e1c81d5
Test: unittest
This reverts commit 0a32e5936b.
Reason for revert: <libsqlite was missing for darwin_x86_64 before>
Change-Id: I2e13e849a503a705ffad425df292380f2f73954e
Add python_test to generate executables for both host and device side.
And change python_library_host to python_library.
installation for python device test will be under
target/product/generic_arm[64]/data/nativetest[64].
Bug: b/69114465
Test: m clean && m -j
Device Test:
adb root; adb push nan_devicetest /data;
adb shell; ./nan_devicetest
print(sys.path)
['/data/nan_devicetest/runfiles', '/data/nan_devicetest', '/data/nan_devicetest/internal', '/data/nan_devicetest/internal/stdlib']
Change-Id: If8317070a3aa1b6dab2e84b8df2d037f495d7247
Also adds checks that the dependencies are android.Modules and
are not disabled.
Test: m checkbuild
Change-Id: I05e945f38915d49cd3c0ab72a86576949bc7eff2
launcher.
For Python2, we bundle embedded launcher as bootstrapper within every
.par file. This feature is only enabled for linux_x86_64 for now. We
provide a user flag: hermetic_enabled within bp file. By default, Pyhon2
still use classic bootstrapping way to construct .par file and relys on
host interpreter. Once embedded_launcher is enabled, launcher will be
used to bootstrap .par file and execute user program.
For Python3, the launcher will be ready soon, and for now it still relys
on classic bootstrapping.
Test: Real example is used to test.
Bug: b/63018041
Change-Id: I28deba413d8ad3af407595e46f77d663e79a3705
Change module factories from returning a blueprint.Module and a list
of property structs to returning an android.Module, which holds the
list of property structs.
Test: build.ninja identical except for Factory: comment lines
Change-Id: Ica1d823f009db812c518f271a386fbff39c9766f
The base module handles all the common functionalites, such as version
compatibilty check, version variations split, source file format check,
source/data file duplicate check.
The library/binary module focuses on how to generate binary build actions,
such as setting up stub script, zipping, filling in __init__.py in
runfiles dir tree.
Bug: b/31676493
Test: go test under python package
Change-Id: I06608369f350f7195873d459e1c8d1bdb811e77e
Paul Duffin