Commit Graph

300 Commits

Author SHA1 Message Date
Colin Cross 72ea641dff Merge changes Ia17b2bcb,Ic71892c3,Id2b23b9e,I435ee7aa,I85112506, ...
* changes:
  Consolidate *MutatorContext and ModuleContext into BaseModuleContext
  Add test for missing defaults modules with AllowMissingDependencies
  Capture missing dependency error rules
  Share buildDir for android/soong/android tests
  Reimplement AddMissingDependencies
  Add GenerateAndroidBuildActions to DefaultsModuleBase
2019-06-12 20:19:55 +00:00
Sasha Smundak 26c6d9e9fb Tune JVM, print full diagnostics on crash.
JVM occasionally (0.5% time) crashes on entering a method with SIGSEGV.
It might be due to excessive load on the host machine. Reduce the number
of the compiler threads to 6, and do not start all the GC threads
immediately. If JVM crashes, copy full diagnostics to stderr.

Bug: 132766811
Test: treehugger
Change-Id: Ib7bfd568f6d32456bf623cbb615ffc990c326a34
2019-06-11 13:30:13 -07:00
Colin Cross 59037628f4 Add GenerateAndroidBuildActions to DefaultsModuleBase
Add an empty GenerateAndroidBuildActiosn to DefaultsModuleBase
so that every defaults module doesn't need to provide one.  This
will also allow adding an implementation in the next patch.

Test: m checkbuild
Change-Id: I13554bdb3a287c2f18e1efab74d4f08a1ba8620c
2019-06-10 13:51:17 -07:00
Pete Gillin 2b1ea2edb3 Don't specify -encoding twice in Jdiff javadoc invocations.
The option '-encoding UTF-8' is included in the opts defined at L1754,
which are passed to transformJdiff (defined at L1594), and from there
to a javadoc rule (defined at L29), where they are included in a
command line string which already has '-encoding UTF-8' in it. The
result is that this appears twice in the command line. This is
apparently ignored by the version of javadoc currently used. Later
upstream versions of javadoc, however, fail with the error 'javadoc:
error - The -encoding option may be specified no more than once.'

Test: make system-api-stubs-docs
Test: make system-api-stubs-docs, using an OpenJDK 11 toolchain via OVERRIDE_ANDROID_JAVA_HOME
Bug: 131683177
Change-Id: Ib9382ada3c8259edfacb2be3ab14d4c09d8753a7
2019-06-10 14:37:41 +01:00
Colin Cross 41955e8895 Support tagged module references
There are cases where a module needs to refer to an intermediate
output of another module instead of its final output.  For example,
a module may want to use the .jar containing .class files from
another module whose final output is a .jar containing classes.dex
files.  Support a new ":module{.tag}" format in any property that
is annotated with `android:"path"`, which will query the target
module for its ".tag" output(s).

Test: path_properties_test.go, paths_test.go
Test: no unexpected changes in build.ninja
Change-Id: Icd3c9b0d83ff125771767c04046fcffb9fc3f65a
2019-06-04 10:22:51 -07:00
Treehugger Robot d5dd534465 Merge "Revert "Print full diagnostics file when Java process crashes."" 2019-05-31 09:37:36 +00:00
Alexander Smundak d71dc6b5e0 Revert "Print full diagnostics file when Java process crashes."
This reverts commit 9d6df2246c.

Reason for revert: This change set the value of JAVA make variable to a string containing "foo bar". When soong_ui is asked to evaluate such string, it drops the quotes, causing bash to treat 'foo' and 'bar' as separate command line arguments.

Change-Id: If4e3c4c950c305bb67a572de3d86119c2da81dc1
2019-05-31 03:15:55 +00:00
Treehugger Robot 7b1ca80579 Merge "Print full diagnostics file when Java process crashes." 2019-05-30 21:16:52 +00:00
Sasha Smundak 9d6df2246c Print full diagnostics file when Java process crashes.
Bug: 132766811
Test: treehugger
Change-Id: I8cca2486f2194a8643007a53e4a695d2a18abbf8
2019-05-29 17:46:22 -07:00
Colin Cross e4a0584f60 Document escaping
Document that strings can contain double quotes escaped as \", and
the droiddoc args argument can contain a dollar signe escaped as $$.

Test: none
Change-Id: I91d3fa73f484094c9ce8cbecc93189850856de84
2019-05-28 10:19:31 -07:00
Colin Cross 3047fa23da Build framework.aidl in Soong
Move the rules to build framework.aidl into Soong, and use it
when compiling aidl files with sdk_version: "current".

Also fixes incorrectly using the aidl includes exported by the
"framework" module when the proguardRaiseDep dependency was
added.

Bug: 130798034
Test: sdk_test.go
Change-Id: I126adf1d9e7b6acb528875ff62b974ba7ad9a337
2019-04-18 13:51:58 -07:00
Colin Cross 8a49795df1 Replace ctx.ExpandSources with android.PathsForModuleSrc
Move the logic from ctx.ExpandSources into android.PathsForModuleSrc
and ctx.ExpandSource into android.PathForModuleSrc, and deprecate
them.  When combined with the pathDepsMutator this will let all
properties that take source paths also take filegroups or genrule
outputs, as long as they are tagged with `android:"path"`.

Test: All soong tests
Change-Id: I01625e76b5da19240e9649bf26a014eeeafcab8f
2019-03-20 19:36:13 +00:00
Colin Cross 07e51619a2 Remove ModuleSrcPath
ModuleSrcPath was designed as a type that ensured that modules only
referenced sources inside the directory that contained the Android.bp
file.  In practice they don't work very well, because allowing
filegroups and genrules as inputs to any module that takes a source
path means that the path might end up being to a file in another
source directory or to a generated file in the output directory.

Remove ModuleSrcPath, replacing it with SourcePath in the places
that need to explicitly refer to a path in the source tree, or
Path where it may be a source path or a generated path.

Make PathForModuleSrc return a Path instead of a SourcePath in
preparation for consolidation with ctx.ExpandSources, which will
make it possibly return paths to generated files.

Test: All soong tests
Change-Id: I973a78470ed14307eea5f6d0cc93942775a65715
2019-03-20 19:36:03 +00:00
Colin Cross 27b922f53e Annotate paths and deprecate ExtractSource(s)Deps
Add `android:"path"` to all properties that take paths to source
files, and remove the calls to ExtractSource(s)Deps, the
pathsDepsMutator will add the necessary SourceDepTag dependency.

Test: All soong tests
Change-Id: I488ba1a5d680aaa50b04fc38acf693e23c6d4d6d
2019-03-07 18:36:35 +00:00
Inseob Kim 38449af64f Ignore missing prebuilt_apis of java_sdk_library
Building java_sdk_library without defining prebuilt_apis has been
failing with weird error messages. So one have to touch empty txt files
and create prebuilt_apis module, even when the java_sdk_library is
brand-new and has no previous versions. This commit fixes it so that API
check against previous version is skipped, in the case of missing
prebuilt_apis.

Note that Current api txt files (placed under api/ directory) are still
needed (make update-api)

Bug: 126259114
Test: tries to build without touching empty api txt files.
Change-Id: I93630f4139cbf502621693ec315dc06c0d07d1c3
2019-03-04 10:56:53 +09:00
Paul Duffin 2fbbfb8630 Remove reference to legacy-test and core-junit from comments
The legacy-test and core-junit are no longer used by the runtime,
core-junit was removed a long time ago, legacy-test will be removed
soon.

Bug: 63127064
Test: TH
Change-Id: I094e7263afcba76b40361c3a681c9fd7a4314133
2019-02-14 10:53:59 +00:00
Inseob Kim c0907f191a Create sysprop_library soong module
A newly introduced sysprop_library soong module will generate a
java_sdk_library and a cc_library from .sysprop description files.
Both Java modules and C++ modules can link against sysprop_library
module, thus giving consistency for using generated sysprop API.

As Java controls accessibility of Internal / System properties with
@hide and @SystemApi, 2 different header files will be created. And
build system will selectively expose depending on the property owner
and the place where the client libraries go into.

Bug: 80125326
Bug: 122170616
Test: 1) Create sysprop_library module.
Test: 2) Create empty txt files under prebuilts/sdk.
Test: 3) Create api directory, make update-api, and see changes.
Test: 4) Try to link against sysprop_library with various clients.
Test: 5) Soc_specific, Device_specific, Product_specific, recovery flags
work as intended.
Change-Id: I78dc5780ccfbb4b69e5c61dec26b94e92d43c333
2019-02-13 23:32:51 +00:00
Paul Duffin 99e4a50202 Fix droiddoc handling of $(location ...)
Bug: 124218911
Test: make checkbuild
Change-Id: I4d9c52b40ce4e8b9ae07f557e079e4f101fbbe87
2019-02-13 10:11:54 +00:00
Colin Cross 897d2ed92f Make java_sdk_library implement Dependency
dexpreopting boot jars is going to need to visit both java_library
and java_sdk_library modules.  Since java_sdk_library is already
a java_library module, move the SdkLibraryDependency methods out
of the way so that it will implement Dependency.  Also requries
re-ording some of the type switches to check for the more-specific
SdkLibraryDependency first.

Test: TestJavaSdkLibrary
Change-Id: I155c9ffaf31689dd150a4d99e07e432ff770b4a7
2019-02-11 15:29:51 -08:00
Colin Cross 5f692ec219 Remove empty DepsMutator methods
Add an empty DepsMutator to ModuleBase so it doesn't have to be
implemented on every module that doesn't need it.

Test: all soong tests
Change-Id: I545a832a0dbf27386d3080377a75ea482cd9ce59
2019-02-01 17:17:58 -08:00
Colin Cross 44c29a8fe1 Clean up srcjar directories after each rule
The srcjar directories just confuse people looking for generated
files, clean them up after each rule.

Test: m checkbuild
Change-Id: If712ce7d4922619fd8420ae8f89830fe7824114d
2019-01-25 00:53:05 +00:00
Colin Cross 39c1679e69 Fix conflicting srcjar dir for current and last apicheck
Each rule needs its own srcJarDir.

Bug: 123222452
Test: m checkbuild
Change-Id: I23c62b75c7532eca9320b12235a102141d6eeefa
2019-01-25 00:52:50 +00:00
Tor Norbye ebcdfed262 122452571: Fix path to android.jar in prebuilts
The --android-jar-pattern flag passed to metalava is used to
find all the prebuilts for previous API levels such that
it can compute the exact API level for each class and member.

At some point the prebuilts must have been moved without this
code getting updated.

This fixes
Bug: 122452571: Can't build app with Q prebuilt SDK with minSdkVersion=21
Test: make sdk; inspect api-versions.xml

Change-Id: I25f6203725391dc5404cb8b6049479c9c3da1699
2019-01-24 11:09:16 -08:00
Tor Norbye 76c875a796 Update signature files to the new format
The new format is described in tools/metalava/FORMAT.md.

Fixes: 116589402 Switch signature files over to the v2 format
Fixes: 112626813 Drop "synchronized" modifier from the signature format
Fixes: 122358225 Omit overriding methods that only differ in final [...]
Fixes: 73088715 API Review: Need doclava to output nullability
Fixes: 79739773 API Review: Metalava enforcing constants are constant
Exempt-From-Owner-Approval: Large-scale tooling change
Test: make sdk, make update-api, make checkapi
Merged-In: I8314f4e7099fa92e4f8ed7d283ccf836cc9a84a0
Change-Id: I828ec5ea2f879ebf8dd4ead651e9c9d2345f52e0
2019-01-19 21:36:07 -08:00
Sundong Ahn 054b19a131 Support core library
To support core library, "Openjdk9", "No_standard_libs" and metalava
properties are added to java_sdk_library.
If core_lib is true, dist paths are changed to
apistubs/core/....
impl library name is changed to {module_name}.jar instead of
{module_name}.impl.jar

Bug: 110404779
Test: m -j
Change-Id: Ieb6248ea714b4260333d8bf61573d4f3413f7f24
Merged-In: Ieb6248ea714b4260333d8bf61573d4f3413f7f24
(cherry picked from commit af4907fed7)
2019-01-16 22:00:17 -08:00
Nan Zhang dee152bfa3 Pass --merge-qualifier-annotations to check-api
As tnorbye@ mentioned, the check-API comptibility needs the annotations
that were merged in so that the behavior will be consistent with
signature file.

Test: check the ninja file.
Bug: N/A
Change-Id: I0c06c838cf7ddfd43f583fc8c1b1ad999a4cc066
2019-01-03 11:28:47 -08:00
Sundong Ahn ba49360c51 Remove dependency on framework.jar
Remove dependency on framework.jar to reduce the build time. So sdk
libraries are not checked API whenever frameowkr.jar changes.

Test: m -j
Bug: 119625999
Change-Id: I7435c429b7857de8c3c1834757c54888091753e5
2018-12-07 02:32:40 +00:00
Nan Zhang 3ffc352545 Change srcjarDir/stubsDirs to a different name.
Otherwise it might have the chance to clobber each other
if the rules ran at the same time.

Test: m -j docs
Bug: b/119671939
Change-Id: Ia57e62f0899e5c61539516833c6e3db88547932e
2018-11-29 10:42:47 -08:00
Pete Gillin c382a5602b Add droidstubs support for --validate-nullability-from-list.
This names an input file for the droidstubs rule, so needs to be
handled in soong.

Test: core-current-stubs-nullability-validation-check-nullability-warnings
Bug: 73448108
Change-Id: I092c89f01c8a448ca486e4b57d43a1c3af133114
2018-11-19 17:49:38 +00:00
Pete Gillin 581d608468 Add check_nullability_warnings support to droidstubs.
This takes the warnings file written by metalava and diffs it against
the expected (checked-in) file, in a manner similar to the checking
and updating of the signature files. This makes it possible for a
developer adding a large API surface (e.g. by moving to a new version
of upstream sources for ojluni) to do so without being blocked on
adding the annotations, while ensuring that at all times there is an
up-to-date record of the API surface which is missing annotations.

Bug: 73448108
Test: See the other change in this topic.
Change-Id: If9ed470ef3355a0d713bb556b5561fb255e4e277
2018-11-01 15:28:04 +00:00
Dan Willemsen 419290aba9 Support setting target_sdk_version separately from sdk_version
Before this change, if targetSdkVersion wasn't set in the
AndroidManifest.xml, we'd set it to the sdk_version from the Android.bp.

But there are cases where you want to compile against a later SDK, but
target an earlier one (especially if you depend on libraries that need
to be compiled against more recent SDKs, like androidx).

Test: build APK with different target_sdk_version.
Change-Id: Iaed36b522955a374a049ef331158cc8fc5798ad2
2018-10-31 21:37:34 -07:00
Pete Gillin a262c05904 Add support for nullability validation to droidstubs.
This detects whether this is enabled, makes sure the other necessary
flag is set to provide a suitable location, and suppresses the
requirement for a previous_api.

Test: making a new libcore target using this
Bug: 73448108
Change-Id: Ifea3a060011a58e3288050c6c5d98b205abe25fc
2018-10-25 19:20:08 +01:00
Neil Fuller b2f14ecfef Add annotation-related args to check api
Without the annotation args the check API fails for
APIs that use visibility annotation args.

Bug: 117936519
Bug: 113148576
Test: build
Change-Id: Ie9d2116c7f255634de72de9a448f5c378c11c457
2018-10-23 20:34:34 +01:00
Sundong Ahn 56dce44269 Add quotes to sourcepath
If Srcs contains only srcjar, sourcepath is null and an error occurs. So
if sourcepath is null, we will use the "-sourcepath "" ".

Test: m -j
Bug: 112397488
Change-Id: I03ac0074fc041203fa1b427d4b4a418af44e85e2
2018-10-08 02:22:08 +00:00
Nan Zhang 40b41b4038 Use SoongJavacWrapper for javadoc
Test: m -j docs; javac_wrapper_test
Bug: b/116874473
Change-Id: I51d440c815d3db1a0a5dc26ad4e376189dc0aefe
2018-10-02 16:45:37 -07:00
Nan Zhang 199645cd76 Add --proguard option to metalava
Also clean-up properties comments

Test: m -j docs
Bug: b/78245848
Change-Id: I43899e9841e0d29e5be7ab2b393cc78186970231
2018-09-25 14:10:00 -07:00
Nan Zhang 5994b620f2 Add no_standard_libs property to droidstubs
mainline-modules need this support to strip default libs.

Test: N/A
Bug: b/78245848
Change-Id: I8605019a55dd8f724c863ddaa705b72f461a7cb2
2018-09-25 17:02:10 +00:00
Pete Gillin 77167904e1 Add droidstubs support for merging show/hide annotations.
This works like the existing annotation merging support, but uses a
different flag, because the merging needs to happen earlier in
metalava (see the other change in this topic).

Bug: 115746226
Test: `make api-stubs-docs`
Test: `make core-platform-api-stubs` with local modifications to trigger this path
Change-Id: I30d6d7993e06b105fb3e9fcefde39c1bf3da998a
2018-09-25 12:37:38 +01:00
Nan Zhang 86b0620dc7 Add comment for naming jdiffdocZip in droiddoc.go
Add comment to provide some cautions in case of we make some changes on
the jdiffdocZip name in the future.

Test: N/A
Bug: b/116221385
Change-Id: I3b612d50465f7feb87bc26ef81e692bf72d6f98f
2018-09-24 12:09:24 -07:00
Nan Zhang 23a1ba687d Fix race condition for jdiff/apicheck
Some build breakages happening on git_master/docs because jdiff rule and
apicheck rule may start in the same time, and jdiff may delete the same
srcjars and out dirs when apicheck trying to read which can cause
apicheck to be failed.

So change the dir names used by jdiff to not mix with dirs used by
apicheck.

Test: m -j docs
Bug: b/114368000
Change-Id: I90a53e0ef5f1ff947db7eb8cc73744919a9b93cb
2018-09-19 11:19:39 -07:00
Nan Zhang 71bbe63526 Add jdiff support for Soong
Test: api-stubs-docs-diff
Bug: b/78245848
Change-Id: I069f6f8b1f9aa74ae1425cd5b792643d05b26820
2018-09-18 14:07:30 -07:00
Nan Zhang 9c69a126d5 Add Api level annotations support for Metalava.
Test: m -j docs with Api level annotations enabled.
Bug: b/78245848

Change-Id: Ife93899cdcf5b54ee0c75003c295aa4b917d3dba
2018-09-12 16:32:01 -07:00
Nan Zhang 1598a9e099 Split doclava run from metalava target.
Originally when metalava sees arg: "--generate-documentation", it will
create a seperate process to invoke javadoc/dokka.

Now this is handled by our build system. Basically metalava target will
only generate a .srcjar file. Any other droiddoc target which generates
docs can depend on this metalava target.

By doing this, if multiple doc targets depending on the same metalava
sources. The metalava won't need to be re-run by multiple times which
could cause quite some timing overhead.

Test: m clean && m -j core-docs and
compare the results using md5sum between the old and new content.
also m clean && m -j core-current-stubs-gen-docs and
compare the results using md5sum  between the old and new content.
Bug: b/78245848

Change-Id: If7deef0da738645efe7d3a8376ff2bb3dec92c01
2018-09-11 15:41:50 -07:00
Colin Cross 42d48b7b8b Enforce dependencies have right architecture
ctx.AddDependency will succeed if the named dependency only has a
single variant, even if that variant is the wrong architecture.
Use ctx.AddVariationDependency(nil, ...) instead, which requires
that all variations of the calling module match the dependency.

Bug: 112707915
Test: no change to out/soong/build.ninja
Test: using a device dependency in a host java module is an error
Change-Id: I70b661a57d4412eb63b8c9841febfb756e9e025d
2018-08-30 16:31:02 +00:00
Colin Cross 86bc9d4622 gofmt -w .
Test: m checkbuild
Change-Id: Ia4aec5ddadf1f1b00f7c567522ca89fd839504f0
2018-08-29 15:37:15 -07:00
Nan Zhang aa43b94107 [java9] Add "--dex-api" option to Metalava.
This allow Metalava to generate public-dex file when user specify
dex_api_filename.

This blocks how we try java9 for droiddoc since "hiddenapi-lists-docs"
targets depends on libcore sources.

Test: m -j hiddenapi-lists-docs
Bug: b/78245848

Change-Id: Ic847c1123b14afb599dbf2a588885fd36e56b2cc
Merged-In: Ic847c1123b14afb599dbf2a588885fd36e56b2cc
2018-08-28 15:52:49 -07:00
Nan Zhang 2760dfc03c Revert "Revert "Make Metalava checkapi a seperate run.""
This reverts commit ec716b4151.

Reason for revert:
This is second attempt for Make Metalava checkapi a seperate run.

The first attept didn't work since Metalava also need sources,
classpath, and etc, not like how old Doclava apicheck works.

Test: cherry-pick go/ag/4832335 in internal master, and do 'm clean && m
-j checkapi', it failed with erros which is expected.
Bug: b/113126618, b/113131828

Change-Id: I664b7838d15615905a2e798b1df832218031dfe7
2018-08-24 11:37:33 -07:00
Nan Zhang ec716b4151 Revert "Make Metalava checkapi a seperate run."
This reverts commit ed0a39dba7.

Reason for revert:
it seems checkapi can be passed successfully even if we didn't do update-api for new changes. roll-back for now and do more investigation.  

Change-Id: Idd54ea8919671d54aa6401233156d6c0bcf5325f
2018-08-24 15:59:17 +00:00
Nan Zhang ed0a39dba7 Make Metalava checkapi a seperate run.
We should not squeeze all the args to one Metalava run, especially for
checkapi.

Bug: b/113126618
Test: m -j checkapi, and check out dir api-stubs-docs.
Change-Id: I8d0c75745c7861b6de9070ef69f8e9284473a5d7
2018-08-23 17:30:38 -07:00
Nan Zhang a05ff57f29 Support API compatibility check in Soong
Test: m -j api-stubs-docs
Bug: b/78245848
Change-Id: Id88a3e3716f9f07f0dc0071aa56a328c4712fcc9
2018-08-22 18:28:58 -07:00
Nan Zhang 66dc236781 Add support for dumping proguard api file.
Test: ref go/ag/4783442
Bug: b/70351683
Change-Id: I0aedd41686535c9a662b935003b80634e56663e0
Merged-In: I0aedd41686535c9a662b935003b80634e56663e0
(cherry-picked from commit: 26c055258b)
2018-08-22 13:59:48 -07:00
Nan Zhang 443fa525df Hardcode "-source 1.8" for Doclava.
Due to various bugs hidden deeply within Doclava, running with
 "-source 1.9" is always failed for it.

So switch to "-source 1.8" for now. This includes the case of running the
combination of Metalava + Doclava. Once we have new documentation tool
in system, we can full deprecate Doclava.

Test: USE_R8=true EXPERIMENTAL_USE_OPENJDK9=true m -j core-docs
Bug: b/70351683
Change-Id: I9553f88d0dd9bc263a249c49075c0b931b5d4927
2018-08-21 11:19:53 -07:00
Nan Zhang d05a436e8b Append previous_api to "--migrate-nullness" option
"--previous-api" option was not recommended after Metalava was
changed on how to consume the api file args.

Test: m -j api-stubs-docs
Bug: b/78245848
Change-Id: I1054173e11b4d47942dda0b95193277758aa8193
2018-08-20 13:28:41 -07:00
Nan Zhang 55e0bf4231 Add update-api support for Metalava
Test: m -j api-stubs-docs-update-current-api
Bug: b/112668343
Change-Id: Ia91ae6969b01c21d88de7ceef4b25a73d0efd4a1
2018-08-17 12:38:15 -07:00
Nan Zhang 86d2d55ade Support Dokka in Soong.
Metalava is supposed to treat all the args after
"--generate-documentation" as either Javadoc or Dokka commands, and it starts
 seperate process to invoke javadoc or java -jar dokka...

Dokka doesn't support --bootclasspath in its args, so treat all the
bootclasspath as classpath.

Also continue to refactor code to seperate Dokka runs from Javadoc or
Metalava.

Test: m -j metalava-dokka-core-docs
Bug: b/72394196
Change-Id: I0f0f3dd80cb2dbb53f19da8fa11ae0b1d92ac5d7
2018-08-10 13:37:24 -07:00
Nan Zhang de860a4c1c Remove additional 'docs' Dir when genearate Soong droiddoc
Test: m -j core-docs
Bug: b/70351683
Change-Id: Icb0b096e24ee8e498532ca32c9e9e74c2803191d
2018-08-09 15:53:33 -07:00
Nan Zhang 77a69ecb70 Use config.DefaultLibraries for droiddoc
Test: m -j ds-static-docs
Bug: b/70351683
Change-Id: I867fb12004e0b6e1bc22f455585386109f0df2d1
2018-08-06 16:58:34 -07:00
Nan Zhang f4936b02b2 Fix the issue that changing merge_annotations_dir doesn't trigger build
We don't have API exported to glob any dir outside of current module.
Any files deps outside of current module should be either a
filegroup/filegroup or a customized module.

We already have similar customized module to track droiddoc-template, so
rename it to be more generic so that it can be used by
merge_annotations_dir also.

Bug: b/111916275, b/70351683
Test: touch manual/android/support/design/widget/annotations.xml, and m
-j metalava-api-stubs-docs, and check the built output srcjar.

Change-Id: I75420ddba69785e46bea75b6dd3f189be7cfe5ad
2018-08-01 15:08:39 -07:00
Nan Zhang a40da04a99 Refactor java/droiddoc.go in Soong.
The GenerateAndroidBuildActions() is too big to be maintained and added-in new
features.

Since the GenerateAndroidBuildActions() is a giant code block having build flags
setting and workflow logic mixed in it. So seperate these two main
stuff to only keep workflow logic in the function, and create other
indipendent flags setting functions to increase the modularity.

Test: m -j api-stubs-docs && m -j metalava-api-stubs-docs, and check
build.ninja file
Bug: b/70351683

Change-Id: I4230d353756aeb4b24640f641b8f8bab6b21204d
2018-08-01 13:18:29 -07:00
Nan Zhang 90fe6746a7 bundle files related to sdk dev tool with generated docs.
bundle files related to sdk dev tool(activity_actions.txt, and etc.) to the
final -docs.zip file since sdk.atree needs to copy these files from
$(OUT_DOCS)/offline-sdk dir.

Test: m -j out/target/common/docs/offline-sdk-timestamp
Bug: b/70351683
Change-Id: I3bfb3c56570dd6728dec272d38c6046e0d44c40f
Merged-In: I3bfb3c56570dd6728dec272d38c6046e0d44c40f
2018-07-26 16:13:23 -07:00
Sundong Ahn 20e998b32c API txt files are dist artifacts
API txt files from a java_sdk_library are automatically registered as
dist artifacts for sdk and win_sdk targets.

They are installed under $(DIST_DIR)/apistubs/<apiscope>/api/<name>.txt
where <apiscope> can be public, system and test.

Bug: 77577799
Test: m -j
Change-Id: I38cd8ee000445ce843ac01ead38001e509228738
2018-07-26 01:40:08 +00:00
Pete Gillin b13a015cee Turn droiddoc's metalava_merge_annotations_dir into a list.
This change replaces droiddoc's metalava_merge_annotations_dir
parameter (which takes a single value) with
metalava_merge_annotations_dirs (which takes a list). This will makes
it possible to merge libcore annotations in from a separate directory
under ojluni, instead of adding them in a preprocessing step.

It is implemented by passing the --merge-annotations argument to
metalava multiple times, which is already supported by metalava's
command line parsing.

Test: `make metalava-api-stubs-docs` with a local change to add a second directory in //frameworks/base/Android.bp
Bug: 111639530
Change-Id: I53d31f1dd45c13405b4511c2b44cbeb7f0e439d1
2018-07-25 11:37:44 +01:00
Nan Zhang 03b733c936 Use Metalava based android.jar for SDK
Define LOCAL_DROIDDOC_ANNOTATIONS_ZIP to export zip file.

Test: m out/target/common/obj/PACKAGING/metalava-api-stubs-docs_annotations.zip
Bug: b/78245848
Change-Id: I66ca2a50adc095cc8a8a1a10620427a91480a9bc
2018-07-24 11:16:56 -07:00
Sundong Ahn 241cd3747c Change classpath for droiddoc
When the module type is SdkLibraryDependency, the classpath has been
gotten from headerJar
Becuase droiddoc uses src files when it builds, we should change to use
ImplementationJars instead of headerJar.

Bug: 77577799
Test: make -j

Change-Id: I1a072be69d7edff5636ea80af700be7796c3b0fc
2018-07-17 23:52:49 +00:00
Nan Zhang e2ba5d4c51 Add more options in droiddoc.go
These new options are required for online/offline-docs. These options
are only for Doclava, not Metalava.

Bug: b/70351683
Test: m -j offline-sdk-docs
Change-Id: Ie229821409d5640adfa6d7d05450e5f4c215b9b4
2018-07-11 15:35:24 -07:00
Mathew Inwood 76c3de1cea Add support for -apiMapping parameter in DocLava.
This is used to output a mapping of class member signature to source
position, which in turn is used for automated addition of annotations in
frameworks code.

Metalava support does not exist (yet), and may not be needed, since the
addition of these annotations in a one-time thing. This change can be
reverted once this is complete.

See go/hidden-api-annotations for more context.

Test: m
Bug: 110868826
Change-Id: I90e409f6ea02b16eb9e03d92382f9bb46a8fdab5
2018-07-10 05:48:49 +00:00
Colin Cross 83bb3167f6 Add support for min_sdk_version
Add min_sdk_version properties and use it for aapt2
--min-sdk-version and --target-sdk-version flags.  Add an sdkContext
interface that any function that needs an sdk version can take in
order to get the values for the current module.

Bug: 110848854
Test: m checkbuild
Change-Id: Ic69f1f935d8b865ec77689350407df08bfac5925
2018-07-09 12:42:01 -07:00
Nan Zhang c94f9d886c Do not pass -system arg to Metalava.
Continue to use -bootclasspath when -source 1.9 is enabled for Metalava
since it doesn't support -system yet.

Test: N/A
Bug: b/110824610
Change-Id: Ie1c047963bd66f1a100255efdcd6ca378cf2e1c4
2018-06-26 12:02:49 -07:00
Nan Zhang 7ca467854e Override javaVersion for Doclava when EXPERIMENTAL_USE_OPENJDK9=true
Test: N/A
Bug: 110786923
Change-Id: I6d1f808e97aad2fe3206f52c8fc93c87b61fdb64
2018-06-25 13:00:25 -07:00
Colin Cross 997262f506 Always use OpenJDK9 for building
Remove support for compiling with javac from OpenJDK8.
We still target 1.8 by default, and OpenJDK8 prebuilts are still
required for the bootclasspath and running robolectric.

Bug: 38418220
Test: m java
Change-Id: I5686deb0ae4f9927192a039d08adc0117b2605dd
2018-06-21 12:25:50 -07:00
Colin Cross a1ce2a0071 Update soong for blueprint change to allow multiple deps
Blueprint allows multiple dependencies on the same module after
https://github.com/google/blueprint/pull/210.

Fix defaults, WalkDeps can now find the same defaults module multiple
times.

Fix droiddoc, if the srcs_lib points to a lib module that is
specified multiple times, for example through explicit properties
and implicit default libraries, the srcs would be listed on the
command line multiple times.  Move srcs_lib to use its own dependency
tag.

Test: m checkbuild
Change-Id: Ia30ce83be1382820d76bca5046ad18cbffe8af1a
2018-06-20 22:56:36 +00:00
Nan Zhang af322cc7c2 Fix the build breakage for droiddoc
Removed the additional "-source $javaVersion" arg in javadoc rule
otherwise javadoc will report error:
javadoc: error - The -source option may be specified no more than once.

Test: m -j ahat-docs
Bug: b/70351683
Change-Id: I7f92236884154075fb4b20cd9a363419b1ecd3f5
2018-06-19 15:23:31 -07:00
Nan Zhang 357466b18a Support Java Lang level 9 for droiddoc in Soong.
Test: m -j docs / m -j metalava-api-stubs-docs
Bug: b/70351683
Change-Id: I7220d04f508f6bb5ffd281cf31bc8c0069c219e5
2018-06-19 10:55:29 -07:00
Nan Zhang 16c0a31b18 Changed droiddoc.go to compile Metalava based android.jar
Test: m -j metalava_android_stubs_current
Bug: b/78245848
Change-Id: Ibce139a93a40e412d8def1b26660446b0a7a24ff
2018-06-15 15:42:59 -07:00
David Brazdil fbe4cc32ed Support new Doclava flag
CL I5f92a54d718749f8ddef9472f3b6387a3debd385 adds a new "-dexApi" flag
to Doclava, which generates the public API as a list of Dex signatures.
Add support for this flag to Soong and expose it in the droiddoc rule.

Bug: 80461842
Bug: 64382372
Test: make out/target/common/obj/PACKAGING/hiddenapi-blacklist.txt
Change-Id: Iadf09f17bb783209a6555ce5289c381bf025a01c
2018-06-07 09:44:31 +00:00
Colin Cross 2d24c1b654 Remove references to SourceDepTag in module types
SourceDepTag is going to become a set of tags of the same type
instead of a single tag, remove references to it outside the
android module.

Bug: 80144045
Test: soong tests
Change-Id: I00b2ea5040e4fc95dfbfdd79e21579853c478fcb
2018-06-06 17:05:20 +00:00
Nan Zhang 79614d1c14 Integrate Metalava to Soong Build
Test: Manually checked-in prebuilt metalava.jar to my local src tree and
      compare the doclava based api file and metalava based api file.
Bug: b/78245848

Change-Id: Ib434f1e15403a78a0ccf89d9aacb70a6f8b05b2b
2018-06-05 14:44:43 -07:00
Nan Zhang 46130975ab Fix droiddoc date error on Darwin
Bug: b/109579193
Test: m -j all-docs
Change-Id: I708f200c8dbd5a135767d0ffb10786f223471850
2018-06-04 15:19:40 -07:00
Colin Cross 86a60ae6fa Add core-lambda-stubs to classpath
Add core-lambda-stubs to the bootclasspath for modules with no
sdk_version or with sdk_version: "current", and to the classpath
for modules that specify a specific sdk_version number.  Use the
prebuilt sdk-core-lambda-stubs when using a prebuilt SDK.

Fixes compiling modules with lambdas against the SDK.

This reapplies I5e700f2dd86f1a6b84b7a55dd9bfe21a448d3fb6 with fixes
and tests for unbundled builds.

Bug: 80428539
Test: java_test.go
Change-Id: Ib7aaeebb93137270d7336a7293a08ed7064a3184
2018-05-30 16:47:00 +00:00
Jiyong Park 1e440683e5 droiddoc accepts aidl files as inputs
droiddoc now accepts aidl files as inputs. This in turn allows us to
feed aidl files to java_sdk_library modules. This is required as some
java_sdk_library internally uses AIDL files and thus we need to specify
*.aidl files in the srcs property. Since the srcs property is internally
given to the droiddoc module as well as the runtime library, droiddoc
should be able to handle aidl files.

Bug: 77575606
Test: java_test.go
Change-Id: If7a8559a2a1d8ac1056b061d24e3a5ee5253453f
2018-05-23 18:42:04 +09:00
Jiyong Park eeb8a6474c Fix some minor bugs for droiddoc
1) typo: check-api -> checkapi
2) newlines in the error messages are correctly emitted

Bug: 77575606
Bug: 78034256
Test: m -j checkapi

Change-Id: Ibeef4d9f4f7f2ca48c29c23b1784efa6aa53a364
2018-05-12 22:22:34 +09:00
Nan Zhang 61819cef14 Apicheck Support in Soong
didn't remove the props: api_filename, removed_api_filename yet since
these two currently are used by java_sdk_library.

Bug: b/78034256
Test: m clean && m checkapi

Change-Id: Iebd014ef227487717b5b3819c80d630c34559983
2018-05-08 16:24:07 -07:00
David Brazdil aac0c3ca2e Add new Doclava flag support to soong
CL I8fe27e8c03334f9c90204db1e69447de65a06a38 adds new "-removedDexApi"
flag to Doclava. This patch adds support for that flag into Soong.
"droiddoc" targets can generate the new API file with property
"removed_dex_api_filename".

Bug: 78182899
Test: make
Change-Id: I04b8f615aa4655167b824d40c3fafa2648beec01
2018-04-25 10:12:40 +00:00
Nan Zhang 30963749fd Fix the undefined CommandDeps for standard Javadoc
When compiling against without customized doclet, Javadoc doesn't need
to depend on Doclava and Jsilver jars. Originally, we leave these two
jars empty in Args so that CommandDeps won't have these two
dependencies. But Ninja doesn't like that with errors:

ninja: error: build.ninja: empty path
because $Jsilver and $Doclava is undefined when compiling Javadoc.

Test: m -j platform_library-docs
Bug: b/70351683
Change-Id: I02fee181490f91e87a1feac74f3a69b5b4acb528
2018-04-24 13:23:04 -07:00
Jiyong Park 82484c0771 Additional fixes for java_sdk_library
This change fixes several problems found while re-defining some libs
using java_sdk_library module type.

1) A phony module whose name is the same as the java_sdk_library name is
defined. It is used to support installing the runtime library via
PRODUCT_PACKAGES (even when there is no dependency from other modules).

2) A bug that package names whose depth is >= 2 is ignored in
droiddoc.srcs_lib_whitelist_pkgs is fixed. A package name foo.bar.baz is
now converted to a path foo/bar/baz following the Java convention.

3) A make variable JAVA_SDK_LIBRARIES that is the list of java_sdk_library
module names is exported to make. This can be used to distinguish SDK
libraries from ordinary libs.

4) When running doclava, some framework sources in the 'android' package
and the same packages as the SDK library are automatically added to the
sources list. This is required for the case when API class is extending
from the framework class. In that case, doclava needs to know whether
the base class is hidde nor not. Since that visibility info is encoded
as @hide string in the comment, we need source files for the classes,
not the compiled ones. Also there are rare cases where part of SDK
library is implemented in the framework (e.g. org.apache.http.legacy).
In that case, we need framework source to make API stubs, though the
sources are not required to build the runtime library.

5) The stubs libs are disabled for PDK builds as well.

Bug: 77575606
Test: m -j

Change-Id: I360cf15fe5339e46644fb16d0dabe2548aa315f9
2018-04-24 09:47:50 +09:00
Jiyong Park c678ad3f71 Adding java_sdk_library
java_sdk_library is to make a Java library that implements optional
SDK APIs to apps. It is actually a wrapper for several modules: 1) stubs
library that clients are linked against to, 2) droiddoc module that
internally generates API stubs source files, 3) the real runtime shared
library that implements the API, and 4) XML file for adding the runtime
lib to the classpath at runtime if requested via <uses-library>.

Note: this is only the initial CL for the feature. Followings are features
currently missing and under development.

1) check for API consistency
2) install stubs libs as the dist artifacts
3) ensuring that apps have appropriate <uses-library> tag
4) disallowing linking to the runtime shared lib
5) HTML generation

Bug: 77575606
Test: m -j

Change-Id: I4a4ccf6e730e041703c63bb275d8860d0de96887
2018-04-19 08:57:30 +09:00
Nan Zhang 853f420ed7 Add bool to control if we need generate stubs src files.
Some modules only need the documentation instead of the stubs.

And fix a small issue for arg name passed to standard javadoc.

Test: m -j dx-docs and check the build.ninja
Bug: b/70351683
Change-Id: Ic47c53db12db6f2632349e48c4a626a4f432f447
2018-04-12 18:05:40 -07:00
Tobias Thierer c9e360ac8a javadoc/droidoc: Don't add .jar files to sourcepath.
javadoc expects directories on the sourcepath, not jar files.
The original commit 88b607994a148f4af5bffee163e39ce8296750c6
in 2009 was already passing the jar files, and every revision
since then seems to have kept this habit through cargo cult.

This CL removes the superfluous jar file arguments from the
sourcepath argument to javadoc.

Test: Treehugger
Bug: 76436487

Change-Id: I3a8503ae089cd6657a698c0552dc194156311849
2018-03-27 00:33:05 +01:00
Nan Zhang 9cbe677024 Fix the issue for incorrect logic of "no_framework_libs"
Fortunately Android stubs jar generation didn't get affected by this
issue since "framework" and "ext" were explicitly added to the "libs"
property:

https://cs.corp.google.com/aosp-master/frameworks/base/Android.bp?rcl=72fa61b3250529504a9f5dab2e927a0d2e873f58&l=876

Test: m -j android_stubs_current and check the build.ninja to see if
addtional "framework" and "lib" are shwon there.
Bug: b/70351683

Change-Id: I9f1eec0d7f8baff6361efef064b7a09a5b2d4a58
2018-03-21 17:56:39 -07:00
Nan Zhang 28c68b9a76 Changes *api_filename properties droiddoc.go
Test: check build.ninja, and compare file content.
Bug: b/70351683
Change-Id: I385100f5c5c5354bec04a6435b735e92834a3512
Merged-In: I385100f5c5c5354bec04a6435b735e92834a3512
2018-03-20 11:40:50 -07:00
Colin Cross 436b76564d Replace extract-srcjars.sh with zipsync tool
extract_srcjars.sh uses zipinfo and unzip, which fail with an
error on an empty zip file.  Instead of trying to hack around
this (which is hard to make guarantees for since they are
non-hermetic host tools), replace them with a go tool to unzip
a set of zip files into a directory.

Bug: 73885582
Test: m checkbuild
Change-Id: I151fed347ed5196726e36866ffc27bc831799afb
2018-03-15 16:42:43 -07:00
Nan Zhang b2b33de3c0 Fetch generated srcs from java libs for droiddoc.
api-stubs, system-api-stubs and etc need generated sources and srcjars from "framework",
so add a property that tell module to fetch srcs and srcjars from its
dependency libraries. The libraries in that property has to be in the
module's classpath.

Also add doc_defaults targets.

Bug: b/70351683
Test: m -j
Change-Id: I05831fbcad488037710950e4f05dc8fb2a12f403
2018-03-09 10:53:42 -08:00
Nan Zhang ccff0f78ba Rename stubsjar to stubsSrcjar for droiddoc
Test: m -j core-docs
Bug: b/70351683
Change-Id: I65f310d68a5a940fe88ccce4a22110548c15c306
2018-03-08 17:29:32 -08:00
Nan Zhang e66c727392 droiddoc supports no_framework_libs.
follow: https://android-review.googlesource.com/c/platform/build/+/618046

Don't build agianst framwork libs when "no_framwork_libs" set to true.

Test: m -j
Bug: b/70351683
Change-Id: If6d37fac20c2b401aa30c31c14f1463300623a0e
2018-03-06 12:59:27 -08:00
Dan Willemsen cc09097359 Add droiddoc_template
We prefer not to use absolute paths in modules, but to reference modules
that have associated paths. This a few benefits:

* it's easier to move a module than to update all the references
* if the module doesn't exist, we treat it as a normal missing
dependency, not having to deal with the missing dependency in path.go
* implementing visibility(etc) in the future would be simpler if there
was a module attached to the reference, so we don't have to do various
path-based lookups to try and match things up.

So define a `droiddoc_template` module, which takes a path, and will run
the glob once in that module. All of the `droiddoc` modules can then
specify it through the `custom_template` property, which will pull the
necessary data.

Also fix that htmldirs should be references from the local path, the
htmldir2 argument never being specified, and complain if more than two
htmldirs are specified, or if the custom template isn't specified.

Test: m core-docs
Test: out/soong/build.ninja is nearly identical
      - line numbers in comments
      - adds directories to droiddoc template dependency lists, which
        is more correct, since we need to rerun on added or removed
	files too.
Change-Id: Iff630bddb3818b8eeed439de7e41fc7fbe7cdcb0
2018-03-01 02:10:51 +00:00
Nan Zhang 3bb14200e2 Change name of droiddoc target output to *-stubs.srcjar
Test: m -j core-docs
Bug: b/70351683
Change-Id: If8c00a9992d262b7cd4c95794073591dcc245bd2
2018-02-28 16:15:05 -08:00
Colin Cross 480cd76672 Use PathForSource instead of PathsForSource
This reapplies If1690a708393964d3030cb908beaf7b6897c0084.

PathForSource does the AllowMissingDependencies check now, use it
instead of PathsForSource.

Test: m checkbuild
Change-Id: I84868c92ee43779f9b5c8b285dac92f5f87b018c
2018-02-23 11:31:45 -08:00
Colin Cross 5a49e1c65e Revert "Use PathForSource instead of PathsForSource"
This reverts commit 9d37831dd3.

Reason for revert: Broke builds with ALLOW_MISSING_DEPENDENCIES=true

Change-Id: Ibff79642f747b83364c83d6b298334121028e02d
2018-02-23 07:08:43 +00:00
Colin Cross 9d37831dd3 Use PathForSource instead of PathsForSource
PathForSource does the AllowMissingDependencies check now, use it
instead of PathsForSource.

Test: m checkbuild
Change-Id: If1894fd98d8d757ebc3c1635d5fcea86f81bfc4a
2018-02-22 14:43:36 -08:00
Nan Zhang 581fd21e91 Droiddoc Support in Soong
Support Droiddoc to Soong based on core/droiddoc.mk. The non-std doclet
based droiddoc compilation output is a "real" stubs.jar instead of a
directory of java files and a timestamp file.

The std doclet based javadoc compilation output is a "empty" stubs.jar
instead of a timestamp file.

The stubs.jar will be exported to
out/target/common/obj/JAVA_LIBRARIES/$(LOCAL_MODULE)_intermediates/classes.jar
and out/target/common/docs/$(LOCAL_MODULE)-stubs.jar

A $(LOCAL_MODULE).zip file will be generated also, and is exported to
out/target/common/docs/$(LOCAL_MODULE)-docs.zip if property: installable is not set
to false.

Bug: b/70351683
Test: unittest + convert libcore docs Android.mk to Soong manually.

Change-Id: I1cffddd138a5d9d445f86a3d4a3fd4de88a2bc0f
(cherry picked from commit 78188ec622cb1ee24171455867fc58ffab91562e)
2018-02-22 11:14:13 -08:00