Commit Graph

43 Commits

Author SHA1 Message Date
Pedro Loureiro 7609182dda make system server modules use filtered lint database
Bug: 188851704
Test: m
Change-Id: Ie8f63286aeeb4cdbec5b9320c2ba41c8e37c7345
2021-07-05 14:01:49 +00:00
Pedro Loureiro f1be9ba5a4 Use trimmed lint database for mainline modules
Lint's NewApi checks currently produce a lot of false positive findings.
The filtered lint database removes information of classes defined by
mainline modules which are the cases that might become a false positive.

This commit updates soong to use this database instead of the normal one
when linting mainline modules.

Test: m lint-check
Fixes: 186478867
Change-Id: Ica646081b9189303c393b36b2f02914d69eee291
Merged-In: Ica646081b9189303c393b36b2f02914d69eee291
2021-06-25 18:09:45 +00:00
Colin Cross fb618c387a Always propagate some environment variables to RBE
Always propagate LANG, LC_MESSAGES and PYTHONDONTWRITEBYTECODE
to RBE to get more consistent behavior between local actions and
RBE.

Bug: 182415460
Bug: 190593001
Test: treehugger
Change-Id: I726e6f02fd3ef77e158baf6fde77ffb7247a1375
Merged-In: I726e6f02fd3ef77e158baf6fde77ffb7247a1375
2021-06-11 12:54:36 +01:00
Jaewoong Jung 476b9d6d17 Strict updatability linting against dependencies.
Propagate strict_updatability_linting to transitive dependencies using a
top-down mutator.

Test: lint_test.go
Bug: 182349282
Change-Id: Ifc9e58f1a597e3c7725ee49b4027afb6f42f45cb
2021-05-10 15:30:00 -07:00
Jaewoong Jung 3c87b1df11 Fix minor issues in updatability lint CLs.
Test: TreeHugger
Bug: 182349282
Change-Id: Ifae282d51b088d0562605b41a09696d75c5138f1
2021-04-22 13:40:41 -07:00
Jaewoong Jung 48de883834 Add lint.strict_updatability_linting
The flag prevents developers from skipping updatability lint checks with
baseline files

Test: lint_test.go & manual
Bug: 182349282
Change-Id: Ia74a2b83c7ef686124128bdf16f7b85a919d9e8d
2021-04-21 16:56:49 -07:00
Jaewoong Jung 79e6f6bfcc Forbid bypassing updatability lint checks.
Test: lint_test.go
Bug: 182349282
Change-Id: Iac7c01493b449c2ddd6df6c68f8a74dfe72dfd7a
2021-04-21 16:56:19 -07:00
Jaewoong Jung 5a4202575e Rename lint-project-xml.py to remove dashes.
So that a future test can import it without a dirty syntax.

Test: m lint-check
Bug: 182349282
Change-Id: I520a5af49543801ab2a8ee888ff235876546dc74
2021-04-20 07:06:03 -07:00
Jaewoong Jung 302c5b8d80 Extract getBaselineFilepath method.
Test: m nothing
Bug: 182349282
Change-Id: Id3ab0f3b7d398af9dcfd66ee3c0bda64d999178d
2021-04-19 08:54:36 -07:00
Pedro Loureiro f4a88b1633 Introduce NewApi lint checks
We are enabling NewApi lint check where the min sdk != compile sdk.
At the same time, we are introducing baseline files for existing
projects that fail this check in order to keep the build running.

At the very least we stop introducing new problems and teams might
realise of risks in their projects they were not aware of.

Bug: 150847901

Test: m lint-check
Change-Id: Icfa5eb98cc6b6708149f0c52fac8fc1440d9c3b0
Merged-In: Icfa5eb98cc6b6708149f0c52fac8fc1440d9c3b0
2021-04-13 15:19:02 +00:00
Pedro Loureiro 9537d594ce Merge "Update soong for new lint version" 2021-04-07 09:30:06 +00:00
Pedro Loureiro 70acc3dc0c Update soong for new lint version
Test: m nothing
Change-Id: Iaa9d37ef73f1fef5f2dbb8a840e774bb86c65bf7
2021-04-06 17:49:19 +00:00
Colin Cross ee4a8b7070 Make lint HTML output deterministic
Remove a date from the lint HTML output to make it deterministic.

Bug: 183509050
Test: m out/soong/.intermediates/packages/apps/DocumentsUI/DocumentsUI/android_common/lint/lint-report.html
Change-Id: I796c0b571780814b657b9427a17c248668053983
2021-04-05 18:38:05 -07:00
Colin Cross 5bedfa2d45 Simplify lint rules using improved RuleBuilder rsp support
With the improved RuleBuilder rsp support a manual resources.list
file is not necessary, use FlagWithRspFileInputList instead.

The switch to RBE support in RuleBuilder in
Iab4e09d961891ef182643583d4d456e413bc5e39 obsoleted tracking remoteInputs
and remoteRSPInputs, remove them.

writeLintProjectXML was written to allow it to be applied to a separate
rule than the one that ran lint, but it is not used that way.  Using
the same rule for both means that manual tracking of the input
dependencies described by the project.xml rule but read by the lint
rule is not necessary, just treat them as inputs to the single rule.

Test: m lint-check
Test: m USE_RBE=true RBE_LINT=true RBE_LINT_EXEC_STRATEGY=remote lint-check
Change-Id: If1827b9dede3ebcd0792b6b4b8114d3199f6570b
2021-03-25 11:06:45 -07:00
Colin Cross 1661aff8be Run lint actions in sbox
Run lint actions in sbox with RuleBuilder.SandboxInputs.  This
copies all input files into the sandbox, which prevents the lint
tool from finding nearby source files that were not presented to it.

Using SandboxInputs requires use of PathForInput or PathForOutput
anywhere a path is used outside of the RuleBuilderCommand methods
that take paths so that they can be translated to the paths that
will be used in the sandbox.

Bug: 181681346
Test: lint_test.go
Test: m lint-check dist
Test: m USE_RBE=true RBE_LINT=true lint-check dist
Test: m USE_RBE=true RBE_LINT=true RBE_LINT_EXEC_STRATEGY=remote lint-check dist
Change-Id: Iab4e09d961891ef182643583d4d456e413bc5e39
2021-03-18 16:17:34 -07:00
Colin Cross 77cdcfdeaf Move android package on top of remotexec
Remove the references to the android package in remotexec so that
the android package can reference the remoteexec package.  This
will allow RuleBuilder to integrate directly with remoteexec.

Bug: 182612695
Test: m checkbuild
Change-Id: I15be5ef126d8aacbd605518638f341daf6f31bb3
2021-03-18 16:17:34 -07:00
Colin Cross 70c4741215 Add explicit rspfile argument to RuleBuilderCommand.FlagWithRspFileInputList
Using $out.rsp as the rsp file adds extra complexity around keeping
the $ unescaped.  Make callers to FlagWithRspFileInputList provide
an explicit path for the rsp file instead.

Bug: 182612695
Test: rule_builder_test.go
Change-Id: I3f531d80c1efa8a9d09aac0a63790c5b11a9f0c6
2021-03-16 16:52:56 -07:00
Colin Cross 9c78cb83d0 Propagate LANG environment variable to lint in RBE
Without LANG, lint's text output is ASCII instead of UTF-8, causing
differences between local and remote execution.

Bug: 181681346
Bug: 182415460
Test: m USE_RBE=true RBE_LINT=true
Change-Id: I0ad54aa731582c9b54abb80f50ba508c75992b91
2021-03-10 18:03:05 -08:00
Colin Cross 9b93af488b Default lint RBE to local exec strategy
Match other tools by defaulting to local exec strategy.

Also use the local absolute path when using the local exec strategy.

Bug: 181681346
Test: m USE_RBE=true RBE_LINT=true
Test: m USE_RBE=true RBE_LINT=true RBE_LINT_EXEC_STRATEGY=remote
Change-Id: I1d6d20ec69663b99d6d9af1d8e5e67b48a5cd050
2021-03-10 18:03:05 -08:00
Colin Cross ad22bc2995 Add comment to lintPaths
Test: none
Change-Id: Ia7db6e27fedd95df2ebfc573e35a44a93acb031b
2021-03-10 10:10:40 -08:00
Colin Cross 31972dc487 Support remoting lint commands with RBE
Bug: 181681346
Bug: 181912787
Test: m USE_RBE=true RBE_LINT=true lint-check
Change-Id: I10596c40dc5e29075ba0cab51ea9a98cc58b3188
2021-03-05 11:13:40 -08:00
Colin Cross 5c113d13eb Remove lint outputs to prevent showing old lint results on error
The lint rules dumped the text output file to stdout on error.  If the
lint binary exited without updating the output file it would show old
results.

Remove the output files before running lint, and only dump the text
output file if it exists.

Bug: 181681346
Test: m lint-check
Change-Id: I4fa962b1212e8715f234912a9a5e049d5c1540e8
2021-03-04 10:01:34 -08:00
Colin Cross 00d93b143f Make common dependencies of lint use restat
Use restat for the api_versions.xml and annotations.zip dependencies
of lint so that frameworks/base changes don't always result in
rerunning lint on every module.

Bug: 181681346
Test: m lint-check
Change-Id: Ic6a540b41cf79b21441311a8baefe528a3d90d8b
2021-03-04 10:00:09 -08:00
Colin Cross 84f3dc1c9d Use repackaged lint binary
Lint references lint-classpath.jar, which does not contain any classes
by has a manifest that points to other jars for the classpath.  This
breaks dependency tracking during the build.  Use a lint tool that
is repackaged into a single jar.

Bug: 181681346
Test: m lint-check
Change-Id: I07d2b7404c18626e03c5af3ef5a75dd7f899cb0e
2021-03-04 09:56:15 -08:00
Pedro Loureiro 8a75058d3f Merge "Add support for lint baseline files" 2021-02-18 15:53:13 +00:00
Pedro Loureiro 5d190cc24e Add support for lint baseline files
Test: m droid
Test: go test ^TestJavaLint # (from soong/build/java)

Change-Id: I249a0a0597b0bf8495460ed283b476ad2eb36edc
2021-02-18 11:15:30 +00:00
Colin Cross dcf71b299c Convert java.Dependency to JavaInfo provider
Export information about java dependencies through a Provider
instead of accessing the module directly.

Test: java_test.go
Test: no changes to build.ninja
Change-Id: Ifc5d566bf6f6ebc0ad399e948effaa1ef6a22876
2021-02-09 15:36:25 -08:00
Colin Cross f1a035e6be Pass pctx and ctx to NewRuleBuilder
Enable the RuleBuilder and RuleBuilderCommand methods to access
the BuilderContext by passing it to NewRuleBuilder instead of
RuleBuilder.Build.

Test: genrule_test.go
Test: rule_builder_test.go
Test: m checkbuild
Change-Id: I63e6597e19167393876dc2259d6f521363b7dabc
2020-12-01 16:22:16 -08:00
Jingwen Chen cda22c9bb9 Refactor 'in_make' to mean Kati is not skipped.
In Nougat and before, Make wrote a marker file to indicate that
soong_build was invoked from Make to change certain behaviors of Soong
at build time.
https://cs.android.com/android/platform/superproject/+/android-7.1.2_r36:build/core/soong.mk;l=70-73;drc=ae18638b0406ad107b0882a02a13cdd8b92f2a4e

Things have changed, and now soong_build is invoked from soong_ui, which
supports a --skip-make configuration flag:
https://cs.android.com/android/platform/superproject/+/master:build/soong/ui/build/build.go;l=31-33;drc=680387bf1d3ce7cbc77f535be7c42cec411b1687

Thus, the various remnants of 'EmbeddedInMake' and 'inMake'
configuration are misleading, since soong_build is no longer invoked
from Make. This CL refactors all instances to actually mean that
Kati is enabled (not skipped with --skip-make), and will run after
soong_build finishes, so Kati-specific behavior like the AndroidMk
singleton should run.

Test: TH presubmit

Change-Id: I576ab8e54f99f5c8ddf9feaf9a828019b279e266
2020-11-23 00:29:18 -05:00
Colin Cross 56a8321c21 Remove global state from apex modules
A global variant was used to store the global mapping between
modules and APEXes.  Replace it with storing pointers to APEX
contents inside each module so that they can query the contents
of any APEXes they belong to.

Bug: 146393795
Test: all Soong tests
Test: single line change to build.ninja host install dependency ordering
Test: no Android-${TARGET_PRODUCT}.mk, make_vars-${TARGET_PRODUCT}.mk or late-${TARGET_PRODUCT}.mk
Change-Id: Id2d7b73ea27f8c3b41d30820bdd86b65c539bfa4
2020-10-06 13:39:57 -07:00
Colin Cross 053fca10c9 Support ninja rsp files in soong_zip
Add a -r argument to soong_zip that reads a list of files from a file
like the -l argument but treats it as a Ninja rsp file with escaping.
Replace the -l arguments in Soong that are using rsp files with -r.

Fixes: 162435077
Test: TestReadRespFile, TestZip
Change-Id: I4605312e99406ab1bd0c37af9c5ad212393f0403
2020-08-19 21:18:56 +00:00
Treehugger Robot fd5978a473 Merge "Increase heap size for lint tool" 2020-08-12 22:07:18 +00:00
Jeongik Cha 816a23a50d Introduce AlwaysUsePrebuiltSdks
Instead of UnbundledBuild, use AlwaysUsePrebuiltSdks
to determine if java modules needs to be built against prebuilt sdks.
And rename UnbundledBuildUsePrebuiltSdks to AlwaysUsePrebuiltSdks to
express its behavior more correctly.(It can be orthgonal to "Unbundled")

Bug: 160390776
Test: TARGET_BUILD_UNBUNDLED_IMAGE=true m vendorimage

Change-Id: I0be7265c1959d8774c295372cd7a9250169f6df9
2020-08-07 12:15:52 +09:00
Liz Kammer 20ebfb46fc Update language to comply with inclusive guidance
See https://source.android.com/setup/contribute/respectful-code for
reference

Bug: 161896447
Test: m nothing
Change-Id: Ie1d0d63d27f1b1b48680922b7202aa6df04f4696
2020-07-28 11:37:31 -07:00
Colin Cross 08dca38eb0 Build transitive lint reports for apex modules
Build and export transitive lint report zips for apex modules.

Bug: 153485543
Test: m TARGET_BUILD_APPS=com.google.android.wifi lint-check dist
Change-Id: I5a1805440452301a7e2c4ca91482b989638b54fb
2020-07-22 20:16:14 -07:00
Colin Cross 988dfcc067 Add environment variables to control lint checks
Set ANDROID_LINT_CHECK to a comma-separated list of lint issues
that should be checked instead of the defaults.  This will disable
all lint checks and enable only the given list.

Set ANDROID_LINT_CHECK_EXTRA_MODULES to a list of modules that provide
lint checks that should added to all modules when ANDROID_LINT_CHECK
is specified.

Bug: 153485543
Test: m ANDROID_LINT_CHECK=JavaKotlinApiUsedByModule ANDROID_LINT_CHECK_EXTRA_MODULES=JavaKotlinApiFinder TARGET_BUILD_APPS=Gallery2 lint-check dist
Change-Id: Ifdf9bf972b8550104315b0f5e98b34ad699dcb67
2020-07-16 17:37:04 -07:00
Colin Cross c0efd1db13 Build a zip of transitive lint reports for apps
Add a rule to build a zip containing the lint reports from transitive
dependencies for apps, and pass it to Make.

Bug: 153485543
Test: m TARGET_BUILD_APPS=Gallery2 lint-check
Change-Id: I523c09016251377ff89d76084769be7401b95425
2020-07-16 12:51:42 -07:00
Colin Cross e22217956f Increase heap size for lint tool
We perodically get java.lang.OutOfMemoryError: Java heap space errors
on the build servers, increase the lint heap size from 2GB to 3GB.

Bug: 161151611
Test: m lint-check
Change-Id: Id6cb97871e01780ac8761647c6d3a504fbfb49f2
2020-07-14 22:41:01 +00:00
Colin Cross 8a6ed3750d Support lint on unbundled builds
Use prebuilts of the annotations.zip and api-versions.xml files
when running lint in an unbundled build.

Test: m TARGET_BUILD_APPS=Gallery2 lint-check
Change-Id: Idacf3758a2769678a635941486183673e95b43f8
2020-07-10 19:30:32 -07:00
Colin Cross c31efeb25c Set root dir in lint project.xml
Set the root dir in the project.xml for lint so that the paths in the
lint report are relative to the top of the tree instead of relative
to the project.xml.

Bug: 153485543
Test: run lint
Change-Id: Ie163a4dadd976e708f798855de73e58084931a91
2020-06-23 10:27:05 -07:00
Colin Cross 977b6a822d Set ANDROID_SDK_HOME when running lint
Lint tries to create ~/.android, set ANDROID_SDK_HOME to keep it
from attempting to write to the home directory, which may not be
writable.

Test: run lint
Bug: 159676171
Change-Id: I16375b88d309a8fa416b3a8efeabe15759889ae3
2020-06-23 10:25:03 -07:00
Colin Cross 92e4b46af5 Support adding extra lint checks
Add a lint.extra_check_modules property to list modules to use as
plugins to Lint.

Bug: 153485543
Test: m checkbuild
Change-Id: I25c7799438cfec43163e757637c65b8657488d36
2020-06-18 15:58:32 -07:00
Colin Cross 014489c1e6 Add support for running Android lint on java and android modules.
Add a rule that runs Android lint on each java and android module
and produces reports in xml, html and text formats.

Bug: 153485543
Test: m out/soong/.intermediates/packages/apps/Settings/Settings-core/android_common/lint-report.html
Change-Id: I5a530975b73ba767fef45b257d4f9ec901a19fcb
2020-06-16 15:44:16 -07:00