This adds the build infrastructure for checking DT binding schema
documents and validating dts files using the binding schema.
Check DT binding schema documents:
make dt_binding_check
Build dts files and check using DT binding schema:
make dtbs_check
Optionally, DT_SCHEMA_FILES can be passed in with a schema file(s) to
use for validation. This makes it easier to find and fix errors
generated by a specific schema.
Currently, the validation targets are separate from a normal build to
avoid a hard dependency on the external DT schema project and because
there are lots of warnings generated.
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mark Rutland <mark.rutland@arm.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: linux-doc@vger.kernel.org
Cc: devicetree@vger.kernel.org
Cc: linux-kbuild@vger.kernel.org
Signed-off-by: Rob Herring <robh@kernel.org>
There is nothing arch specific about building dtb files other than their
location under /arch/*/boot/dts/. Keeping each arch aligned is a pain.
The dependencies and supported targets are all slightly different.
Also, a cross-compiler for each arch is needed, but really the host
compiler preprocessor is perfectly fine for building dtbs. Move the
build rules to a common location and remove the arch specific ones. This
is done in a single step to avoid warnings about overriding rules.
The build dependencies had been a mixture of 'scripts' and/or 'prepare'.
These pull in several dependencies some of which need a target compiler
(specifically devicetable-offsets.h) and aren't needed to build dtbs.
All that is really needed is dtc, so adjust the dependencies to only be
dtc.
This change enables support 'dtbs_install' on some arches which were
missing the target.
Acked-by: Will Deacon <will.deacon@arm.com>
Acked-by: Paul Burton <paul.burton@mips.com>
Acked-by: Ley Foon Tan <ley.foon.tan@intel.com>
Acked-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Michal Marek <michal.lkml@markovi.net>
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: James Hogan <jhogan@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Chris Zankel <chris@zankel.net>
Cc: Max Filippov <jcmvbkbc@gmail.com>
Cc: linux-kbuild@vger.kernel.org
Cc: linux-snps-arc@lists.infradead.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: uclinux-h8-devel@lists.sourceforge.jp
Cc: linux-mips@linux-mips.org
Cc: nios2-dev@lists.rocketboards.org
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-xtensa@linux-xtensa.org
Signed-off-by: Rob Herring <robh@kernel.org>
Major changes are I2C and SPI bus checks, YAML output format (for
future validation), some new libfdt functions, and more libfdt
validation of dtbs.
The YAML addition adds an optional dependency on libyaml. pkg-config is
used to test for it and pkg-config became a kconfig dependency in 4.18.
This adds the following commits from upstream:
c86da84d30e4 Add support for YAML encoded output
361b5e7d8067 Make type_marker_length helper public
bfbfab047e45 pylibfdt: Add a means to add and delete notes
9005f4108e7c pylibfdt: Allow delprop() to return errors
b94c056b137e Make valgrind optional
fd06c54d4711 tests: Better testing of dtc -I fs mode
c3f50c9a86d9 tests: Allow dtbs_equal_unordered to ignore mem reserves
0ac9fdee37c7 dtc: trivial '-I fs -O dts' test
0fd1c8c783f3 pylibfdt: fdt_get_mem_rsv returns 2 uint64_t values
04853cad18f4 pylibfdt: Don't incorrectly / unnecessarily override uint64_t typemap
9619c8619c37 Kill bogus TYPE_BLOB marker type
ac68ff92ae20 parser: add TYPE_STRING marker to path references
90a190eb04d9 checks: add SPI bus checks
53a1bd546905 checks: add I2C bus checks
88f18909db73 dtc: Bump version to v1.4.7
85bce8b2f06d tests: Correction to vg_prepare_blob()
57f7f9e7bc7c tests: Don't call memcmp() with NULL arguments
c12b2b0c20eb libfdt: fdt_address_cells() and fdt_size_cells()
3fe0eeda0b7f livetree: Set phandle properties type to uint32
853649acceba pylibfdt: Support the sequential-write interface
9b0e4fe26093 tests: Improve fdt_resize() tests
1087504bb3e8 libfdt: Add necessary header padding in fdt_create()
c72fa777e613 libfdt: Copy the struct region in fdt_resize()
32b9c6130762 Preserve datatype markers when emitting dts format
6dcb8ba408ec libfdt: Add helpers for accessing unaligned words
42607f21d43e tests: Fix incorrect check name 'prop_name_chars'
9d78c33bf8a1 tests: fix grep for checks error messages
b770f3d1c13f pylibfdt: Support setting the name of a node
2f0d07e678e0 pylibfdt: Add functions to set and get properties as strings
354d3dc55939 pylibfdt: Update the bytearray size with pack()
3c374d46acce pylibfdt: Allow reading integer values from properties
49d32ce40bb4 pylibfdt: Use an unsigned type for fdt32_t
481246a0c13a pylibfdt: Avoid accessing the internal _fdt member in tests
9aafa33d99ed pylibfdt: Add functions to update properties
5a598671fdbf pylibfdt: Support device-tree creation/expansion
483e170625e1 pylibfdt: Add support for reading the memory reserve map
29bb05aa4200 pylibfdt: Add support for the rest of the header functions
582a7159a5d0 pylibfdt: Add support for fdt_next_node()
f0f8c9169819 pylibfdt: Reorder functions to match libfdt.h
64a69d123935 pylibfdt: Return string instead of bytearray from getprop()
4d09a83420df fdtput: Add documentation
e617cbe1bd67 fdtget: Add documentation
180a93924014 Use <inttypes.h> format specifiers in a bunch of places we should
b9af3b396576 scripts/dtc: Fixed format mismatch in fprintf
4b8fcc3d015c libfdt: Add fdt_check_full() function
c14223fb2292 tests: Use valgrind client requests for better checking
5b67d2b955a3 tests: Better handling of valgrind errors saving blobs
e2556aaeb506 tests: Remove unused #define
fb9c6abddaa8 Use size_t for blob lengths in utilfdt_read*
0112fda03bf6 libfdt: Add fdt_header_size()
6473a21d8bfe Consolidate utilfdt_read_len() variants
d5db5382c5e5 libfdt: Safer access to memory reservations
719d582e98ec libfdt: Propagate name errors in fdt_getprop_by_offset()
70166d62a27f libfdt: Safer access to strings section
eb890c0f77dc libfdt: Make fdt_check_header() more thorough
899d6fad93f3 libfdt: Improve sequential write state checking
04b5b4062ccd libfdt: Clean up header checking functions
44d3efedc816 Preserve datatype information when parsing dts
f0be81bd8de0 Make Property a subclass of bytearray
24b1f3f064d4 pylibfdt: Add a method to access the device tree directly
Signed-off-by: Rob Herring <robh@kernel.org>
It is tedious to specify extra compiler options for every file.
HOST_EXTRACFLAGS is useful to add options to all files in a
directory.
-I$(src)/libfdt is needed for all the files in this directory
to include libfdt_env.h etc. from scripts/dtc/libfdt/.
On the other hand, -I$(src) is used to include check-in headers
from generated C files. Thus, I added it only to dtc-lexer.lex.o
and dtc-parser.tab.o .
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Files generated by if_changed* must be added to 'targets' to include
*.cmd files. Otherwise, they would be regenerated every time.
The build system automatically adds objects to 'targets' where
appropriate, such as obj-y, extra-y, etc. but does nothing for
intermediate files. So, each Makefile needs to add them by itself.
There are some common cases where objects are generated by chained
rules. Lexers and parsers are compiled like follows:
%.lex.o <- %.lex.c <- %.l
%.tab.o <- %.tab.c <- %.y
They are common patterns, so it is reasonable to take care of them
in the core Makefile instead of requiring each Makefile to do so.
At this moment, you cannot delete 'target += zconf.lex.c' in the
Kconfig Makefile because zconf.lex.c is included from zconf.tab.c
instead of being compiled separately. It should be deleted after
Kconfig is more refactored.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>
Files suffixed by .lex.c, .tab.[ch] are generated lexers, parsers,
respectively. Clean them up globally from the top Makefile.
Some of the final host programs those lexer/parser are linked into
are necessary for building external modules, but the intermediates
are unneeded. They can be cleaned away by 'make clean' instead of
'make mrproper'.
Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: Frank Rowand <frowand.list@gmail.com>
Now that the kernel build supports flex and bison, remove the _shipped
files and generate them during the build instead.
Based on Masahiro's original patch.
Reviewed-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: Rob Herring <robh@kernel.org>
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Fix "make distclean" to clean up generated dtc files.
Without this patch the following files are left around:
- dtc-lexer.lex.c
- dtc-parser.tab.c
- dtc-parser.tab.h
Signed-off-by: Magnus Damm <damm@opensource.se>
Reviewed-by: Simon Horman <horms@verge.net.au>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
Pull in recent changes from the main dtc repository. These changes
primarily allow multiple device trees to be declared which are merged
by dtc. This feature allows us to include a basic dts file and then
provide more information for the specific system through the merging
functionality.
Changes pulled from git://git.jdl.com/software/dtc.git
commit id: 37c0b6a0, "dtc: Add code to make diffing trees easier"
Signed-off-by: John Bonesio <bones@secretlab.ca>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>
The powerpc kernel always requires an Open Firmware like device tree
to supply device information. On systems without OF, this comes from
a flattened device tree blob. This blob is usually generated by dtc,
a tool which compiles a text description of the device tree into the
flattened format used by the kernel. Sometimes, the bootwrapper makes
small changes to the pre-compiled device tree blob (e.g. filling in
the size of RAM). To do this it uses the libfdt library.
Because these are only used on powerpc, the code for both these tools
is included under arch/powerpc/boot (these were imported and are
periodically updated from the upstream dtc tree).
However, the microblaze architecture, currently being prepared for
merging to mainline also uses dtc to produce device tree blobs. A few
other archs have also mentioned some interest in using dtc.
Therefore, this patch moves dtc and libfdt from arch/powerpc into
scripts, where it can be used by any architecture.
The vast bulk of this patch is a literal move, the rest is adjusting
the various Makefiles to use dtc and libfdt correctly from their new
locations.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Rob Herring