The its command buffer must be page aligned, but kzalloc() is not
guaranteed to be (though it is mostly when allocating 64k). Use
__get_free_pages() as this is used for other buffers as well.
Signed-off-by: Robert Richter <rrichter@cavium.com>
[Marc: fixed the error path]
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Back in the days when the GICv3/v4 architecture was drafted,
the command to an event to an LPI number was called MAPVI.
Later on, and to avoid confusion with the GICv4 command VMAPI,
it was renamed MAPTI. We've carried the old name for a long
time, but it gets in the way of people reading the code in
the light of the public architecture specification.
Just repaint all the references and kill the old definition.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
During the development of the GICv3/v4 architecture, it was
envisaged to have a CPU table, though the use for it was
never completely clear (the collection table serves that role
pretty well). It ended being dropped before the specification
was published, though it lived on in the driver.
In order to avoid people scratching their head too much, let's do
the same in the kernel.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The way we encode the various ITS command fields is both tedious
and error prone. Let's introduce a helper function that performs
the encoding, and convert the existing encoders to use that
helper. It also has the advantage of expressing the encoding in
a way that matches the architecture specification.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Read-allocation hints are not enabled for both the GIC-ITS and GICR
tables. This forces the hardware to always read the table contents
from an external memory (DDR) which is slow compared to cache memory.
Most of the tables are often read by hardware. So, it's better to
enable Read-allocate hints in addition to Write-allocate hints in
order to improve the GICR_PEND, GICR_PROP, Collection, Device, and
vCPU tables lookup time.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The GICv3 ITS is MSI remapping capable. Let's advertise
this property so that VFIO passthrough can assess IRQ safety.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Tomasz Nowicki <tomasz.nowicki@caviumnetworks.com>
Tested-by: Tomasz Nowicki <tomasz.nowicki@caviumnetworks.com>
Tested-by: Bharat Bhushan <bharat.bhushan@nxp.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
readq and writeq type of assessors are not supported in AArch32, so we
need to specialise them and glue later with series of 32-bit accesses
on AArch32 side.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
It'd be better to switch to CMA... but before that done redirect
flush_dcache operation, so 32-bit implementation could be wired
latter.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
GITS_BASER<n>'s Entry Size is much smaller than 64-bit, but when it
used as a divider it forces compiler to generate __aeabi_uldivmod if
build in 32-bit mode. So, casting it to int (like it is done in other
places) where used as a divider would give a hint to compiler that
32-bit division can be used.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Make sure that constants which are supposed to be applied on 64-bit
data is actually unsigned long long, so they won't be truncated when
used in 32-bit mode.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The GICv3 architecture specification mentions that a 64bit
register can be accessed using two 32bit accesses. What it
doesn't mention is that this is only guaranteed on a system
that implements AArch32, and a pure AArch64 system is allowed
not to support this. This causes issues with the GICR_TYPER
and GITS_TYPER registers, which are both RO 64bit registers.
In order to solve this, this patch switches the TYPER accesses
to the gic_read_typer macro already used in other parts of the
driver. This makes sure that we always use a 64bit access on
64bit systems, and two 32bit accesses on 32bit system.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Including:
* Support for interrupt virtualization in the AMD IOMMU driver.
These patches were shared with the KVM tree and are already
merged through that tree.
* Generic DT-binding support for the ARM-SMMU driver. With this
the driver now makes use of the generic DMA-API code. This
also required some changes outside of the IOMMU code, but
these are acked by the respective maintainers.
* More cleanups and fixes all over the place.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIcBAABAgAGBQJX/PmUAAoJECvwRC2XARrjyloP/1hymxXC2yXZ4EIBTHSO5X+c
jSJaGTIbQAdQDpllscSNJ0Au43L3vGtJcHo4JqwEERNlwLsU82LH7QJhq+q1La/b
5cPaY5gI3E++qxQt8umuZJAIUQthFYrfGoS5lJc5t5r/d8iVsLWbW4VkR19/1o7A
4/Uz7ETmi9VVy8Hkvumx+PQ0VHJet381KB7ud9LU5Spim0En2AAGwZXLMkmxXd2W
uDQ+O1rlDVc2/ka3+GmfZEml5EASWRqS/MTNoU/ZbQGYWKCWygXbuiqt6gLudWjx
dCR1Knh68b0gN6k/QAj8XY/1gkfmZ3YkfS0AHIMLYTFRT51BuxOrkXrBdkYnWEBv
UirmaiV87SlR1j83yb3ZmjpBPvd2sGWYFDqY1P0riLutjGUS6zycWWs13olvbfbz
SFrH7PT7JPQGYprI1oVn4ihszjN1NZ4+Gj7QBhyFW6FtvqTzmaFVsMOlDIeg1FwR
k8cOzov4NG33Bp4IpsHK8e0/qV6K3oJOiOQgCyQp9kPKK+UWv9v9+HaEA7npJuRV
c+lTE6j3G4LjEoVybkqm8TiPKxTMVNjUjgA3kwB2yNkCQT7hTCNYIAFrtfCYjYdo
B1dnFE7feVqtoimnu2qvkVs59hWlF7Hc3RRHoBxMmO8DLwl9n2OcmoQIeCTsviss
i9aNwC9bzBs+Hd3X/psB
=1hFE
-----END PGP SIGNATURE-----
Merge tag 'iommu-updates-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu
Pull IOMMU updates from Joerg Roedel:
- support for interrupt virtualization in the AMD IOMMU driver. These
patches were shared with the KVM tree and are already merged through
that tree.
- generic DT-binding support for the ARM-SMMU driver. With this the
driver now makes use of the generic DMA-API code. This also required
some changes outside of the IOMMU code, but these are acked by the
respective maintainers.
- more cleanups and fixes all over the place.
* tag 'iommu-updates-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (40 commits)
iommu/amd: No need to wait iommu completion if no dte irq entry change
iommu/amd: Free domain id when free a domain of struct dma_ops_domain
iommu/amd: Use standard bitmap operation to set bitmap
iommu/amd: Clean up the cmpxchg64 invocation
iommu/io-pgtable-arm: Check for v7s-incapable systems
iommu/dma: Avoid PCI host bridge windows
iommu/dma: Add support for mapping MSIs
iommu/arm-smmu: Set domain geometry
iommu/arm-smmu: Wire up generic configuration support
Docs: dt: document ARM SMMU generic binding usage
iommu/arm-smmu: Convert to iommu_fwspec
iommu/arm-smmu: Intelligent SMR allocation
iommu/arm-smmu: Add a stream map entry iterator
iommu/arm-smmu: Streamline SMMU data lookups
iommu/arm-smmu: Refactor mmu-masters handling
iommu/arm-smmu: Keep track of S2CR state
iommu/arm-smmu: Consolidate stream map entry state
iommu/arm-smmu: Handle stream IDs more dynamically
iommu/arm-smmu: Set PRIVCFG in stage 1 STEs
iommu/arm-smmu: Support non-PCI devices with SMMUv3
...
When an MSI doorbell is located downstream of an IOMMU, attaching
devices to a DMA ops domain and switching on translation leads to a rude
shock when their attempt to write to the physical address returned by
the irqchip driver faults (or worse, writes into some already-mapped
buffer) and no interrupt is forthcoming.
Address this by adding a hook for relevant irqchip drivers to call from
their compose_msi_msg() callback, to swizzle the physical address with
an appropriatly-mapped IOVA for any device attached to one of our DMA
ops domains.
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
ITS is prepared for being initialized different than DT,
therefore we can initialize it in ACPI way. We collect register base
address from MADT table and pass mandatory info to firmware-agnostic
ITS init call.
Use here IORT lib to register ITS domain which then can be found and
used on to build another PCI MSI domain in hierarchical stack domain.
NOTE: Waiting for proper ITS and NUMA node relation description in IORT
table, we pass around NUMA_NO_NODE to the its_probe_one init call.
This means that Cavium ThunderX erratum 23144 (pass1.1 only)
is not supported for ACPI boot method yet.
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
In order to add ACPI support we need to isolate ACPI&DT common code and
move DT logic to corresponding functions. To achieve this we are using
firmware agnostic handle which can be unpacked to either DT or ACPI node.
No functional changes other than a very minor one:
1. Terminate its_init call with -ENODEV for non-DT case which allows
to remove hack from its-gic-v3.c.
2. Fix ITS base register address type (from 'unsigned long' to 'phys_addr_t'),
as a bonus we get nice string formatting.
3. Since there is only one of ITS parent domain convert it to static global
variable and drop the parameter from its_probe_one. Users can refer to it
in more convenient way then.
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
There is no point to initialize ITS without having msi-controller
property in corresponding DT node. However, its_probe is checking
msi-controller presence at the end, so we can save our time and do that
check prior to its_probe call. Also, for the code clarity purpose,
we put domain initialization to separate function.
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
When starting a kexec/kdump kernel, the GIC ITS will already have been
enabled. According to the ARM Generic Interrupt Controller
Architecture Specification (GIC architecture Version 3.0 and version
4.0), writing to GITS_BASER<n> or GITS_CBASER is "UNPREDICTABLE" when
the ITS is enabled. On Cavium Thunder systems, this prevents the ITS
from being initializing in the kexec/kdump kernel, resulting in
failure to register/enable interrupts for all devices.
The fix is to disable the ITS if it is not already in the disabled
state. This allows the ITS to be properly initialized and then
re-enabled in the kexec/kdump kernel.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: David Daney <david.daney@cavium.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Since device IDs are extremely sparse, the single, a.k.a flat table is
not sufficient for the following two reasons.
1) According to ARM-GIC spec, ITS hw can access maximum of 256(pages)*
64K(pageszie) bytes. In the best case, it supports upto DEVid=21
sparse with minimum device table entry size 8bytes.
2) The maximum memory size that is possible without memblock depends on
MAX_ORDER. 4MB on 4K page size kernel with default MAX_ORDER, so it
supports DEVid range 19bits.
The two-level device table feature brings us two advantages, the first
is a very high possibility of supporting upto 32bit sparse, and the
second one is the best utilization of memory allocation.
The feature is enabled automatically during driver probe if the memory
requirement is more than 2*ITS-pages and the hardware is capable of
two-level table walk.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
No references to argument 'node_name' after modifying pr_xxx()
messages to include ITS base address instead of 'node_name'.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The function is getting out of control, it has too many goto
statements and would be too complicated for adding a feature
two-level device table. So, it is time for us to cleanup and
move some of the logic to a separate function without affecting
the existing functionality.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Only the device table BASERn needs to be handled differently as
compared to all other tables. So, adding a separate function for
easy code maintenance and improved code readability.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
This patch adds the two handy helper functions for reading and writing
ITS BASERn register.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
[Marc: Folded its_write_baser_cache into its_write_baser]
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The erratum fixes the hang of ITS SYNC command by avoiding inter node
io and collections/cpu mapping on thunderx dual-socket platform.
This fix is only applicable for Cavium's ThunderX dual-socket platform.
Reviewed-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
We are not checking whether the requested device identifier fits into
the device table memory or not. The function its_create_device()
assumes that enough memory has been allocated for whole DevID space
(reported by ITS_TYPER.Devbits) during the ITS probe() and continues
to initialize ITS hardware.
This assumption is not perfect, sometimes we reduce memory size either
because of its size crossing MAX_ORDER-1 or BASERn max size limit. The
MAPD command fails if 'Device ID' is outside of device table range.
Add a simple validation check to avoid MAPD failures since we are
not handling ITS command errors. This change also helps to return an
error -ENOMEM instead of success to caller.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
gicv3_init_bases() is the only caller for its_init(),
also it is a __init function, so mark its_init() as __init too,
then recursively mark the functions called as __init.
This will help to introduce ITS initialization using ACPI tables as
we will use acpi_table_parse_entries family functions there which
belong to __init section as well.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Tomasz Nowicki <tn@semihalf.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The gic_root_node variable defined in ITS driver is not actually
used, so just remove it.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Hanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Function its_alloc_tables() maintains two local variables, "order" and
and "alloc_size", to hold memory size that has been allocated to
ITS_BASEn. We don't always refresh the variable alloc_size whenever
value of the variable order changes, causing the following two
problems.
- Cache flush operation with size more than required.
- Information reported by pr_info is not correct.
Use a helper macro that converts page order to size in bytes instead of
variable "alloc_size" to fix both the problems.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
When the GIC is using EOImode==1, the EOI is done immediately,
leaving the deactivation to be performed when the EOI was
previously done.
Unfortunately, the ITS is not aware of the EOImode at all, and
blindly EOIs the interrupt again. On most systems, this is ignored
(despite being a programming error), but some others do raise a
SError exception as there is no priority drop to perform for this
interrupt.
The fix is to stop trying to be clever, and always call into the
underlying GIC to perform the right access, irrespective of the
more we're in.
[Marc: Reworked commit message]
Fixes: 0b996fd359 ("irqchip/GICv3: Convert to EOImode == 1")
Cc: stable@vger.kernel.org
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ashok Kumar <ashoks@broadcom.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
The current ITS driver has a memory leak in its_free_tables(). It
happens on tear down path of the driver when its_probe() call fails.
its_free_tables() should free the exact number of pages that have
been allocated, not just a single page as current code does.
This patch records the memory size for each ITS_BASERn at the time of
page allocation and uses the same size information when freeing pages
to fix the issue.
Signed-off-by: Shanker Donthineni <shankerd@codeaurora.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Vikram Sethi <vikrams@codeaurora.org>
Cc: linux-arm-kernel@lists.infradead.org
Link: http://lkml.kernel.org/r/1454379584-21772-1-git-send-email-shankerd@codeaurora.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
When the programming of a GITS_BASERn register fails because of
an unsupported ITS page size, we retry it with a smaller page size.
Unfortunately, we don't recompute the number of allocated ITS pages,
indicating the wrong value computed in the original allocation.
A convenient fix is to free the pages we allocated, update the
page size, and restart the allocation. This will ensure that
we always allocate the right amount in the case of a device
table, specially if we have to reduce the allocation order
to stay within the boundaries of the ITS maximum allocation.
Reported-and-tested-by: Ma Jun <majun258@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1453818255-1289-1-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Since we now have a generic data structure to express an
interrupt specifier, convert all hierarchical irqchips that
are OF based to use a fwnode_handle as part of their alloc
and xlate (which becomes translate) callbacks.
As most of these drivers have dependencies (they exchange IRQ
specifiers), change them all in a single, massive patch...
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-and-tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: Tomasz Nowicki <tomasz.nowicki@linaro.org>
Cc: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Graeme Gregory <graeme@xora.org.uk>
Cc: Jake Oshins <jakeo@microsoft.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Link: http://lkml.kernel.org/r/1444737105-31573-6-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The struct irq_domain contains a "struct device_node *" field
(of_node) that is almost the only link between the irqdomain
and the device tree infrastructure.
In order to prepare for the removal of that field, convert all
users to use irq_domain_get_of_node() instead.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-and-tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Tested-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: Tomasz Nowicki <tomasz.nowicki@linaro.org>
Cc: Suravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Graeme Gregory <graeme@xora.org.uk>
Cc: Jake Oshins <jakeo@microsoft.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Link: http://lkml.kernel.org/r/1444737105-31573-2-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
More agressive inlining in recent versions of GCC have uncovered
a new set of warnings:
drivers/irqchip/irq-gic-v3-its.c: In function its_msi_prepare:
drivers/irqchip/irq-gic-v3-its.c:1148:26: warning: lpi_base may be used
uninitialized in this function [-Wmaybe-uninitialized]
dev->event_map.lpi_base = lpi_base;
^
drivers/irqchip/irq-gic-v3-its.c:1116:6: note: lpi_base was declared here
int lpi_base;
^
drivers/irqchip/irq-gic-v3-its.c:1149:25: warning: nr_lpis may be used
uninitialized in this function [-Wmaybe-uninitialized]
dev->event_map.nr_lpis = nr_lpis;
^
drivers/irqchip/irq-gic-v3-its.c:1117:6: note: nr_lpis was declared here
int nr_lpis;
^
The warning is fairly benign (there is no code path that could
actually use uninitialized variables), but let's silence it anyway
by zeroing the variables on the error path.
Reported-by: Alex Shi <alex.shi@linaro.org>
Tested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: David Daney <ddaney.cavm@gmail.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1443800646-8074-2-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
This implements two gicv3-its errata workarounds for ThunderX. Both
with small impact affecting only ITS table allocation.
erratum 22375: only alloc 8MB table size
erratum 24313: ignore memory access type
The fixes are in ITS initialization and basically ignore memory access
type and table size provided by the TYPER and BASER registers.
Signed-off-by: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Reviewed-by: Marc Zygnier <marc.zyngier@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Tirumalesh Chalamarla <tchalamarla@cavium.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Will Deacon <will.deacon@arm.com>
Link: http://lkml.kernel.org/r/1442869119-1814-6-git-send-email-rric@kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Some GIC revisions require an individual configuration to esp. add
workarounds for HW bugs. This patch implements generic code to parse
the hw revision provided by an IIDR register value and runs specific
code if hw matches. A function is added that reads the IIDR registers
for ITS (GITS_IIDR) and then goes through a list of init functions to
be called for specific versions. Same could be done for GICV3
(GICD_IIDR), but there are no users yet for it.
The patch is needed to implement workarounds for HW errata in Cavium's
ThunderX GICV3 ITS.
Signed-off-by: Robert Richter <rrichter@cavium.com>
Reviewed-by: Marc Zygnier <marc.zyngier@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Tirumalesh Chalamarla <tchalamarla@cavium.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1442869119-1814-5-git-send-email-rric@kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
No need to read the typer register in the loop. Values do not change.
This patch is basically a prerequisite for a follow-on patch that adds
errata code for Cavium ThunderX. It moves the calculation of the
number of id entries to the beginning of the function close to other
setup values that are needed to allocate the its table. Now we have a
central location to modify the setup parameters and the errata code
can be implemented in a single block.
Signed-off-by: Robert Richter <rrichter@cavium.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: Tirumalesh Chalamarla <tchalamarla@cavium.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1442869119-1814-4-git-send-email-rric@kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The number of pages for the its table may exceed the maximum of 256.
Adding a range check and limitting the number to its maximum.
Based on a patch from Tirumalesh Chalamarla <tchalamarla@cavium.com>.
Signed-off-by: Tirumalesh Chalamarla <tchalamarla@cavium.com>
Signed-off-by: Robert Richter <rrichter@cavium.com>
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1442869119-1814-2-git-send-email-rric@kernel.org
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
When the ITS is configured for non-cacheable transactions, make sure
that the allocated, zeroed memory is flushed to the Point of
Coherency, allowing the ITS to observe the zeros instead of random
garbage (or even get its own data overwritten by zeros being evicted
from the cache...).
Fixes: 241a386c7d "irqchip: gicv3-its: Use non-cacheable accesses when no shareability"
Reported-and-tested-by: Stuart Yoder <stuart.yoder@freescale.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Pavel Fedin <p.fedin@samsung.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1442142873-20213-3-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
We can now lookup the base ITS domain, making it possible to
initialize the PCI/MSI code independently from the main ITS
subsystem.
This allows us to remove all the previously add hooks.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: Yijing Wang <wangyijing@huawei.com>
Cc: Ma Jun <majun258@huawei.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Duc Dang <dhdang@apm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1438091186-10244-15-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The GICv3 ITS only uses the msi_controller structure as a way
to match the host bridge with its MSI HW, and thus the msi_domain.
But now that we can directly associate an msi_domain with a device,
there is no use keeping this msi_controller around.
Just remove all traces of msi_controller from the driver.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: Yijing Wang <wangyijing@huawei.com>
Cc: Ma Jun <majun258@huawei.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Duc Dang <dhdang@apm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1438091186-10244-14-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Now that we can distinguish between multiple domains carrying the
same device_node, tag the raw ITS domain with DOMAIN_BUS_NEXUS.
This will allow MSI providers built on top of the raw ITS domain
to identify it.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: Yijing Wang <wangyijing@huawei.com>
Cc: Ma Jun <majun258@huawei.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Duc Dang <dhdang@apm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1438091186-10244-13-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
It is becoming obvious that having the PCI/MSI code in the same
file as the the core ITS code is giving people implementing non-PCI
MSI support the wrong kind of idea.
In order to make things a bit clearer, let's move the PCI/MSI code
out to its own file. Hopefully it will make it clear that whoever
thinks of hooking into the core ITS better have a very strong point.
We use a temporary entry point that will get removed in a subsequent
patch, once the proper infrastructure is added.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: Yijing Wang <wangyijing@huawei.com>
Cc: Ma Jun <majun258@huawei.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Duc Dang <dhdang@apm.com>
Cc: Hanjun Guo <hanjun.guo@linaro.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1438091186-10244-12-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The GICv3 ITS architecture allows a given [DevID, EventID] pair to be
translated to a [LPI, Collection] pair, where DevID is the device writing
the MSI, EventID is the payload being written, LPI is the actual
interrupt number, and Collection is roughly equivalent to a target CPU.
Each LPI can be mapped to a separate collection, but the ITS driver
insists on maintaining the collection on a device basis, instead of doing
it on a per interrupt basis.
This is obviously flawed, and this patch fixes it by adding a per interrupt
index that indicates which collection number is in use.
Reported-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: stable@vger.kernel.org # 4.1, 4.0
Link: http://lkml.kernel.org/r/1437126402-11677-1-git-send-email-marc.zyngier@arm.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
When allocating a device table, if the requested allocation is smaller
than the default granule size of the ITS then, we need to round up to
the default size.
Signed-off-by: Minghuan Lian <Minghuan.Lian@freescale.com>
[ stuart: Added comments and massaged changelog ]
Signed-off-by: Stuart Yoder <stuart.yoder@freescale.com>
Reviewed-by: Marc Zygnier <marc.zyngier@arm.com>
Cc: <linux-arm-kernel@lists.infradead.org>
Cc: <jason@lakedaemon.net>
Link: http://lkml.kernel.org/r/1432134795-661-1-git-send-email-stuart.yoder@freescale.com
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
If the ITS or the redistributors report their shareability as zero,
then it is important to make sure they will no generate any cacheable
traffic, as this is unlikely to produce the expected result.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1427465705-17126-5-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The ITS driver sometime mixes up the use of GICR_PROPBASE bitfields
for the GICR_PENDBASE register, and GITS_BASER for GICR_CBASE.
This does not lead to any observable bug because similar bits are
at the same location, but this just make the code even harder to
understand...
This patch provides the required #defines and fixes the mixup.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1427465705-17126-4-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
When building ITS commands which have the device ID in it, we
should mask off the whole upper 32 bits of the first command word
before inserting the new value in there.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1427465705-17126-3-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
With a monolithic GICv3, redistributors are addressed using a linear
number, while a distributed implementation uses physical addresses.
When encoding a target address into a command, we strip the lower
16 bits, as redistributors are always 64kB aligned. This works
perfectly well with a distributed implementation, but has the
silly effect of always encoding target 0 in the monolithic case
(unless you have more than 64k CPUs, of course).
The obvious fix is to shift the linear target number by 16 when
computing the target address, so that we don't loose any precious
bit.
Reported-by: Andre Przywara <andre.przywara@arm.com>
Tested-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1427465705-17126-2-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
It's unsafe to change the configurations of an activated ITS directly
since this will lead to unpredictable results. This patch guarantees
the ITSes being initialized are quiescent.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Yun Wu <wuyun.wu@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-12-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
Define macros for GITS_CTLR fields to avoid using magic numbers.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Yun Wu <wuyun.wu@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-11-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
When required size of Device Table is out of the page allocator's
capability, the whole ITS will fail in probing. This actually is
not the hardware's problem and is mainly a limitation of the kernel
page allocator. This patch will keep ITS going on to the next
initializaion stage with an explicit warning.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Yun Wu <wuyun.wu@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-10-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The field of page size in register GITS_BASERn might be read-only
if an implementation only supports a single, fixed page size. But
currently the ITS driver will throw out an error when PAGE_SIZE
is less than the minimum size supported by an ITS. So addressing
this problem by using 64KB pages as default granule for all the
ITS base tables.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
[maz: fixed bug breaking non Device Table allocations]
Signed-off-by: Yun Wu <wuyun.wu@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-9-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
Some kind of brain-dead implementations chooses to insert ITEes in
rapid sequence of disabled ITEes, and an un-zeroed ITT will confuse
ITS on judging whether an ITE is really enabled or not. Considering
the implementations are still supported by the GICv3 architecture,
in which ITT is not required to be zeroed before being handled to
hardware, we do the favor in ITS driver.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Yun Wu <wuyun.wu@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-8-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
When compiled with CONFIG_LOCKDEP, the kernel shouts badly, saying
that my locking is unsafe. I'm afraid the kernel is right:
CPU0 CPU1
---- ----
lock(&its->lock);
local_irq_disable();
lock(&irq_desc_lock_class);
lock(&its->lock);
<Interrupt>
lock(&irq_desc_lock_class);
*** DEADLOCK ***
The fix is to always take its->lock with interrupts disabled.
Reported-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-5-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The current PCI/MSI support in the GICv3 ITS doesn't really deal
with systems where different PCI devices end-up using the same
RequesterID (as it would be the case with non-transparent bridges,
for example). It is likely that none of these devices would
actually generate any interrupt, as the ITS is programmed with
the device's own ID, and not that of the bridge.
A solution to this is to iterate over the PCI hierarchy to
discover what the device aliases too. We also use this
to discover the upper bound of the number of MSIs that this
sub-hierarchy can generate.
With this in place, PCI aliases can be supported.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-4-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The ITS table allocator is only allocating a single page per table.
This works fine for most things, but leads to silent lack of
interrupt delivery if we end-up with a device that has an ID that is
out of the range defined by a single page of memory. Even worse, depending
on the page size, behaviour changes, which is not a very good experience.
A solution is actually to allocate memory for the full range of ID that
the ITS supports. A massive waste memory wise, but at least a safe bet.
Tested on a Phytium SoC.
Tested-by: Chen Baozi <chenbaozi@kylinos.com.cn>
Acked-by: Chen Baozi <chenbaozi@kylinos.com.cn>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-3-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
We skip initialisation of ITS in case the device-tree has no
corresponding description, but we are still accessing to ITS bits while
setting CPU interface what leads to the kernel panic:
ITS: No ITS available, not enabling LPIs
CPU0: found redistributor 0 region 0:0x000000002f100000
Unable to handle kernel NULL pointer dereference at virtual address 00000000
pgd = ffffffc0007fb000
[00000000] *pgd=00000000fc407003, *pud=00000000fc407003, *pmd=00000000fc408003, *pte=006000002f000707
Internal error: Oops: 96000005 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.19.0-rc2+ #318
Hardware name: FVP Base (DT)
task: ffffffc00077edb0 ti: ffffffc00076c000 task.ti: ffffffc00076c000
PC is at its_cpu_init+0x2c/0x320
LR is at gic_cpu_init+0x168/0x1bc
It happens in gic_rdists_supports_plpis() because gic_rdists is NULL.
The gic_rdists is set to non-NULL only when ITS node is presented in
the device-tree.
Fix this by moving the call to gic_rdists_supports_plpis() inside the
!list_empty(&its_nodes) block, because it is that list that guards the
validity of the rest of the information in this driver.
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1425659870-11832-2-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
arm64 defconfig spits out the following compiler warning from the ITS
driver:
In file included from include/linux/bitmap.h:9:0,
from drivers/irqchip/irq-gic-v3-its.c:18:
drivers/irqchip/irq-gic-v3-its.c: In function ‘its_create_device’:
include/linux/kernel.h:716:17: warning: comparison of distinct pointer types lacks a cast
(void) (&_max1 == &_max2); \
^
drivers/irqchip/irq-gic-v3-its.c:1056:12: note: in expansion of macro ‘max’
nr_ites = max(2, roundup_pow_of_two(nvecs));
Fix the warning by specifying the decimal constant `2' explicitly as an
unsigned long type.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Will Deacon <will.deacon@arm.com>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1418825469-30529-1-git-send-email-will.deacon@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
When issuing a MAPD command, one of the parameters passed to the ITS
is the number of EventID bits used to index the per-device Interrupt
Translation Table (ITT). Crucially, this is the number of bits
*minus one*.
This has two consequences:
- The size of the ITT has to be a strict power of two, no matter
how many different events the device is actually going to generate.
- It is impossible to express an ITT with a single entry, as you
would have to tell the ITS to "use zero bit from the EventID",
and that clashes with "minus one" above.
Fix this by allocating the ITT with the number of vectors rounded up
to the next power of two, with a minimum of two entries.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Yun Wu (Abel) <wuyun.wu@huawei.com>
Cc: Robert Richter <robert.richter@caviumnetworks.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The ITS code could do a bit less in the alloc/free paths, and a bit
more in the activate/deactivate methods, giving a better separation
between software allocation and HW programing.
Suggested-by: Wuyun Wu (Abel) <wuyun.wu@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Yun Wu (Abel) <wuyun.wu@huawei.com>
Cc: Robert Richter <robert.richter@caviumnetworks.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Fix stupid thinko on the path freeing the interrupts, where only
the first interrupt would get reset, and none of the others.
This should only affect multi-MSI allocations.
Reported-by: Wuyun Wu (Abel) <wuyun.wu@huawei.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Cc: Robert Richter <robert.richter@caviumnetworks.com>
Cc: Jason Cooper <jason@lakedaemon.net>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
The ITS has a notion of "device" that can write to it in order to
generate an interrupt.
Conversly, the driver maintains a per-ITS list of devices, together
with their configuration information, and uses this to configure
the HW.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1416839720-18400-9-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The interrupt translation is driven by a set of tables (device,
ITT, and collection) to be in the end delivered to a CPU. Also,
the redistributors rely on a couple of tables (configuration, and
pending) to deliver the interrupts to the CPUs.
This patch adds the required allocators for these tables.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1416839720-18400-8-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
LPIs are the type of interrupts that are used by the ITS. Given
the size of the namespace (anywhere between 16 and 32bit), interrupt
IDs are allocated in chunks of 32.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1416839720-18400-7-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
The ITS is configured through a number commands that the driver
issues to the HW using a memory-based circular buffer.
This patch implements the subset of commands that are required
for Linux.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Link: https://lkml.kernel.org/r/1416839720-18400-5-git-send-email-marc.zyngier@arm.com
Signed-off-by: Jason Cooper <jason@lakedaemon.net>
Robert Richter