Adds support for:
- 64KiB/2MiB big page sizes (128KiB not supported by HW with new PT layout).
- System-memory PTs.
- LPTE "invalid" state.
- (Tegra) Use of video memory aperture.
- Sparse PDEs/PTEs.
- Additional blocklinear kinds.
- 49-bit address-space.
GP100 supports an entirely new 5-level page table layout that provides
an expanded 49-bit address-space. It also supports the layout present
on previous generations, which we've been making do with until now.
This commit implements support for the new layout, and enables it by
default.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
This is the common code to support a rework of the VMM backends.
It adds support for more than 2 levels of page table nesting, which
is required to be able to support GP100's MMU layout.
Sparse mappings (that don't cause MMU faults when accessed) are now
supported, where the backend provides it.
Dual-PT handling had to become more sophisticated to support sparse,
but this also allows us to support an optimisation the MMU provides
on GK104 and newer.
Certain operations can now be combined into a single page tree walk
to avoid some overhead, but also enables optimsations like skipping
PTE unmap writes when the PT will be destroyed anyway.
The old backend has been hacked up to forward requests onto the new
backend, if present, so that it's possible to bisect between issues
in the backend changes vs the upcoming frontend changes.
Until the new frontend has been merged, new backends will leak BAR2
page tables on module unload. This is expected, and it's not worth
the effort of hacking around this as it doesn't effect runtime.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
We previously required each VMM user to allocate their own page directory
and fill in the instance block themselves.
It makes more sense to handle this in a common location.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Adds support for:
- Selection of old/new-style page table layout (GP100MmuLayout=0/1).
- System-memory PDs.
New layout disabled by default for the moment, as we don't have a
backend that can handle it yet.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
This is the first chunk of the new VMM code that provides the structures
needed to describe a GPU virtual address-space layout, as well as common
interfaces to handle VMM creation, and connecting instances to a VMM.
The constructor now allocates the PD itself, rather than having the user
handle that manually. This won't/can't be used until after all backends
have been ported to these interfaces, so a little bit of memory will be
wasted on Fermi and newer for a couple of commits in the series.
Compatibility has been hacked into the old code to allow each GPU backend
to be ported individually.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
GP100 "big" (which is a funny name, when it supports "even bigger") page
tables are small enough that we want to be able to suballocate them from
a larger block of memory.
This builds on the previous page table cache interfaces so that the VMM
code doesn't need to know the difference.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Builds up and maintains a small cache of each page table size in order
to reduce the frequency of expensive allocations, particularly in the
pathological case where an address range ping-pongs between allocated
and free.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Removes the need to expose internals outside of MMU, and GP100 is both
different, and a lot harder to deal with.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Another transition step to allow finer-grained patches transitioning to
new MMU backends.
Old backends will continue operate as before (accessing nvkm_mem::tag),
and new backends will get a reference to the tags allocated here.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Upcoming MMU changes use nvkm_memory as its basic representation of memory,
so we need to be able to allocate VRAM like this.
The code is basically identical to the current chipset-specific allocators,
minus support for compression tags (which will be handled elsewhere anyway).
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Adds support for 64-bit writes, and optimised filling of buffers with
fixed 32/64-bit values.
These will all be used by the upcoming MMU changes.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
We need to be able to prevent memory from being freed while it's still
mapped in a GPU's address-space.
Will be used by upcoming MMU changes.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Needed by VMM code to determine whether an allocation is compatible with
a given page size (ie. you can't map 4KiB system memory pages into 64KiB
GPU pages).
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Map flags (access, kind, etc) are currently defined in either the VMA,
or the memory object, which turns out to not be ideal for things like
suballocated buffers, etc.
These will become per-map flags instead, so we need to support passing
these arguments in nvkm_memory_map().
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
nvkm_memory is going to be used by the upcoming mmu rework for the basic
representation of a memory allocation, as such, this commit adds support
for comptag allocation to nvkm_memory.
This is very simple for now, in that it requires comptags for the entire
memory allocation even if only certain ranges are compressed.
Support for tracking ranges will be added at a later date.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
We're moving towards having a central place to handle comptag allocation,
and as some GPUs don't have a ram submodule (ie. Tegra), we need to move
the mm somewhere else.
It probably never belonged in ram anyways.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Different sections of VRAM may have different properties (ie. can't be used
for compression/display, can't be mapped, etc).
We currently already support this, but it's a bit magic. This change makes
it more obvious where we're allocating from.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
The accessor functions can change as a result of acquire()/release() calls,
and are protected by any refcounting done there.
Other functions must remain constant, as they can be called any time.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Discovered by accident while working to use BAR2 access to instmem objects
on more paths.
We've apparently been relying on luck up until now!
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Upcoming changes will remove the nvkm_vmm pointer from nvkm_vma, instead
requiring it to be explicitly specified on each operation.
It's not currently possible to get this information for BAR1 mappings,
so let's fix that ahead of time.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Will prevent spurious MMU fault interrupts if something decides to touch
BAR1 after we've unloaded the driver.
Exposed external to BAR so that INSTMEM can use it to better control the
suspend/resume fast-path access.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
If we want to be able to hit the instmem fast-path in a few trickier cases,
we need to be more flexible with when we can initialise BAR2 access.
There's probably a decent case to be made for merging BAR/INSTMEM into BUS,
but that's something to ponder another day.
Flushes have been added after the write to bind the instance block,
as later commits will reveal the need for them.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
RM appears to do this really early in its initialisation, before DEVINIT.
We currently do this before BAR2 initialisation for some reason.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
MMU will be needing this to specify kind info on BAR mappings.
We have no userspace currently using these interfaces, so break the ABI
instead of supporting both. NVIF version bump so any future use can be
guarded.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
v2:
- add nv138 and drop nv13b chipsets (Ilia Mirkin)
- refactor out status variable and instead mask tsensor (Ilia Mirkin)
- switch SHADOWed state message away from nvkm_error() (Ilia Mirkin)
- rename internal temperature variable (Karol Herbst)
v3:
- use nvkm_trace() for SHADOWed state message (Ben Skeggs)
Signed-off-by: Rhys Kidd <rhyskidd@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Most of these errors seem to be WFD related. Official documentation
says dcb type 8 is reserved. It's probably used for WFD. Silence
the warning in either case.
Connector type 70 is stated to be a virtual connector for WiFi
display. Since we know this, don't warn that we don't.
Signed-off by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
This makes use of all the additional routing and state added in previous
commits, making it possible to deal with GM20x macro link routing, while
also sharing code between the NV50 and GF119 implementations.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
These exist to give NVKM information on the set of display paths that
the DD needs to be active at any given time.
Previously, the supervisor attempted to determine this solely from OR
state, but there's a few configurations where this information on its
own isn't enough to determine the specific display paths in question:
- ANX9805, where the PIOR protocol for both DP and TMDS is TMDS.
- On a device using DCB Switched Outputs.
- On GM20x and newer, with a crossbar between the SOR and macro links.
After this commit, the DD tells NVKM *exactly* which display path it's
attempting a modeset on.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Also removes the user-facing methods to these controls, as they're not
currently utilised by the DD anyway.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
This hasn't been used since atomic.
We may want to re-implement "fast" DPMS at some point, but for now,
this just gets in the way.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Upcoming commits make supervisor handling share code between the NV50
and GF119 implementations. Because of this, and a few other cleanups,
we need to allow some additional customisation.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
In order to properly support the SOR -> SOR + pad macro separation
that occurred with GM20x GPUs, we need to separate OR handling out
of the output path code.
This will be used as the base to support ORs (DAC, SOR, PIOR).
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Primarily intended as a way to pass per-head state around during
supervisor handling, and share logic between NV50/GF119.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
This is to allow hw-specific code to instantiate output resources first,
so we can cull unsupported output paths based on them.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
As of DCB 4.1, these are not the same thing.
Compatibility temporarily in place until callers have been updated.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
nvkm_timer_alarm() already handles this as part of protecting against
callers passing in no timeout value.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
The nouveau driver, in the Linux 3.7 days, used to try and set the
AVI InfoFrame based on the selected display mode. These days, it
uses a fixed set of InfoFrames. Start to correct that, by
providing a mechanism whereby InfoFrame data may be passed to the
NVKM functions that do the actual configuration.
At this point, only establish the new parameters and their parsing,
don't actually use the data anywhere yet (since it's not supplied
anywhere).
Signed-off-by: Alastair Bridgewater <alastair.bridgewater@gmail.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reusing the list_head for both is a bad idea. Callback execution is done
with the lock dropped so that alarms can be rescheduled from the callback,
which means that with some unfortunate timing, lists can get corrupted.
The execution list should not require its own locking, the single function
that uses it can only be called from a single context.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Cc: stable@vger.kernel.org
GP10B's power is managed by generic PM domains, so it does not require a
VDD regulator. Add this option into the chip function structure.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
GR is similar to GP100, with a few unavailable registers.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
GP10B requires a specific initialization sequence due to the absence of
devinit.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
GP10B's MC is compatible with GP100's, but engines need to be explicitly
put out of ELPG during init.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
GP10B's FB is largely compatible with the GP100 implementation.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
GP10B's FIFO is similar to GP100's, but only allows 512 channels.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
GP10B's secboot is largely similar to GM20B's. Only differences are MC
base address and the fact that GPCCS is also securely managed.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Having access to the secboot instance loading a LS firmware can be
useful to LS firmware handlers. At least more useful than just having an
out-of-context subdev pointer.
GP10B's firmware will also need to know the WPR address, which can be
obtained from the secboot instance.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Change the secboot and msgqueue interfaces to take a mask of falcons to
reset instead of a single falcon. The GP10B firmware interface requires
FECS and GPCCS to be booted in a single firmware command.
For firmwares that only support single falcon boot, it is trivial to
loop over the mask and boot each falcons individually.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
This commit reworks the RAM detection algorithm, using RAM-per-LTC to
determine whether a board has a mixed-memory configuration instead of
using RAM-per-FBPA. I'm not certain the algorithm is perfect, but it
should handle all currently known configurations in the very least.
This should fix GTX 970 boards with 4GiB of RAM where the last 512MiB
isn't fully accessible, as well as only detecting half the VRAM on
GF108 boards.
As a nice side-effect, GP10x memory detection now reuses the majority
of the code from earlier chipsets.
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Apparently sinks are allows to respond with ACK even if they didn't
fully complete a transaction... It seems like a missed opportunity
for DEFER to me, but what do I know :)
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Differences from GP100:
- 3 PPCs/GPC.
- Another random reg to calculate/write.
- Attrib CB setup a little different.
- PascalB
- PascalComputeB
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
These gp10x chips are supporting using (roughly) the same firmware.
Compared to previous secure chips, ACR runs on SEC2 and so does the
low-secure msgqueue.
ACR for these chips is based on r367.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
On some secure boot instances (e.g. gp10x) the load and unload blobs do
not run on the same falcon. Support this case by introducing a new
member to the ACR structure and making related functions take the falcon
to use as an argument instead of assuming the boot falcon is to be used.
The rule is that the load blob can be run on either the SEC or PMU
falcons, but the unload blob must be always run on PMU.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Add support for running the ACR binary on the SEC falcon.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
On SEC, DMEM is unaccessible by the CPU when the falcon is running in LS
mode. This makes communication with the firmware using DMEM impossible.
For this purpose, a new kind of memory (EMEM) has been added. It works
similarly to DMEM, with the difference that its address space starts at
0x1000000. For this reason, it makes sense to treat it like a special
case of DMEM.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
SEC2 is the name given by NVIDIA to the SEC engine post-Fermi (reasons
unknown). Even though it shares the same address range as SEC, its usage
is quite different and this justifies a new engine. Add this engine and
make TOP use it all post-TOP devices should use this implementation and
not the older SEC.
Also quickly add the short gp102 implementation which will be used for
falcon booting purposes.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
gp10x' secure boot requires a blob to be run on NVDEC. Expose the falcon
through a dummy device.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Reading registers at device construction time can be harmful, as there
is no guarantee the underlying engine will be up, or in its runtime
configuration. Defer register reading to the oneinit() hook and update
users accordingly.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
NVIDIA-provided PMU firmware is controlled by a msgqueue. Add a member
to the PMU structure as well as the required cleanup code if this
feature is used.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
A message queue firmware implements a specific protocol allowing the
host to send "commands" to a falcon, and the falcon to reply using
"messages". This patch implements the common part of this protocol and
defines the interface that the host can use.
Due to the way the firmware is developped internally at NVIDIA (where
kernel driver and firmware evolve in lockstep), firmwares taken at
different points in time can have frustratingly subtle differences that
must be taken into account. This code is architectured to make
implementing such differences as easy as possible.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
The falcon library may be used concurrently, especially after the
introduction of the msgqueue interface. Make it safe to use it that way.
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>
Ben Skeggs