When page alloc debugging is not enabled, we essentially accept any
virtual address for linear kernel TLB misses. But with kgdb, kernel
address probing, and other facilities we can try to access arbitrary
crap.
So, make sure the address we miss on will translate to physical memory
that actually exists.
In order to make this work we have to embed the valid address bitmap
into the kernel image. And in order to make that less expensive we
make an adjustment, in that the max physical memory address is
decreased to "1 << 41", even on the chips that support a 42-bit
physical address space. We can do this because bit 41 indicates
"I/O space" and thus covers non-memory ranges.
The result of this is that:
1) kpte_linear_bitmap shrinks from 2K to 1K in size
2) we need 64K more for the valid address bitmap
We can't let the valid address bitmap be dynamically allocated
once we start using it to validate TLB misses, otherwise we have
crazy issues to deal with wrt. recursive TLB misses and such.
If we're in a TLB miss it could be the deepest trap level that's legal
inside of the cpu. So if we TLB miss referencing the bitmap, the cpu
will be out of trap levels and enter RED state.
To guard against out-of-range accesses to the bitmap, we have to check
to make sure no bits in the physical address above bit 40 are set. We
could export and use last_valid_pfn for this check, but that's just an
unnecessary extra memory reference.
On the plus side of all this, since we load all of these translations
into the special 4MB mapping TSB, and we check the TSB first for TLB
misses, there should be absolutely no real cost for these new checks
in the TLB miss path.
Reported-by: heyongli@gmail.com
Signed-off-by: David S. Miller <davem@davemloft.net>
This allows the callers to now pass down the full set of FAULT_FLAG_xyz
flags to handle_mm_fault(). All callers have been (mechanically)
converted to the new calling convention, there's almost certainly room
for architectures to clean up their code and then add FAULT_FLAG_RETRY
when that support is added.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Commit b696fdc259 ("sparc64: Defer
cpu_data() setup until end of per-cpu data initialization.") broke
bootup for UP builds because the cpu_data() initialization only
occurs in setup_per_cpu_areas() which is never compiled in nor
called in UP builds.
Fix this up by calling the setups directly from init_64.c when
non-SMP.
Reported-by: Alexander Beregalov <a.beregalov@gmail.com>
Tested-by: Alexander Beregalov <a.beregalov@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
CPU_MASK_ALL is the (deprecated) "all bits set" cpumask, defined as so:
#define CPU_MASK_ALL (cpumask_t) { { ... } }
Taking the address of such a temporary is questionable at best,
unfortunately 321a8e9d (cpumask: add CPU_MASK_ALL_PTR macro) added
CPU_MASK_ALL_PTR:
#define CPU_MASK_ALL_PTR (&CPU_MASK_ALL)
Which formalizes this practice. One day gcc could bite us over this
usage (though we seem to have gotten away with it so far).
[Description by Rusty Russell]
Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: David S. Miller <davem@davemloft.net>
Now that we defer the cpu_data() initializations to the end of per-cpu
setup, we can get rid of this local hack we had to setup the per-cpu
areas eary.
This is a necessary step in order to support HAVE_DYNAMIC_PER_CPU_AREA
since the per-cpu setup must run when page structs are available.
Signed-off-by: David S. Miller <davem@davemloft.net>
We need to split up the cpu present mask setup from the cpu_data
initialization, and this is a first step towards that.
Signed-off-by: David S. Miller <davem@davemloft.net>
It's theoretically possible that there are exception table entries
which point into the (freed) init text of modules. These could cause
future problems if other modules get loaded into that memory and cause
an exception as we'd see the wrong fixup. The only case I know of is
kvm-intel.ko (when CONFIG_CC_OPTIMIZE_FOR_SIZE=n).
Amerigo fixed this long-standing FIXME in the x86 version, but this
patch is more general.
This implements trim_init_extable(); most archs are simple since they
use the standard lib/extable.c sort code. Alpha and IA64 use relative
addresses in their fixups, so thier trimming is a slight variation.
Sparc32 is unique; it doesn't seem to define ARCH_HAS_SORT_EXTABLE,
yet it defines its own sort_extable() which overrides the one in lib.
It doesn't sort, so we have to mark deleted entries instead of
actually trimming them.
Inspired-by: Amerigo Wang <amwang@redhat.com>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Cc: linux-alpha@vger.kernel.org
Cc: sparclinux@vger.kernel.org
Cc: linux-ia64@vger.kernel.org
Use debug_kmap_atomic in kmap_atomic, kmap_atomic_pfn, and
iomap_atomic_prot_pfn.
Signed-off-by: Akinobu Mita <akinobu.mita@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Otherwise it might interrupt switch_to() midstream and use
half-cooked register window state.
Reported-by: Chris Torek <chris.torek@windriver.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Impact: cleanup, futureproof
In fact, all cpumask ops will only be valid (in general) for bit
numbers < nr_cpu_ids. So use that instead of NR_CPUS in various
places.
This is always safe: no cpu number can be >= nr_cpu_ids, and
nr_cpu_ids is initialized to NR_CPUS at boot.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Acked-by: Ingo Molnar <mingo@elte.hu>
Impact: cleanup, futureproof
In fact, all cpumask ops will only be valid (in general) for bit
numbers < nr_cpu_ids. So use that instead of NR_CPUS in various
places.
This is always safe: no cpu number can be >= nr_cpu_ids, and
nr_cpu_ids is initialized to NR_CPUS at boot.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Mike Travis <travis@sgi.com>
Acked-by: Ingo Molnar <mingo@elte.hu>
This builds upon eeabac7386
("sparc64: Validate kernel generated fault addresses on sparc64.")
Upon further consideration, we actually should never see any
fault addresses for 32-bit tasks with the upper 32-bits set.
If it does every happen, by definition it's a bug. Whatever
context created that fault would only have that fault satisfied
if we used the full 64-bit address. If we truncate it, we'll
always fault the wrong address and we'll always loop faulting
forever.
So catch such conditions and mark them as errors always. Log
the error and fail the fault.
Signed-off-by: David S. Miller <davem@davemloft.net>
In order to handle all of the cases of address calculation overflow
properly, we run sparc 32-bit processes in "address masking" mode
when running on a 64-bit kernel.
Address masking mode zeros out the top 32-bits of the address
calculated for every load and store instruction.
However, when we're in privileged mode we have to run with that
address masking mode disabled even when accessing userspace from
the kernel.
To "simulate" the address masking mode we clear the top-bits by
hand for 32-bit processes in the fault handler.
It is the responsibility of code in the compat layer to properly
zero extend addresses used to access userspace. If this isn't
followed properly we can get into a fault loop.
Say that the user address is 0xf0000000 but for whatever reason
the kernel code sign extends this to 64-bit, and then the kernel
tries to access the result.
In such a case we'll fault on address 0xfffffffff0000000 but the fault
handler will process that fault as if it were to address 0xf0000000.
We'll loop faulting forever because the fault never gets satisfied.
So add a check specifically for this case, when the kernel is faulting
on a user address access and the addresses don't match up.
This code path is sufficiently slow path, and this bug is sufficiently
painful to diagnose, that this kind of bug check is warranted.
Signed-off-by: David S. Miller <davem@davemloft.net>
Move all applicable EXPORT_SYMBOL()s to the file where the respective
symbol is defined.
Removed all the includes that are no longer needed in sparc_ksyms_64.c
Comment all remaining EXPORT_SYMBOL()s in sparc_ksyms_64.c
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Additions by Julian Calaby:
* Moved EXPORT_SYMBOL()s for prom functions to their rightful places.
* Made some minor cleanups to the includes and comments of sparc_ksyms_64.c
* Updated and tidied commit message.
* Rebased patch over sparc-2.6.git HEAD.
* Ensured that all modified files have the correct includes.
Signed-off-by: Julian Calaby <julian.calaby@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Move all applicable EXPORT_SYMBOL()s to the file where the respective
symbol is defined.
Removed all the includes that are no longer needed in sparc_ksyms_32.c
Comment all remaining EXPORT_SYMBOL()s in sparc_ksyms_32.c
Two symbols are shared with sparc64 thus the exports were removed from
the sparc_ksyms_64.c too, along with the include their ommission made
redundant.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Additions by Julian Calaby:
* Moved EXPORT_SYMBOL()s for prom functions to their rightful places.
* Made some minor cleanups to the includes and comments of sparc_ksyms_32.c
* Made another subtraction from sparc_ksyms_64.c
* Updated and tidied commit message.
* Rebased patch over sparc-2.6.git HEAD.
* Ensured that all modified files have the correct includes.
Signed-off-by: Julian Calaby <julian.calaby@gmail.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Andrew Morton wrote:
People keep on doing
printk("%llu", some_u64);
testing it only on x86_64 and this generates a warning storm on
powerpc, sparc64, etc. Because they use `long', not `long long'.
Quite a few 64-bit architectures are using `long' for their
s64/u64 types. We should convert them all to `long long'.
Update types.h so we use unsigned long long for u64 and
fix all warnings in sparc64 code.
Tested with an allnoconfig, defconfig and allmodconfig builds.
This patch introduces additional warnings in several drivers.
These will be dealt with in separate patches.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The sparc allmodconfig build broke due to enabling of the
branch_tracer that does some very clever things with
all if conditions. This caused my gcc 3.4.5 to be so confused that
it emitted a warning:
arch/sparc/mm/fault_32.c: In function `do_sparc_fault':
arch/sparc/mm/fault_32.c:176: warning: 'fixup' might be used uninitialized in this function
And with -Werror this broke the build.
Refactor code so it:
1) becomes more readable
2) no longer emit a warning with the branch_tracer enabled
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The sparc64 allmodconfig build broke due to enabling of the
branch_tracer that does some very clever things with
all if conditions. This caused my gcc 3.4.5 to be so confused that
it emitted two warnings:
arch/sparc/mm/init_64.c: In function `update_mmu_cache':
arch/sparc/mm/init_64.c:271: warning: 'pg_flags' might be used uninitialized in this function
arch/sparc/mm/init_64.c:272: warning: 'page' might be used uninitialized in this function
And with -Werror this broke the build.
Refactor code so it:
1) becomes more readable
2) no longer emit a warning with the branch_tracer enabled
The refactoring uses a small helper function (flush_dcache()).
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
SUN_IO is always 'y' so drop it and thus killing an ifdef/endif pair
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
While doing this use standard names for start/end
so we could use definitions straight from asm-generic
for all the typical symbols.
This also allowed us to drop the use of PROVIDE in the linker
script so sprc is less non-standard on this area.
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Use sparc64 version of scatterlist.h.
There are three main differences:
dma_addr_t replaces __u32
dma_address replaces dvma_address
dma_length replaces dvma_length
dma_addr_t is a u32 on sparc32.
Boot tested on sparc32.
Signed-off-by: Robert Reif <reif@earthlink.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
So that we can profile code even in a local_irq_disable() section,
only write 14 (instead of 15) into the %pil register to disable IRQs.
This allows PIL level 15 to serve as a pseudo NMI.
Signed-off-by: David S. Miller <davem@davemloft.net>
Add a sysctl to tweak the RSS limit used to decide when to grow
the TSB for an address space.
In order to avoid expensive divides and multiplies only simply
positive and negative powers of two are supported.
The function computed takes the number of TSB translations that will
fit at one time in the TSB of a given size, and either adds or
subtracts a percentage of entries. This final value is the
RSS limit.
See tsb_size_to_rss_limit().
Signed-off-by: David S. Miller <davem@davemloft.net>
- move all sparc64/mm/ files to arch/sparc/mm/
- commonly named files are named _64.c
- add files to sparc/mm/Makefile preserving link order
- delete now unused sparc64/mm/Makefile
- sparc64 now finds mm/ in sparc
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
- rename files where sparc64 has similar files to _32.c
- Restructure Makefile
- Sneak in -Werror as we have for sparc64
Signed-off-by: Sam Ravnborg <sam@ravnborg.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
All noise since we don't have CPU hotplug there. However, they
did expose something very odd-looking in there - poke_viking()
does a bunch of identical btfixup each time it's called (i.e.
for each CPU). That one is left alone for now; just the trivial
misannotation fixes.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This patch adds init memory poisoning. It looks like
totalram_pages was not updated properly in free_initrd_mem
so I fixed that as well.
Signed-off-by: Robert Reif <reif@earthlink.net>
Signed-off-by: David S. Miller <davem@davemloft.net>
While doing some easy cleanups on the sparc code I noticed that the
CONFIG_SUN4 code seems to be worse than the rest - there were some
"I don't know how it should work, but the current code definitely cannot
work." places.
And while I have seen people running Linux on machines like a
SPARCstation 5 a few years ago I don't recall having seen sun4
machines, even less ones running Linux.
Signed-off-by: Adrian Bunk <bunk@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
In order to make this week I also had to add an include
of linux/dma-mapping.h to asm/pci_32.h because drivers/pci/pci.c
really depends upon getting this header somehow.
Signed-off-by: David S. Miller <davem@davemloft.net>
The individual SBUS IOMMU arch code now sets the IOMMU information
directly into the OF device objects.
Signed-off-by: David S. Miller <davem@davemloft.net>
This lets us kill this "map it in every IOMMU" crazy code, and also
some of the final references to sbus_root.
Signed-off-by: David S. Miller <davem@davemloft.net>
And stick the iommu archdata pointer into the generic OF device tree
of_device struct as well.
We still have to pass the sbus_bus object down into the routines so
that the SBUS bus objects get the iommu cookies set properly. After
drivers get converted to being pure OF drivers, that can go away.
Signed-off-by: David S. Miller <davem@davemloft.net>
free_area_init_node() gets passed in the node id as well as the node
descriptor. This is redundant as the function can trivially get the node
descriptor itself by means of NODE_DATA() and the node's id.
I checked all the users and NODE_DATA() seems to be usable everywhere
from where this function is called.
Signed-off-by: Johannes Weiner <hannes@saeurebad.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This patch removes the CVS keywords that weren't updated for a long time
from comments.
Signed-off-by: Adrian Bunk <bunk@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
The PROM library function prom_meminit() builds a table,
prom_phys_avail[], just so that probe_memory() in
arch/sparc/mm/fault.c can copy it into sp_banks[].
Just have prom_meminit() fill in the sp_banks[] array directly, and
remove duplicated sort() function.
Signed-off-by: David S. Miller <davem@davemloft.net>
David S. Miller