openkylin
/
linux
mirror of https://gitee.com/openkylin/linux.git
9
0
Fork 0
Code Issues Projects Releases Wiki Activity
linux/arch/sparc/kernel/head_64.S

970 lines
23 KiB
ArmAsm
Raw Normal View History

[SPARC64]: Mark most of initial bootup asm as .text.init.ref_ok We can't mark the whole thing init because there are dependencies in bootloaders that assume that _start, or whatever the image entry value, is 2 instructions before the "HdrS" signature. In fact, TILO assumes this entry is always at 0x4000, yikes! Also, right after the bootloader info area there are OBP strings and values that get used later in the boot process, and those are not all provably .init yet. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-07-25 06:17:33 +08:00
/* head.S: Initial boot code for the Sparc64 port of Linux.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*
[SPARC64]: Mark most of initial bootup asm as .text.init.ref_ok We can't mark the whole thing init because there are dependencies in bootloaders that assume that _start, or whatever the image entry value, is 2 instructions before the "HdrS" signature. In fact, TILO assumes this entry is always at 0x4000, yikes! Also, right after the bootloader info area there are OBP strings and values that get used later in the boot process, and those are not all provably .init yet. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-07-25 06:17:33 +08:00
* Copyright (C) 1996, 1997, 2007 David S. Miller (davem@davemloft.net)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
* Copyright (C) 1996 David Sitsky (David.Sitsky@anu.edu.au)
[SPARC64]: Mark most of initial bootup asm as .text.init.ref_ok We can't mark the whole thing init because there are dependencies in bootloaders that assume that _start, or whatever the image entry value, is 2 instructions before the "HdrS" signature. In fact, TILO assumes this entry is always at 0x4000, yikes! Also, right after the bootloader info area there are OBP strings and values that get used later in the boot process, and those are not all provably .init yet. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-07-25 06:17:33 +08:00
* Copyright (C) 1997, 1998 Jakub Jelinek (jj@sunsite.mff.cuni.cz)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
* Copyright (C) 1997 Miguel de Icaza (miguel@nuclecu.unam.mx)
*/
#include <linux/version.h>
#include <linux/errno.h>
[SPARC64]: Make smp_processor_id() functional before start_kernel() Uses of smp_processor_id() get pushed earlier and earlier in the start_kernel() sequence. So just get it working before we call start_kernel() to avoid all possible problems. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-05-31 16:24:02 +08:00
#include <linux/threads.h>
[SPARC64]: Mark most of initial bootup asm as .text.init.ref_ok We can't mark the whole thing init because there are dependencies in bootloaders that assume that _start, or whatever the image entry value, is 2 instructions before the "HdrS" signature. In fact, TILO assumes this entry is always at 0x4000, yikes! Also, right after the bootloader info area there are OBP strings and values that get used later in the boot process, and those are not all provably .init yet. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-07-25 06:17:33 +08:00
#include <linux/init.h>
sparc64: Use ENTRY/ENDPROC in hypervisor asm. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-09-01 18:13:17 +08:00
#include <linux/linkage.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
#include <asm/thread_info.h>
#include <asm/asi.h>
#include <asm/pstate.h>
#include <asm/ptrace.h>
#include <asm/spitfire.h>
#include <asm/page.h>
#include <asm/pgtable.h>
#include <asm/errno.h>
#include <asm/signal.h>
#include <asm/processor.h>
#include <asm/lsu.h>
#include <asm/dcr.h>
#include <asm/dcu.h>
#include <asm/head.h>
#include <asm/ttable.h>
#include <asm/mmu.h>
[SPARC64]: Elminate all usage of hard-coded trap globals. UltraSPARC has special sets of global registers which are switched to for certain trap types. There is one set for MMU related traps, one set of Interrupt Vector processing, and another set (called the Alternate globals) for all other trap types. For what seems like forever we've hard coded the values in some of these trap registers. Some examples include: 1) Interrupt Vector global %g6 holds current processors interrupt work struct where received interrupts are managed for IRQ handler dispatch. 2) MMU global %g7 holds the base of the page tables of the currently active address space. 3) Alternate global %g6 held the current_thread_info() value. Such hardcoding has resulted in some serious issues in many areas. There are some code sequences where having another register available would help clean up the implementation. Taking traps such as cross-calls from the OBP firmware requires some trick code sequences wherein we have to save away and restore all of the special sets of global registers when we enter/exit OBP. We were also using the IMMU TSB register on SMP to hold the per-cpu area base address, which doesn't work any longer now that we actually use the TSB facility of the cpu. The implementation is pretty straight forward. One tricky bit is getting the current processor ID as that is different on different cpu variants. We use a stub with a fancy calling convention which we patch at boot time. The calling convention is that the stub is branched to and the (PC - 4) to return to is in register %g1. The cpu number is left in %g6. This stub can be invoked by using the __GET_CPUID macro. We use an array of per-cpu trap state to store the current thread and physical address of the current address space's page tables. The TRAP_LOAD_THREAD_REG loads %g6 with the current thread from this table, it uses __GET_CPUID and also clobbers %g1. TRAP_LOAD_IRQ_WORK is used by the interrupt vector processing to load the current processor's IRQ software state into %g6. It also uses __GET_CPUID and clobbers %g1. Finally, TRAP_LOAD_PGD_PHYS loads the physical address base of the current address space's page tables into %g7, it clobbers %g1 and uses __GET_CPUID. Many refinements are possible, as well as some tuning, with this stuff in place. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-27 15:24:22 +08:00
#include <asm/cpudata.h>
sparc64: Split entry.S up into seperate files. entry.S was a hodge-podge of several totally unrelated sets of assembler routines, ranging from FPU trap handlers to hypervisor call functions. Split it up into topic-sized pieces. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-04-28 15:47:20 +08:00
#include <asm/pil.h>
#include <asm/estate.h>
#include <asm/sfafsr.h>
#include <asm/unistd.h>
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/* This section from from _start to sparc64_boot_end should fit into
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
* 0x0000000000404000 to 0x0000000000408000.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*/
.text
.globl start, _start, stext, _stext
_start:
start:
_stext:
stext:
! 0x0000000000404000
b sparc64_boot
flushw /* Flush register file. */
/* This stuff has to be in sync with SILO and other potential boot loaders
* Fields should be kept upward compatible and whenever any change is made,
* HdrS version should be incremented.
*/
.global root_flags, ram_flags, root_dev
.global sparc_ramdisk_image, sparc_ramdisk_size
.global sparc_ramdisk_image64
.ascii "HdrS"
.word LINUX_VERSION_CODE
/* History:
*
* 0x0300 : Supports being located at other than 0x4000
* 0x0202 : Supports kernel params string
* 0x0201 : Supports reboot_command
*/
.half 0x0301 /* HdrS version */
root_flags:
.half 1
root_dev:
.half 0
ram_flags:
.half 0
sparc_ramdisk_image:
.word 0
sparc_ramdisk_size:
.word 0
.xword reboot_command
.xword bootstr_info
sparc_ramdisk_image64:
.xword 0
.word _end
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
/* PROM cif handler code address is in %o4. */
sparc64_boot:
[SPARC64]: Kill no-remapping-needed code in head.S It branches around some necessary prom calls, which we would need to do even if we are mapped at the correct location already. So it doesn't work. The idea was that this sort of thing could be used for the eventual kexec implementation, but it is clear that this will need to be done differently. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-12-12 13:06:55 +08:00
mov %o4, %l7
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
Fix common misspellings Fixes generated by 'codespell' and manually reviewed. Signed-off-by: Lucas De Marchi <lucas.demarchi@profusion.mobi>
2011-03-31 09:57:33 +08:00
/* We need to remap the kernel. Use position independent
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
* code to remap us to KERNBASE.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
* SILO can invoke us with 32-bit address masking enabled,
* so make sure that's clear.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*/
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
rdpr %pstate, %g1
andn %g1, PSTATE_AM, %g1
wrpr %g1, 0x0, %pstate
ba,a,pt %xcc, 1f
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
.globl prom_finddev_name, prom_chosen_path, prom_root_node
.globl prom_getprop_name, prom_mmu_name, prom_peer_name
.globl prom_callmethod_name, prom_translate_name, prom_root_compatible
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
.globl prom_map_name, prom_unmap_name, prom_mmu_ihandle_cache
.globl prom_boot_mapped_pc, prom_boot_mapping_mode
.globl prom_boot_mapping_phys_high, prom_boot_mapping_phys_low
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
.globl prom_compatible_name, prom_cpu_path, prom_cpu_compatible
[SPARC64]: Fix lockdep, particularly on SMP. As noted by Al Viro, when we try to call prom_set_trap_table() in the SMP trampoline code we try to take the PROM call spinlock which doesn't work because the current thread pointer isn't valid yet and lockdep depends upon that being correct. Furthermore, we cannot set the current thread pointer register because it can't be properly dereferenced until we return from prom_set_trap_table(). Kernel TLB misses only work after that call. So do the PROM call to set the trap table directly instead of going through the OBP library C code, and thus avoid the lock altogether. These calls are guarenteed to be serialized fully. Since there are now no calls to the prom_set_trap_table{_sun4v}() library functions, they can be deleted. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-09-17 02:51:15 +08:00
.globl is_sun4v, sun4v_chip_type, prom_set_trap_table_name
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
prom_peer_name:
.asciz "peer"
prom_compatible_name:
.asciz "compatible"
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
prom_finddev_name:
.asciz "finddevice"
prom_chosen_path:
.asciz "/chosen"
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
prom_cpu_path:
.asciz "/cpu"
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
prom_getprop_name:
.asciz "getprop"
prom_mmu_name:
.asciz "mmu"
prom_callmethod_name:
.asciz "call-method"
prom_translate_name:
.asciz "translate"
prom_map_name:
.asciz "map"
prom_unmap_name:
.asciz "unmap"
[SPARC64]: Fix lockdep, particularly on SMP. As noted by Al Viro, when we try to call prom_set_trap_table() in the SMP trampoline code we try to take the PROM call spinlock which doesn't work because the current thread pointer isn't valid yet and lockdep depends upon that being correct. Furthermore, we cannot set the current thread pointer register because it can't be properly dereferenced until we return from prom_set_trap_table(). Kernel TLB misses only work after that call. So do the PROM call to set the trap table directly instead of going through the OBP library C code, and thus avoid the lock altogether. These calls are guarenteed to be serialized fully. Since there are now no calls to the prom_set_trap_table{_sun4v}() library functions, they can be deleted. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-09-17 02:51:15 +08:00
prom_set_trap_table_name:
.asciz "SUNW,set-trap-table"
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
prom_sun4v_name:
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
.asciz "sun4v"
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
prom_niagara_prefix:
.asciz "SUNW,UltraSPARC-T"
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
prom_sparc_prefix:
sparc64: Future proof Niagara cpu detection. Recognize T4 and T5 chips. Treating them both as "T2 plus other stuff" should be extremely safe and make sure distributions will work when those chips actually ship to customers. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-12 01:42:20 +08:00
.asciz "SPARC-"
sparc64: correctly recognize SPARC64-X chips The following patch adds support for correctly recognizing SPARC-X chips. cpu : Unknown SUN4V CPU fpu : Unknown SUN4V FPU pmu : Unknown SUN4V PMU Signed-off-by: Katayama Yoshihiro <kata1@jp.fujitsu.com> Signed-off-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-06 07:47:59 +08:00
prom_sparc64x_prefix:
.asciz "SPARC64-X"
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
.align 4
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
prom_root_compatible:
.skip 64
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
prom_cpu_compatible:
.skip 64
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
prom_root_node:
.word 0
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
prom_mmu_ihandle_cache:
.word 0
prom_boot_mapped_pc:
.word 0
prom_boot_mapping_mode:
.word 0
.align 8
prom_boot_mapping_phys_high:
.xword 0
prom_boot_mapping_phys_low:
.xword 0
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
is_sun4v:
.word 0
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
sun4v_chip_type:
.word SUN4V_CHIP_INVALID
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
1:
rd %pc, %l0
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
mov (1b - prom_peer_name), %l1
sub %l0, %l1, %l1
mov 0, %l2
/* prom_root_node = prom_peer(0) */
stx %l1, [%sp + 2047 + 128 + 0x00] ! service, "peer"
mov 1, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 1
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 1
stx %l2, [%sp + 2047 + 128 + 0x18] ! arg1, 0
stx %g0, [%sp + 2047 + 128 + 0x20] ! ret1
call %l7
add %sp, (2047 + 128), %o0 ! argument array
ldx [%sp + 2047 + 128 + 0x20], %l4 ! prom root node
mov (1b - prom_root_node), %l1
sub %l0, %l1, %l1
stw %l4, [%l1]
mov (1b - prom_getprop_name), %l1
mov (1b - prom_compatible_name), %l2
mov (1b - prom_root_compatible), %l5
sub %l0, %l1, %l1
sub %l0, %l2, %l2
sub %l0, %l5, %l5
/* prom_getproperty(prom_root_node, "compatible",
* &prom_root_compatible, 64)
*/
stx %l1, [%sp + 2047 + 128 + 0x00] ! service, "getprop"
mov 4, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 4
mov 1, %l3
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 1
stx %l4, [%sp + 2047 + 128 + 0x18] ! arg1, prom_root_node
stx %l2, [%sp + 2047 + 128 + 0x20] ! arg2, "compatible"
stx %l5, [%sp + 2047 + 128 + 0x28] ! arg3, &prom_root_compatible
mov 64, %l3
stx %l3, [%sp + 2047 + 128 + 0x30] ! arg4, size
stx %g0, [%sp + 2047 + 128 + 0x38] ! ret1
call %l7
add %sp, (2047 + 128), %o0 ! argument array
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
mov (1b - prom_finddev_name), %l1
mov (1b - prom_chosen_path), %l2
mov (1b - prom_boot_mapped_pc), %l3
sub %l0, %l1, %l1
sub %l0, %l2, %l2
sub %l0, %l3, %l3
stw %l0, [%l3]
sub %sp, (192 + 128), %sp
/* chosen_node = prom_finddevice("/chosen") */
stx %l1, [%sp + 2047 + 128 + 0x00] ! service, "finddevice"
mov 1, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 1
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 1
stx %l2, [%sp + 2047 + 128 + 0x18] ! arg1, "/chosen"
stx %g0, [%sp + 2047 + 128 + 0x20] ! ret1
call %l7
add %sp, (2047 + 128), %o0 ! argument array
ldx [%sp + 2047 + 128 + 0x20], %l4 ! chosen device node
mov (1b - prom_getprop_name), %l1
mov (1b - prom_mmu_name), %l2
mov (1b - prom_mmu_ihandle_cache), %l5
sub %l0, %l1, %l1
sub %l0, %l2, %l2
sub %l0, %l5, %l5
/* prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu") */
stx %l1, [%sp + 2047 + 128 + 0x00] ! service, "getprop"
mov 4, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 4
mov 1, %l3
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 1
stx %l4, [%sp + 2047 + 128 + 0x18] ! arg1, chosen_node
stx %l2, [%sp + 2047 + 128 + 0x20] ! arg2, "mmu"
stx %l5, [%sp + 2047 + 128 + 0x28] ! arg3, &prom_mmu_ihandle_cache
mov 4, %l3
stx %l3, [%sp + 2047 + 128 + 0x30] ! arg4, sizeof(arg3)
stx %g0, [%sp + 2047 + 128 + 0x38] ! ret1
call %l7
add %sp, (2047 + 128), %o0 ! argument array
mov (1b - prom_callmethod_name), %l1
mov (1b - prom_translate_name), %l2
sub %l0, %l1, %l1
sub %l0, %l2, %l2
lduw [%l5], %l5 ! prom_mmu_ihandle_cache
stx %l1, [%sp + 2047 + 128 + 0x00] ! service, "call-method"
mov 3, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 3
mov 5, %l3
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 5
stx %l2, [%sp + 2047 + 128 + 0x18] ! arg1: "translate"
stx %l5, [%sp + 2047 + 128 + 0x20] ! arg2: prom_mmu_ihandle_cache
[SPARC64]: Fix net booting on Ultra5 We were not doing alignment properly when remapping the kernel image. What we want is a 4MB aligned physical address to map at KERNBASE. Mistakedly we were 4MB aligning the virtual address where the kernel initially sits, that's wrong. Instead, we should PAGE align the virtual address, then 4MB align the physical address result the prom gives to us. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-12 06:45:16 +08:00
/* PAGE align */
srlx %l0, 13, %l3
sllx %l3, 13, %l3
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
stx %l3, [%sp + 2047 + 128 + 0x28] ! arg3: vaddr, our PC
stx %g0, [%sp + 2047 + 128 + 0x30] ! res1
stx %g0, [%sp + 2047 + 128 + 0x38] ! res2
stx %g0, [%sp + 2047 + 128 + 0x40] ! res3
stx %g0, [%sp + 2047 + 128 + 0x48] ! res4
stx %g0, [%sp + 2047 + 128 + 0x50] ! res5
call %l7
add %sp, (2047 + 128), %o0 ! argument array
ldx [%sp + 2047 + 128 + 0x40], %l1 ! translation mode
mov (1b - prom_boot_mapping_mode), %l4
sub %l0, %l4, %l4
stw %l1, [%l4]
mov (1b - prom_boot_mapping_phys_high), %l4
sub %l0, %l4, %l4
ldx [%sp + 2047 + 128 + 0x48], %l2 ! physaddr high
stx %l2, [%l4 + 0x0]
ldx [%sp + 2047 + 128 + 0x50], %l3 ! physaddr low
[SPARC64]: Fix net booting on Ultra5 We were not doing alignment properly when remapping the kernel image. What we want is a 4MB aligned physical address to map at KERNBASE. Mistakedly we were 4MB aligning the virtual address where the kernel initially sits, that's wrong. Instead, we should PAGE align the virtual address, then 4MB align the physical address result the prom gives to us. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-12 06:45:16 +08:00
/* 4MB align */
sparc64: Use 'ILOG2_4MB' instead of constant '22'. Signed-off-by: David S. Miller <davem@davemloft.net>
2014-05-04 13:52:50 +08:00
srlx %l3, ILOG2_4MB, %l3
sllx %l3, ILOG2_4MB, %l3
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
stx %l3, [%l4 + 0x8]
/* Leave service as-is, "call-method" */
mov 7, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 7
mov 1, %l3
[SPARC64]: Fix comment typo in head.S Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:31:29 +08:00
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 1
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
mov (1b - prom_map_name), %l3
sub %l0, %l3, %l3
stx %l3, [%sp + 2047 + 128 + 0x18] ! arg1: "map"
/* Leave arg2 as-is, prom_mmu_ihandle_cache */
mov -1, %l3
stx %l3, [%sp + 2047 + 128 + 0x28] ! arg3: mode (-1 default)
[SPARC64]: Remove most limitations to kernel image size. Currently kernel images are limited to 8MB in size, and this causes problems especially when enabling features that take up a lot of kernel image space such as lockdep. The code now will align the kernel image size up to 4MB and map that many locked TLB entries. So, the only practical limitation is the number of available locked TLB entries which is 16 on Cheetah and 64 on pre-Cheetah sparc64 cpus. Niagara cpus don't actually have hw locked TLB entry support. Rather, the hypervisor transparently provides support for "locked" TLB entries since it runs with physical addressing and does the initial TLB miss processing. Fully utilizing this change requires some help from SILO, a patch for which will be submitted to the maintainer. Essentially, SILO will only currently map up to 8MB for the kernel image and that needs to be increased. Note that neither this patch nor the SILO bits will help with network booting. The openfirmware code will only map up to a certain amount of kernel image during a network boot and there isn't much we can to about that other than to implemented a layered network booting facility. Solaris has this, and calls it "wanboot" and we may implement something similar at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-03-22 08:01:38 +08:00
/* 4MB align the kernel image size. */
set (_end - KERNBASE), %l3
set ((4 * 1024 * 1024) - 1), %l4
add %l3, %l4, %l3
andn %l3, %l4, %l3
stx %l3, [%sp + 2047 + 128 + 0x30] ! arg4: roundup(ksize, 4MB)
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
sethi %hi(KERNBASE), %l3
stx %l3, [%sp + 2047 + 128 + 0x38] ! arg5: vaddr (KERNBASE)
stx %g0, [%sp + 2047 + 128 + 0x40] ! arg6: empty
mov (1b - prom_boot_mapping_phys_low), %l3
sub %l0, %l3, %l3
ldx [%l3], %l3
stx %l3, [%sp + 2047 + 128 + 0x48] ! arg7: phys addr
call %l7
add %sp, (2047 + 128), %o0 ! argument array
add %sp, (192 + 128), %sp
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
sethi %hi(prom_root_compatible), %g1
or %g1, %lo(prom_root_compatible), %g1
sethi %hi(prom_sun4v_name), %g7
or %g7, %lo(prom_sun4v_name), %g7
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
mov 5, %g3
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
90: ldub [%g7], %g2
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
ldub [%g1], %g4
cmp %g2, %g4
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
bne,pn %icc, 80f
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
add %g7, 1, %g7
subcc %g3, 1, %g3
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
bne,pt %xcc, 90b
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
add %g1, 1, %g1
sethi %hi(is_sun4v), %g1
or %g1, %lo(is_sun4v), %g1
mov 1, %g7
stw %g7, [%g1]
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
/* cpu_node = prom_finddevice("/cpu") */
mov (1b - prom_finddev_name), %l1
mov (1b - prom_cpu_path), %l2
sub %l0, %l1, %l1
sub %l0, %l2, %l2
sub %sp, (192 + 128), %sp
stx %l1, [%sp + 2047 + 128 + 0x00] ! service, "finddevice"
mov 1, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 1
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 1
stx %l2, [%sp + 2047 + 128 + 0x18] ! arg1, "/cpu"
stx %g0, [%sp + 2047 + 128 + 0x20] ! ret1
call %l7
add %sp, (2047 + 128), %o0 ! argument array
ldx [%sp + 2047 + 128 + 0x20], %l4 ! cpu device node
mov (1b - prom_getprop_name), %l1
mov (1b - prom_compatible_name), %l2
mov (1b - prom_cpu_compatible), %l5
sub %l0, %l1, %l1
sub %l0, %l2, %l2
sub %l0, %l5, %l5
/* prom_getproperty(cpu_node, "compatible",
* &prom_cpu_compatible, 64)
*/
stx %l1, [%sp + 2047 + 128 + 0x00] ! service, "getprop"
mov 4, %l3
stx %l3, [%sp + 2047 + 128 + 0x08] ! num_args, 4
mov 1, %l3
stx %l3, [%sp + 2047 + 128 + 0x10] ! num_rets, 1
stx %l4, [%sp + 2047 + 128 + 0x18] ! arg1, cpu_node
stx %l2, [%sp + 2047 + 128 + 0x20] ! arg2, "compatible"
stx %l5, [%sp + 2047 + 128 + 0x28] ! arg3, &prom_cpu_compatible
mov 64, %l3
stx %l3, [%sp + 2047 + 128 + 0x30] ! arg4, size
stx %g0, [%sp + 2047 + 128 + 0x38] ! ret1
call %l7
add %sp, (2047 + 128), %o0 ! argument array
add %sp, (192 + 128), %sp
sethi %hi(prom_cpu_compatible), %g1
or %g1, %lo(prom_cpu_compatible), %g1
sethi %hi(prom_niagara_prefix), %g7
or %g7, %lo(prom_niagara_prefix), %g7
mov 17, %g3
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
90: ldub [%g7], %g2
ldub [%g1], %g4
cmp %g2, %g4
bne,pn %icc, 89f
add %g7, 1, %g7
subcc %g3, 1, %g3
bne,pt %xcc, 90b
add %g1, 1, %g1
ba,pt %xcc, 91f
nop
89: sethi %hi(prom_cpu_compatible), %g1
or %g1, %lo(prom_cpu_compatible), %g1
sethi %hi(prom_sparc_prefix), %g7
or %g7, %lo(prom_sparc_prefix), %g7
sparc64: Future proof Niagara cpu detection. Recognize T4 and T5 chips. Treating them both as "T2 plus other stuff" should be extremely safe and make sure distributions will work when those chips actually ship to customers. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-12 01:42:20 +08:00
mov 6, %g3
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
90: ldub [%g7], %g2
ldub [%g1], %g4
cmp %g2, %g4
bne,pn %icc, 4f
add %g7, 1, %g7
subcc %g3, 1, %g3
bne,pt %xcc, 90b
add %g1, 1, %g1
sethi %hi(prom_cpu_compatible), %g1
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
or %g1, %lo(prom_cpu_compatible), %g1
sparc64: Future proof Niagara cpu detection. Recognize T4 and T5 chips. Treating them both as "T2 plus other stuff" should be extremely safe and make sure distributions will work when those chips actually ship to customers. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-12 01:42:20 +08:00
ldub [%g1 + 6], %g2
cmp %g2, 'T'
be,pt %xcc, 70f
cmp %g2, 'M'
sparc64: recognize and support Sonoma CPU type Add code to recognize SPARC-Sonoma cpu correctly and update cpu hardware caps and cpu distribution map. SPARC-Sonoma is based upon SPARC-M7 core along with additional PCI functions added on and is reported by firmware as "SPARC-SN". Signed-off-by: Khalid Aziz <khalid.aziz@oracle.com> Acked-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-20 01:12:54 +08:00
be,pt %xcc, 70f
cmp %g2, 'S'
sparc64: correctly recognize SPARC64-X chips The following patch adds support for correctly recognizing SPARC-X chips. cpu : Unknown SUN4V CPU fpu : Unknown SUN4V FPU pmu : Unknown SUN4V PMU Signed-off-by: Katayama Yoshihiro <kata1@jp.fujitsu.com> Signed-off-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-06 07:47:59 +08:00
bne,pn %xcc, 49f
sparc64: Future proof Niagara cpu detection. Recognize T4 and T5 chips. Treating them both as "T2 plus other stuff" should be extremely safe and make sure distributions will work when those chips actually ship to customers. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-12 01:42:20 +08:00
nop
70: ldub [%g1 + 7], %g2
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
cmp %g2, '3'
be,pt %xcc, 5f
mov SUN4V_CHIP_NIAGARA3, %g4
sparc64: Future proof Niagara cpu detection. Recognize T4 and T5 chips. Treating them both as "T2 plus other stuff" should be extremely safe and make sure distributions will work when those chips actually ship to customers. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-12 01:42:20 +08:00
cmp %g2, '4'
be,pt %xcc, 5f
mov SUN4V_CHIP_NIAGARA4, %g4
cmp %g2, '5'
be,pt %xcc, 5f
mov SUN4V_CHIP_NIAGARA5, %g4
sparc64: correctly recognise M6 and M7 cpu type The following patch adds support for correctly recognising M6 and M7 cpu type. Signed-off-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-09-08 14:18:53 +08:00
cmp %g2, '6'
be,pt %xcc, 5f
mov SUN4V_CHIP_SPARC_M6, %g4
cmp %g2, '7'
be,pt %xcc, 5f
mov SUN4V_CHIP_SPARC_M7, %g4
sparc64: recognize and support Sonoma CPU type Add code to recognize SPARC-Sonoma cpu correctly and update cpu hardware caps and cpu distribution map. SPARC-Sonoma is based upon SPARC-M7 core along with additional PCI functions added on and is reported by firmware as "SPARC-SN". Signed-off-by: Khalid Aziz <khalid.aziz@oracle.com> Acked-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-20 01:12:54 +08:00
cmp %g2, 'N'
be,pt %xcc, 5f
mov SUN4V_CHIP_SPARC_SN, %g4
sparc64: correctly recognize SPARC64-X chips The following patch adds support for correctly recognizing SPARC-X chips. cpu : Unknown SUN4V CPU fpu : Unknown SUN4V FPU pmu : Unknown SUN4V PMU Signed-off-by: Katayama Yoshihiro <kata1@jp.fujitsu.com> Signed-off-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-06 07:47:59 +08:00
ba,pt %xcc, 49f
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
nop
91: sethi %hi(prom_cpu_compatible), %g1
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
or %g1, %lo(prom_cpu_compatible), %g1
ldub [%g1 + 17], %g2
cmp %g2, '1'
be,pt %xcc, 5f
mov SUN4V_CHIP_NIAGARA1, %g4
cmp %g2, '2'
be,pt %xcc, 5f
mov SUN4V_CHIP_NIAGARA2, %g4
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
4:
sparc64: correctly recognize SPARC64-X chips The following patch adds support for correctly recognizing SPARC-X chips. cpu : Unknown SUN4V CPU fpu : Unknown SUN4V FPU pmu : Unknown SUN4V PMU Signed-off-by: Katayama Yoshihiro <kata1@jp.fujitsu.com> Signed-off-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-06 07:47:59 +08:00
/* Athena */
sethi %hi(prom_cpu_compatible), %g1
or %g1, %lo(prom_cpu_compatible), %g1
sethi %hi(prom_sparc64x_prefix), %g7
or %g7, %lo(prom_sparc64x_prefix), %g7
mov 9, %g3
41: ldub [%g7], %g2
ldub [%g1], %g4
cmp %g2, %g4
bne,pn %icc, 49f
add %g7, 1, %g7
subcc %g3, 1, %g3
bne,pt %xcc, 41b
add %g1, 1, %g1
ba,pt %xcc, 5f
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
mov SUN4V_CHIP_SPARC64X, %g4
sparc64: correctly recognize SPARC64-X chips The following patch adds support for correctly recognizing SPARC-X chips. cpu : Unknown SUN4V CPU fpu : Unknown SUN4V FPU pmu : Unknown SUN4V PMU Signed-off-by: Katayama Yoshihiro <kata1@jp.fujitsu.com> Signed-off-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2013-03-06 07:47:59 +08:00
49:
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
mov SUN4V_CHIP_UNKNOWN, %g4
5: sethi %hi(sun4v_chip_type), %g2
or %g2, %lo(sun4v_chip_type), %g2
stw %g4, [%g2]
80:
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
BRANCH_IF_SUN4V(g1, jump_to_sun4u_init)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
BRANCH_IF_CHEETAH_BASE(g1,g7,cheetah_boot)
BRANCH_IF_CHEETAH_PLUS_OR_FOLLOWON(g1,g7,cheetah_plus_boot)
ba,pt %xcc, spitfire_boot
nop
cheetah_plus_boot:
/* Preserve OBP chosen DCU and DCR register settings. */
ba,pt %xcc, cheetah_generic_boot
nop
cheetah_boot:
mov DCR_BPE | DCR_RPE | DCR_SI | DCR_IFPOE | DCR_MS, %g1
wr %g1, %asr18
sethi %uhi(DCU_ME|DCU_RE|DCU_HPE|DCU_SPE|DCU_SL|DCU_WE), %g7
or %g7, %ulo(DCU_ME|DCU_RE|DCU_HPE|DCU_SPE|DCU_SL|DCU_WE), %g7
sllx %g7, 32, %g7
or %g7, DCU_DM | DCU_IM | DCU_DC | DCU_IC, %g7
stxa %g7, [%g0] ASI_DCU_CONTROL_REG
membar #Sync
cheetah_generic_boot:
mov TSB_EXTENSION_P, %g3
stxa %g0, [%g3] ASI_DMMU
stxa %g0, [%g3] ASI_IMMU
membar #Sync
mov TSB_EXTENSION_S, %g3
stxa %g0, [%g3] ASI_DMMU
membar #Sync
mov TSB_EXTENSION_N, %g3
stxa %g0, [%g3] ASI_DMMU
stxa %g0, [%g3] ASI_IMMU
membar #Sync
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
ba,a,pt %xcc, jump_to_sun4u_init
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
spitfire_boot:
/* Typically PROM has already enabled both MMU's and both on-chip
* caches, but we do it here anyway just to be paranoid.
*/
mov (LSU_CONTROL_IC|LSU_CONTROL_DC|LSU_CONTROL_IM|LSU_CONTROL_DM), %g1
stxa %g1, [%g0] ASI_LSU_CONTROL
membar #Sync
[SPARC64]: Rewrite bootup sequence. Instead of all of this cpu-specific code to remap the kernel to the correct location, use portable firmware calls to do this instead. What we do now is the following in position independant assembler: chosen_node = prom_finddevice("/chosen"); prom_mmu_ihandle_cache = prom_getint(chosen_node, "mmu"); vaddr = 4MB_ALIGN(current_text_addr()); prom_translate(vaddr, &paddr_high, &paddr_low, &mode); prom_boot_mapping_mode = mode; prom_boot_mapping_phys_high = paddr_high; prom_boot_mapping_phys_low = paddr_low; prom_map(-1, 8 * 1024 * 1024, KERNBASE, paddr_low); and that replaces the massive amount of by-hand TLB probing and programming we used to do here. The new code should also handle properly the case where the kernel is mapped at the correct address already (think: future kexec support). Consequently, the bulk of remap_kernel() dies as does the entirety of arch/sparc64/prom/map.S We try to share some strings in the PROM library with the ones used at bootup, and while we're here mark input strings to oplib.h routines with "const" when appropriate. There are many more simplifications now possible. For one thing, we can consolidate the two copies we now have of a lot of cpu setup code sitting in head.S and trampoline.S. This is a significant step towards CONFIG_DEBUG_PAGEALLOC support. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-23 11:11:33 +08:00
jump_to_sun4u_init:
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/*
* Make sure we are in privileged mode, have address masking,
* using the ordinary globals and have enabled floating
* point.
*
* Again, typically PROM has left %pil at 13 or similar, and
* (PSTATE_PRIV | PSTATE_PEF | PSTATE_IE) in %pstate.
*/
wrpr %g0, (PSTATE_PRIV|PSTATE_PEF|PSTATE_IE), %pstate
wr %g0, 0, %fprs
set sun4u_init, %g2
jmpl %g2 + %g0, %g0
nop
sparc: cleanup references to deprecated .text.init* sections. The section .text.init.refok is deprecated and __REF (.ref.text) should be used in assembly files instead. This patch cleans up a few uses of .text.init.refok in the sparc architecture. Also fix a reference to .text.init in a comment that wasn't updated to .init.text. Signed-off-by: Tim Abbott <tabbott@mit.edu> Cc: David S. Miller <davem@davemloft.net> Acked-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-04-28 02:02:26 +08:00
__REF
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sun4u_init:
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
BRANCH_IF_SUN4V(g1, sun4v_init)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/* Set ctx 0 */
[SPARC64]: Patch up mmu context register writes for sun4v. sun4v uses ASI_MMU instead of ASI_DMMU Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 14:13:05 +08:00
mov PRIMARY_CONTEXT, %g7
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
stxa %g0, [%g7] ASI_DMMU
[SPARC64]: Patch up mmu context register writes for sun4v. sun4v uses ASI_MMU instead of ASI_DMMU Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 14:13:05 +08:00
membar #Sync
mov SECONDARY_CONTEXT, %g7
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
stxa %g0, [%g7] ASI_DMMU
membar #Sync
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
ba,a,pt %xcc, sun4u_continue
[SPARC64]: Patch up mmu context register writes for sun4v. sun4v uses ASI_MMU instead of ASI_DMMU Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 14:13:05 +08:00
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
sun4v_init:
/* Set ctx 0 */
mov PRIMARY_CONTEXT, %g7
[SPARC64]: Patch up mmu context register writes for sun4v. sun4v uses ASI_MMU instead of ASI_DMMU Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 14:13:05 +08:00
stxa %g0, [%g7] ASI_MMU
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
membar #Sync
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
mov SECONDARY_CONTEXT, %g7
stxa %g0, [%g7] ASI_MMU
membar #Sync
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
ba,a,pt %xcc, niagara_tlb_fixup
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Fix sun4v early bootup. prom_sun4v_name should be "sun4v" not "SUNW,sun4v" Also, this is too early to make use of the .sun4v_Xinsn_patch code patching, so just check things manually. This gets us at least to prom_init() on Niagara. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-12 02:56:43 +08:00
sun4u_continue:
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
BRANCH_IF_ANY_CHEETAH(g1, g7, cheetah_tlb_fixup)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
ba,a,pt %xcc, spitfire_tlb_fixup
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Niagara copy/clear page. Happily we have no D-cache aliasing issues on these chips, so the implementation is very straightforward. Add a stub in bootup which will be where the patching calls will be made for niagara/sun4v/hypervisor. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 08:09:12 +08:00
niagara_tlb_fixup:
mov 3, %g2 /* Set TLB type to hypervisor. */
sethi %hi(tlb_type), %g1
stw %g2, [%g1 + %lo(tlb_type)]
/* Patch copy/clear ops. */
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
sethi %hi(sun4v_chip_type), %g1
lduw [%g1 + %lo(sun4v_chip_type)], %g1
cmp %g1, SUN4V_CHIP_NIAGARA1
be,pt %xcc, niagara_patch
cmp %g1, SUN4V_CHIP_NIAGARA2
[SPARC64]: Niagara-2 optimized copies. The bzero/memset implementation stays the same as Niagara-1. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-16 16:47:25 +08:00
be,pt %xcc, niagara2_patch
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
nop
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
cmp %g1, SUN4V_CHIP_NIAGARA3
sparc64: Future proof Niagara cpu detection. Recognize T4 and T5 chips. Treating them both as "T2 plus other stuff" should be extremely safe and make sure distributions will work when those chips actually ship to customers. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-12 01:42:20 +08:00
be,pt %xcc, niagara2_patch
nop
cmp %g1, SUN4V_CHIP_NIAGARA4
sparc64: Add SPARC-T4 optimized memcpy. Before After -------------- -------------- bw_tcp: 1288.53 MB/sec 1637.77 MB/sec bw_pipe: 1517.18 MB/sec 2107.61 MB/sec bw_unix: 1838.38 MB/sec 2640.91 MB/sec make -s -j128 allmodconfig 5min 49sec 5min 31sec Signed-off-by: David S. Miller <davem@davemloft.net>
2012-09-27 12:11:01 +08:00
be,pt %xcc, niagara4_patch
sparc64: Future proof Niagara cpu detection. Recognize T4 and T5 chips. Treating them both as "T2 plus other stuff" should be extremely safe and make sure distributions will work when those chips actually ship to customers. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-09-12 01:42:20 +08:00
nop
cmp %g1, SUN4V_CHIP_NIAGARA5
sparc64: correctly recognise M6 and M7 cpu type The following patch adds support for correctly recognising M6 and M7 cpu type. Signed-off-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-09-08 14:18:53 +08:00
be,pt %xcc, niagara4_patch
nop
cmp %g1, SUN4V_CHIP_SPARC_M6
be,pt %xcc, niagara4_patch
nop
cmp %g1, SUN4V_CHIP_SPARC_M7
sparc64: recognize and support Sonoma CPU type Add code to recognize SPARC-Sonoma cpu correctly and update cpu hardware caps and cpu distribution map. SPARC-Sonoma is based upon SPARC-M7 core along with additional PCI functions added on and is reported by firmware as "SPARC-SN". Signed-off-by: Khalid Aziz <khalid.aziz@oracle.com> Acked-by: Allen Pais <allen.pais@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-20 01:12:54 +08:00
be,pt %xcc, niagara4_patch
nop
cmp %g1, SUN4V_CHIP_SPARC_SN
sparc64: Add SPARC-T4 optimized memcpy. Before After -------------- -------------- bw_tcp: 1288.53 MB/sec 1637.77 MB/sec bw_pipe: 1517.18 MB/sec 2107.61 MB/sec bw_unix: 1838.38 MB/sec 2640.91 MB/sec make -s -j128 allmodconfig 5min 49sec 5min 31sec Signed-off-by: David S. Miller <davem@davemloft.net>
2012-09-27 12:11:01 +08:00
be,pt %xcc, niagara4_patch
sparc: Detect and handle UltraSPARC-T3 cpu types. The cpu compatible string we look for is "SPARC-T3". As far as memset/memcpy optimizations go, we treat this chip the same as Niagara-T2/T2+. Use cache initializing stores for memset, and use perfetch, FPU block loads, cache initializing stores, and block stores for copies. We use the Niagara-T2 perf support, since T3 is a close relative in this regard. Later we'll add support for the new events T3 can report, plus enable T3's new "sample" mode. For now I haven't added any new ELF hwcap flags. We probably need to add a couple, for example: T2 and T3 both support the population count instruction in hardware. T3 supports VIS3 instructions, including support (finally) for partitioned shift. One can also now move directly between float and integer registers. T3 supports instructions meant to help with Galois Field and other HPC calculations, such as XOR multiply. Also there are "OP and negate" instructions, for example "fnmul" which is multiply-and-negate. T3 recognizes the transactional memory opcodes, however since transactional memory isn't supported: 1) 'commit' behaves as a NOP and 2) 'chkpt' always branches 3) 'rdcps' returns all zeros and 4) 'wrcps' behaves as a NOP. So we'll need about 3 new elf capability flags in the end to represent all of these things. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-07-28 12:06:16 +08:00
nop
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
call generic_patch_copyops
nop
call generic_patch_bzero
nop
call generic_patch_pageops
nop
ba,a,pt %xcc, 80f
sparc64: Add SPARC-T4 optimized memcpy. Before After -------------- -------------- bw_tcp: 1288.53 MB/sec 1637.77 MB/sec bw_pipe: 1517.18 MB/sec 2107.61 MB/sec bw_unix: 1838.38 MB/sec 2640.91 MB/sec make -s -j128 allmodconfig 5min 49sec 5min 31sec Signed-off-by: David S. Miller <davem@davemloft.net>
2012-09-27 12:11:01 +08:00
niagara4_patch:
call niagara4_patch_copyops
nop
sparc64: Niagara-4 bzero/memset, plus use MRU stores in page copy. This adds optimized memset/bzero/page-clear routines for Niagara-4. We basically can do what powerpc has been able to do for a decade (via the "dcbz" instruction), which is use cache line clearing stores for bzero and memsets with a 'c' argument of zero. As long as we make the cache initializing store to each 32-byte subblock of the L2 cache line, it works. As with other Niagara-4 optimized routines, the key is to make sure to avoid any usage of the %asi register, as reads and writes to it cost at least 50 cycles. For the user clear cases, we don't use these new routines, we use the Niagara-1 variants instead. Those have to use %asi in an unavoidable way. A Niagara-4 8K page clear costs just under 600 cycles. Add definitions of the MRU variants of the cache initializing store ASIs. By default, cache initializing stores install the line as Least Recently Used. If we know we're going to use the data immediately (which is true for page copies and clears) we can use the Most Recently Used variant, to decrease the likelyhood of the lines being evicted before they get used. Signed-off-by: David S. Miller <davem@davemloft.net>
2012-10-06 04:45:26 +08:00
call niagara4_patch_bzero
sparc64: Add SPARC-T4 optimized memcpy. Before After -------------- -------------- bw_tcp: 1288.53 MB/sec 1637.77 MB/sec bw_pipe: 1517.18 MB/sec 2107.61 MB/sec bw_unix: 1838.38 MB/sec 2640.91 MB/sec make -s -j128 allmodconfig 5min 49sec 5min 31sec Signed-off-by: David S. Miller <davem@davemloft.net>
2012-09-27 12:11:01 +08:00
nop
call niagara4_patch_pageops
nop
ba,a,pt %xcc, 80f
[SPARC64]: Niagara-2 optimized copies. The bzero/memset implementation stays the same as Niagara-1. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-16 16:47:25 +08:00
niagara2_patch:
call niagara2_patch_copyops
nop
call niagara_patch_bzero
nop
sparc: Minor tweaks to Niagara page copy/clear. Don't use floating point on Niagara2, use the traditional plain Niagara code instead. Unroll Niagara loops to 128 bytes for copy, and 256 bytes for clear. Signed-off-by: David S. Miller <davem@davemloft.net>
2011-08-02 09:18:57 +08:00
call niagara_patch_pageops
[SPARC64]: Niagara-2 optimized copies. The bzero/memset implementation stays the same as Niagara-1. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-16 16:47:25 +08:00
nop
ba,a,pt %xcc, 80f
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
niagara_patch:
[SPARC64]: Niagara copy/clear page. Happily we have no D-cache aliasing issues on these chips, so the implementation is very straightforward. Add a stub in bootup which will be where the patching calls will be made for niagara/sun4v/hypervisor. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 08:09:12 +08:00
call niagara_patch_copyops
nop
[SPARC64]: Niagara optimized memset/bzero/clear_user. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-22 06:29:42 +08:00
call niagara_patch_bzero
nop
[SPARC64]: Niagara copy/clear page. Happily we have no D-cache aliasing issues on these chips, so the implementation is very straightforward. Add a stub in bootup which will be where the patching calls will be made for niagara/sun4v/hypervisor. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 08:09:12 +08:00
call niagara_patch_pageops
nop
[SPARC64]: Do not assume sun4v chips have load-twin/store-init support. Check the cpu type in the OBP device tree before committing to using the optimized Niagara memcpy and memset implementation. If we don't recognize the cpu type, use a completely generic version. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-09 08:11:39 +08:00
80:
[SPARC64]: Niagara copy/clear page. Happily we have no D-cache aliasing issues on these chips, so the implementation is very straightforward. Add a stub in bootup which will be where the patching calls will be made for niagara/sun4v/hypervisor. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 08:09:12 +08:00
/* Patch TLB/cache ops. */
call hypervisor_patch_cachetlbops
nop
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
ba,a,pt %xcc, tlb_fixup_done
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
cheetah_tlb_fixup:
mov 2, %g2 /* Set TLB type to cheetah+. */
BRANCH_IF_CHEETAH_PLUS_OR_FOLLOWON(g1,g7,1f)
mov 1, %g2 /* Set TLB type to cheetah. */
1: sethi %hi(tlb_type), %g1
stw %g2, [%g1 + %lo(tlb_type)]
[SPARC64]: Replace cheetah+ code patching with variables. Instead of code patching to handle the page size fields in the context registers, just use variables from which we get the proper values. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-05 06:23:20 +08:00
/* Patch copy/page operations to cheetah optimized versions. */
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
call cheetah_patch_copyops
nop
[SPARC64]: Kill BRANCH_IF_ANY_CHEETAH() from copy page. Just patch the branch at boot time instead. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-08-31 02:26:15 +08:00
call cheetah_patch_copy_page
nop
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
call cheetah_patch_cachetlbops
nop
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
ba,a,pt %xcc, tlb_fixup_done
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
spitfire_tlb_fixup:
/* Set TLB type to spitfire. */
mov 0, %g2
sethi %hi(tlb_type), %g1
stw %g2, [%g1 + %lo(tlb_type)]
tlb_fixup_done:
sethi %hi(init_thread_union), %g6
or %g6, %lo(init_thread_union), %g6
ldx [%g6 + TI_TASK], %g4
wr %g0, ASI_P, %asi
mov 1, %g1
sllx %g1, THREAD_SHIFT, %g1
sub %g1, (STACKFRAME_SZ + STACK_BIAS), %g1
add %g6, %g1, %sp
/* Set per-cpu pointer initially to zero, this makes
* the boot-cpu use the in-kernel-image per-cpu areas
* before setup_per_cpu_area() is invoked.
*/
clr %g5
wrpr %g0, 0, %wstate
wrpr %g0, 0x0, %tl
/* Clear the bss */
sethi %hi(__bss_start), %o0
or %o0, %lo(__bss_start), %o0
sethi %hi(_end), %o1
or %o1, %lo(_end), %o1
call __bzero
sub %o1, %o0, %o1
call prom_init
mov %l7, %o0 ! OpenPROM cif handler
sparc64: Fix register corruption in top-most kernel stack frame during boot. Meelis Roos reported that kernels built with gcc-4.9 do not boot, we eventually narrowed this down to only impacting machines using UltraSPARC-III and derivitive cpus. The crash happens right when the first user process is spawned: [ 54.451346] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 [ 54.451346] [ 54.571516] CPU: 1 PID: 1 Comm: init Not tainted 3.16.0-rc2-00211-gd7933ab #96 [ 54.666431] Call Trace: [ 54.698453] [0000000000762f8c] panic+0xb0/0x224 [ 54.759071] [000000000045cf68] do_exit+0x948/0x960 [ 54.823123] [000000000042cbc0] fault_in_user_windows+0xe0/0x100 [ 54.902036] [0000000000404ad0] __handle_user_windows+0x0/0x10 [ 54.978662] Press Stop-A (L1-A) to return to the boot prom [ 55.050713] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Further investigation showed that compiling only per_cpu_patch() with an older compiler fixes the boot. Detailed analysis showed that the function is not being miscompiled by gcc-4.9, but it is using a different register allocation ordering. With the gcc-4.9 compiled function, something during the code patching causes some of the %i* input registers to get corrupted. Perhaps we have a TLB miss path into the firmware that is deep enough to cause a register window spill and subsequent restore when we get back from the TLB miss trap. Let's plug this up by doing two things: 1) Stop using the firmware stack for client interface calls into the firmware. Just use the kernel's stack. 2) As soon as we can, call into a new function "start_early_boot()" to put a one-register-window buffer between the firmware's deepest stack frame and the top-most initial kernel one. Reported-by: Meelis Roos <mroos@linux.ee> Tested-by: Meelis Roos <mroos@linux.ee> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-10-24 03:58:13 +08:00
/* To create a one-register-window buffer between the kernel's
* initial stack and the last stack frame we use from the firmware,
* do the rest of the boot from a C helper function.
[SPARC64]: Make smp_processor_id() functional before start_kernel() Uses of smp_processor_id() get pushed earlier and earlier in the start_kernel() sequence. So just get it working before we call start_kernel() to avoid all possible problems. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-05-31 16:24:02 +08:00
*/
sparc64: Fix register corruption in top-most kernel stack frame during boot. Meelis Roos reported that kernels built with gcc-4.9 do not boot, we eventually narrowed this down to only impacting machines using UltraSPARC-III and derivitive cpus. The crash happens right when the first user process is spawned: [ 54.451346] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 [ 54.451346] [ 54.571516] CPU: 1 PID: 1 Comm: init Not tainted 3.16.0-rc2-00211-gd7933ab #96 [ 54.666431] Call Trace: [ 54.698453] [0000000000762f8c] panic+0xb0/0x224 [ 54.759071] [000000000045cf68] do_exit+0x948/0x960 [ 54.823123] [000000000042cbc0] fault_in_user_windows+0xe0/0x100 [ 54.902036] [0000000000404ad0] __handle_user_windows+0x0/0x10 [ 54.978662] Press Stop-A (L1-A) to return to the boot prom [ 55.050713] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Further investigation showed that compiling only per_cpu_patch() with an older compiler fixes the boot. Detailed analysis showed that the function is not being miscompiled by gcc-4.9, but it is using a different register allocation ordering. With the gcc-4.9 compiled function, something during the code patching causes some of the %i* input registers to get corrupted. Perhaps we have a TLB miss path into the firmware that is deep enough to cause a register window spill and subsequent restore when we get back from the TLB miss trap. Let's plug this up by doing two things: 1) Stop using the firmware stack for client interface calls into the firmware. Just use the kernel's stack. 2) As soon as we can, call into a new function "start_early_boot()" to put a one-register-window buffer between the firmware's deepest stack frame and the top-most initial kernel one. Reported-by: Meelis Roos <mroos@linux.ee> Tested-by: Meelis Roos <mroos@linux.ee> Signed-off-by: David S. Miller <davem@davemloft.net>
2014-10-24 03:58:13 +08:00
call start_early_boot
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
nop
/* Not reached... */
[SPARC64]: Mark most of initial bootup asm as .text.init.ref_ok We can't mark the whole thing init because there are dependencies in bootloaders that assume that _start, or whatever the image entry value, is 2 instructions before the "HdrS" signature. In fact, TILO assumes this entry is always at 0x4000, yikes! Also, right after the bootloader info area there are OBP strings and values that get used later in the boot process, and those are not all provably .init yet. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-07-25 06:17:33 +08:00
.previous
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
/* This is meant to allow the sharing of this code between
* boot processor invocation (via setup_tba() below) and
* secondary processor startup (via trampoline.S). The
* former does use this code, the latter does not yet due
* to some complexities. That should be fixed up at some
* point.
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
*
* There used to be enormous complexity wrt. transferring
trivial: Fix misspelling of firmware Fix misspelling of firmware. Signed-off-by: Nick Andrew <nick@nick-andrew.net> Signed-off-by: Jiri Kosina <jkosina@suse.cz>
2009-01-26 18:06:57 +08:00
* over from the firmware's trap table to the Linux kernel's.
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
* For example, there was a chicken & egg problem wrt. building
* the OBP page tables, yet needing to be on the Linux kernel
* trap table (to translate PAGE_OFFSET addresses) in order to
* do that.
*
* We now handle OBP tlb misses differently, via linear lookups
* into the prom_trans[] array. So that specific problem no
* longer exists. Yet, unfortunately there are still some issues
* preventing trampoline.S from using this code... ho hum.
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
*/
.globl setup_trap_table
setup_trap_table:
save %sp, -192, %sp
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
/* Force interrupts to be disabled. */
[SPARC64]: Make sure IRQs are disabled properly during early boot. Else we trigger the new irqs_disable() assertion in start_kernel(). Signed-off-by: David S. Miller <davem@davemloft.net>
2006-07-14 07:05:26 +08:00
rdpr %pstate, %l0
andn %l0, PSTATE_IE, %o1
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
wrpr %o1, 0x0, %pstate
[SPARC64]: Make sure IRQs are disabled properly during early boot. Else we trigger the new irqs_disable() assertion in start_kernel(). Signed-off-by: David S. Miller <davem@davemloft.net>
2006-07-14 07:05:26 +08:00
rdpr %pil, %l1
sparc64: Make %pil level 15 a pseudo-NMI. So that we can profile code even in a local_irq_disable() section, only write 14 (instead of 15) into the %pil register to disable IRQs. This allows PIL level 15 to serve as a pseudo NMI. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-11-24 13:55:29 +08:00
wrpr %g0, PIL_NORMAL_MAX, %pil
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
/* Make the firmware call to jump over to the Linux trap table. */
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 07:39:51 +08:00
sethi %hi(is_sun4v), %o0
lduw [%o0 + %lo(is_sun4v)], %o0
brz,pt %o0, 1f
nop
TRAP_LOAD_TRAP_BLOCK(%g2, %g3)
add %g2, TRAP_PER_CPU_FAULT_INFO, %g2
stxa %g2, [%g0] ASI_SCRATCHPAD
/* Compute physical address:
*
* paddr = kern_base + (mmfsa_vaddr - KERNBASE)
*/
sethi %hi(KERNBASE), %g3
sub %g2, %g3, %g2
sethi %hi(kern_base), %g3
ldx [%g3 + %lo(kern_base)], %g3
add %g2, %g3, %o1
[SPARC64]: Fix lockdep, particularly on SMP. As noted by Al Viro, when we try to call prom_set_trap_table() in the SMP trampoline code we try to take the PROM call spinlock which doesn't work because the current thread pointer isn't valid yet and lockdep depends upon that being correct. Furthermore, we cannot set the current thread pointer register because it can't be properly dereferenced until we return from prom_set_trap_table(). Kernel TLB misses only work after that call. So do the PROM call to set the trap table directly instead of going through the OBP library C code, and thus avoid the lock altogether. These calls are guarenteed to be serialized fully. Since there are now no calls to the prom_set_trap_table{_sun4v}() library functions, they can be deleted. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-09-17 02:51:15 +08:00
sethi %hi(sparc64_ttable_tl0), %o0
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 07:39:51 +08:00
[SPARC64]: Fix lockdep, particularly on SMP. As noted by Al Viro, when we try to call prom_set_trap_table() in the SMP trampoline code we try to take the PROM call spinlock which doesn't work because the current thread pointer isn't valid yet and lockdep depends upon that being correct. Furthermore, we cannot set the current thread pointer register because it can't be properly dereferenced until we return from prom_set_trap_table(). Kernel TLB misses only work after that call. So do the PROM call to set the trap table directly instead of going through the OBP library C code, and thus avoid the lock altogether. These calls are guarenteed to be serialized fully. Since there are now no calls to the prom_set_trap_table{_sun4v}() library functions, they can be deleted. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-09-17 02:51:15 +08:00
set prom_set_trap_table_name, %g2
stx %g2, [%sp + 2047 + 128 + 0x00]
mov 2, %g2
stx %g2, [%sp + 2047 + 128 + 0x08]
mov 0, %g2
stx %g2, [%sp + 2047 + 128 + 0x10]
stx %o0, [%sp + 2047 + 128 + 0x18]
stx %o1, [%sp + 2047 + 128 + 0x20]
sethi %hi(p1275buf), %g2
or %g2, %lo(p1275buf), %g2
ldx [%g2 + 0x08], %o1
call %o1
add %sp, (2047 + 128), %o0
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 07:39:51 +08:00
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
ba,a,pt %xcc, 2f
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 07:39:51 +08:00
[SPARC64]: Fix lockdep, particularly on SMP. As noted by Al Viro, when we try to call prom_set_trap_table() in the SMP trampoline code we try to take the PROM call spinlock which doesn't work because the current thread pointer isn't valid yet and lockdep depends upon that being correct. Furthermore, we cannot set the current thread pointer register because it can't be properly dereferenced until we return from prom_set_trap_table(). Kernel TLB misses only work after that call. So do the PROM call to set the trap table directly instead of going through the OBP library C code, and thus avoid the lock altogether. These calls are guarenteed to be serialized fully. Since there are now no calls to the prom_set_trap_table{_sun4v}() library functions, they can be deleted. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-09-17 02:51:15 +08:00
1: sethi %hi(sparc64_ttable_tl0), %o0
set prom_set_trap_table_name, %g2
stx %g2, [%sp + 2047 + 128 + 0x00]
mov 1, %g2
stx %g2, [%sp + 2047 + 128 + 0x08]
mov 0, %g2
stx %g2, [%sp + 2047 + 128 + 0x10]
stx %o0, [%sp + 2047 + 128 + 0x18]
sethi %hi(p1275buf), %g2
or %g2, %lo(p1275buf), %g2
ldx [%g2 + 0x08], %o1
call %o1
add %sp, (2047 + 128), %o0
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
/* Start using proper page size encodings in ctx register. */
[SPARC64]: Use ASI_SCRATCHPAD address 0x0 properly. This is where the virtual address of the fault status area belongs. To set it up we don't make a hypervisor call, instead we call OBP's SUNW,set-trap-table with the real address of the fault status area as the second argument. And right before that call we write the virtual address into ASI_SCRATCHPAD vaddr 0x0. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-11 07:39:51 +08:00
2: sethi %hi(sparc64_kern_pri_context), %g3
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
ldx [%g3 + %lo(sparc64_kern_pri_context)], %g2
[SPARC64]: Patch up mmu context register writes for sun4v. sun4v uses ASI_MMU instead of ASI_DMMU Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 14:13:05 +08:00
mov PRIMARY_CONTEXT, %g1
661: stxa %g2, [%g1] ASI_DMMU
.section .sun4v_1insn_patch, "ax"
.word 661b
stxa %g2, [%g1] ASI_MMU
.previous
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
membar #Sync
[SPARC64]: Do not touch %tick_cmpr on sun4v cpus. This register is not a part of the sun4v architecture. Niagara 1 and 2 happened to leave it around. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-08-16 16:52:44 +08:00
BRANCH_IF_SUN4V(o2, 1f)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/* Kill PROM timer */
sethi %hi(0x80000000), %o2
sllx %o2, 32, %o2
wr %o2, 0, %tick_cmpr
[SPARC64]: Detect sun4v early in boot process. We look for "SUNW,sun4v" in the 'compatible' property of the root OBP device tree node. Protect every %ver register access, to make sure it is not touched on sun4v, as %ver is hyperprivileged there. Lock kernel TLB entries using hypervisor calls instead of calls into OBP. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-09 18:52:44 +08:00
BRANCH_IF_ANY_CHEETAH(o2, o3, 1f)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
sparc64: Fix bootup regressions on some Kconfig combinations. The system call tracing bug fix mentioned in the Fixes tag below increased the amount of assembler code in the sequence of assembler files included by head_64.S This caused to total set of code to exceed 0x4000 bytes in size, which overflows the expression in head_64.S that works to place swapper_tsb at address 0x408000. When this is violated, the TSB is not properly aligned, and also the trap table is not aligned properly either. All of this together results in failed boots. So, do two things: 1) Simplify some code by using ba,a instead of ba/nop to get those bytes back. 2) Add a linker script assertion to make sure that if this happens again the build will fail. Fixes: 1a40b95374f6 ("sparc: Fix system call tracing register handling.") Reported-by: Meelis Roos <mroos@linux.ee> Reported-by: Joerg Abraham <joerg.abraham@nokia.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2016-04-28 05:27:37 +08:00
ba,a,pt %xcc, 2f
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/* Disable STICK_INT interrupts. */
1:
sethi %hi(0x80000000), %o2
sllx %o2, 32, %o2
wr %o2, %asr25
2:
wrpr %g0, %g0, %wstate
call init_irqwork_curcpu
nop
[SPARC64]: Make sure IRQs are disabled properly during early boot. Else we trigger the new irqs_disable() assertion in start_kernel(). Signed-off-by: David S. Miller <davem@davemloft.net>
2006-07-14 07:05:26 +08:00
/* Now we can restore interrupt state. */
wrpr %l0, 0, %pstate
wrpr %l1, 0x0, %pil
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
ret
restore
.globl setup_tba
[SPARC64]: Kill sole argument passed to setup_tba(). No longer used, and move extern declaration to a header file. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-01 10:33:37 +08:00
setup_tba:
[SPARC64]: Fix Ultra5, Ultra60, et al. boot failures. On the boot processor, we need to do the move onto the Linux trap table a little bit differently else we'll take unhandlable faults in the firmware address space. Previously we would do the following: 1) Disable PSTATE_IE in %pstate. 2) Set %tba by hand to sparc64_ttable_tl0 3) Initialize alternate, mmu, and interrupt global trap registers. 4) Call prom_set_traptable() That doesn't work very well actually with the way we boot the kernel VM these days. It worked by luck on many systems because the firmware accesses for the prom_set_traptable() call happened to be loaded into the TLB already, something we cannot assume. So the new scheme is this: 1) Clear PSTATE_IE in %pstate and set %pil to 15 2) Call prom_set_traptable() 3) Initialize alternate, mmu, and interrupt global trap registers. and this works quite well. This sequence has been moved into a callable function in assembler named setup-trap_table(). The idea is that eventually trampoline.S can use this code as well. That isn't possible currently due to some complications, but eventually we should be able to do it. Thanks to Meelis Roos for the Ultra5 boot failure report. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-11 07:12:13 +08:00
save %sp, -192, %sp
/* The boot processor is the only cpu which invokes this
* routine, the other cpus set things up via trampoline.S.
* So save the OBP trap table address here.
*/
rdpr %tba, %g7
sethi %hi(prom_tba), %o1
or %o1, %lo(prom_tba), %o1
stx %g7, [%o1]
call setup_trap_table
nop
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
ret
restore
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
sparc64_boot_end:
sparc,sparc64: unify kernel/ o Move all files from sparc64/kernel/ to sparc/kernel - rename as appropriate o Update sparc/Makefile to the changes o Update sparc/kernel/Makefile to include the sparc64 files NOTE: This commit changes link order on sparc64! Link order had to change for either of sparc32 and sparc64. And assuming sparc64 see more testing than sparc32 change link order on sparc64 where issues will be caught faster. Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2008-12-03 19:11:52 +08:00
#include "etrap_64.S"
#include "rtrap_64.S"
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
#include "winfixup.S"
sparc64: Split entry.S up into seperate files. entry.S was a hodge-podge of several totally unrelated sets of assembler routines, ranging from FPU trap handlers to hypervisor call functions. Split it up into topic-sized pieces. Signed-off-by: David S. Miller <davem@davemloft.net>
2008-04-28 15:47:20 +08:00
#include "fpu_traps.S"
#include "ivec.S"
#include "getsetcc.S"
#include "utrap.S"
#include "spiterrs.S"
#include "cherrs.S"
#include "misctrap.S"
#include "syscalls.S"
#include "helpers.S"
#include "hvcalls.S"
[SPARC64]: Sun4v interrupt handling. Sun4v has 4 interrupt queues: cpu, device, resumable errors, and non-resumable errors. A set of head/tail offset pointers help maintain a work queue in physical memory. The entries are 64-bytes in size. Each queue is allocated then registered with the hypervisor as we bring cpus up. The two error queues each get a kernel side buffer that we use to quickly empty the main interrupt queue before we call up to C code to log the event and possibly take evasive action. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-08 18:53:50 +08:00
#include "sun4v_tlb_miss.S"
#include "sun4v_ivec.S"
[SPARC64]: Fix two bugs wrt. kernel 4MB TSB. 1) The TSB lookup was not using the correct hash mask. 2) It was not aligned on a boundary equal to it's size, which is required by the sun4v Hypervisor. wasn't having it's return value checked, and that bug will be fixed up as well in a subsequent changeset. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-05-29 16:58:31 +08:00
#include "ktlb.S"
#include "tsb.S"
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
/*
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
* The following skip makes sure the trap table in ttable.S is aligned
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
* on a 32K boundary as required by the v9 specs for TBA register.
[SPARC64]: Increase swapper_tsb size to 32K. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-01 10:33:49 +08:00
*
* We align to a 32K boundary, then we have the 32K kernel TSB,
[SPARC64]: Fix two bugs wrt. kernel 4MB TSB. 1) The TSB lookup was not using the correct hash mask. 2) It was not aligned on a boundary equal to it's size, which is required by the sun4v Hypervisor. wasn't having it's return value checked, and that bug will be fixed up as well in a subsequent changeset. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-05-29 16:58:31 +08:00
* the 64K kernel 4MB TSB, and then the 32K aligned trap table.
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
*/
[SPARC64]: Fix boot failures on SunBlade-150 The sequence to move over to the Linux trap tables from the firmware ones needs to be more air tight. It turns out that to be %100 safe we do need to be able to translate OBP mappings in our TLB miss handlers early. In order not to eat up a lot of kernel image memory with static page tables, just use the translations array in the OBP TLB miss handlers. That solves the bulk of the problem. Furthermore, to make sure the OBP TLB miss path will work even before the fixed MMU globals are loaded, explicitly load %g1 to TLB_SFSR at the beginning of the i-TLB and d-TLB miss handlers. To ease the OBP TLB miss walking of the prom_trans[] array, we sort it then delete all of the non-OBP entries in there (for example, there are entries for the kernel image itself which we're not interested in at all). We also save about 32K of kernel image size with this change. Not a bad side effect :-) There are still some reasons why trampoline.S can't use the setup_trap_table() yet. The most noteworthy are: 1) OBP boots secondary processors with non-bias'd stack for some reason. This is easily fixed by using a small bootup stack in the kernel image explicitly for this purpose. 2) Doing a firmware call via the normal C call prom_set_trap_table() goes through the whole OBP enter/exit sequence that saves and restores OBP and Linux kernel state in the MMUs. This path unfortunately does a "flush %g6" while loading up the OBP locked TLB entries for the firmware call. If we setup the %g6 in the trampoline.S code properly, that is in the PAGE_OFFSET linear mapping, but we're not on the kernel trap table yet so those addresses won't translate properly. One idea is to do a by-hand firmware call like we do in the early bootup code and elsewhere here in trampoline.S But this fails as well, as aparently the secondary processors are not booted with OBP's special locked TLB entries loaded. These are necessary for the firwmare to processes TLB misses correctly up until the point where we take over the trap table. This does need to be resolved at some point. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-10-13 03:22:46 +08:00
1:
.skip 0x4000 + _start - 1b
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Fix two bugs wrt. kernel 4MB TSB. 1) The TSB lookup was not using the correct hash mask. 2) It was not aligned on a boundary equal to it's size, which is required by the sun4v Hypervisor. wasn't having it's return value checked, and that bug will be fixed up as well in a subsequent changeset. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-05-29 16:58:31 +08:00
! 0x0000000000408000
[SPARC64]: Increase swapper_tsb size to 32K. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-01 10:33:49 +08:00
.globl swapper_tsb
swapper_tsb:
.skip (32 * 1024)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Fix two bugs wrt. kernel 4MB TSB. 1) The TSB lookup was not using the correct hash mask. 2) It was not aligned on a boundary equal to it's size, which is required by the sun4v Hypervisor. wasn't having it's return value checked, and that bug will be fixed up as well in a subsequent changeset. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-05-29 16:58:31 +08:00
.globl swapper_4m_tsb
swapper_4m_tsb:
.skip (64 * 1024)
! 0x0000000000420000
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Fix two bugs wrt. kernel 4MB TSB. 1) The TSB lookup was not using the correct hash mask. 2) It was not aligned on a boundary equal to it's size, which is required by the sun4v Hypervisor. wasn't having it's return value checked, and that bug will be fixed up as well in a subsequent changeset. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-05-29 16:58:31 +08:00
/* Some care needs to be exercised if you try to move the
* location of the trap table relative to other things. For
* one thing there are br* instructions in some of the
* trap table entires which branch back to code in ktlb.S
* Those instructions can only handle a signed 16-bit
* displacement.
*
* There is a binutils bug (bugzilla #4558) which causes
* the relocation overflow checks for such instructions to
* not be done correctly. So bintuils will not notice the
* error and will instead write junk into the relocation and
* you'll have an unbootable kernel.
*/
sparc64: renamed ttable.S to ttable_64.S To allow us to add ttable_32.S Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2012-05-20 04:02:44 +08:00
#include "ttable_64.S"
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
[SPARC64]: Fix two bugs wrt. kernel 4MB TSB. 1) The TSB lookup was not using the correct hash mask. 2) It was not aligned on a boundary equal to it's size, which is required by the sun4v Hypervisor. wasn't having it's return value checked, and that bug will be fixed up as well in a subsequent changeset. Signed-off-by: David S. Miller <davem@davemloft.net>
2007-05-29 16:58:31 +08:00
! 0x0000000000428000
sparc,sparc64: unify kernel/ o Move all files from sparc64/kernel/ to sparc/kernel - rename as appropriate o Update sparc/Makefile to the changes o Update sparc/kernel/Makefile to include the sparc64 files NOTE: This commit changes link order on sparc64! Link order had to change for either of sparc32 and sparc64. And assuming sparc64 see more testing than sparc32 change link order on sparc64 where issues will be caught faster. Signed-off-by: Sam Ravnborg <sam@ravnborg.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2008-12-03 19:11:52 +08:00
#include "systbls_64.S"
[SPARC64]: Put syscall tables after trap table. Otherwise with too much stuff enabled in the kernel config we can end up with an unaligned trap table. Signed-off-by: David S. Miller <davem@davemloft.net>
2006-02-23 18:28:25 +08:00
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
.data
.align 8
.globl prom_tba, tlb_type
prom_tba: .xword 0
tlb_type: .word 0 /* Must NOT end up in BSS */
.section ".fixup",#alloc,#execinstr
[SPARC64]: Fix fault handling in unaligned trap handler. We were not calling kernel_mna_trap_fault() correctly. Instead of being fancy, just return 0 vs. -EFAULT from the assembler stubs, and handle that return value as appropriate. Create an "__retl_efault" stub for assembler exception table entries and use it where possible. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-29 11:41:45 +08:00
sparc64: Kill .fixup section bloat. This is an implementation of a suggestion made by Chris Torek: -------------------- Something else I noticed in passing: the EX and EX_LD/EX_ST macros scattered throughout the various .S files make a fair bit of .fixup code, all of which does the same thing. At the cost of one symbol in copy_in_user.S, you could just have one common two-instruction retl-and-mov-1 fixup that they all share. -------------------- The following is with a defconfig build: text data bss dec hex filename 3972767 344024 584449 4901240 4ac978 vmlinux.orig 3968887 344024 584449 4897360 4aba50 vmlinux Signed-off-by: David S. Miller <davem@davemloft.net>
2009-02-09 14:00:55 +08:00
.globl __ret_efault, __retl_efault, __ret_one, __retl_one
ENTRY(__ret_efault)
Linux-2.6.12-rc2 Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
2005-04-17 06:20:36 +08:00
ret
restore %g0, -EFAULT, %o0
sparc64: Kill .fixup section bloat. This is an implementation of a suggestion made by Chris Torek: -------------------- Something else I noticed in passing: the EX and EX_LD/EX_ST macros scattered throughout the various .S files make a fair bit of .fixup code, all of which does the same thing. At the cost of one symbol in copy_in_user.S, you could just have one common two-instruction retl-and-mov-1 fixup that they all share. -------------------- The following is with a defconfig build: text data bss dec hex filename 3972767 344024 584449 4901240 4ac978 vmlinux.orig 3968887 344024 584449 4897360 4aba50 vmlinux Signed-off-by: David S. Miller <davem@davemloft.net>
2009-02-09 14:00:55 +08:00
ENDPROC(__ret_efault)
ENTRY(__retl_efault)
[SPARC64]: Fix fault handling in unaligned trap handler. We were not calling kernel_mna_trap_fault() correctly. Instead of being fancy, just return 0 vs. -EFAULT from the assembler stubs, and handle that return value as appropriate. Create an "__retl_efault" stub for assembler exception table entries and use it where possible. Signed-off-by: David S. Miller <davem@davemloft.net>
2005-09-29 11:41:45 +08:00
retl
mov -EFAULT, %o0
sparc64: Kill .fixup section bloat. This is an implementation of a suggestion made by Chris Torek: -------------------- Something else I noticed in passing: the EX and EX_LD/EX_ST macros scattered throughout the various .S files make a fair bit of .fixup code, all of which does the same thing. At the cost of one symbol in copy_in_user.S, you could just have one common two-instruction retl-and-mov-1 fixup that they all share. -------------------- The following is with a defconfig build: text data bss dec hex filename 3972767 344024 584449 4901240 4ac978 vmlinux.orig 3968887 344024 584449 4897360 4aba50 vmlinux Signed-off-by: David S. Miller <davem@davemloft.net>
2009-02-09 14:00:55 +08:00
ENDPROC(__retl_efault)
ENTRY(__retl_one)
retl
mov 1, %o0
ENDPROC(__retl_one)
sparc64: fix FP corruption in user copy functions Short story: Exception handlers used by some copy_to_user() and copy_from_user() functions do not diligently clean up floating point register usage, and this can result in a user process seeing invalid values in floating point registers. This sometimes makes the process fail. Long story: Several cpu-specific (NG4, NG2, U1, U3) memcpy functions use floating point registers and VIS alignaddr/faligndata to accelerate data copying when source and dest addresses don't align well. Linux uses a lazy scheme for saving floating point registers; It is not done upon entering the kernel since it's a very expensive operation. Rather, it is done only when needed. If the kernel ends up not using FP regs during the course of some trap or system call, then it can return to user space without saving or restoring them. The various memcpy functions begin their FP code with VISEntry (or a variation thereof), which saves the FP regs. They conclude their FP code with VISExit (or a variation) which essentially marks the FP regs "clean", ie, they contain no unsaved values. fprs.FPRS_FEF is turned off so that a lazy restore will be triggered when/if the user process accesses floating point regs again. The bug is that the user copy variants of memcpy, copy_from_user() and copy_to_user(), employ an exception handling mechanism to detect faults when accessing user space addresses, and when this handler is invoked, an immediate return from the function is forced, and VISExit is not executed, thus leaving the fprs register in an indeterminate state, but often with fprs.FPRS_FEF set and one or more dirty bits. This results in a return to user space with invalid values in the FP regs, and since fprs.FPRS_FEF is on, no lazy restore occurs. This bug affects copy_to_user() and copy_from_user() for NG4, NG2, U3, and U1. All are fixed by using a new exception handler for those loads and stores that are done during the time between VISEnter and VISExit. n.b. In NG4memcpy, the problematic code can be triggered by a copy size greater than 128 bytes and an unaligned source address. This bug is known to be the cause of random user process memory corruptions while perf is running with the callgraph option (ie, perf record -g). This occurs because perf uses copy_from_user() to read user stacks, and may fault when it follows a stack frame pointer off to an invalid page. Validation checks on the stack address just obscure the underlying problem. Signed-off-by: Rob Gardner <rob.gardner@oracle.com> Signed-off-by: Dave Aldridge <david.j.aldridge@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2015-12-23 14:24:49 +08:00
ENTRY(__retl_one_fp)
VISExitHalf
retl
mov 1, %o0
ENDPROC(__retl_one_fp)
sparc64: Kill .fixup section bloat. This is an implementation of a suggestion made by Chris Torek: -------------------- Something else I noticed in passing: the EX and EX_LD/EX_ST macros scattered throughout the various .S files make a fair bit of .fixup code, all of which does the same thing. At the cost of one symbol in copy_in_user.S, you could just have one common two-instruction retl-and-mov-1 fixup that they all share. -------------------- The following is with a defconfig build: text data bss dec hex filename 3972767 344024 584449 4901240 4ac978 vmlinux.orig 3968887 344024 584449 4897360 4aba50 vmlinux Signed-off-by: David S. Miller <davem@davemloft.net>
2009-02-09 14:00:55 +08:00
ENTRY(__ret_one_asi)
wr %g0, ASI_AIUS, %asi
ret
restore %g0, 1, %o0
ENDPROC(__ret_one_asi)
ENTRY(__retl_one_asi)
wr %g0, ASI_AIUS, %asi
retl
mov 1, %o0
ENDPROC(__retl_one_asi)
sparc64: fix FP corruption in user copy functions Short story: Exception handlers used by some copy_to_user() and copy_from_user() functions do not diligently clean up floating point register usage, and this can result in a user process seeing invalid values in floating point registers. This sometimes makes the process fail. Long story: Several cpu-specific (NG4, NG2, U1, U3) memcpy functions use floating point registers and VIS alignaddr/faligndata to accelerate data copying when source and dest addresses don't align well. Linux uses a lazy scheme for saving floating point registers; It is not done upon entering the kernel since it's a very expensive operation. Rather, it is done only when needed. If the kernel ends up not using FP regs during the course of some trap or system call, then it can return to user space without saving or restoring them. The various memcpy functions begin their FP code with VISEntry (or a variation thereof), which saves the FP regs. They conclude their FP code with VISExit (or a variation) which essentially marks the FP regs "clean", ie, they contain no unsaved values. fprs.FPRS_FEF is turned off so that a lazy restore will be triggered when/if the user process accesses floating point regs again. The bug is that the user copy variants of memcpy, copy_from_user() and copy_to_user(), employ an exception handling mechanism to detect faults when accessing user space addresses, and when this handler is invoked, an immediate return from the function is forced, and VISExit is not executed, thus leaving the fprs register in an indeterminate state, but often with fprs.FPRS_FEF set and one or more dirty bits. This results in a return to user space with invalid values in the FP regs, and since fprs.FPRS_FEF is on, no lazy restore occurs. This bug affects copy_to_user() and copy_from_user() for NG4, NG2, U3, and U1. All are fixed by using a new exception handler for those loads and stores that are done during the time between VISEnter and VISExit. n.b. In NG4memcpy, the problematic code can be triggered by a copy size greater than 128 bytes and an unaligned source address. This bug is known to be the cause of random user process memory corruptions while perf is running with the callgraph option (ie, perf record -g). This occurs because perf uses copy_from_user() to read user stacks, and may fault when it follows a stack frame pointer off to an invalid page. Validation checks on the stack address just obscure the underlying problem. Signed-off-by: Rob Gardner <rob.gardner@oracle.com> Signed-off-by: Dave Aldridge <david.j.aldridge@oracle.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2015-12-23 14:24:49 +08:00
ENTRY(__retl_one_asi_fp)
wr %g0, ASI_AIUS, %asi
VISExitHalf
retl
mov 1, %o0
ENDPROC(__retl_one_asi_fp)
sparc64: Kill .fixup section bloat. This is an implementation of a suggestion made by Chris Torek: -------------------- Something else I noticed in passing: the EX and EX_LD/EX_ST macros scattered throughout the various .S files make a fair bit of .fixup code, all of which does the same thing. At the cost of one symbol in copy_in_user.S, you could just have one common two-instruction retl-and-mov-1 fixup that they all share. -------------------- The following is with a defconfig build: text data bss dec hex filename 3972767 344024 584449 4901240 4ac978 vmlinux.orig 3968887 344024 584449 4897360 4aba50 vmlinux Signed-off-by: David S. Miller <davem@davemloft.net>
2009-02-09 14:00:55 +08:00
ENTRY(__retl_o1)
retl
mov %o1, %o0
ENDPROC(__retl_o1)