Add support for `{LDX,STX,ST} | PROBE_MEM32 | {B,H,W,DW}` instructions.
They are similar to PROBE_MEM instructions with the following differences:
* PROBE_MEM32 supports store.
* PROBE_MEM32 relies on the verifier to clear upper 32-bit of the
src/dst register
* PROBE_MEM32 adds 64-bit kern_vm_start address (which is stored in S6
in the prologue). Due to bpf_arena constructions such S6 + reg +
off16 access is guaranteed to be within arena virtual range, so no
address check at run-time.
* S6 is a free callee-saved register, so it is used to store arena_vm_start
* PROBE_MEM32 allows ST and STX. If they fault the store is a nop. When
LDX faults the destination register is zeroed.
To support these on LoongArch, we employ the t2/t3 registers to store the
intermediate results of reg_arena + src/dst reg and use the t2/t3 registers
as the new src/dst reg. This allows us to reuse most of the existing code.
Acked-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Tested-by: Vincent Li <vincent.mc.li@gmail.com>
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
Use bpf_jit_binary_pack_alloc() for BPF JIT binaries. The BPF prog pack
allocator creates a pair of RW and RX buffers. The BPF JIT writes the
program into the RW buffer. When the JIT is done, the program is copied
to the final RX buffer with bpf_jit_binary_pack_finalize().
Acked-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Tested-by: Vincent Li <vincent.mc.li@gmail.com>
Signed-off-by: Hengqi Chen <hengqi.chen@gmail.com>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
In commit a759e37fb467 ("err.h: add ERR_PTR_PCPU(), PTR_ERR_PCPU() and
IS_ERR_PCPU() macros"), specialized macros were added to check an error
within a __percpu pointer, so use them instead of manually casting with
__force, like all other users of register_wide_hw_breakpoint().
Signed-off-by: Carlos López <clopez@suse.de>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
According to Documentation/dev-tools/kasan.rst, software KASAN modes use
compiler instrumentation to insert validity checks. Such instrumentation
might be incompatible with some parts of the kernel, and therefore needs
to be disabled, just use the attribute __no_sanitize_address to disable
instrumentation for the low level function setup_ptwalker().
Otherwise bringing up the secondary CPUs failed when CONFIG_KASAN is set
(especially when PTW is enabled), here are the call chains:
smpboot_entry()
start_secondary()
cpu_probe()
per_cpu_trap_init()
tlb_init()
setup_tlb_handler()
setup_ptwalker()
The reason is the PGD registers are configured in setup_ptwalker(), but
KASAN instrumentation may cause TLB exceptions before that.
Cc: stable@vger.kernel.org
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
After commit 88fd2b70120d ("LoongArch: Fix sleeping in atomic context for
PREEMPT_RT"), it should guard percpu handler under !CONFIG_PREEMPT_RT to
avoid redundant operations.
Cc: stable@vger.kernel.org
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
Currently, use %p to prevent leaking information about the kernel memory
layout when printing the PC address, but the kernel log messages are not
useful to debug problem if bt_address() returns 0. Given that the type of
"pc" variable is unsigned long, it should use %px to print the unmodified
unwinding address.
Cc: stable@vger.kernel.org
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
Currently we use bottom-up allocation after sparse_init(), the reason is
sparse_init() need a lot of memory, and bottom-up allocation may exhaust
precious low memory (below 4GB). On the other hand, SWIOTLB and CMA need
low memories for DMA32, so swiotlb_init() and dma_contiguous_reserve()
need bottom-up allocation.
Since swiotlb_init() and dma_contiguous_reserve() are both called in
arch_mem_init(), we no longer need bottom-up allocation after that. So
we set the allocation policy to top-down at the end of arch_mem_init(),
in order to avoid later memory allocations (such as KASAN) exhaust low
memory.
This solve at least two problems:
1. Some buggy BIOSes use 0xfd000000~0xfe000000 for secondary CPUs, but
didn't reserve this range, which causes smpboot failures.
2. Some DMA32 devices, such as Loongson-DRM and OHCI, cannot work with
KASAN enabled.
Cc: stable@vger.kernel.org
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
For benchmarking or debugging purpose, we usually want to control SMT
via boot parameter and sysfs knobs. So add HOTPLUG_SMT implementation.
1. Boot parameters:
nosmt: Disable SMT, can be enabled via sysfs knobs.
nosmt=force: Disable SMT, cannot be enabled via sysfs knobs.
2. Runtime sysfs controls:
Write "on", "off", "forceoff" or the number of SMT threads (1, 2, ...)
to /sys/devices/system/cpu/smt/control.
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
LoongArch supports ARCH_HAS_SET_DIRECT_MAP, therefore wire up the
memfd_secret system call, which just depends on it.
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Red Hat) <david@kernel.org>
Signed-off-by: Lain "Fearyncess" Yang <fearyncess@aosc.io>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
Fix warnings like: "Prefer seq_puts to seq_printf" by checkpatch.pl.
Replace seq_printf() calls with seq_puts() in show_cpuinfo() when
outputting simple constant strings without format specifiers.
This improves performance slightly as seq_puts() avoids parsing the
format string.
Signed-off-by: George Guo <guodongtai@kylinos.cn>
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
LoongArch has already implemented cmpxchg_local(), this_cpu_cmpxchg()
and similar functions, so select HAVE_CMPXCHG_LOCAL in Kconfig to avoid
incurring the overhead of local_irq_save()/local_irq_restore() for page
state helpers in mm/vmstat.c.
Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
On MIPS, __current_thread_info is defined as global register variable
locating in $gp, and is simply assigned with new address during kernel
relocation.
This however is broken with LLVM, which always restores $gp if it finds
$gp is clobbered in any form, including when intentionally through a
global register variable. This is against GCC's documentation[1], which
requires a callee-saved register used as global register variable not to
be restored if it's clobbered.
As a result, $gp will continue to point to the unrelocated kernel after
the epilog of relocate_kernel(), leading to an early crash in init_idle,
[ 0.000000] CPU 0 Unable to handle kernel paging request at virtual address 0000000000000000, epc == ffffffff81afada8, ra == ffffffff81afad90
[ 0.000000] Oops[#1]:
[ 0.000000] CPU: 0 UID: 0 PID: 0 Comm: swapper Tainted: G W 6.19.0-rc5-00262-gd3eeb99bbc99-dirty #188 VOLUNTARY
[ 0.000000] Tainted: [W]=WARN
[ 0.000000] Hardware name: loongson,loongson64v-4core-virtio
[ 0.000000] $ 0 : 0000000000000000 0000000000000000 0000000000000001 0000000000000000
[ 0.000000] $ 4 : ffffffff80b80ec0 ffffffff80b53d48 0000000000000000 00000000000f4240
[ 0.000000] $ 8 : 0000000000000100 ffffffff81d82f80 ffffffff81d82f80 0000000000000001
[ 0.000000] $12 : 0000000000000000 ffffffff81776f58 00000000000005da 0000000000000002
[ 0.000000] $16 : ffffffff80b80e40 0000000000000000 ffffffff80b81614 9800000005dfbe80
[ 0.000000] $20 : 00000000540000e0 ffffffff81980000 0000000000000000 ffffffff80f81c80
[ 0.000000] $24 : 0000000000000a26 ffffffff8114fb90
[ 0.000000] $28 : ffffffff80b50000 ffffffff80b53d40 0000000000000000 ffffffff81afad90
[ 0.000000] Hi : 0000000000000000
[ 0.000000] Lo : 0000000000000000
[ 0.000000] epc : ffffffff81afada8 init_idle+0x130/0x270
[ 0.000000] ra : ffffffff81afad90 init_idle+0x118/0x270
[ 0.000000] Status: 540000e2 KX SX UX KERNEL EXL
[ 0.000000] Cause : 00000008 (ExcCode 02)
[ 0.000000] BadVA : 0000000000000000
[ 0.000000] PrId : 00006305 (ICT Loongson-3)
[ 0.000000] Process swapper (pid: 0, threadinfo=(____ptrval____), task=(____ptrval____), tls=0000000000000000)
[ 0.000000] Stack : 9800000005dfbf00 ffffffff8178e950 0000000000000000 0000000000000000
[ 0.000000] 0000000000000000 ffffffff81970000 000000000000003f ffffffff810a6528
[ 0.000000] 0000000000000001 9800000005dfbe80 9800000005dfbf00 ffffffff81980000
[ 0.000000] ffffffff810a6450 ffffffff81afb6c0 0000000000000000 ffffffff810a2258
[ 0.000000] ffffffff81d82ec8 ffffffff8198d010 ffffffff81b67e80 ffffffff8197dd98
[ 0.000000] ffffffff81d81c80 ffffffff81930000 0000000000000040 0000000000000000
[ 0.000000] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 0.000000] 0000000000000000 000000000000009e ffffffff9fc01000 0000000000000000
[ 0.000000] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 0.000000] 0000000000000000 ffffffff81ae86dc ffffffff81b3c741 0000000000000002
[ 0.000000] ...
[ 0.000000] Call Trace:
[ 0.000000] [<ffffffff81afada8>] init_idle+0x130/0x270
[ 0.000000] [<ffffffff81afb6c0>] sched_init+0x5c8/0x6c0
[ 0.000000] [<ffffffff81ae86dc>] start_kernel+0x27c/0x7a8
This bug has been reported to LLVM[2] and affects version from (at
least) 18 to 21. Let's work around this by using inline assembly to
assign $gp before a fix is widely available.
Cc: stable@vger.kernel.org
Link: https://gcc.gnu.org/onlinedocs/gcc-15.2.0/gcc/Global-Register-Variables.html # [1]
Link: https://github.com/llvm/llvm-project/issues/176546 # [2]
Signed-off-by: Yao Zi <me@ziyao.cc>
Acked-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
Loongson2ef reserves io range below 0x4000 (LOONGSON_PCI_IO_START) while
ISA-mode only IDE controller on the south bridge still has a hard
dependency on ISA IO ports.
The reservation was done by lifting loongson_pci_io_resource.start onto
0x4000. Prior to commit ae81aad5c2e1 ("MIPS: PCI: Use
pci_enable_resources()"), the arch specific pcibios_enable_resources()
did not check if the resources were claimed, which diverges from what
PCI core checks, effectively hiding the fact that IDE IO resources were
not properly within the resource tree. After starting to use
pcibios_enable_resources() from PCI core, enabling IDE controller fails:
pata_cs5536 0000:00:0e.2: BAR 0 [io 0x01f0-0x01f7]: not claimed; can't enable device
pata_cs5536 0000:00:0e.2: probe with driver pata_cs5536 failed with error -22
MIPS PCI code already has support for enforcing lower bounds using
PCIBIOS_MIN_IO in pcibios_align_resource() without altering the IO
window start address itself. Make Loongson2ef PCI code use
PCIBIOS_MIN_IO too.
Fixes: ae81aad5c2e1 ("MIPS: PCI: Use pci_enable_resources()")
Cc: stable@vger.kernel.org
Tested-by: Beiyan Yun <root@infi.wang>
Tested-by: Yao Zi <me@ziyao.cc>
Signed-off-by: Rong Zhang <rongrong@oss.cipunited.com>
Acked-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
We are about to set loongson_pci_io_resource.start to 0 and adopt
PCIBIOS_MIN_IO. As the first step, PCI controller needs to be registered
in early stage to make it the root of other resources (e.g., i8259) and
prevent resource conflicts.
Register it in plat_mem_setup() instead of arch_initcall().
Fixes: ae81aad5c2e1 ("MIPS: PCI: Use pci_enable_resources()")
Cc: stable@vger.kernel.org
Tested-by: Beiyan Yun <root@infi.wang>
Tested-by: Yao Zi <me@ziyao.cc>
Signed-off-by: Rong Zhang <rongrong@oss.cipunited.com>
Acked-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Signed-off-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
The `interrupts` property indicates an RTC alarm interrupt, which is
required for RTCs that support the alarm feature, which is not supported
by the Loongson-1C RTC. We exclude it for a more accurate description.
Changing the `allowed` property is ABI-breaking behavior, but
throughout the existing Loongson DTS{i}, the description of the RTC
nodes conforms to the modified bingding rules.
Thus, the existing Loongson DTS{i} will not be affected.
Fixes: 487ef32caebe ("dt-bindings: rtc: Split loongson,ls2x-rtc into SoC-based compatibles")
Reviewed-by: Huacai Chen <chenhuacai@loongson.cn>
Signed-off-by: Binbin Zhou <zhoubinbin@loongson.cn>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://patch.msgid.link/b6295c907410f6708115cba4df0959ee6629f8a5.1768616276.git.zhoubinbin@loongson.cn
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
With multiple virtio crypto devices supported with different NUMA
nodes, when crypto session is created, it will search virtio crypto
device with the same numa node of current CPU.
Here API topology_physical_package_id() is replaced with cpu_to_node()
since package id is physical concept, and one package id have multiple
memory numa id.
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <20260113030556.3522533-4-maobibo@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
When VM boots with one virtio-crypto PCI device and builtin backend,
run openssl benchmark command with multiple processes, such as
openssl speed -evp aes-128-cbc -engine afalg -seconds 10 -multi 32
openssl processes will hangup and there is error reported like this:
virtio_crypto virtio0: dataq.0:id 3 is not a head!
It seems that the data virtqueue need protection when it is handled
for virtio done notification. If the spinlock protection is added
in virtcrypto_done_task(), openssl benchmark with multiple processes
works well.
Fixes: fed93fb62e05 ("crypto: virtio - Handle dataq logic with tasklet")
Cc: stable@vger.kernel.org
Signed-off-by: Bibo Mao <maobibo@loongson.cn>
Acked-by: Jason Wang <jasowang@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <20260113030556.3522533-2-maobibo@loongson.cn>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
The divider_round_rate() function is now deprecated, so let's migrate
to divider_determine_rate() instead so that this deprecated API can be
removed.
Note that when the main function itself was migrated to use
determine_rate, this was mistakenly converted to:
req->rate = divider_round_rate(...)
This is invalid in the case when an error occurs since it can set the
rate to a negative value.
Fixes: bb40a2ef4fc9 ("clk: loongson1: convert from round_rate() to determine_rate()")
Reviewed-by: Keguang Zhang <keguang.zhang@gmail.com>
Tested-by: Keguang Zhang <keguang.zhang@gmail.com> # on LS1B & LS1C
Signed-off-by: Brian Masney <bmasney@redhat.com>
Signed-off-by: maoxiaochuan <maoxiaochuan@loongson.cn>
maoxiaochuan