Is there any upper memory bound for initramfs or opensbi or kernel?

[NazerkeT]

Hi there,

I was recently running OpenSBI + Linux on Spike and root launches as expected with regular small rootfs payload. However, when I rerun simulation with 3.6GB of data added to the rootfs overlay, I hit kernel panic at the very last moment with it not being able to mount rootfs. I tried to modify maximum memory amount passed with -mN as well, but it did not work. Is there anything I need to know more to be able to diagnose this problem better?

Particularly, the following are my logs.

`
OpenSBI v1.2

---

/ __ \ / | _ _ |
| | | | __ ___ _ __ | ( | |) || |
| | | | '_ \ / _ \ '_ \ ___ | _ < | |
| || | |) | __/ | | |) | |) || |
_/| ./ _|| ||/|____/|
| |
|_|

Platform Name : ucbbar,spike-bare
Platform Features : medeleg
Platform HART Count : 1
Platform IPI Device : aclint-mswi
Platform Timer Device : aclint-mtimer @ 10000000Hz
Platform Console Device : htif
Platform HSM Device : ---
Platform PMU Device : ---
Platform Reboot Device : htif
Platform Shutdown Device : htif
Firmware Base : 0x80000000
Firmware Size : 212 KB
Runtime SBI Version : 1.0

Domain0 Name : root
Domain0 Boot HART : 0
Domain0 HARTs : 0*
Domain0 Region00 : 0x0000000002080000-0x00000000020bffff (I)
Domain0 Region01 : 0x0000000080000000-0x000000008003ffff ()
Domain0 Region02 : 0x0000000002000000-0x000000000207ffff (I)
Domain0 Region03 : 0x0000000000000000-0xffffffffffffffff (R,W,X)
Domain0 Next Address : 0x0000000080200000
Domain0 Next Arg1 : 0x0000000082200000
Domain0 Next Mode : S-mode
Domain0 SysReset : yes

Boot HART ID : 0
Boot HART Domain : root
Boot HART Priv Version : v1.11
Boot HART Base ISA : rv64imafdc
Boot HART ISA Extensions : none
Boot HART PMP Count : 16
Boot HART PMP Granularity : 4
Boot HART PMP Address Bits: 54
Boot HART MHPM Count : 0
Boot HART MIDELEG : 0x0000000000000222
Boot HART MEDELEG : 0x000000000000b109
[ 0.000000] Linux version 6.1.14 ([email protected]) (riscv64-buildroot-linux-gnu-gcc.br_real (Buildroot 2023.11-1184-gd8ea853b8c-dirty) 12.3.0, GNU ld (GNU Binutils) 2.40) #1 SMP Tue Feb 20 03:37:46 UTC 2024
[ 0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[ 0.000000] Machine model: ucbbar,spike-bare
[ 0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[ 0.000000] printk: bootconsole [sbi0] enabled
[ 0.000000] efi: UEFI not found.
[ 0.000000] Zone ranges:
[ 0.000000] DMA32 [mem 0x0000000080200000-0x00000000ffffffff]
[ 0.000000] Normal [mem 0x0000000100000000-0x000000047fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000080200000-0x000000047fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x000000047fffffff]
[ 0.000000] SBI specification v1.0 detected
[ 0.000000] SBI implementation ID=0x1 Version=0x10002
[ 0.000000] SBI TIME extension detected
[ 0.000000] SBI IPI extension detected
[ 0.000000] SBI RFENCE extension detected
[ 0.000000] SBI SRST extension detected
[ 0.000000] SBI HSM extension detected
[ 0.000000] riscv: base ISA extensions acdfim
[ 0.000000] riscv: ELF capabilities acdfim
[ 0.000000] percpu: Embedded 18 pages/cpu s34744 r8192 d30792 u73728
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 4128264
[ 0.000000] Kernel command line: console=hvc0 earlycon=sbi
[ 0.000000] Dentry cache hash table entries: 2097152 (order: 12, 16777216 bytes, linear)
[ 0.000000] Inode-cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)
[ 0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off
[ 0.000000] software IO TLB: area num 1.
[ 0.000000] software IO TLB: mapped [mem 0x00000000fbfff000-0x00000000fffff000] (64MB)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] fixmap : 0xffff8d7ffee00000 - 0xffff8d7fff000000 (2048 kB)
[ 0.000000] pci io : 0xffff8d7fff000000 - 0xffff8d8000000000 ( 16 MB)
[ 0.000000] vmemmap : 0xffff8d8000000000 - 0xffff8f8000000000 (2048 GB)
[ 0.000000] vmalloc : 0xffff8f8000000000 - 0xffffaf8000000000 ( 32 TB)
[ 0.000000] modules : 0xffffffff01362000 - 0xffffffff80000000 (2028 MB)
[ 0.000000] lowmem : 0xffffaf8000000000 - 0xffffaf83ffe00000 ( 15 GB)
[ 0.000000] kernel : 0xffffffff80000000 - 0xffffffffffffffff (2047 MB)
[ 0.000000] Memory: 16401492K/16775168K available (7467K kernel code, 4915K rwdata, 4096K rodata, 2176K init, 461K bss, 373676K reserved, 0K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] rcu: Hierarchical RCU implementation.
[ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
[ 0.000000] rcu: RCU debug extended QS entry/exit.
[ 0.000000] Tracing variant of Tasks RCU enabled.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] riscv-intc: 64 local interrupts mapped
[ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.000000] riscv-timer: riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[ 0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[ 0.000005] sched_clock: 64 bits at 10MHz, resolution 100ns, wraps every 4398046511100ns
[ 0.000610] Console: colour dummy device 80x25
[ 0.000880] printk: console [hvc0] enabled
[ 0.000880] printk: console [hvc0] enabled
[ 0.001370] printk: bootconsole [sbi0] disabled
[ 0.001370] printk: bootconsole [sbi0] disabled
[ 0.001930] Calibrating delay loop (skipped), value calculated using timer frequency.. 20.00 BogoMIPS (lpj=40000)
[ 0.002540] pid_max: default: 32768 minimum: 301
[ 0.002865] LSM: Security Framework initializing
[ 0.003270] Mount-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.003760] Mountpoint-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.005860] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.006290] cblist_init_generic: Setting shift to 0 and lim to 1.
[ 0.006775] riscv: ELF compat mode unsupported
[ 0.006785] ASID allocator disabled (0 bits)
[ 0.007460] rcu: Hierarchical SRCU implementation.
[ 0.007750] rcu: Max phase no-delay instances is 1000.
[ 0.008355] EFI services will not be available.
[ 0.008940] smp: Bringing up secondary CPUs ...
[ 0.009210] smp: Brought up 1 node, 1 CPU
[ 0.010575] devtmpfs: initialized
[ 0.011385] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.011965] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[ 0.012470] pinctrl core: initialized pinctrl subsystem
[ 0.014195] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.014650] DMA: preallocated 2048 KiB GFP_KERNEL pool for atomic allocations
[ 0.015110] DMA: preallocated 2048 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[ 0.015605] audit: initializing netlink subsys (disabled)
[ 0.016280] cpuidle: using governor menu
[ 0.022690] audit: type=2000 audit(0.012:1): state=initialized audit_enabled=0 res=1
[ 0.023605] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.024010] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.024750] iommu: Default domain type: Translated
[ 0.025075] iommu: DMA domain TLB invalidation policy: strict mode
[ 0.025640] SCSI subsystem initialized
[ 0.026130] usbcore: registered new interface driver usbfs
[ 0.026480] usbcore: registered new interface driver hub
[ 0.026815] usbcore: registered new device driver usb
[ 0.027725] vgaarb: loaded
[ 0.027945] clocksource: Switched to clocksource riscv_clocksource
[ 0.036695] NET: Registered PF_INET protocol family
[ 0.037515] IP idents hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.079080] tcp_listen_portaddr_hash hash table entries: 8192 (order: 6, 262144 bytes, linear)
[ 0.079835] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.080350] TCP established hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[ 0.082295] TCP bind hash table entries: 65536 (order: 10, 4194304 bytes, linear)
[ 0.086190] TCP: Hash tables configured (established 131072 bind 65536)
[ 0.086660] UDP hash table entries: 8192 (order: 7, 786432 bytes, linear)
[ 0.087650] UDP-Lite hash table entries: 8192 (order: 7, 786432 bytes, linear)
[ 0.088755] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.089575] RPC: Registered named UNIX socket transport module.
[ 0.089930] RPC: Registered udp transport module.
[ 0.090215] RPC: Registered tcp transport module.
[ 0.090500] RPC: Registered tcp NFSv4.1 backchannel transport module.
[ 0.090895] PCI: CLS 0 bytes, default 64
[ 0.092090] workingset: timestamp_bits=46 max_order=22 bucket_order=0
[ 0.099435] NFS: Registering the id_resolver key type
[ 0.099775] Key type id_resolver registered
[ 0.100075] Key type id_legacy registered
[ 0.100385] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[ 0.100790] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[ 0.101350] 9p: Installing v9fs 9p2000 file system support
[ 0.101990] NET: Registered PF_ALG protocol family
[ 0.102310] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
[ 0.102755] io scheduler mq-deadline registered
[ 0.103035] io scheduler kyber registered
[ 0.155450] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[ 0.160570] loop: module loaded
[ 0.161830] e1000e: Intel(R) PRO/1000 Network Driver
[ 0.162125] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[ 0.162825] usbcore: registered new interface driver uas
[ 0.163175] usbcore: registered new interface driver usb-storage
[ 0.163695] mousedev: PS/2 mouse device common for all mice
[ 0.164370] sdhci: Secure Digital Host Controller Interface driver
[ 0.164740] sdhci: Copyright(c) Pierre Ossman
[ 0.165020] sdhci-pltfm: SDHCI platform and OF driver helper
[ 0.165475] usbcore: registered new interface driver usbhid
[ 0.165810] usbhid: USB HID core driver
[ 0.166115] riscv-pmu-sbi: SBI PMU extension is available
[ 0.166540] riscv-pmu-sbi: 16 firmware and 2 hardware counters
[ 0.166890] riscv-pmu-sbi: Perf sampling/filtering is not supported as sscof extension is not available
[ 0.168070] NET: Registered PF_INET6 protocol family
[ 0.169240] Segment Routing with IPv6
[ 0.169540] In-situ OAM (IOAM) with IPv6
[ 0.169815] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[ 0.170570] NET: Registered PF_PACKET protocol family
[ 0.170990] 9pnet: Installing 9P2000 support
[ 0.171410] Key type dns_resolver registered
[ 0.178600] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 0.181025] /dev/root: Can't open blockdev
[ 0.181275] VFS: Cannot open root device "(null)" or unknown-block(0,0): error -6
[ 0.181720] Please append a correct "root=" boot option; here are the available partitions:
[ 0.182215] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[ 0.182705] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.1.14 #1
[ 0.183060] Hardware name: ucbbar,spike-bare (DT)
[ 0.183345] Call Trace:
[ 0.183495] [] dump_backtrace+0x1c/0x24
[ 0.183825] [] show_stack+0x2c/0x38
[ 0.184140] [] dump_stack_lvl+0x3c/0x54
[ 0.184470] [] dump_stack+0x14/0x1c
[ 0.184780] [] panic+0x106/0x29e
[ 0.185075] [] mount_block_root+0x170/0x222
[ 0.185425] [] mount_root+0x1f0/0x220
[ 0.185745] [] prepare_namespace+0x142/0x184
[ 0.186100] [] kernel_init_freeable+0x23e/0x260
[ 0.186470] [] kernel_init+0x1e/0x10a
[ 0.186790] [] ret_from_exception+0x0/0x16
[ 0.187135] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]-`

[ddwolf]

mine got stuck after

Boot HART ID : 0
Boot HART Domain : root
Boot HART Priv Version : v1.11
Boot HART Base ISA : rv64imafdc
Boot HART ISA Extensions : none
Boot HART PMP Count : 16
Boot HART PMP Granularity : 4
Boot HART PMP Address Bits: 54
Boot HART MHPM Count : 0
Boot HART MIDELEG : 0x0000000000000222
Boot HART MEDELEG : 0x000000000000b109