RPi4: Kernel clock_gettime64() vDSO disabled on 32-bit kernel due to firmware not initializing timer registers

[rsaxvc]

Is this the right place for my bug report?

I think so. But, it's an integration issue between firmware and kernel, so it might belong there.

Describe the bug
Currently, the Raspberry-Pi Linux kernel does not trust that the firmware has configured the BCM2711's timer registers, so it disables clock_gettime64 vDSO. This is a request that if the RaspberryPi isn't configuring this timer, to do so.

To reproduce

1. strace the following program

#include <time.h>

int main(){
struct timespec tp;
return clock_gettime(CLOCK_MONOTONIC, &tp);
}

2.strace shows clock-gettime64 systemcall is in use instead of vDSO.

Expected behaviour
The firmware should configure the timer registers.
Once done, Linux would no longer mark them with cpu-registers-not-fw-configured, then the clock_gettime64() vDSO works.

Actual behaviour
The Raspberry Pi foundation's kernel believes the firmware has not configured the timer registers. Did the firmware configure them? If not, could it? It seems to be working, but I couldn't find any references to fixing it in the github issues list. It appeared to have been broken in ~2020: #3579

System

$ cat /etc/issue
Raspbian GNU/Linux 11 \n \l
$ vcgencmd version 
Jan  5 2023 10:46:54 
Copyright (c) 2012 Broadcom
version 8ba17717fbcedd4c3b6d4bce7e50c7af4155cba9 (clean) (release) (start)
$ uname -a
Linux raspberrypi 6.1.21-v7l+ raspberrypi/firmware#1642 SMP Mon Apr  3 17:22:30 BST 2023 armv7l GNU/Linux
$ cat /proc/cpuinfo | tail -4
Hardware	: BCM2711
Revision	: d03114
Serial		: 10000000701809ed
Model		: Raspberry Pi 4 Model B Rev 1.4

[pelwell]

Is this bug report completely accurate? I agree that a 64-bit userland (which implies a 64-bit kernel) does not require clock_gettime64 calls, and that a 32-bit kernel (and therefore 32-bit userland) does require them, but I also find that a 32-bit userland with a 64-bit kernel does need them. I therefore think that the problem correlates with the flavour of userland in use.

[rsaxvc]

Is this bug report completely accurate?

I believe so, and I believe it's kernel-related, not userland-related, but these are the facts I'm certain of:

• Applications on my Raspberry Pi 2 Model B Rev 1.1, 5.10.103-v7+, Firmware May 9 2023 12:17:54
  , do use the clock_gettime64() vDSO.
• Applications on my Pi4B with raspberrypi kernel and device tree doesn't use the clock_gettime64() vDSO, and use the syscall instead.
• If I remove cpu-registers-not-fw-configured from the timer in the device tree, then applications use the vDSO, and it appears to work now.
• cpu-registers-not-fw-configured is Only supported for 32-bit systems which follow the ARMv7 architected reset values.
• As of kernel commit 688837b, cpu-registers-not-fw-configured is not mentioned in arch/arm64, only in arch/arm/kernel/vdso.c, where its presence triggers calling vdso_nullpatch_one() on "__vdso_gettimeofday", "__vdso_clock_gettime", and "__vdso_clock_gettime64" - I think that's what disables the vDSO.

Edits: added RPi2 test

[pelwell]

My results are slightly confusing:

• With not-fw-configured:

  arm32 kernel, arm32 app: ioctl
  arm64 kernel, arm32 app: ioctl
  arm64 kernel, arm64 app: vdso

• Without not-fw-configured:

  arm32 kernel, arm32 app: vdso
  arm64 kernel, arm32 app: ioctl
  arm64 kernel, arm64 app: vdso

This shows that the removal of the property (which is the right thing to do) only improves matters for a 32-bit kernel running on a Pi 4, which is not the default behaviour on a current firmware and OS - you need to set arm_64bit=0 (the Pi 4 default is 1).

Do you agree?

See #6602 for a fix. In about 40 minutes you will be able to install a trial 6.6 kernel with the patched DTB by running sudo rpi-update pulls/6602. You should back up any important data first.

[rsaxvc]

You're right - I did have "kernel=kernel7l.img" in my config.txt. When I comment that out I get the Arm V8 kernel, and syscall instead of vDSO as you did.

When I try to install pulls/6604, it refuses with BOOTLOADER_REV:54d9c333a9d39941b4fc881275f433821c7b 5cde *** Invalid hash given. I'm not sure if that hash is of part of my current system, which has been modified from stock, or of the incoming update.

Is there any worry about running a new device tree on a RPi4 with older firmware that may not initialize the timer? I was uncertain if the firmware that sets up the timer is the start.elf that could be easily updated or something else.

[popcornmix]

When I try to install pulls/6604

it is 6602.