Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ui: refactor spinner using raylib #33738

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

deanlee
Copy link
Contributor

@deanlee deanlee commented Oct 7, 2024

  1. Add selfdrive/ui/raylib/util.cc for Helper Functions: Introduced a new raylib/util.cc file containing utility functions to manage common tasks such as font handling, application initialization, and other reusable operations.
  2. refactor spinner using raylib:
    Screenshot from 2024-10-07 02-23-09
    Screenshot from 2024-10-07 02-22-38

Copy link
Contributor

github-actions bot commented Oct 7, 2024

UI Preview

All Screenshots

@deanlee deanlee force-pushed the raylib_spinner branch 5 times, most recently from c8fdc0c to 9c35827 Compare October 8, 2024 04:47
Copy link
Contributor

@adeebshihadeh adeebshihadeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

It's even a (slight) red diff, even with setting up all the new raylib infrastructure.

Comment on lines +59 to +60
Vector2 textSize = MeasureTextEx(getFont(), userInput.c_str(), kFontSize, 1.0);
DrawTextEx(getFont(), userInput.c_str(), {center.x - textSize.x / 2, yPos}, kFontSize, 1.0, WHITE);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make a helper for this?

}
}

EndDrawing();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raylib handles the FPS here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, raylib hand FPS and poll input events in EndDrawing().

@deanlee
Copy link
Contributor Author

deanlee commented Oct 9, 2024

@adeebshihadeh: Raylib failed to initialize GLFW on the C3X:

INFO: Initializing raylib 5.5-dev
INFO: Platform backend: DESKTOP (GLFW)
INFO: Supported raylib modules:
INFO:     > rcore:..... loaded (mandatory)
INFO:     > rlgl:...... loaded (mandatory)
INFO:     > rshapes:... loaded (optional)
INFO:     > rtextures:. loaded (optional)
INFO:     > rtext:..... loaded (optional)
INFO:     > rmodels:... loaded (optional)
INFO:     > raudio:.... loaded (optional)
WARNING: GLFW: Error: 65544 Description: Wayland: Failed to load libwayland-client entry point
WARNING: GLFW: Failed to initialize GLFW

The error occurs because two functions, wl_proxy_get_tag and wl_proxy_set_tag, are missing from the /lib/aarch64-linux-gnu/libwayland-client.so :

https://github.com/raysan5/raylib/blob/3fb1ba25aca0a6b023ca254a0910df36b0744e64/src/external/glfw/src/wl_init.c#L605-L626

To resolve this issue, we need to update Wayland on the device to the correct version and ensure raylib is compiled with the appropriate build options ( https://github.com/raysan5/raylib/blob/3fb1ba25aca0a6b023ca254a0910df36b0744e64/CMakeOptions.txt)

@adeebshihadeh
Copy link
Contributor

Unfortunately, we can't easily update our Weston. Can we just patch those out in raylib?

@adeebshihadeh
Copy link
Contributor

Getting this error building on device:

comma@comma-5d0ac7f2:/data/openpilot$ scons -j8 selfdrive/ui/
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
clang++ -o selfdrive/ui/raylib/spinner.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq selfdrive/ui/raylib/spinner.cc
clang++ -o selfdrive/ui/raylib/util.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq selfdrive/ui/raylib/util.cc

clang++ -o selfdrive/ui/qt/text.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_MESSAGELOGCONTEXT -Wno-deprecated-declarations -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq -I/usr/include -I/usr/include/aarch64-linux-gnu/qt5 -I/usr/include/aarch64-linux-gnu/qt5/QtGui/5.12.8/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtWidgets -I/usr/include/aarch64-linux-gnu/qt5/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtCore -I/usr/include/aarch64-linux-gnu/qt5/QtNetwork -I/usr/include/aarch64-linux-gnu/qt5/QtConcurrent -I/usr/include/aarch64-linux-gnu/qt5/QtDBus -I/usr/include/aarch64-linux-gnu/qt5/QtXml -Ithird_party/qrcode selfdrive/ui/qt/text.cc
clang++ -o selfdrive/ui/qt/network/networking.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_MESSAGELOGCONTEXT -Wno-deprecated-declarations -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq -I/usr/include -I/usr/include/aarch64-linux-gnu/qt5 -I/usr/include/aarch64-linux-gnu/qt5/QtGui/5.12.8/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtWidgets -I/usr/include/aarch64-linux-gnu/qt5/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtCore -I/usr/include/aarch64-linux-gnu/qt5/QtNetwork -I/usr/include/aarch64-linux-gnu/qt5/QtConcurrent -I/usr/include/aarch64-linux-gnu/qt5/QtDBus -I/usr/include/aarch64-linux-gnu/qt5/QtXml -Ithird_party/qrcode selfdrive/ui/qt/network/networking.cc
clang++ -o selfdrive/ui/qt/network/wifi_manager.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_MESSAGELOGCONTEXT -Wno-deprecated-declarations -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq -I/usr/include -I/usr/include/aarch64-linux-gnu/qt5 -I/usr/include/aarch64-linux-gnu/qt5/QtGui/5.12.8/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtWidgets -I/usr/include/aarch64-linux-gnu/qt5/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtCore -I/usr/include/aarch64-linux-gnu/qt5/QtNetwork -I/usr/include/aarch64-linux-gnu/qt5/QtConcurrent -I/usr/include/aarch64-linux-gnu/qt5/QtDBus -I/usr/include/aarch64-linux-gnu/qt5/QtXml -Ithird_party/qrcode selfdrive/ui/qt/network/wifi_manager.cc
/usr/bin/moc -o selfdrive/ui/qt/network/moc_networking.cc selfdrive/ui/qt/network/networking.h
/usr/bin/moc -o selfdrive/ui/qt/network/moc_wifi_manager.cc selfdrive/ui/qt/network/wifi_manager.h
selfdrive/ui/update_translations.py
clang++ -o selfdrive/ui/qt/network/moc_networking.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_MESSAGELOGCONTEXT -Wno-deprecated-declarations -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq -I/usr/include -I/usr/include/aarch64-linux-gnu/qt5 -I/usr/include/aarch64-linux-gnu/qt5/QtGui/5.12.8/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtWidgets -I/usr/include/aarch64-linux-gnu/qt5/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtCore -I/usr/include/aarch64-linux-gnu/qt5/QtNetwork -I/usr/include/aarch64-linux-gnu/qt5/QtConcurrent -I/usr/include/aarch64-linux-gnu/qt5/QtDBus -I/usr/include/aarch64-linux-gnu/qt5/QtXml -Ithird_party/qrcode selfdrive/ui/qt/network/moc_networking.cc
clang++ -o selfdrive/ui/qt/network/moc_wifi_manager.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_MESSAGELOGCONTEXT -Wno-deprecated-declarations -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq -I/usr/include -I/usr/include/aarch64-linux-gnu/qt5 -I/usr/include/aarch64-linux-gnu/qt5/QtGui/5.12.8/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtWidgets -I/usr/include/aarch64-linux-gnu/qt5/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtCore -I/usr/include/aarch64-linux-gnu/qt5/QtNetwork -I/usr/include/aarch64-linux-gnu/qt5/QtConcurrent -I/usr/include/aarch64-linux-gnu/qt5/QtDBus -I/usr/include/aarch64-linux-gnu/qt5/QtXml -Ithird_party/qrcode selfdrive/ui/qt/network/moc_wifi_manager.cc
ld -r -b binary -o selfdrive/ui/installer/continue_openpilot.o selfdrive/ui/installer/continue_openpilot.sh
clang++ -o selfdrive/ui/installer/installers/installer_openpilot.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_MESSAGELOGCONTEXT -Wno-deprecated-declarations -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -DBRANCH='"release3"' -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq -I/usr/include -I/usr/include/aarch64-linux-gnu/qt5 -I/usr/include/aarch64-linux-gnu/qt5/QtGui/5.12.8/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtWidgets -I/usr/include/aarch64-linux-gnu/qt5/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtCore -I/usr/include/aarch64-linux-gnu/qt5/QtNetwork -I/usr/include/aarch64-linux-gnu/qt5/QtConcurrent -I/usr/include/aarch64-linux-gnu/qt5/QtDBus -I/usr/include/aarch64-linux-gnu/qt5/QtXml -Ithird_party/qrcode selfdrive/ui/installer/installer.cc
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_en.ts'...
    Found 254 source text(s) (250 new and 4 already existing)
Stripping non plural forms in 'selfdrive/ui/translations/main_en.ts'...
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_de.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_fr.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_pt-BR.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_es.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_tr.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_ar.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_th.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_zh-CHT.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_zh-CHS.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_ko.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
Scanning directory '/data/openpilot/selfdrive/ui'...
Updating 'selfdrive/ui/translations/main_ja.ts'...
    Found 254 source text(s) (0 new and 254 already existing)
/usr/bin/moc -o selfdrive/ui/installer/moc_installer.cc selfdrive/ui/installer/installer.h
clang++ -o selfdrive/ui/installer/moc_installer.o -c -std=c++1z -DQCOM2 -mcpu=cortex-a57 -D_REENTRANT -DQT_NO_DEBUG -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -DQT_MESSAGELOGCONTEXT -Wno-deprecated-declarations -g -fPIC -O2 -Wunused -Werror -Wshadow -Wno-unknown-warning-option -Wno-inconsistent-missing-override -Wno-c99-designator -Wno-reorder-init-list -Wno-vla-cxx-extension -DQCOM2 -mcpu=cortex-a57 -Ithird_party/opencl/include -I. -Ithird_party/acados/include -Ithird_party/acados/include/blasfeo/include -Ithird_party/acados/include/hpipm/include -Ithird_party/catch2/include -Ithird_party/libyuv/include -Ithird_party/json11 -Ithird_party/linux/include -Ithird_party/snpe/include -Ithird_party -Imsgq -I/usr/include -I/usr/include/aarch64-linux-gnu/qt5 -I/usr/include/aarch64-linux-gnu/qt5/QtGui/5.12.8/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtWidgets -I/usr/include/aarch64-linux-gnu/qt5/QtGui -I/usr/include/aarch64-linux-gnu/qt5/QtCore -I/usr/include/aarch64-linux-gnu/qt5/QtNetwork -I/usr/include/aarch64-linux-gnu/qt5/QtConcurrent -I/usr/include/aarch64-linux-gnu/qt5/QtDBus -I/usr/include/aarch64-linux-gnu/qt5/QtXml -Ithird_party/qrcode selfdrive/ui/installer/moc_installer.cc
ar rc selfdrive/ui/libraylib_lib.a selfdrive/ui/raylib/util.o
ranlib selfdrive/ui/libraylib_lib.a
clang++ -o selfdrive/ui/_spinner -Wl,--as-needed -Wl,--no-undefined -Wl,-rpath=/data/openpilot/third_party/acados/larch64/lib -Wl,-rpath=/usr/local/lib selfdrive/ui/raylib/spinner.o -L/usr/local/lib -L/system/vendor/lib64 -Lthird_party/acados/larch64/lib -Lthird_party/snpe/larch64 -Lthird_party/libyuv/larch64/lib -L/usr/lib/aarch64-linux-gnu -Lmsgq_repo -Lthird_party -Lselfdrive/pandad -Lcommon -Lrednose/helpers -Lthird_party/raylib/larch64 -lraylib selfdrive/ui/libraylib_lib.a -lOpenCL
/usr/bin/ld: selfdrive/ui/libraylib_lib.a(util.o): in function `HardwareTici::set_brightness(int)':
/data/openpilot/./system/hardware/tici/hardware.h:55:(.text._ZN12HardwareTici14set_brightnessEi[_ZN12HardwareTici14set_brightnessEi]+0x80): undefined reference to `util::read_file(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
scons: *** [selfdrive/ui/_spinner] Error 1

@deanlee
Copy link
Contributor Author

deanlee commented Oct 15, 2024

Getting this error building on device:

@adeebshihadeh: fixed this build issue

@adeebshihadeh
Copy link
Contributor

adeebshihadeh commented Oct 15, 2024

@maxime-desroches can you get this working on-device? It's likely easiest to patch raylib to work with our existing weston.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants