Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GX12 hangs while flying #5810

Open
1 task done
arkarkark opened this issue Jan 21, 2025 · 35 comments
Open
1 task done

GX12 hangs while flying #5810

arkarkark opened this issue Jan 21, 2025 · 35 comments
Labels
bug 🪲 Something isn't working triage Bug report awaiting review / sorting

Comments

@arkarkark
Copy link

arkarkark commented Jan 21, 2025

Is there an existing issue for this problem?

  • I have searched the existing issues

What part of EdgeTX is the focus of this bug?

Transmitter firmware

Current Behavior

While flying my model called Pumpkin (model02.yml in attached zip file). My Radiomaster GX12 has got into a locked state where luckily the sticks still control the plane, but none of the other buttons/screens work. everything appears locked up.

This has happened three times over two days flying. The only way to get my radio back appears to be to pull the battery out and put it back in.
I moved this model over from my Jumper T-Pro v2 (I have another issue #5805 about that not being the smoothest process)

gx12.2025-01-20.zip

here is a video of the radio when locked up

here's the log file from a time that it locked up
Pumpkin-2025-01-20-103633.csv
I've also made this cvs available in a google sheet

luckily this was all line of sight, but if it had happened when I need the switches to change flight modes or return to home I would be stuck.

  1. is there a way to reboot the GX12 radio (when holding the power button doesn't do it) without pulling the battery out?
  2. it's possible that some missing telemetry scripts are casuing the problem? I had forgot to copy them over from my old radio. I. have put them on now and if I don't experience the problem next week I'll comment back here. The scripts are TOOLS/GPS Stats X9L.lua and Telemetry/GPSx9L.lua from moschotto/OpenTX_GPS_Telemetry

The plane is a scythe delta wing with a cheap ELRS 2.4Ghz rx connected to a matek 6.ch CRSF to pwm board and a cheap GPS unit attached to the second uart.

the GX12 is running this firmware from the factory
2.11.0-selfbuild .
GIT# d5552c80
2024-10-04 10:17:34

Expected Behavior

radio should not lock up

Steps To Reproduce

  1. fly plane with a GX12 transmitter.
  2. at some point notice that radio isn't talking to me. work out no buttons or switches work on the radio. on the sticks.

Version

Other (Please specify below)

Transmitter

RadioMaster GX12

Operating System (OS)

macOS

OS Version

sequoia 15.2

Anything else?

No response

@arkarkark arkarkark added bug 🪲 Something isn't working triage Bug report awaiting review / sorting labels Jan 21, 2025
@pfeerick
Copy link
Member

pfeerick commented Jan 21, 2025

Since you are running factory/pre-release firmware, can you try updating to a nightly firmware for this radio, and see if this is any different. Double check first if there is a copy of the firmware saved in the FIRMWARE folder on the radios onboard memory, as then you can revert back to that if you have any issues. If it's not there, I can ask RM for a copy of what they are shipping with.

@pfeerick
Copy link
Member

is there a way to reboot the GX12 radio (when holding the power button doesn't do it) without pulling the battery out?

If it is just a UI lock up, holding down the power button for about 10 seconds should forcibly power down.

@arkarkark
Copy link
Author

is there a way to reboot the GX12 radio (when holding the power button doesn't do it) without pulling the battery out?

If it is just a UI lock up, holding down the power button for about 10 seconds should forcibly power down.

ok. I tried holding it down for a lot longer than 10 seconds and it didn't power down. I show it for a while in my video, but also tried it for much longer off camera. I'll try updating to a nightly build.

@pfeerick
Copy link
Member

Yeah, that looks like 15 seconds easily... ouch! Hm... strange... if you have controls working, mixer is working, so the isForcePowerOffRequested() check should have caught that...

@arkarkark
Copy link
Author

arkarkark commented Jan 21, 2025

Thanks, I have the nightly on there now so I'll be able to report back if it still has this problem with the nightly.
It sure is nice to have companion to set up this radio (and my MT12 too!)

@pfeerick
Copy link
Member

pfeerick commented Jan 21, 2025 via email

@arkarkark
Copy link
Author

I tried it again today and it locked up on my talon model, luckily before I had taken off. I was running the c4646cf nightly gx12 firmware. (md5sum 6dbe68f08b86987c2c81183809e2b7e5)

@philmoz
Copy link
Collaborator

philmoz commented Jan 26, 2025

I've flown at least a dozen times with the GX12 on various builds of 2.11 without any issues.

In you .zip file model02.yml is loading a telemetry screen using a script name GPSx9L which isn't in the zip file as far as I can see. Can you upload this script if you are still using it.

@arkarkark
Copy link
Author

I mention GPSx9L in my original report. it was indeed not present.
I had copied over the .yml files from my old t-pro v2 and neglected to copy the telemetry scripts. but I had since added it and today the lockup happened with that file present.
Usually the problem happened with my plane called "Pumpkin", but today it happened with "Talon"
The GPS telemetry was all working really well today, so that was nice.

@arkarkark
Copy link
Author

@philmoz I wonder if you have telemetry logging turned on with "SD Logs" special function?
It's only a feeling, but I suspect writing to the flash storage is not as reliable as it could be.

I've also flown my GX12 more than a dozen times with no issues, but I've also flown 4 times where I've had to pull the battery out before I could get my radio to work again. I even have video of it locked up.

@philmoz
Copy link
Collaborator

philmoz commented Jan 26, 2025

Things to try:

  • check radio storage for errors (mount via USB and run a disk check)
  • turn off logging and remove the GPSx9L telemetry screen (which also does logging).
  • run bench tests to see if the problem is still reproducible

@arkarkark
Copy link
Author

oof, I just spent ages changing the led color of my switches to match my plane color (on the radio using the dial since I couldn't find how to do it in companion) and when I went back to cycle through my planes I see my Pumpkin has been corrupted and I lost all my colors and my inputs and mixes. Luckily I made a backup of the model before setting the colors and have attached it here since it seems to be the plane that causes the most trouble (It's also the plane I use the most). This does point to maybe an "SD Card"/storage problem, I have yet to run fsck on it and I'm tempted to just format it and start from scratch (unless you think that's a bad idea). I'll still make sure to run fsck first though.

model02.yml.txt

@arkarkark
Copy link
Author

I ran fsk.fat on my linux system and it found filesystem errors. I then fixed it. and checked it again on my mac and all was well.

sudo fsck.fat -n  /dev/sdc1
fsck.fat 4.2 (2021-01-31)
/MODELS/._model00.yml
  Duplicate directory entry.
  First    Size 4096 bytes, date 21:07:02 Jan 20 2025
  Second   Size 4096 bytes, date 21:07:06 Jan 20 2025
  Auto-renaming second.
  Renamed to FSCK0000.000
/SCRIPTS/TOOLS/GPS Stats X9L.lua
  Duplicate directory entry.
  First    Size 6438 bytes, date 18:52:36 Jan 20 2025
  Second   Size 4680 bytes, date 18:52:38 Jan 20 2025
  Auto-renaming second.
  Renamed to FSCK0000.000
Label in boot sector is 'WTWFDOTCOM', but there is no volume label in root directory.
  Auto-removing label from boot sector.
Reclaimed 3 unused clusters (98304 bytes).

Leaving filesystem unchanged.
/dev/sdc1: 1427 files, 2165/15365 clusters

But then I copied some model files back and over various mounting an unmounting my bee model got corrupted. Then I copied the bee file back and my diana jet model was corrupted. and I copied that back and my bee model is corrupted again. I am just using terminal cp on my macbook pro.

Next step is to format the sdcard and try again I guess.

@philmoz
Copy link
Collaborator

philmoz commented Jan 27, 2025

Does the same thing happen if you start in bootloader mode and connect the USB there?

I would also make sure the boatloader is up to date first - do a DFU flash of the entire firmware or update the bootloader from the firmware.

@arkarkark
Copy link
Author

I formatted the drive using my ubuntu system and put back the sdcard contents and have been playing around with it tonight and haven't got it to corrupt any files. I did notice that some of my models are unix newline only files and some were dos crlf files. But I went through each of them and hit the trim button and now they're all crlf (apart from the readme.txt which was still unix style)

I'm not flying for a couple of weeks, but I'll report back if I have more problems with it.

@arkarkark
Copy link
Author

spoke too soon. I was playing around with the ELRS script lua and my 'dronesim' model just got cleared out :(

@arkarkark
Copy link
Author

and to be clear, I only ever unplug the radio from usb AFTER it has been ejected (the fs is unmounted).

@arkarkark
Copy link
Author

Alright I can reproduce this constantly now and I'm wondering if it's related to timers? I've attached my simdrone model 23 and it has a flight timer that's always increasing. every time I plug in the USB this file gets corrupted down to exactly 1K I've attached the corrupted file as well. notice how it's 1024 bytes long. each corrupted file is the same length and the only difference between them is the flight timer value.

model23.yml.txt
broken.model23.yml.txt

now this might not be related to the original bug (GX12 hangs during flight) this might be a new bug, model file corrupted when plugging in USB?

@arkarkark
Copy link
Author

I just selected a model with a timer, but the timer was not incrementing and plugged in the USB and that model got corrupted. so it seems only the currently selected model is corrupted when you plug in the USB.

@philmoz
Copy link
Collaborator

philmoz commented Jan 28, 2025

I just selected a model with a timer, but the timer was not incrementing and plugged in the USB and that model got corrupted. so it seems only the currently selected model is corrupted when you plug in the USB.

Did you do anything after plugging into USB (e.g. copy files)?
I've tried on two different Macs (Intel with USB-A to USB-C cable, and M3 with USB-C to USB-C cable).
Not seeing any corruption.

@arkarkark
Copy link
Author

I plug in the usb and ls /volumes/gx12/models/ and see the model file of the currently selected model is 1k in size. all other models look o.k. (size wise)

@philmoz
Copy link
Collaborator

philmoz commented Jan 28, 2025

When you connect the USB, the firmware saves any pending changes to the model file, then shutdown the SD card so it can be connected as a drive on the USB port.

If a directory listing is showing a truncated file then there is something very strange going on - I can't reproduce this on my GX12.

Can you try with a new blank model that has no timers, scripts or module enabled.

@MEBeans
Copy link

MEBeans commented Jan 29, 2025

Just like Arkarkark discovered, I too just discovered that it is something about plugging in the usb that is messing with my models. This fault also changes the channel order. Also learned that pressing one of the menu navigation keys advances the screen through whatever faults it is experiencing when you end up with a red power button, then puts the radio into calibration mode. So far, any models that I have selected to backup on the "SD" seem to be stable on the sd, in spite of the usb doing something to their names. This fault changes the names to "MODELXX". Once the model is reloaded, the name is restored.
I'm sorry if my newness to all this is missing the mark for devs who might be looking into this, but I am just doing exactly what researching EdgeTX said you do when having problems. I am not a programmer and I have almost no experience with navigating this stuff, but I assure you, I can learn!

@arkarkark
Copy link
Author

sorry @philmoz , not going to have time to do this before a trip, I'll get back to it in a week.

@3djc
Copy link
Collaborator

3djc commented Jan 29, 2025

I also tried to replicate here, without success. Here is what I did:

  • loaded GX12 with current nightly
  • copy model23.yml on my gx12
  • run that model, I can see a persistant timer, all is fine
  • plug USB, select Joystick mode, leave it connected for 2 minutes
  • unplug USB
    All is fine there, no issue to report. Is that exactly the right procedure that you can fail 100% @arkarkark ?

@arkarkark
Copy link
Author

@3djc I have mine automatically set to be usb drive mode. which is why I'm able to see that file file is only 1K in length. I think yours is the first mention of joystick mode in this issue.

@3djc
Copy link
Collaborator

3djc commented Jan 29, 2025

Hmm storage mode and Mac do add a lot of unknowns. MacOSX is well known to mess up with sd cards, so it is hard to isolate the issue if you add that complexity into the mix. Would be nice to try to replicate with joystick mode (since the name of the model was simdrone, I wrongly assumed it was used for sim)

@arkarkark
Copy link
Author

oof, (again!), well I have a linux laptop that I use in the field and when I get back I can exclusively connect to that to do my configuration and see if it pops up again. and I do use joystick mode with my mac but via bluetooth ELRS, I can switch to using plugged in joystick mode when I play sim drones.

I hadn't heard about OSX messing with sdcards, but I can believe it. I've found the most reliabl;e way to do sdcards is to format them from my linux machione. OSX formatted ones (still FAT fs) sometimes cause trouble with my HDZero goggles.

I have had a Jumper T-Pro and T-Pro v2 for years and never had this problem with them.

@3djc
Copy link
Collaborator

3djc commented Jan 29, 2025

Spotlight indexing engine part of MacOSX leaves (and therefore has to write) tons of hidden files. Not suggesting that it is linked, just want to remove variables from the equation. It would be nice to see if you can replicate using joystick mode instead of storage

@MEBeans
Copy link

MEBeans commented Jan 29, 2025

I am curious about these "nightlys". What are these? I haven't been able to nail down a pattern with the USB problem on mine yet, things still seem to have a random quality to them. The problem affects the models as well as radio configuration. Every time it happens to me , the channel order gets rearranged.

@MEBeans
Copy link

MEBeans commented Jan 29, 2025

3djc, I have had my radio reset models and controls on it's own over night. I was flying my drone and flying on the sim. Turned the radio off, went to bed. Next morning the radio and model data were reset. I'm sorry if my info isn't helping much, but I figure every little bit of info pointing to the problem should help get it sorted out.

@arkarkark
Copy link
Author

I’ve previously turned off spotlight and removed the spotlight from that drive

@3djc
Copy link
Collaborator

3djc commented Jan 29, 2025

sudo fsck.fat -n /dev/sdc1 fsck.fat 4.2 (2021-01-31) /MODELS/._model00.yml

That ._ file doesn't come from Etx AFAIK, so unsure where it comes from. Anyhow, using joystick instead of storage will tell us a lot of what might be happening, since we are then sure nothing is modifying files on Etx back

@arkarkark
Copy link
Author

That might have been before I turned off and removed spotlight. But I still see the problem with no spotlight and a freshly formatted “sdcard”

Will try the joystick thing in a week.

@pfeerick
Copy link
Member

I am curious about these "nightlys". What are these?

The builds of the main development branch that are built every night, available for download here => https://github.com/EdgeTX/edgetx/releases/tag/nightly, and can also be selected in EdgeTX Buddy via the Filters -> "Include pre-releases" option.

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🪲 Something isn't working triage Bug report awaiting review / sorting
Projects
None yet
Development

No branches or pull requests

5 participants