Crash on Fedora with SDRplay

[fjl82]

I've just built this project from source using qmake and make on Fedora 34, and it starts up and detects my SDRplay RSP1a fine. But when I hit 'start', the only things that happen is the gain counting down, then the UI freezing up and a few seconds later a crash with message "Illegal instruction (core dumped)". No other output in the terminal whatsoever. Disabling AGC does not help.

Any way to fix this? Do you need more info?

[Oleg-Malyutin]

I do not speak English. I use translator.
Initially, the project was not designed for receiving from an antenna.
Good signal strength and a matched antenna are required.
In practice, the gain reduction is at least 45 (gain:> 45).
It is necessary to improve the algorithm for determining the P1 symbol. This is where a weak signal crashes and a sharp consumption of CPU resources.

[fjl82]

I tried a few different gain values, but this doesn't seem to work at all. At least with autogain, I get a few seconds of constellation diagrams. When I enter gain manually, everything stays empty, and it doesn't seem to make an attempt at receiving.

I don't think it's a problem with bad signal. The signal looks fine to me in GQRX:

That's a pity, I was hoping this was going to work. This is the only project so far that seems capable of receiving DVB-T2 from an SDR. The RTL sticks only support DVB-T, which has been replaced in most of Europe years ago by T2, and the Linux DVB kernel stack relies on the hardware doing all the decoding (which an SDR is not going to do). I hope someday you can get this working properly.

[Oleg-Malyutin]

I meant 45dB with automatic gain. This is an indication of the signal strength at the antenna.
With a worse signal, the P1 symbol may be incorrectly assigned, and also there may not be enough performance for the LDPC decoder.
There have already been consultations with a similar problem and this was solved by a good antenna.
This is a test project and does not include CPU overload protection.

[Oleg-Malyutin]

P.S. SDRPlay no gain in db, it is gain reduction db

[rapbando]

@Oleg-Malyutin Hopw difficult would be to add compatibility for Hack-RF SDR?

Cheers

[Oleg-Malyutin]

I don't have HackRF.
I think it's not difficult. I tried SdrPlay on 8 bit and it worked.
The condition is an unprocessed signal with HackRF , i.e. without DC offset and IQ imbalance remove.

Regards.

[rapbando]

Is possible for you to integrate with Soapy so HackRF and other SDRs can be used?

[Oleg-Malyutin]

Unknown.
As I wrote earlier, an unprocessed signal is needed (the driver's own DC offset and IQ imbalance remove functions are not suitable for DVB-T2).
Soapy may not contain any driver settings to disable internal enhancements.

[rapbando]

@Oleg-Malyutin

I added the QT modifications nedded for HackRF SDR. Now before compiling I need to create rx_hackrf.hheader and rx_hackrf.cpp source code, could you help me since you already know all the requirements? I would very gladly send you a support donation if you can.
Here and here I link for your reference the osmosdr analogues files for intefacing with hackrf.

Cheers

[Oleg-Malyutin]

I'll see. I don't promise a quick answer.
This will be without a sanity check.

[rapbando]

@Oleg-Malyutin

Great, no rush whatsoever!
When you will have some progress I can test and debug with my hackrf...
Where can I send a donation?

Lawrence

[andre1968]

I have the chinese SDRplay (DE BG7YZF) with libmirisdr-4 from f3exv/libmirisdr-4. I made my own rx_sdrplay.cpp and rx_sdrplay.h because the original was not working. This come because mine sdr is not working synchonous. I make it like the rx_airspy manner. My question is about the 8 bit samples. What values expect the software? Is it from 0-225 or from -128-127?
Thank in advance.
Andre

[Oleg-Malyutin]

I am writing through a translator.
8-bit -127 128.
I strongly do not advise the driver. Had experience with a similar msi driver on Linux. This is reverse engineering and a lot of mistakes. I did not set the exact frequency.
Chinese SDRPlay should work well with native drivers (I have both the original and from China). I recommend a version no newer than 2.x.
Regards.
Oleg.

[andre1968]

Hello Oleg, Thank you for your quick response. I use this driver from https://github.com/f4exb/libmirisdr-4 [https://opengraph.githubassets.com/547824d9ba890242d9975c1068698e4a26f9dc156330d2d02c81cf7d49e4702d/f4exb/libmirisdr-4]<https://github.com/f4exb/libmirisdr-4> GitHub - f4exb/libmirisdr-4: Support of Mirics MSi001 + MSi2500 SDR devices<https://github.com/f4exb/libmirisdr-4> Support of Mirics MSi001 + MSi2500 SDR devices. Contribute to f4exb/libmirisdr-4 development by creating an account on GitHub. github.com For other projects I use also this driver. I think this works well. I didn't see mistakes because I hadn't made no many projects. Thank you very much. Andre

…

________________________________ Van: Oleg-Malyutin ***@***.***> Verzonden: donderdag 4 mei 2023 16:51 Aan: Oleg-Malyutin/sdr_receiver_dvb_t2 ***@***.***> CC: andre1968 ***@***.***>; Comment ***@***.***> Onderwerp: Re: [Oleg-Malyutin/sdr_receiver_dvb_t2] Crash on Fedora with SDRplay (Issue #4) I am writing through a translator. 8-bit -127 128. I strongly do not advise the driver. Had experience with a similar msi driver on Linux. This is reverse engineering and a lot of mistakes. I did not set the exact frequency. Chinese SDRPlay should work well with native drivers (I have both the original and from China). I recommend a version no newer than 2.x. Regards. Oleg. — Reply to this email directly, view it on GitHub<#4 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AHSZ4F73UPU6Z7YTFE3IS5TXEPM75ANCNFSM5H4VNUGQ>. You are receiving this because you commented.Message ID: ***@***.***>

[andre1968]

Hello Oleg,
What documents did you use? I'm reading this one: ETSI EN 302 755 V1.3.1 (2012-04). Have you used also other documents?
Andre

[Oleg-Malyutin]

Hello.
Yes, this document and more:

• ETSI TS 102 831

• "Improved synchronization, channel estimation,
  and simplified LDPC decoding for the physical
  layer of the DVB-T2 receiver"
  Doaa H Sayed 1* , Maha Elsabrouty 2 and Ahmed F Shalash 3

• "Optimum Receiver Design for Wireless
  Broad-Band Systems Using OFDM—Part I"
  Michael Speth, Student Member, IEEE, Stefan A. Fechtel, Member, IEEE, Gunnar Fock, and Heinrich Meyr, Fellow, IEEE

• "Optimum Receiver Design for OFDM-Based Broadband
  Transmission—Part II: A Case Study"
  Michael Speth, Stefan Fechtel, Gunnar Fock, and Heinrich Meyr, Fellow, IEEE

• "A Low-complexity Feed-forward I/Q Imbalance
  Compensation Algorithm"
  Niels A. Moseley and Cornelis H. Slump
  University of Twente
  Signals and Systems Group
  Department of Electrical Engineering
  Mathematics and Computer Sciences (EEMCS)
  Enschede
  The Netherlands

Regards.
Oleg.

[vladisslav2011]

Hello, everybody.
I've forked this project some time ago and have added HackRF and generic MSi SDR support.
https://github.com/vladisslav2011/sdr_receiver_dvb_t2/tree/mirisdr
Updated libmirisdr-4, that should work a bit better is here https://github.com/vladisslav2011/libmirisdr-4
I've made MSi SDR actually work yesterday - got it running for ~30minutes or so.
MSi SDR works a bit worse, than HackRF (with TCXO). It shows more bit error artifacts when playing the stream with vlc player.
My laptop (i7-8550u) is barely enough to process 10Msps/256QAM.
And this receiver segfaults too often (mostly during P1 symbol acquisition).

[Oleg-Malyutin]

Hello.
8 months ago the project was significantly changed (there was time)
I hope the changes are for the better. But this is not a release yet.
I will try to find time and continue the work.

[vladisslav2011]

I've pulled recent changes, re-added HackRF/mirisdr support, fixed segfaults and some memory leaks (not all, I think) and fixed SMP bugs (it was designed as multi-threaded application, but it wasn't using more than 1 CPU core due to incorrect mutex/condition variable usage).
My laptop is able to process the 9Msps IQ stream with CPU "turbo" disabled, but it can't play the UDP stream in parallel as dvbt2_demodulator thread consumes ~95% of a CPU core (64QAM) and is very sensitive to context switches.
I think, running rotator/interpolator in another thread may improve the performance.
Disabling full P1 symbol acquisition after initial synchronization may help to improve the performance too.

[Oleg-Malyutin]

I'll look at your changes. If you tell me where to look, I'll be grateful.
When measuring performance, the maximum was at the QAM decoder, and with a bad signal at the LDPC.
I agree with the streams - it was done incorrectly (I started programming on this project).

[vladisslav2011]

My changes are here: https://github.com/vladisslav2011/sdr_receiver_dvb_t2/tree/mirisdr Feel free to cherry-pick, rebase, rewrite or just take ideas and implement differently.
It may be not the best code, that I had ever written, but it builds and runs. 😆
I've made it run without dropouts and buffer growing in the background while vlc is playing the UDP stream fullscreen even when my laptop's CPU has "turbo" disabled. No fan hiss and cold CPU. It consumes about 35% of Core i7-8550U @1800MHz while processing 10Msps 64QAM staream.
It still sometimes crash when running low on CPU, but it is in "usable" state to me now.
And sometimes it loses lock, fails to recover and runs out of lock for some time showing rotating P1 constellation and large sample rate offset...

HackRF constellation looks fuzzy, but it runs with a lot less LDPC errors, than mirisdr with a clean constellation.

Miri constellation is a lot cleaner...

HackRF: 2 LDPC errors in ~20 minutes, miri: 8 LDPC errors in ~2 minutes

Thanks a lot for an interesting project 👍

[Oleg-Malyutin]

Ok.
I'll take a look soon.
As for Msi and HackRF - this is one of the features of this project. It is believed that with the Zero-IF architecture it is impossible to receive DVB-T2 due to the leakage of the local oscillator at zero frequency. It turned out that it is possible. I think TCXO HackRF is the reason for better job, where the leakage is narrower.

[vladisslav2011]

It is believed that with the Zero-IF architecture it is impossible to receive DVB-T2

Hmmm. Lo leakage may slightly damage the central pilot, that can be ignored without any problems. So, Zero-IF frontends should not have any issues receiving DVB-T2 or any other properly designed OFDM (LTE for example).

I think TCXO HackRF is the reason for better job
My MSi SDR has a TCXO too (I have soldered it myself, replacing the stock crystal), so it should work more or less like HackRF does.

I've pushed some more fixes to reduce CPU consumption and improve stability and added Ettus USRP frontend support.
B200 works a bit better, than MSi SDR, but has similar problems with high order mixing products (interference from cellular towers)...

The constellation is not perfect, but it shows 31db SNR!

With recent modifications (P1 correlation result scaling) it is possible to receive a very weak signal. B200 shows 30db SNR even at 0db gain setting (see the P1 constellation).

[Oleg-Malyutin]

"central pilot" - that's the problem.
There is no central pilot, but an active cell.
Also, with a strong imbalance, it affects neighboring cells.
You disabled "DC offset remove" and "IQ imbalance remove" in vain.
Own corrections in drivers work worse.
LDPC restores everything, but consumes a lot of resources.

[vladisslav2011]

There is no central pilot, but an active cell.

I thought, there is second pilot acquisition:
https://github.com/Oleg-Malyutin/sdr_receiver_dvb_t2/blob/master/src/DVB_T2/data_symbol.cpp#L224-L256
But I've re-read the code and found, that there are the same calculations as in the loop, but without equalizer estimation...
Anyways, it is easy to tune the frontend slightly off the center to put the DC peak between cells and correct this offset in the AFC loop (not implemented yet).

You disabled "DC offset remove" and "IQ imbalance remove" in vain.

I've tried to do so, found, that things are not working well and I had to put both corrections back here (May be not the best place for that template definition...):
https://github.com/vladisslav2011/sdr_receiver_dvb_t2/blob/usrp/src/DVB_T2/dvbt2_demodulator.h#L59-L136

This template is used in the signal source blocks:
https://github.com/vladisslav2011/sdr_receiver_dvb_t2/blob/usrp/src/rx_hackrf.cpp#L310
https://github.com/vladisslav2011/sdr_receiver_dvb_t2/blob/usrp/src/rx_airspy.cpp#L233
and so on.
This moves some small fraction of CPU load from dvbt2_demodulator thread to rx_* thread.

[Oleg-Malyutin]

DC offset remove" and IQ imbalance remove must work with phase and frequency synchronization and timing synchronization. That is, with a restored signal.
Only then the effect is achieved.
A little offset will not do anything. Frequency synchronization and timing synchronization will return everything back.

[vladisslav2011]

Why?
These corrections do not depend on the symbol demodulator output and, I think, may be applied out of sync.
DC offset correction is just a narrow high-pass IIR filter. There is no reason to make it work in sync. Or it may be applied after the FFT to the central bin only.
Elliptic distortion correction (gain and angle) may be applied out of sync too as mixer imperfections and LO phase offset are changing slowly as the device heats up.

The sample rate correction and phase angle correction depend on the demodulator output, so delaying these by arbitrary number of sample breaks tracking loops. That's why I have not moved these corrections to rx_* blocks.

My tests are showing that the central FFT bin was having a small random magnitude offset before my changes and much the same after my changes. The elliptic distortion correction is working a bit better after my changes - the HackRF demapped constellation was showing some distortion before my changes and almost no distortion after my changes.

[Oleg-Malyutin]

If simply:
To remove distortions introduced only by the SDR device (and not the sum of distortions caused by frequency detuning), precise synchronization is needed.
For example, how to remove DC of the device if the signal is not in the center (not symmetrical) and has a constant offset that can appear when selecting the envelope. (The example is, of course, rough)
Correction in the driver is sufficient if you have, for example, only symbolic synchronization. Here the signal is very dense - the theoretical limit of bit transmission per hertz. Synchronization is necessary not only of the carrier frequency, but also of the sampling frequency.

[Oleg-Malyutin]

What is happening with the central FFT bin can be seen on the "Miscellaneous plot" tab in the equalizer operation graphs.

[Oleg-Malyutin]

I launched your version and mine.
I did not notice any difference in performance and loading of all CPU cores. (I roughly estimated it using System Monitor Mate).
The constellations are also the same.
But LDPC triggering - never for me, constantly for you.
(I don't have time for a long test yet)
Still, I think all corrections should be in the synchronization loop. For the same DC offset, there are three reasons for its occurrence. I think no less for IQ imbalance and it is difficult to take into account all the influences (or non-influences) of synchronization.

[vladisslav2011]

I see increase in LDPC decoding failures even when IQ corrections are performed in the dvbt2_demodulator in sync (before vladisslav2011@22ee9e6), so it's different issue.
It is related either to l1_postsignalling state desynchronization (starting of vladisslav2011@d8b2697) or due to reduced number of LDPC decoding trials (from 25 to 15 in my fork).

[Oleg-Malyutin]

"LDPC decoding trials (from 25 to 15 in my fork)".
Yes, of course. I didn't notice that.
I'll do the tests again.

[Oleg-Malyutin]

preprint_DC.pdf
Here DC offset is described outside the loop and in the synchronization loop. Advantages and disadvantages are at the beginning of the article..

[vladisslav2011]

The article is interesting, but tracking power distribution grid frequency is completely different challenge...
I think, I've fixed the "LDPC error bursts" issue by changing the sample rate offset tracking algorithm and... by making a low-pass hardware filter (a very bad filter, made from scrap components, sort of ~2GHz rejector+3db attenuator instead of a real filter 😃 ).
One multiplex is being received really well, but another multiplex is polluted by MSi SDR internal spurs and still produces some LDPC errors (one per ~5 minutes or so).
USRP works without any issues, HackRF does not produce LDPC error even without a filter due to double conversion architecture.
And I have figured the main CPU eater in the dvbt2_demodulator block - it is the resampler, that consumes ~60% of time.
I'll try to replace it with a lightweight MMSE resampler (8-tap FIR, that requires 2 aligned AVX multiplications per sample).

[Oleg-Malyutin]

I had to remember my project.
I also have DC-offset and IQ-imbalance not in the synchronization loop, but I conducted tests for 15 minutes on AirSpy (with trials = 25) and in your version LDPC triggered 5 times more often. I don't see any fundamental difference, maybe there are still errors in your version...
As for the performance, there is a place that I couldn't optimize - llr_demaper. Namely, de-interleaving (in QAM64 this is pp. 487-508).
I also didn't try interpolation without increasing the frequency. Then the decimator is not needed. This option just worked.

[vladisslav2011]

I'm testing commit 7a51d45 using MSi SDR frontend now and it runs for hours receiving the first multiplex without LDPC decoding failures, but it fails in several minutes, receiving the second multiplex... Even commit 9026004 with minimal modifications fails quickly. And the second multiplex shows very bad frequency estimation stability and demapped constellation stability. The constellation is "breathing" and eventually breaks completely for a duration of one symbol, but instantly recovers. It may be possible that some pulsed interference is not fully cancelled by the LPF. There may be a bug in the receiver (likely) or an issue with the transmitter (clipping or distortion, unlikely as the conventional TV receives it quite well).

The llr_demapper performance is acceptable to me. There are 2 main CPU eaters, that want decent single thread performance: dvbt2_demodulator and ldpc_decoder. I've tried removing the resampler completely and setting the input rate to 9142857Hz - it works well (the dvbt2_demodulator consumes ~ 40% CPU) when the UDP stream is not played even in the "battery mode", but the LDPC decoder starts eating the whole CPU core upon VLC startup when running in the "battery mode".
It is even possible to play the stream with the resampler in place (commit 7a51d45) in "battery mode", but only when VLC is in full-screen state. When the VLC is windowed, dvbt2_demodulator starts eating the CPU and growing buffer... I think, this is memory throughput bottleneck, so improving cache locality and reducing the number of allocations may help.

[vladisslav2011]

I think, I've fixed the "LDPC decoding failures" issue. There was a race between time_deinterleaver and llr_demapper.
It demodulates both the 1st and the 2nd multiplexes without too many LDPC decoding failures now.

I've experimented with smaller FIR filters in the decimator and found, that small 16-tap filter works well at least for the 64QAM.
It is possible to play 3+channels at once using ffplay instead of vlc even in the "battery" mode as for some unknown reason ffplay does not interfere with the demodulator thread...
I've fixed some more segfaults and added LDPC decoder statistics plot.
Most recent changes are here: https://github.com/vladisslav2011/sdr_receiver_dvb_t2/tree/dev