Using ADC to get multiple audio streams

[hpmax]

I am trying to make an audio mixer and trying to figure out if arduino-audio-tools and/or the ESP32 are appropriate for this function.

I plan to run audio to four different ADC1 pins. The sampler will presumably have to run at 4x the normal sampling rate and do a "round robin" on which input pin I'm using.

The goal is to determine whether sound is present on each input pin -- effectively creating a 4 bit number covering all the possible scenarios. A gain factor for each input pin is then looked up in a table based on that 4 bit number. Each input pin data is multiplied by its respective gain factor, summed, and then output to a MAX98357A I2S Class D auto amplifier. Audio quality needs to be decent, but high fidelity music replication is unnecessary, the sound will be beeps, and tones, and voice not music. Latency is an issue, and more than .1 seconds of latency would be undesirable.

Is this practical? Possible? I didn't really see any mention of doing the round robin with the input stream

[pschatzmann]

I see quite a few challenges here.

If you want to feed a signal directly to the ESP32, you need to make sure that the input voltage is in the expected range between 0 and 3.3V - oscillating around 1.6V. Dependent on your audio source you might get quite different values and in the worst case you might see positive and negative voltages oscillating around 0V!

I found that the only easy way to do audio is to use the I2S interface (e.g. from the internal ADC. But this way you can get max 2 input channels). Collecting ADC values with a timer is slow and just a pain! In addition I would expect the audio quality to be quite poor!

The best results you can get with some external audio ADCs. The ESP32 has 2 I2S ports, so you can attach max 2 of these which will give max 4 input channels (2 with separate left and right). But here as well you still need to make sure that your input is a line level.

One of these I2S ports you can run in duplex mode to do the output to the MAX98357A,

But to be sure what you really get using the internal ADCs driven by a timer, you would need to implement a prototype. The TimerCallbackAudioStream class might help to get started and you just need to implement the callback that provides the data.
Just don't expect high sampling rates (I think you can go up to max 10k samples per second)...

[pschatzmann]

4 single ended channels....
Yes that's what you would get from an ES8388 as well, including the software controlled gain...

[pcbartists-admin]

Sorry, I meant 8 single ended channels (4 diff) on the PCM chip.

[pschatzmann]

It's just a pity that the PCM186x based eval boards are so expensive. But I agree, for a professional project it makes more sense. I am doing audio as a hobby so I am looking for affordable modules...

[hpmax]

That eval board is bizarrely expensive. I mean, you'd think manufacturers would make them cheap to sell chips. The big issue I see with the PCM186x is it burns a LOT of power. The datasheet shows 85mW for 2 channel mode, and 145 mW for 4 channel. By comparison, the TLV320ADC6140 part burns about 9.5mW/channel for very similar performance with an I2C interface, making it simpler to interface (and less wires to be routed).

Any reason to prefer the STM32 over the ESP32. I'd think if you were using an external ADC then it really wouldn't matter which processor you use?

The ESP32 only has 12 bit ADCs as far as I'm aware of, which gives the external ADC a huge advantage because the externals are either 24 or 32 bit. Although it would seem that noise is the limiting factor and you can't really get better than 20 bits before being limited by noise.

@pcbartists-admin Do you have any example code of reading out 4 ADC audio channels on the ESP32?

[pcbartists-admin]

ESP32 has its own set of issues when you are working with audio ADCs. The noise spikes from ESP32 pretty much always find a way to the codec in my experience. If not that, the I2S module of the ESP32 is not as good/flexible as the ones on the STM32 chips. For example, you can use the TDM mode on TI chips using STM32. With ESP32, it can be a problem unless you want to mess with register level I2S configuration, which in turn might end up breaking IDF I2S drivers.
The ESP32 TDM code I worked on is confidential, part of some work we did for a client. But that is not something I would look forward to doing unless it was the last resort (the client had thousands of boards sitting and ESP32 + PCM186x was the only way).
As for power usage, you can save power with the STM32L4 series and make up for additional power consumption in the ADC I guess.

And the ESP32 internal ADC - I wouldn't use it with audio applications. It can barely read levels on a battery pack! You'd be way better off using 2x ES8388 on 2x I2S ports of the ESP32.
On any codec (ES8388, SGTL5000, etc), simply route

• line in -> line out
• line in -> i2s

You can then read data via I2S and set line in to line out gain accordingly. That makes a good mixer with gain being processed by the codec's analog circuits rather than discrete math. I am assuming you need analog out and not necessarily digital output? Doing math on I2S audio streams can get complicated if you want good audio quality.

[pschatzmann]

I am not sure if there is an Arduino library for theses chips.
To write a new one is usually a major undertaking....

Did you measure effective input voltage for your case ?

[hpmax]

My bigger concern is that I'll be able to use your software to get 4 separate streams from the 2 chips, and then merge them and send them out to the DAC without an issue.

[pschatzmann]

I suggest to make one step back and that you try the built in ADC with my suggested class: There you would merge the samples in your code by just adding them up and then scaling the result to fit into the int16_t range.

If you go the I2S way it you will end up using the panning function of the VolumeStream to weight the different inputs separately...

[hpmax]

Now I'm thoroughly confused. Didn't you just say that I couldn't reliably use the built in ADC because of my need to get accurate timing if I needed to multiplex the converters to sample all 4 channels on 2 converters?

[pschatzmann]

For high sampling rates yes. But you mentioned that it's only speech, so I assumed that you might be ok with a sample rate of only 8000. In the end it's just a question what quality you can accept...

And to figure out what you really get I was recommending that you just program a simple prototype...

If you go with one of this complicated ADCs that require configuration via i2C, you might even do the mixing in the audio chip...

[hpmax]

I don't know if it's possible to do the mixing on the chip, or how you would since each chip only has two inputs channels. Even if you could mix two, you'd have to get two signals into the DAC chip from two separate sources. You can probably do the gain within the chip though since they have programmable filters built in.

8000 samples per second is probably acceptable but it's on the very low end of what I'd consider acceptable. It also makes me wonder if I'd need anti-aliasing filters and if they're builtin to the ADC chips (possibly) or the ESP32 (seems unlikely).

I just spent all weekend re-laying out my board to include the ADCs. :(

[pschatzmann]

Before doing anything with a custom hardware board, I would suggest to build a hardware prototype with some modules: e.g. https://pcbartists.com/product/es8388-module/

[hpmax]

I hate to say it, but my go to usually is custom hardware. In this case, I'd likely still need transformers and possibly resistors and capacitors to filter before being able to just sample it from the part. Plus, I have this awful wiring mess which could create all sorts of noise and reliability problems. If I have a 3.5mm audio connector I might need to buy a receptacle anyway.

I don't like debugging hardware and software at the same time, but I feel like I'm in for that no matter what.

Can I use your software to take on left and right audio from the ADCs, combine them and output to a DAC? Can I use it to read the internal ADC and multiplex across 4 channels either with 1 or both ADCs? What really worries me is that I'm designing a board that relies on the ESP doing something it can't do (hooking something up to a pin that can't be used for that purpose for instance) or assuming the software supports a feature it doesn't.

[pschatzmann]

I never used that module, but what I liked was that all the audio pins seem to be available and you don't need to go via the audio connector.

I would pay the price of the risk of noise and messy wires when you trade in the certainty that your selected hardware and software scenario will work. Since I have never done this myself, I only know what should work theoretically..,

• Can I use it to read the internal ADC and multiplex across 4 channels either with 1 or both ADCs.
  • With the AnalogAudioStream (which uses DMA via I2S) you can process max 2 channels on I2S port 0, so the answer is no!
  • With the TimerCallbackAudioStream you will do the reading and combining of the signal yourself and you can process as many input pins as the ESP32 can support. But as I have written, the sample rate is limited and you need to decide if the quality is sufficient. But the beauty of this variant is low software and hardware complexity. I would try this variant and it might be just to get a baseline to which any other variant can be compared with.
• Can I use your software to take on left and right audio from the (external) ADCs, combine them and output to a DAC
  • My proposed processing chain will be for each I2S ADC: I2SStream-> VolumeStream -> ChannelFormatConverterStream. You get 2 input channels. With the VolumeStream you can modify the volume of each channel separatly and with the ChannelFormatConverterStream you can combine the 2 stereo channels to 1 mono output.
  • Last you can combine the 2 mono singnals from the 2 ADCs with the InputMixer.

[hpmax]

To me the big cost of doing custom hardware is the time to perform layout and the chance of getting things wrong because of a lack of understanding. With software, if you do something wrong you can reconfigure it, with hardware, often times you have to chuck it. You may be able to salvage it to continue testing, but in the end you probably need to replace it to get fully correct functionality. But now adays, the cost is pretty low.

JLCPCB will make you a 4 layer board for around $5. As long as you limit yourself to parts they stock you can get it assembled at a cost of about $2 per unique part that isn't a "basic" part (most passives will be "basic"). So, in my case, I probably have 10 or 12 "extended parts", so figure about $24. So I'd guess it costs about $30 + cost of parts, plus $15 shipping and I have something that does EXACTLY what I want about 2 weeks after I order it. That compares pretty favorably to the generic PCM186x eval module, right? And of course the price amortizes down if I need a few boards, since the fixed costs are no different if I want 1 or 5, and the parts costs may actually decrease a bit (on a per board basis) becausecomponent price goes down with quantity.

In response to your answer. Thanks for explaining more clearly.

For "AnalogAudioStream" you say I can process a max 2 channels on I2S port 0, so the answer is no! Could I process 2 channels on I2S port 0, and another 2 channels on port 1?

For "TimerCallbackAudioStream", again I'd think by splitting it up over the two ADCs, I could essentially run two TimerCallBackAudio streams and get pretty decent performance?

The external ADC solution sounds pretty straightforward, just a fair bit more hardware.

[pschatzmann]

No, Analog is only supported on port 0 and not on 1.
You just need one TimerCallbackAudioStream and there you can read as many pins as you want. I never tried, so I don't know if it helps to split it up by ADC. However if I remember right if you plan to use the Wifi Module, you can't use the second ADC...

ps. PCMArtist mentioned TDM. This seems to be supported by some of the new ESP32 variants using IDF > 5.0.

[pschatzmann]

For a start I tried to capure audio with just one pin using the following sketch

#include "AudioTools.h"
#include "esp_adc_cal.h"

CallbackStream in;
CsvStream<int16_t> out;
StreamCopy copier(out, in); 

size_t read(uint8_t *data, size_t len) {
  int16_t v1 = analogRead(13);
  //int16_t v2 = 0.5 * analogRead(13);
  int16_t *p_result = (int16_t *)data;
  p_result[0] = v1;
  return sizeof(int16_t);
}

void setup() {
  Serial.begin(119200);
  // configure ADC
  //analogSetAttenuation(ADC_0db);
  analogReadResolution(12); //  max 12 bits resulution
  analogSetClockDiv(1);

  // Setup CallbackStream
  in.setReadCallback(read);
  AudioBaseInfo cfg;
  cfg.channels = 1;
  cfg.sample_rate = 8000;
  cfg.bits_per_sample = 16;
  in.begin(cfg);

  out.begin(cfg);
}

void loop() {
  copier.copy();
}

Feeding a sine signal generated from the pc was giving this

So, this does not look good at all...

[theSchoofseggl]

Hi,

I was excited to come across a thread where someone had faced a similar challenge, and I hope it's okay to reopen the discussion. If it’s an inconvenience, I’m happy to start a new question instead!

I’m curious to know if and how you eventually solved the problem you were facing. Any insights or advice would be greatly appreciated!

I'm working on a project where I need to manage and prioritize three audio channels from three different radios. The audio quality doesn't need to be high-definition, but it’s crucial that the channels are sampled and prioritized correctly. The highest-priority channel should then be output via Bluetooth. Essentially, I'm aiming to create a multiplexer.

Given the complexity of managing multiple channels, my idea is to use three analog Schmitt triggers to monitor the signals and send an "Audio ON" signal to a digital pin. A digital output would then switch the three analog signals to an I2C ADC. That would be avoid the problems of to many inputs. The software would handle the switching between the signals, similar to this project: Audio Switcher Arduino.

[pschatzmann]

please open a new discussion since you topic has nothing to do with this discussion!