Add listener for background noise to dynamically update levels

[ajbogh]

Speech is hardcoded for a level below -2000 and above 2000 but not more than 7000. Background noise is determined to be above 7000. This leaves little room to detect speech, and the user has to speak clearly and loudly over the background noise.

These numbers should be variables depending on a small background noise sample. A few seconds should give an average upon startup, and perhaps when silence is detected it could be updated -- at some interval.

This would allow louder backgrounds to be omitted as well as quieter speech to be understood.

[ajbogh]

Perhaps some background noise data can be stored in memory for comparison against the incoming mic data.

Another thought is to use RNN (neural network) to reduce noise in the audio files. https://github.com/xiph/rnnoise/

I believe the most interesting idea would be to add RNN to the incoming data directly, but instead of using sample data, it could build its own knowledgebase of background noise from the user's microphone itself.

Consider other approaches as well: https://pdfs.semanticscholar.org/fc57/e1146a8906e4e59cbb048ad15dbf4950299c.pdf

The easiest may be filtering using a library like DSP: https://github.com/corbanbrook/dsp.js

[ajbogh]

I've added xiph/rnnoise to the project using napa. In order for it to work, you must install ffmpeg (instructions in README.md) and run npm install after pulling the latest code from master.

The change is in this commit: 9448592

[mateolan]

downloadd latest source, but still getting ".../norman/lib/../node_modules/rnnoise/examples/rnnoise_demo: No such file or directory" after running 'npm run listener'

[ajbogh]

Hi @mateolan. I'd be happy to merge in a fix if you would be willing to help out with your bug.

The best suggestion that I have is to run npm install once again, but if that doesn't fix the rnnoise problem then we may have to do more development.

I would also welcome new PRs on the project to add more capabilities and better understanding of speech if you're willing to help out.

[vandonsel]

I'm also getting that error every time I run the listener and say something. There doesn't appear to be rnnoise in the node_modules folder. I've grabbed the latest build I believe and I've run npm install several times:

[d@zen norman]npm run listener -- --mic hw:0,0

> [email protected] listener
> node ./lib/listener.js "--mic" "hw:0,0"

{ _: [], mic: 'hw:0,0', '$0': 'lib/listener.js' }
Got SIGNAL startComplete
Got SIGNAL silence
Speech blocks found: 4216
Got SIGNAL stopComplete
Saved audio
Cleaning up background noise...
/bin/sh: line 1: /home/d/builds/norman/lib/../node_modules/rnnoise/examples/rnnoise_demo: No such file or directory

Then the program quits

[ajbogh]

@vandonsel I may have had the package.json incorrectly set up. If you look at the scripts section you'll see an "install" script which calls other scripts, such as "install:rnnoise". These should really be under a "postInstall" script.

I recommend creating a PR to change the "install" script to "postInstall".

[vandonsel]

@vandonsel I may have had the package.json incorrectly set up. If you look at the scripts section you'll see an "install" script which calls other scripts, such as "install:rnnoise". These should really be under a "postInstall" script.

I recommend creating a PR to change the "install" script to "postInstall".

Sorry, I meant to delete that comment. I don't think it was appearing because I kept running "audit" like a dummy. You can re-close this I think.

EDIT: I'd recommend putting in a note in the installation instructions to ignore the audit prompts from npm, though.