Realtime WebRTC Prototype

[sipsorcery]

I've created a prototype dotnet app that works with the OpenAPI WebRTC Realtime endpoint and is available here.

Thanks to @lostromb's C# OPUS port the prototpye is completely dotnet, no native libraries required.

The OpenAPI Realtime server event models currently seem to be incomplete in this openai-dotnet SDK. Hopefully they'll be added in the future. I only wired up a couple in the prototype.

A shortcoming of the app is lack of software echo cancellation. The consequence of which is that ChatGPT starts talking to itself and gets very creative with the number of ways it can say hello etc.

[21:58:41 INF] Transcript done: Hey there! How's it going? What can I help you with today?.
[21:58:44 INF] Transcript done: Hey! What's up? How can I assist you today?.
[21:58:46 INF] Transcript done: Hey there! How can I help you today?.
[21:58:49 INF] Transcript done: Hello! What's on your mind today?.
[21:58:51 INF] Transcript done: Hi! How can I assist you today?.
[21:58:54 INF] Transcript done: Hey! What can I do for you today?.
[21:58:56 INF] Transcript done: Hey there! How's it going?.
[21:58:59 INF] Transcript done: Hello again! What's new with you?.
[21:59:02 INF] Transcript done: Hi there! How can I help you today?.
[21:59:05 INF] Transcript done: Hello! What's on your mind today?.
[21:59:09 INF] Transcript done: I'm here to help with whatever you need! What's on your mind?.
[21:59:12 INF] Transcript done: Great! What can I do for you?.
[21:59:14 INF] Transcript done: Awesome! How can I assist you today?.
[21:59:17 INF] Transcript done: What's on your mind? .

[lostromb]

I had played around a bit with acoustic echo cancellation in general using code from the Alanta media client but was never really able to get it to work - you need your own way of estimating the feedback delay, and it doesn't seem to like setting a delay of zero. But maybe someone else could pick up the thread....