- Pallavi Das - [email protected]
- Ryan Gertz - [email protected]
- The frontend React app initializes by displaying a landing page where it lists calls with the "active" attribute set to true from an AWS DynamoDB table.
- Clicking on a call box leads to a call page where a WebSocket connection is established with AWS using the react-use-websocket library.
- The app displays the top 3 predictions made by the AI model in real-time, represented by dynamically sized bubbles.
- Once the AI model's prediction stabilizes, the WebSocket sends a boolean value, and the UI transitions to show the dominant prediction and instructions for the operator.
- Node.js
- Vite
- Public facing S3 bucket
To install project dependencies:
npm installRun the application locally with:
npm run devBuild and package the project using:
npm run buildDeploy to a public S3 bucket with:
aws s3 sync <build directory> <s3 destination link>- Overflow issues with lengthy call instructions.
- Hardcoded placeholder text in various components.
- The backend of the SPD Care Project is constructed using AWS serverless technologies which are orchestrated to handle data-intensive operations and real-time interactions.
- Key functionalities include audio processing, data transcription via AWS Transcribe, and real-time prediction delivery through WebSocket connections managed by AWS Lambda functions.
- The system is designed for high availability and scalability, ensuring it can handle an increasing load of data and user interactions without performance bottlenecks.
The backend is responsible for audio processing, transcription via AWS Transcribe, storing call data, and sending predictions to the frontend through WebSocket connections managed by AWS Lambda.
This Python script records audio from a local microphone, sends the audio stream to AWS Transcribe, and writes the transcription to the current_calls DynamoDB table in real time, excluding the 911 call operator's speech.
- AWS CLI
- Python 3.x
- Libraries: Boto3, Sounddevice, Amazon Transcribe Streaming Service
Before running the script, ensure that the AWS credentials are set up with the required permissions to access Transcribe services and DynamoDB.
Start processing audio streams with:
python3 audio_diarization.pyTriggered by updates to PredictionsTable, it evaluates prediction stability and communicates with the frontend.
Manage WebSocket connections, adding or removing entries in ConnectionIdMappingTable.
This table stores connection IDs for WebSocket clients, which are used to send prediction data to the frontend.
This table holds the real-time transcription data, with each row representing a single call. The audio_diarization.py script updates this table.
This table captures prediction data from the machine learning model. The SendSessionData Lambda function reads from this table to process and send predictions.
This interface provides full-duplex communication channels over a single TCP connection. It enables real-time data exchange between the client and server.
connect: Invoked when a client establishes a new WebSocket connection.disconnect: Invoked when a client's WebSocket connection is closed.
- Hardcoded 'Seattle' as call location.
- Lack of support for concurrent call handling.
- Enable multiple simultaneous call sessions.
- Dynamically determine the operation location.
- Accurate stability modeling.
Use AWS Serverless Application Model (SAM) for deployment with the template.yaml file:
sam build
sam deploy --guidedFor any queries or issues, please contact:
- Darren Kraker, Sr Solutions Architect - [email protected]
- Pallavi Das, Software Developer Intern - [email protected]
- Ryan Gertz, Software Developer Intern - [email protected]