This project addresses the need for personalized attention and safety monitoring in elderly care facilities. Leveraging IoT devices, computer vision, and AI, our system aims to reduce risks associated with mental health issues and physical accidents, such as falls, by providing real-time alerts for caregivers. This allows for early intervention, reducing the reliance on in-person monitoring.
According to the World Health Organization (WHO), by 2050, one in six people globally will be over 65. In Mexico, the population over 60 is expected to double by 2030, emphasizing the need for improved elder care. With only one caregiver for every ten elderly individuals, there is a risk of undetected mental health issues, such as depression or anxiety, and physical accidents. This project seeks to address these issues through automated detection and alert systems.
Our system is designed to:
- Detect Falls: Using data from accelerometers and gyroscopes on an ESP32 module, combined with computer vision on a cloud server, the system detects falls in real-time.
- Monitor Emotional Health: Employing YOLO (You Only Look Once) for emotion detection, the system captures facial expressions to identify signs of mental distress, such as anxiety or depression.
- Real-Time Monitoring and Alerts: WebSockets enable the ESP32 to stream live video feeds to a server for processing and display, ensuring caregivers can monitor the elderly in real-time.
-
Hardware:
- ESP32 Microcontroller: Captures sensor data and transmits video frames for processing.
- MPU6050 Accelerometer and Gyroscope: Tracks movements to detect falls.
- Camera: Provides real-time facial data for emotion detection.
-
Software and Communication:
- MQTT Protocol: Ensures secure and efficient data transfer for sensor data.
- WebSockets: Allows real-time video feed transmission from the ESP32, enabling immediate processing and display on the caregiver’s dashboard.
- Ubuntu Server: Hosts the AI models, processes data, and provides real-time visualization.
- Computer Vision with YOLO: Detects falls and evaluates facial expressions to monitor emotional states.
- WhatsApp Notifications: Using
wwebjsandjspybridge, the system sends immediate alerts to designated WhatsApp contacts when a fall is detected or if an emotional distress signal is identified, allowing caregivers to act quickly.
To further enhance real-time monitoring, our system utilizes wwebjs and jspybridge to send automated alerts through WhatsApp. This functionality is triggered by the server upon detecting a fall or significant emotional shift, delivering notifications directly to the caregivers’ smartphones. This ensures that they are promptly informed of any potential risks, even if they are away from the monitoring station.
- When a fall is detected or an emotional distress signal is identified, the server triggers a notification via
jspybridge, which communicates with a JavaScript file usingwwebjs(WhatsApp Web JavaScript API). - The message is automatically sent to a pre-configured list of contacts, providing details of the detected event and ensuring a fast response.
Our modular IoT architecture divides the system into three main components:
- Sensing: Collects data on body movements and facial expressions.
- Communication: Transfers data securely using MQTT for sensor data and WebSockets for real-time video feed.
- Processing: Analyzes data on the server, triggering alerts when necessary and displaying results in real-time on a dashboard.
Below are images demonstrating our system’s current progress in detecting falls, monitoring emotional states, and analyzing sensor data.
Image 1: An example of the fall detection system accurately identifying a simulated fall.
Image 2: The YOLO-based emotion recognition system analyzing facial expressions to detect emotional states.
Image 3: Visualization of sensor data from MPU6050 accelerometer and gyroscope.
This system aims to enhance the quality of life for elderly individuals in care facilities by offering proactive monitoring and reducing risks associated with both physical and emotional health. Real-time data and video feeds empower caregivers with timely insights, enabling effective resource allocation and rapid intervention.