MediBot: A Continuous AI-Powered Patient Monitoring for Hospitals.

# MediBot: A Continuous AI-Powered Patient Monitoring for Hospitals. ## The Problem In many Nigerian hospital wards, one nurse monitors between 10 and 20 patients at the same time. That is not negligence. That is the reality of what the healthcare system has to work with. The consequence is a monitoring gap that no amount of dedication can close by hand. Oxygen saturation can fall from a healthy level to a life-threatening one in under two minutes. A patient losing consciousness may go unnoticed for 20 minutes or more if the nurse is attending to another emergency. Pain in sedated or non-verbal patients often goes completely undetected because there is simply no continuous system watching closely enough. The standard solution to this problem costs millions of naira per ward. ICU-grade monitors are expensive, difficult to maintain, and built around assumptions that do not match the realities of many Nigerian hospitals. MediBot was built around those realities from the beginning. The goal was never to build another expensive hospital machine. The goal was to build a system that could realistically exist in Nigerian hospitals using affordable hardware, open-source software, and infrastructure that hospitals already have access to. --- ## What MediBot Does MediBot continuously monitors patients without requiring a nurse to remain physically at the bedside. The system combines: - Contactless temperature monitoring - Heart rate monitoring - Blood oxygen saturation monitoring - AI-powered facial analysis - Pain detection - Eye-state analysis - Real-time patient state classification - Remote dashboard monitoring to determine what condition the patient is likely in at any given moment. When everything is stable, the system stays quiet and keeps watching. When something becomes dangerous, it alerts medical staff immediately. The purpose is not to replace nurses or doctors. The purpose is to help them see more patients safely with the limited manpower already available. --- ## Why Contactless Monitoring Matters One of the biggest focuses of MediBot is contactless monitoring. Many hospital systems only react when a patient manually reports distress or when a nurse physically checks them. That creates long periods where deterioration can happen unnoticed. MediBot reduces that gap by continuously observing the patient using both sensors and AI vision. The MLX90614 infrared sensor measures temperature without touching the patient. The camera system continuously watches facial expressions, eye state, and movement patterns. The AI models estimate pain levels and determine whether the patient appears awake, asleep, distressed, or unconscious. The only contact-based sensor currently in the system is the MAX30102 pulse oximeter used for heart rate and blood oxygen saturation monitoring. That decision was not because a contactless approach was preferred less. It was because a reliable and affordable contactless alternative that could realistically be sourced locally in Nigeria was not available during development. Rather than delay the entire system waiting for hardware that may be difficult or expensive to obtain locally, the project prioritised building a working monitoring pipeline around components that could actually be sourced and deployed within the Nigerian ecosystem. Everything else in the monitoring pipeline was intentionally designed to minimise physical interaction with the patient as much as possible. This matters especially in situations where patients: - Cannot communicate clearly - Are sedated - Are unconscious - Are post-operative - Are critically ill - Are being monitored remotely The system keeps watching even when nobody else can. --- ## Innovation Built Around Nigerian Realities Most medical monitoring systems are designed for hospitals with stable electricity, strong internet infrastructure, large budgets, and dedicated biomedical engineering teams. MediBot was designed around the opposite assumptions. That constraint shaped every engineering decision in the system. --- ## Low-Cost Embedded AI The entire system runs on ESP32 hardware costing only a fraction of traditional hospital monitoring equipment. Instead of requiring expensive GPUs or cloud servers, MediBot pushes intelligence directly onto embedded devices. The ESP32 handles: - Sensor readings - Patient-state logic - Alert management - Communication with the dashboard while the vision system handles AI inference separately. This two-board architecture exists because memory is the real constraint. Splitting responsibilities across devices allows the system to remain responsive while still running AI models and live monitoring simultaneously. --- ## Browser-Based AI Vision One of the most important innovations in MediBot is the browser-based AI vision pipeline. Instead of requiring expensive dedicated AI hardware, the current production prototype uses a smartphone camera running `face-api.js` directly inside the browser. That means: - No cloud inference - No GPU server - No installation - No expensive AI workstation A regular smartphone becomes the AI vision system. The browser continuously performs: - Face detection - Expression classification - Eye-state detection - Pain estimation and sends those results directly into the patient-state engine. This is important because smartphones are already widely available in Nigerian hospitals. MediBot turns existing devices into clinical monitoring tools instead of requiring hospitals to buy entirely new infrastructure. ```javascript // Browser-based AI vision pipeline using face-api.js const dets = await faceapi .detectAllFaces( vid, new faceapi.TinyFaceDetectorOptions({ inputSize: 224, scoreThreshold: 0.4 }) ) .withFaceLandmarks(true) .withFaceExpressions(); // Smooth emotion scores across frames Object.keys(phoneExprBuf).forEach(k => { phoneExprBuf[k].push(expr[k] || 0); if (phoneExprBuf[k].length > PHONE_SMOOTH) { phoneExprBuf[k].shift(); } }); // Weighted pain score estimation let pain = 0; Object.keys(PAIN_W).forEach(e => { pain += (smoothed[e] || 0) * PAIN_W[e]; }); pain = Math.round( Math.max(0, Math.min(100, pain * 160)) ); // Eye state detection using Eye Aspect Ratio const avgEAR = (ear(left) + ear(right)) / 2; eyeState = avgEAR < 0.22 ? 'closed' : 'open'; ``` --- ## Designed for Usefulness, Not Demonstration Many prototypes work only in ideal conditions. MediBot was built around usefulness first. The system focuses heavily on: - Fast deployment - Reliability - Low hardware cost - Low bandwidth usage - Minimal maintenance - Real-time usefulness The dashboard works on phones because phones are the most available computing devices in many wards. The alert system runs independently of the dashboard because internet access cannot always be trusted. The vision system continues running locally even during network instability. Every part of the system was designed around making sure it remains clinically useful under imperfect conditions. --- ## Reading the Patient MediBot combines multiple forms of monitoring together because no single signal is reliable enough on its own. The system reads: - Heart rate - Blood oxygen saturation - Temperature - Facial expression - Eye state - Motion patterns and combines them to determine what state the patient is likely in. The patient-state engine classifies conditions such as: - Awake - Sleeping - Distressed - In pain - Unconscious - Critical emergency Emergency logic always runs first. If vitals become critically dangerous, the system immediately escalates regardless of what the camera sees. ```cpp // Emergency classification logic bool hrCritical = hrPresent && (hr < 30 || hr > 150); bool spo2Critical = spo2Present && (spo2 < 85); bool tempCritical = tempPresent && (tempStable < 34.0f || tempStable > 41.0f); if (hrCritical || spo2Critical || tempCritical) { patientState = "unconscious"; distressLevel = "CRITICAL"; painScore = 100; return; } ``` When vitals are stable, the AI vision system helps determine whether the patient appears comfortable, asleep, or in visible distress. --- ## Sensor Detection and Reliability The firmware scans the I2C bus during startup and confirms which sensors are connected before monitoring begins. This makes hardware debugging significantly easier in real deployment environments. ```cpp // Boot-time I2C sensor scan void scanI2CBus() { for (byte addr = 1; addr < 127; addr++) { Wire.beginTransmission(addr); if (Wire.endTransmission() == 0) { Serial.printf("Found 0x%02X", addr); if (addr == 0x57) Serial.print(" <- MAX30102"); if (addr == 0x5A) Serial.print(" <- MLX90614"); Serial.println(); } } } ``` The MAX30102 sensor also performs a raw infrared-value validation before attempting a reading. If no finger is detected, the collection cycle is skipped entirely to keep the system responsive. --- ## The Dashboard The MediBot dashboard is a single HTML application deployed as a Progressive Web App. It works on: - Phones - Tablets - Laptops - Desktop browsers without requiring installation. The dashboard displays: - Heart rate - SpO₂ - Temperature - Pain score - Emotion analysis - Consciousness state - ECG waveform - Alert history - Camera feed in real time. Expression labels are mapped into clinically meaningful categories before entering the patient-state engine. ```javascript // Clinical expression mapping const CLINICAL_EXPR = { pain: { icon: '😣', label: 'Pain', color: '#ef4444' }, disgusted: { icon: '😣', label: 'Pain', color: '#ef4444' }, fearful: { icon: '😣', label: 'Pain', color: '#ef4444' }, angry: { icon: '😠', label: 'Pain', color: '#ef4444' }, happy: { icon: '😊', label: 'Happy', color: '#10b981' }, neutral: { icon: '😐', label: 'Neutral', color: '#4a6480' } }; ``` The interface was designed mobile-first because mobile phones are often the most accessible devices in Nigerian clinical environments. --- ## Project Gallery ### Machine Learning Training  ---  --- ## The Technology Behind It MediBot is built entirely on open tools and affordable hardware. ### Hardware - ESP32 Dev Module - ESP32-CAM - MAX30102 pulse oximeter - MLX90614 infrared temperature sensor - Active buzzer - WiFi communication ### AI and Machine Learning - TensorFlow Lite - Edge Impulse - face-api.js - MobileNetV1 - TinyFaceDetector - FOMO object detection ### Backend - Supabase - PostgreSQL - Real-time subscriptions - Role-based access control ### Frontend - Vanilla JavaScript - HTML - CSS - Progressive Web App deployment --- ## What Makes This Important MediBot is not important because it uses artificial intelligence. It is important because it demonstrates that useful healthcare technology can be built locally around Nigerian realities instead of imported assumptions. The system shows that: - Contactless patient monitoring can be affordable - Embedded AI can run on low-cost hardware - Real-time monitoring does not require expensive infrastructure - Local engineering can solve local healthcare problems The monitoring gap in Nigerian hospitals is not only a funding problem. It is also an engineering problem. MediBot was built as an engineering response to that gap. --- ## Current Development Status | Component | Status | |-----------|--------| | ESP32 vitals firmware | Complete | | Supabase data pipeline | Complete | | Dashboard | Complete | | Phone camera AI monitoring | Complete | | Login system | Complete | | Real-time alert system | Complete | | ESP32-CAM integration | In progress | | Bed occupancy model | In training | | Expression classifier | Dataset collection ongoing | | Offline local mode | Planned | | Clinical validation | Planned | --- ## What Comes Next The immediate roadmap is completing the ESP32-CAM vision pipeline, improving embedded AI models, and beginning clinical validation in real patient environments. Long term, the goal is broader deployment in Nigerian hospitals and further development of locally built robotics and embedded AI systems for healthcare. The talent to build these systems already exists here. What has been missing is infrastructure, research continuity, and systems designed specifically for the realities we actually operate in. MediBot is part of building that foundation. --- > The monitoring gap in Nigerian hospitals cannot be solved by importing more expensive equipment alone. It requires systems designed specifically for the environments where the need is real. MediBot was built around that reality from the beginning.