Skip to content

© 2026 Rami Kronbi · Beirut, Lebanon

Back
AI · support tool

Edge emotion-recognition prototype for children with autism

Reads facial expressions in real time on a small device, as a support tool for children with autism.

My partIntern at VIBE: developed the pipeline and its data

GitHub — CNN-Emotion-Detection ↗Read the write-up →

CameraA live view of the child’s face.
Face & landmarksFacial landmarks describe the shape of the face.
CNNClassifies the expression on a Jetson Orin Nano.
Training dataCALMED, Kaggle, and other image and video sources, combined and cleaned.
Real timeRuns live on the device; ~92% development accuracy.
SupportA support tool, not a diagnosis.
Facial landmarks and a CNN run in real time on a Jetson Orin Nano, trained on combined and cleaned data from CALMED, Kaggle, and other sources. The output is there to support, not to diagnose.

In short

A real-time emotion-recognition pipeline I developed in 2022–2023 as a support tool for children with autism. It pairs facial landmarks with a convolutional neural network and runs on a Jetson Orin Nano. After I combined and cleaned image and video data from CALMED, Kaggle, and other sources, it reached about 92% development accuracy. The prototype and the concept were later acquired following a Gulf-region competition.

What it is
A real-time emotion-recognition prototype for children with autism, running on a Jetson Orin Nano.
Why it matters
A support tool that reads facial expressions has to work for the children it is meant for, which means learning from data that includes them, and it has to keep up in real time on a small device.
My part
I developed the pipeline in 2022–2023, as a computer-vision intern at VIBE, a startup, in a small team with a mentor, and prepared the data it learned from, a collection of datasets around CALMED. The prototype and the concept were later acquired following a Gulf-region competition.

Step by step

  1. 1

    Gather

    Image and video data from CALMED, Kaggle, and other sources, combined and cleaned into one training set.

  2. 2

    Train

    A convolutional neural network learns expressions from the cleaned data, reaching about 92% development accuracy.

  3. 3

    Run

    Facial landmarks and the CNN run live on a Jetson Orin Nano.

  4. 4

    Support

    The result is meant to support, never to diagnose.

Results

~92%
Development accuracy
after combining and cleaning the data
Jetson Orin Nano
Hardware
real time
Acquired
Outcome
prototype and concept, following a Gulf-region competition
For engineersHow it is built, its limits, and its keywords.

How it works

Facial landmarks and a convolutional neural network run together on a Jetson Orin Nano in real time. I trained the network on image and video data from CALMED, Kaggle, and other sources that I combined and cleaned, and it reached about 92% development accuracy.

Limits

  • The analysis panels are verified on simulated signals with known noise, vibration and distortion (the Noise Lab recovers the simulated white noise to within 1 %); long captures from the real unit are still to come.
  • The ~92% is development accuracy on the data I combined. It doesn’t tell you how it will do with one particular child in one particular room.
  • It reads facial expressions, and an expression is not always what someone feels. It is a support tool, not a diagnosis.

Keywords

  • emotion recognition
  • autism
  • Jetson Orin Nano
  • CNN
  • facial landmarks
  • CALMED
  • edge inference

Read the write-up →

BackAll projects →