ICPR2026_Presentation

1.

Real-Time Emotion Recognition
System with Emotionalize Score
Nikita Podstrelov
Hadas Chassidim
Irina Rabaev
podstrelov99@gmail.com
hadasch@ac.sce.ac.il
irinar@ac.sce.ac.il
Department of Software Engineering, Shamoon
College of Engineering, Be’er Sheva, Israel
BRAIN ICPR 2026

2.

Part 1 – Motivation and
Problem Statement

3.

Motivation
Current FER systems recognize WHAT emotion is expressed but
ignore HOW STRONGLY it is expressed
Happy
Happy
Happy
──────────────────────────────────────────────────────────────────────────────────────
Weak Medium
Mid expression
High expression

4.

Limitations
of Existing
FER Systems
Current limitations
- Discrete emotion classification only
- No continuous intensity estimation
- Prediction instability in video streams
- Poor recognition of subtle expressions
Our solution
- Continuous Emotionalize score
- Real-time stable inference
- Lightweight CNN +Face Mesh
framework

5.

Key Idea
CNN
Emotion category
Face Mash
Facial Geometry
(eyes, mouth...)
(happy, sad...)
Emotionalize score
Emotion +Intensity
✓ Continuous intensity ✓ Stable predictions ✓ Real-time inference

6.

Part 2 – Proposed Method

7.

Overall pipeline

8.

Emotion Classification
EfficientNet-B0
Happy ███████████████████ 0.91
Surprise ██ 0.05
Neutral █ 0.02
Sad █ 0.01
Others <0.01
CNN tells us what emotion is expressed

9.

Facial Geometry Features
Face Mesh tells us how it is expressed
MediaPipe Face Mesh
Geometric Facial Features
(Eyebrows Eyes Mouth)
Expression Intensity Cues
Emotion Energy

10.

Emotion Energy
Instead of using only the predicted class, we utilize the entire probability distribution.
pi — predicted probability of emotion i produced by EfficientNet-B0.
αi — weight reflecting the contribution of emotion i to the overall emotion energy.
Positive emotions (e.g., Happy) and negative emotions (e.g., Angry, Fear, Sad, Disgust)
contribute positively to E.
The Neutral class contributes negatively, reducing the overall emotion energy.

11.

Emotionalize combines both
Emotianalize score
•