In this work, we propose a technique for facial expression recognition to bridge the semantic gap among the features that can be extracted in a content-based video retrieval system. The paper aims to provide accurate and reliable facial expression recognition of a dominant person in video frames using deterministic binary cellular automata (DBCA). Both geometric and appearance-based features are used. Efficient dimension reduction techniques for face detection and recognition are applied. Using the facial action coding system (FACS), one can code automatically nearly any anatomically possible facial expression, deconstructing it into what are called as action units (AUs). By employing two-dimensional deterministic binary cellular automaton systems (2D-DBCA), a scheme is developed to classify the facial expressions representing various emotions to retrieve video scenes/shots. Extensive experiments on Cohn–Kanade database, Yale database, and large movie videos show the superiority of the proposed method, in comparison with support vector machines (SVMs), hidden Markov models (HMMs), and neural network (NN) classifiers.

References

1.
Lisetti
,
C. L.
, and
Schiano
,
D. J.
,
2000
, “
Automatic Facial Expression on Facial Interpretation: Where Human-Computer Interaction, AI and Cognitive Science Intersect
,”
Pragmatics Cognit.
,
8
(
1
), pp.
185
235
.10.1075/pc.8.1.09lis
2.
Bente
,
G.
,
Krämer
,
N. C.
, and
Eschenburg
,
F.
,
2008
, “
Is There Anybody Out There? Analyzing the Effects of Embodiment and Nonverbal Behavior in Avatar-Mediated Communication
,”
Mediated Interpersonal Communication
,
Routledge
,
New York
, pp.
131
157
.
3.
Ahn
,
S. J.
,
Jabon
,
M. E.
, and
Bailenson
,
J. N.
,
2009
, “
Judging a Book by the Cover: Using Facial Expressions to Predict Performance
,”
59th Annual International Communication Association Conference
, May 21–25,
Chicago, IL
.
4.
Matsumoto
,
D.
, and
Willingham
,
B.
,
2009
, “
Spontaneous Facial Expressions of Emotion in Congenitally and Non-congenitally Blind Individuals
,”
J. Pers. Soc. Psychol.
,
96
(
1
), pp.
1
10
.10.1037/a0014037
5.
Mehrabian
,
A.
,
1968
, “
Communication Without Words
,”
Psychol. Today
,
2
(
4
), pp.
53
56
.
6.
Deng
,
H. B.
,
Jin
,
L. W.
,
Zhen
,
L. X.
, and
Huang
,
J. C.
,
2005
, “
A New Facial Expression Recognition Method Based on Local Gabor Filter Bank and PCA Plus LDA
,”
Int. J. Inf. Technol.
,
11
(11), pp. 86–96.
7.
Patnic
,
M.
, and
Rothkrantz
,
J.
,
2000
, “
Automatic Analysis of Facial Expressions: The State of Art
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
22
(
12
), pp.
1424
1445
.10.1109/34.895976
8.
Singh
,
S. Kr.
,
Chauhan
,
D. S.
,
Vatsa
,
M.
, and
Singh
,
R.
,
2003
, “
A Robust Skin Color Based Face Detection Algorithm
,”
Tamkang J. Sci. Eng.
,
6
(
4
), pp.
227
234
.
9.
Belhumeur
,
P. N.
,
Hespanha
,
J. P.
, and
Kriegman
,
D. J.
,
1997
, “
Eigen Faces vs. Fisherfaces: Recognition Using Class Specific Linear Projection
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
19
(
7
), pp.
711
720
.10.1109/34.598228
10.
Fasel
,
B.
, and
Luettin
,
J.
,
2003
, “
Automatic Facial Expression Analysis: A Survey
,”
Pattern Recognit.
,
36
(
1
), pp.
259
275
.10.1016/S0031-3203(02)00052-3
11.
Ekman
,
P.
, and
Friesen
,
W.
,
1978
,
The Facial Action Coding System: A Technique for the Measurement of Facial Movement Consulting
,
Psychologist Press
,
San Francisco, CA
.
12.
Donato
,
G.
,
Bartlett
,
M. S.
,
Hager
,
J. C.
,
Ekman
,
P.
, and
Sejnowski
,
T. J.
,
1999
, “
Classifying Facial Actions
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
21
(
10
), pp.
974
989
.10.1109/34.799905
13.
Nayak
,
B. K.
,
Sahoo
,
S.
, and
Rout
,
S. K.
,
2008
, “
Color Graphs: An Efficient Model for Two Dimensional Cellular Automata Linear Rules
,”
2008 Orissa Mathematical Society Conference
, pp.
1
14
.
14.
Wolfram
,
S.
,
1983
, “
Statistical Mechanics of Cellular Automata
,”
Rev. Mod. Phys.
,
55
(3), pp.
601
644
.10.1103/RevModPhys.55.601
15.
Cohen
,
I.
,
Sebe
,
N.
,
Cozman
,
F.
,
Cirelo
,
M.
, and
Huang
,
T.
,
2003
, “
Coding, Analysis, Interpretation, and Recognition of Facial Expressions
,”
J. Comput. Vision Image Understanding
, (Special Issue).
16.
Fawcett
,
T.
,
2008
, “
Data Mining With Cellular Automata
,”
SIGKDD
,
10
(
1
), pp.
32
39
.10.1145/1412734.1412738
17.
Khademi
,
M.
,
Kiapour
,
M. H.
,
Manzuri-Shalmani
,
M. T.
, and
Kiaei
,
A. A.
,
2010
, “
Analysis, Interpretation, and Recognition of Facial Action Units and Expressions Using Neuro-Fuzzy Modeling
,”
ANNPR'10, 4th IAPR TC3 Conference on Artificial Neural Networks in Pattern Recognition
,
Springer-Verlag, Berlin, Heidelberg
, pp.
161
172
, Paper No. LNAI 5998.
18.
Tian
,
Y.
,
Kanade
,
T.
, and
Cohn
,
F.
,
2001
, “
Recognizing Action Units for Facial Expression Analysis
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
23
(
2
), pp.
97
115
.10.1109/34.908962
You do not currently have access to this content.