Database for hand gesture recognition

---

Our database for hand gesture recognition (HGR) contains the gestures from Polish Sign Language ('P' in the gesture's ID) and American Sign Language ('A'). In addition, it some special signs ('S') were included as well. The database was developed as a part of the hand detection and pose estimation project, supported by the Polish Ministry of Science and Higher Education under research grant no. IP2011 023071 from the Science Budget 2012-2013.

The database consists of three series (termed HGR1, HGR2A and HGR2B) which include the subsequent data:

• original RGB images (jpg files),
• ground truth binary skin presence masks (bmp files),
• hand feature points location (xml files).

For benchmarking purposes, we publish the results obtained using several different methods developed during our research project. Bayesian classifier was trained using images from the ECU skin database (click here to see the list of images used for training). The classifier was trained in the RGB color space, using 64 bins per channel.

In addition, we have prepared the binaries which implement our methods. The software is available upon request.

In case of any problems or inquries concerning the database, please do not hesitate to contact us.

The file names store the following (e.g. '0_A_hgr2B_id01_1'):

• gesture's identifier, e.g. '0_A' - 3 characters,
• series name ('hgr1', 'hgr2A1', 'hgr2A2', 'hgr2B') - 4-6 characters,
• identifier of an individual presenting the gesture(e.g. 'id01') - 4 characters,
• gesture counter (incremented for the same gesture presented multiple times by the same individual) - 1 character.

The data are organized into three series acquired in different conditions:

1. HGR1:
   • Number of images: 899.
   • Number of individuals: 12.
   • Number of gestures: 25.
   • Image dimensions: varied (from 174x131 up to 640x480).
   • Background: uncontrolled.
   • Lighting conditions: uncontrolled.
   • Data: original images, skin masks, hand feature points.
   • Visualized data (for presentation purposes only): visualized skin masks, visualized hand feature points.
   • Results obtained using the following methods:
     • (Jones and Rehg, 2002): skin probability maps, binarized masks;
     • (Kawulok et al., 2013) [1]: skin probability maps, binarized masks;
     • (Kawulok, 2013) [2]: skin probability maps, binarized masks.
2. HGR2A:
   • Number of images: 85.
   • Number of individuals: 3.
   • Number of gestures: 13.
   • Image dimensions: 4672x3104.
   • Background: gray (subseries 'hgr2A1' - 44 images) and uncontrolled (subseries 'hgr2A2' - 41 images).
   • Lighting conditions: uniform, flash.
   • Data: original images, skin masks, hand feature points.
   • Data (downscaled to 602x400): original images, skin masks, hand feature points.
   • Visualized data (for presentation purposes only, 602x400): visualized skin masks, visualized hand feature points.
   • Results obtained using the following methods:
     • (Jones and Rehg, 2002): skin probability maps, binarized masks;
     • (Kawulok et al., 2013) [1]: skin probability maps, binarized masks;
     • (Kawulok, 2013) [2]: skin probability maps, binarized masks.
3. HGR2B:
   • Number of images: 574.
   • Number of individuals: 18.
   • Number of gestures: 32.
   • Image dimensions: 3264x4928.
   • Background: controlled (green tone).
   • Lighting conditions: uniform, flash.
   • Data: original images, skin masks, hand feature points.
   • Data (downscaled to 400x604): original images, skin masks, hand feature points.
   • Visualized data (for presentation purposes only, 400x604): visualized skin masks, visualized hand feature points.
   • Results obtained using the following methods:
     • (Jones and Rehg, 2002): skin probability maps, binarized masks;
     • (Kawulok et al., 2013) [1]: skin probability maps, binarized masks;
     • (Kawulok, 2013) [2]: skin probability maps, binarized masks.

The database is free to use for research, non-commercial purposes, provided that the following papers are cited in the works that exploit the database (here are the BibTeX entries).

1. M. Kawulok, J. Kawulok, J. Nalepa, and B. Smolka, “Self-adaptive algorithm for segmenting skin regions,” EURASIP Journal on Advances in Signal Processing, vol. 2014, no. 170, 2014. (BibTeX)
2. J. Nalepa and M. Kawulok, “Fast and accurate hand shape classification,” in Beyond Databases, Architectures, and Structures, S. Kozielski, D. Mrozek, P. Kasprowski, B. Malysiak-Mrozek, and D. Kostrzewa, Eds., vol. 424 of Communications in Computer and Information Science, pp. 364-373. Springer, 2014. (BibTeX)
3. T. Grzejszczak, M. Kawulok, and A. Galuszka, “Hand landmarks detection and localization in color images,” Multimedia Tools and Applications, vol. 75, no. 23, pp. 16363-16387, 2016. (BibTeX)

Example:

Original input image (2_A_hgr2B_id01_1.jpg)	Skin mask	Visualized hand feature points	Visualized skin region

Feature points locations in XML: <IMAGE WIDTH="400" HEIGHT="604" COLOR="TRUE"> <hand> <FeaturePoint x="212" y="277" category="Wrist" type="WristThumb"/> <FeaturePoint x="162" y="295" category="Wrist" type="WristPinky"/> <FeaturePoint x="191" y="215" category="Thumb" type="Knuckle1"/> <FeaturePoint x="176" y="189" category="Thumb" type="Knuckle2"/> <FeaturePoint x="154" y="182" category="Thumb" type="Fingertip"/> <FeaturePoint x="188" y="192" category="IndexFinger" type="Knuckle1"/> <FeaturePoint x="187" y="164" category="IndexFinger" type="Knuckle2"/> <FeaturePoint x="186" y="147" category="IndexFinger" type="Knuckle3"/> <FeaturePoint x="182" y="125" category="IndexFinger" type="Fingertip"/> <FeaturePoint x="158" y="167" category="MiddleFinger" type="Knuckle2"/> <FeaturePoint x="153" y="147" category="MiddleFinger" type="Knuckle3"/> <FeaturePoint x="144" y="125" category="MiddleFinger" type="Fingertip"/> <FeaturePoint x="150" y="182" category="RingFinger" type="Knuckle2"/> <FeaturePoint x="163" y="203" category="RingFinger" type="Knuckle3"/> <FeaturePoint x="169" y="218" category="RingFinger" type="Fingertip"/> <FeaturePoint x="144" y="216" category="Pinky" type="Knuckle1"/> <FeaturePoint x="140" y="204" category="Pinky" type="Knuckle2"/> <FeaturePoint x="146" y="192" category="Pinky" type="Knuckle3"/> <FeaturePoint x="160" y="197" category="Pinky" type="Fingertip"/> </hand> </IMAGE>

The database was created by:
Michal Kawulok
Tomasz Grzejszczak
Jakub Nalepa
Mateusz Knyc

Here, we would also like to acknowledge the help of a number of students in creating the ground-truth annotations.

Related content:

1. Michal Kawulok, Jolanta Kawulok, Jakub Nalepa, Maciej Papiez: Skin detection using spatial analysis with adaptive seed - additional data
2. Michal Kawulok, Jolanta Kawulok, Jakub Nalepa: Spatial-based skin detection using discriminative skin-presence features - additional data

---

Contact:
Michal Kawulok, D.Sc.
e-mail: michal.kawulok@polsl.pl

Silesian University of Technology
Institute of Informatics
Akademicka 16, room 504, 44-100 Gliwice, Poland