Master's Thesis: Scouter
Category: EE & Hardware (back to list)
Project Page: Master's Thesis: Scouter project page
Summary: Applications of Convolutional Neural Networks to Facial Detection and Recognition in Wearable Computing and Augmented Reality
Begin: May 1, 2009
Completed: May 6, 2010
Welcome to Hackaday visitors.
Table of Contents
1. Overview >>
2.Thesis and Poster Downloads >>
3. Images and Diagrams >>
4. Videos >>

1. Overview

Facial detection and recognition are among the most heavily researched fields of computer vision and image processing. However, the computation necessary for most facial processing tasks has historically made it unfit for real-time applications. The constant pace of technological progress has made current computers powerful enough to perform near-real-time image processing and light enough to be carried as wearable computing systems. Facial detection within an augmented reality framework has myriad applications, including potential uses for law enforcement, medical personnel, and patients with post-traumatic or degenerative memory loss or visual impairments. Although the hardware is now available, few portable or wearable computing systems exist that can localize and identify individuals for real-time or near-real-time augmented reality.

The author presents a system design and implementation that performs robust facial detection and recognition robust to variations in lighting, pose, and scale. Scouter combines a commodity netbook computer, a high-resolution webcam, and display glasses into a light and powerful wearable computing system platform for real-time augmented reality and near-real-time facial processing. A convolutional neural network performs precise facial localization, a Haar cascade object detector is used for facial feature registration, and a Fisherface implementation recognizes size-normalized faces. A novel multiscale voting and overlap removal algorithm is presented to boost face localization accuracy; a failure-resilient normalization method is detailed that can perform rotation and scale normalization on faces with occluded or undetectable facial features. The development, implementation, and positive performance results of this system are discussed at length.

2. Thesis and Poster Downloads

download Master's Thesis Full Text (3.1MB, PDF)
download Technical Poster (4.2MB, PDF)

3. Images and Diagrams

Scouter netbook Scouter netbook Scouter netbook Scouter HUD Full hardware platform (without human) Final realtime performance sample Planned final interface Full view of Scouter wearable computing platform View of my head with Scouter HUD and camera Architecture of Lenet-5 CNN Distribution of CPU usage Block flowchart of Scouter functionality
Click any image to enlarge. Left to right, top to bottom: eeePC901 used for computing (2), Vuzix VR920 heads-up display with camera (2), full hardware platform without human, realtime sample view inside HUD, planned final interface, full platform view with human, view of HUD and camera with human, CNN architecture, CPU usage distribution, Scouter flowchart.

4. Videos

Formal Overview of the Scouter Project

Realtime Screencast of Project in Operation