Eye Detection Program (Source Code Included)

Amazon My eye finding program, given an image with a human's eyes in it, is supposed to tell you where eyes of people are located. As you can see finding and locating eyes reliably is extremely easy for people to do but difficult for machines to do.

I used Matlab to develop my eye detection algorithms because Matlab has a very useful image toolbox that I can use to my advantage.

My eye detection program is broken down into two stages: find the face, and then find the eyes. To find the face, I designed two face finding algorithms: one utilizing the high symmetry of a human face and the other using average face matching.

In the first algorithm I scan the image for the region that has the best symmetry, and then perform some optimization to get better results. Also I apply a skin filter (2 types the user can choose from) to the face detection window to further shrink down the search space.

In the second algorithm I scan the image for the region that has the best correlation with the given average face template. To find eyes I also implemented two ways: rule based one and average eye matching.

The rule-based method simply means the program tries to find two clusters of dark pixels (eyes are dark...) with some constraints applied to them. It takes too long and is highly error prone.

The average eye matching, however, is extremely fast and gives accurate results, but user must supply the approximate ratio of the face size to the image size. This requirement renders this method less practical because ideally, user shouldn't tell the program anything other than the image itself. The following is a snapshot of running my eyes locating program (symmetry for face finding and average-eye matching for eye finding):



The following is a snippet of the source code for symeye.m, the main function for the eye detection program.

function [coordinates]=symeye(imfile,display,exh_search,method,eye_method);
%imfile: file name of the given image, which must be 3-colored
%display: 1 if you want to see the process of reaching the eye location;
%   other values if you do not (1 makes it run more slowly)
%exh_search: 1 if you want exhaustive search in the image for face; other
%   values if you want only horizontal search, assuming face appears on the
%   top of the image (1 makes it run more slowly)
%method: skin filter method; 0 for skin texture map (slower but better results) 
%   and others for thresholding in YCbCr color space (much faster but worse
%   results)
%eye_method: eye finding method; 0 for average eye matching method and
%   others for rule-based method

fprintf('Your arguments:
');

fprintf('File to process: %s
',imfile);
if display==1
    fprintf('Display the finding process.
');
else
    fprintf('Do not display the finding process.
');
end

if exh_search==0
    fprintf('Do only horizontal face window search.
');
else
    fprintf('Do exhaustive horizontal and vertical face window search.
');
end

if method==0
    fprintf('Use skin texture map to filter out non-skin regions.
');
else
    fprintf('Use YCbCr color space to filter out non-skin regions.
');
end

if eye_method==0
    fprintf('Use eye template matching to locate eyes.
');
else
    fprintf('Use rule-based method to locate eyes.
');
end

fprintf('
');

%dark weights we generated
dark_weights=[
.05+.09i .053+.085i  .06+.08i  .07+.075i  .08+.07i  .025+.19i   .08+.22i   .09+.39i  .11+.38i  .18+.36i;
.18+.4i  .21+.4i  .14+.47i  .12+.53i  .11+.54i  .11+.55i  .12+.56i  .14+.52i  .16+.52i  .155+.54i;
.155+.55i  .155+.56i  .21+.49i  .19+.5i  .21+.5i  .2+.51i  .2+.52i  .19+.54i  .2+.54i  .19+.55i;
.26+.42i  .25+.44i  .25+.455i  .25+.49i  .24+.5i  .255+.51i  .22+.51i  .23+.53i  .225+.53i  .26+.46i;
.26+.47i  .27+.48i  .275+.445i .28+.46i  .29+.47i .31+.495i  .33+.505i .325+.51i .320+.515i .315+.52i;
.95+.09i  .947+.085i   .94+.08i  .93+.075i .92+.07i  .975+.19i .92+.22i  .91+.39i  .89+.38i  .82+.36i;
.82+.4i   .79+0.4i   .86+.47i   .88+.53i   .89+.54i  .89+.55i .88+.56i  .86+.52i   .84+.52i  .845+.54i;
.845+.55i  .8450+.56i .79+.49i   .81+.5i   .79+.5i  .8+.51i .8+.52i   .81+.54i   .8+.54i   .81+.55i;
.74+.42i   .75+.44i   .75+.455i  .75+.49i   .76+.5i  .745+.51i .78+.51i  .77+.53i  .775+.53i  .74+.46i;
.74+.47i   .73+.48i  .725+.445i   .72+.46i   .71+.47i  .69+.495i  .67+.505i  .675+.51i  .68+.515i .685+.52i
];

%bright weights we generated
bright_weights=[
.06+.75i  .09+.805i  .11+.84i  .12+.82i  .12+.87i  .14+.86i  .16+.84i  .15+.89i  .17+.87i  .175+.89i;
.21+.835i  .21+.9i  .22+.87i  .345+.84i  .475+.875i  .49+.86i  .16+.62i  .155+.56i  .17+.54i  .2+.555i;
.2+.54i  .205+.51i  .22+.575i  .22+.55i  .23+.535i  .255+.575i  .275+.55i  .475+.475i  .49+.44i  .485+.39i;
.465+.37i  .43+.37i  .42+.325i  .39+.325i  .325+.34i  .35+.31i  .375+.3i  .36+.29i  .39+.27i  .4+.25i;
.3+.32i  .32+.29i  .335+.275i  .35+.25i  .355+.255i  .36+.22i  .37+.24i  .375+.2i  .345+.275i  .325+.29i;
.94+.75i .91+.805i .89+.84i  .88+.82i .88+.87i  .86+.86i .84+.84i  .85+.89i .83+.87i .825+.89i;
.79+.835i .79+.9i .78+.87i   .655+.84i  .525+.875i .51+.86i  .84+.62i  .845+.56i  .83+.54i  .8+.555i;
.8+.54i  .795+.51i .78+.575i  .78+.55i  .77+.535i   .745+.575i .725+.55i  .525+.475i  .51+.44i  .515+.39i;
.535+.37i  .57+.37i  .58+.325i  .61+.325i  .675+.34i .65+.31i .625+.3i  .64+.29i  .61+.27i  .6+.25i;
.7+.32i  .68+.29i  .665+.275i  .65+.25i  .645+.255i  .64+.22i .63+.24i  .625+.2i  .655+.275i  .675+.29i;
];

gray255map=gray(255);

[coordinates,bw_map]=recur(imfile, bright_weights, dark_weights, gray255map, ...
    0, display, exh_search, method, eye_method);

function [coordinates,bw_map]=recur(imfile, bright_weights, dark_weights, ...
    gray255map, recur_factor, display, exh_search, ...
    method, eye_method);

%clock starts time~
tic;

%store pixel values of different format in arrays
imrgb=im2double(imread(imfile));
imrgb_255=double(imread(imfile));
[imh, imw, imnc]=size(imrgb);

%report error if image is not 3-color channeled
if imnc~=3
   fprintf('%s is not a 3-color image; cannot use skin filter, forced exit
',...
       imfile);
   coordinates=[];
   bw_map=[];
   return;
end

max_face=0.7; %max percentage size of face in input image
min_face=0.2; %min percentage size of face in input image
big=min(imh,imw); %get the smaller value
max2=round(big*max_face);%max pixel width/height of face in input image
max2=max2+mod(max2,2); %make even for faceval
min2=round(big*min_face); %min pixel width/height of face in input image
jump= round(imw*0.015); 
jump= jump+mod(jump,2); %make even for faceval

%use image() instead of imshow() because image() displays calibration
figure, image(imrgb),title('Orignal Image');
axis([0 imw 0 imh]); axis ij; axis equal; axis tight; 

y=0; x=0;
%if user wants to see the process, get graphics window's handler
if display==1
    rh=rectangle('Position',[x,y,max2,max2]);
    set(rh,'EraseMode','xor');
    %set window's color to black
    set(rh,'EdgeColor','k'); 
end

detection=-10; %highest value for faceness that have obtained
detection2=-10;
faceness=-10; %value returned by faceval
winsize=max2; %biggest winsize (max)
winsize=uint32(0.7*min(imh,imw));
winsize=winsize+mod(winsize,2);
stop_threshold=100;

%begin searching for the window with best symmetry value
y=1;
x=1;
if display==1
    title('Searching for best symmetry...');
end

%either exhaustive search or just top horizontal search
if exh_search==1
    while x+winsize<imw
        y=1;
        while y+winsize<imh
            if display==1
                set(rh,'Position',[x,y,winsize,winsize]);
                drawnow;
            end
            faceness=symmetry(imrgb,x,y,winsize,winsize);
            %try to get the best window
            if faceness>detection 
                detection=faceness;
                coordinates=[detection,x,y,winsize,winsize];
            end
            %try to get the second best window
            if faceness>detection2 & detection>faceness 
                detection2=faceness;
                coordinates2=[detection2,x,y,winsize,winsize];
            end
            y=y+jump;
        end
        x=x+jump;
    end
    
else
    while x+winsize<imw
  
        if display==1
            set(rh,'Position',[x,y,winsize,winsize]);
            drawnow;
        end
        
        faceness=symmetry(imrgb,x,y,winsize,winsize);
        %try to get the best window
        if faceness>detection 
            detection=faceness;
            coordinates=[detection,x,y,winsize,winsize];
        end

        %try to get second best window
        if faceness>detection2 & detection>faceness
            detection2=faceness;
            coordinates2=[detection2,x,y,winsize,winsize];
        end

        x=x+jump;
    end

end

...

I wrote two functions that use two different eye locating algorithms. In the first approach, symeye.m, you only need to specify some control parameters (type help symeye for help). You can only input 3-channel color image in this function due to skin filtering method. In the second function, avgeye.m, you need to specify the approximate ratio of face size to image size (type help avgeye for help). You can use color or gray-scale images in this function.

I assume that each image contains one face which covers at least 20% of the image and at most 70% of the total image size so that it is not too big or small (although face violating these size constraints may nonetheless be detected successfully).

I assume frontal, upright view of the face. Image size should be around 300 by 300 pixels to allow efficiency.

In the first function, the face should be located higher than the center horizontal line of the image to allow maximum efficiency. If the face is very low, use option exh_search = 1 to enable exhaustive search vertically and horizontally. Add path of the source code directory via addpath command. For example if our source code is in C:ye_detection\, then do addpath C:ye_detection\images\, addpath C:ye_detection\code\, addpath C:ye_detection\code\m1, and addpath C:ye_detection\code\m2.

The picture files should reside in C:ye_detection\images\ and function 1 should reside in C:ye_detection\code\m1\ and function 2 should reside in C:ye_detection\code\m2\.

Do symeye('1.jpg',1,0,0,0) to run the first function and avgeye('1.jpg',0.4) to run the second function on '1.jpg'.

You can run the script I wrote as following to run a range of pictures: Run function 1: eyescript(0,1,30,[0,0,0,0],'.jpg'); This means you run the function 1 on 1.jpg, 2.jpg, ..., 30.jpg with args [0,0,0,0].

Run function 2: eyescript(1,1,30,[.5,0],'.jpg'); This means you run the function 2 on 1.jpg, 2.jpg, ..., 30.jpg with args [.5,0].

For further help please type 'help eyescript' in matlab. In function 2 if your image is too large you will get a warning saying the image is too large and is displayed at a certain size. In that case you may not see the eye detection window clearly; you will need to make the matlab window larger to see it.

Occasionally you may get an error due to the image you select. If it happens simply use another image.

Here's the entire source code for my eye finding algorithms. Use at your own risks.
Source Code for Eye Detection Program in Matlab