Computer vision with iOS Part 2: Face tracking in live video

by Robin Summerhill on September 30, 2011

Update: Nov 28 2011 – The OpenCV framework has been rebuilt using opencv svn revision 7017

Introduction

Hot on the heels of our last article, in which we showed you how to build an OpenCV framework for iOS, we are turning our attention to capturing live video and processing video frames with OpenCV. This is the foundation for augmented reality, the latest buzz topic in computer vision. The article is accompanied by a demo app that detects faces in a real-time video feed from your iOS device’s camera. You can check out the source code for the app at GitHub or follow the direct download link at the end of the article.
The FaceTracker demo app
As shown in our last article, OpenCV supports video capture on iOS devices using the cv::VideoCapture class from the highgui module. Calling the grab method of this class allows you to capture a single video frame and return it as a cv::Mat object for processing. However, the class is not optimized for processing live video:

  • Each video frame is copied several times before being made available to your app for processing.
  • You are required to ‘pull’ frames from cv::VideoCapture at a rate that you decide rather than being ‘pushed’ frames in real time as they become available.
  • No video preview is supported. You are required to display frames manually in your UI.

In designing image processing apps for iOS devices we recommend that you use OpenCV for what it excels at – image processing – but use standard iOS support for accessing hardware and implementing UI. It may be a philosophical standpoint, but we find that cross-platform layers such as OpenCV’s highgui module always incur performance and design restrictions in trying to support the lowest common denominator. With that in mind, we have implemented a re-useable view controller subclass (VideoCaptureViewController) that enables high performance processing of live video using video capture support provided by the AVFoundation framework. The controller automatically manages a video preview layer and throttles the rate at which video frames are supplied to your processing implementation to accomodate processing load. The components of the underlying AVFoundation video capture stack are also made available to you so that you can tweak behaviour to match your exact requirements.

The Video Capture View Controller

The AVFoundation video capture stack and video preview layer are conveniently wrapped up in the VideoCaptureViewController class provided with the demo source code. This class handles creation of the video capture stack, insertion of the view preview layer into the controller’s view hierarchy and conversion of video frames to cv::Mat instances for processing with OpenCV. It also provides convenience methods for turning the iPhone 4′s torch on and off, switching between the front and back cameras while capturing video and displaying the current frames per second.

The details of how to set up the AVFoundation video capture stack are beyond the scope of this article and we refer you to the documentation from Apple and the canonical application sample AVCam. If you are interested in how the stack is created, however, then take a look at the implementation of the createCaptureSessionForCamera:qualityPreset:grayscale: method, which is called from viewDidLoad. There are a number of interesting aspects of the implementation, which we will go into next.

Hardware-acceleration of grayscale capture

For many image processing applications the first processing step is to reduce the full-color BGRA data received from the video hardware to a grayscale image to maximize processing speed when color information is not required. With OpenCV, this is usually achieved using the cv::cvtColor function, which produces a single channel image by calculating the weighted average of the R, G and B components of the original image. In VideCaptureViewController we perform this conversion in hardware using a little trick and save processor cycles for the more interesting parts of your image processing pipeline.

If grayscale mode is enabled then the video format is set to kCVPixelFormatType_420YpCbCr8BiPlanarFullRange. The video hardware will then supply YUV formatted video frames in which the Y channel contains luminance data and the color information is encoding in the U and V chrominance channels. The luminance channel is used by the controller to create a single-channel grayscale image and the chrominance channels are ignored. Note that the video preview layer will still display the full-color video feed whether grayscale mode is enabled or not.

Processing video frames

VideoCaptureViewController implements the AVCaptureVideoDataOutputSampleBufferDelegate protocol and is set as the delegate for receiving video frames from AVFoundation via the captureOutput:didOutputSampleBuffer:fromConnection: method. This method takes the supplied sample buffer containing the video frame and creates a cv::Mat object. If grayscale mode is enabled then a single-channel cv::Mat is created; for full-color mode a BGRA format cv::Mat is created. This cv::Mat object is then passed on to processFrame:videoRect:videoOrientation: where the OpenCV heavy-lifting is implemented. Note that no video data is copied here: the cv::Mat that is created points right into the hardware video buffer and must be processed before captureOutput:didOutputSampleBuffer:fromConnection: returns. If you need to keep references to video frames then use the cv::Mat::clone method to create a deep copy of the video data.

Note that captureOutput:didOutputSampleBuffer:fromConnection: is called on a private GCD queue created by the view controller. Your overridden processFrame:videoRect:videoOrientation: method is also called on this queue. If you need to update UI based on your frame processing then you will need to use dispatch_sync or dispatch_async to dispatch those updates on the main application queue.

VideoCaptureViewController also monitors video frame timing information and uses it to calculate a running average of performance measured in frames per second. Set the showDebugInfo property of the controller to YES to display this information in an overlay on top of the video preview layer.

The FaceTracker App screenshot

Video orientation and the video coordinate system

Video frames are supplied by the iOS device hardware in landscape orientation irrespective of the physical orientation of the device. Specifically, the front camera orientation is AVCaptureVideoOrientationLandscapeLeft (as if you were holding the device in landscape with the Home button on the left) and the back camera orientation is AVCaptureVideoOrientationLandscapeRight (as if you were holding the device in landscape with the Home button on the left). The video preview layer automatically rotates the video feed to the upright orientation and also mirrors the feed from the front camera to give the reflected image that we are used to seeing when we look in a mirror. The preview layer also scales the video according to its current videoGravity mode: either stretching the video to fill its full bounds or fitting the video while maintaining the aspect ratio.

All these transformations create a problem when we need to map from a coordinate in the original video frame to the corresponding coordinate in the view as seen by the user and vice versa. For instance, you may have the location of a feature detected in the video frame and need to draw a marker at the corresponding position in the view. Or a user may have tapped on the view and you need to convert that view coordinate into the corresponding coordinate in the video frame.

All this complexity is handled in -[VideoCaptureController affineTransformForVideoRect:orientation:], which creates an affine transform that you can use to convert CGPoints and CGRects between the video coordinate system and the view coordinate system. If you need to convert in the opposite direction then create the inverse transform using the CGAffineTransformInvert function. If you are not sure what an affine transform is then just look at the following code snippet for how to use them to convert CGPoints and CGRects between different coordinate systems.

// Create the affine transform for converting from the video coordinate system to the view coordinate system
CGAffineTransform t = [self affineTransformForVideoRect:videoRect orientation:videoOrientation];
 
// Convert CGPoint from video coordinate system to view coordinate system
viewPoint = CGPointApplyAffineTransform(videoPoint, t);
 
// Convert CGRect from video coordinate system to view coordinate system
viewRect = CGRectApplyAffineTransform(videoRect, t);
 
// Create inverse transform for converting from view coordinate system to video coordinate system
CGAffineTransform invT = CGAffineTransformInvert(t);
videoPoint = CGPointApplyAffineTransform(viewPoint, t);
videoRect = CGRectApplyAffineTransform(viewRect, t);

Using VideoCaptureViewController in your own projects

VideoCaptureViewController is designed to be re-useable in your own projects by subclassing it just as you would subclass Apple-provided controllers like UIViewController and UITableViewController. Add the header and implementation files (VideoCaptureViewController.h and VideoCaptureViewController.mm) to your project and modify your application-specific view controller(s) to derive from VideoCaptureViewController instead of UIViewController. If you want to add additional controls over the top of the video preview you can use Interface Builder and connect up IBOutlets as usual. See the demo app for how this is done to overlay the video preview with UIButtons. You implement your application-specific video processing by overriding the processFrame:videoRect:videoOrientation: method in your controller. Which leads us to face tracking…

Face tracking

Face tracking seems to be the ‘Hello World’ of computer vision and judging by the number of questions about it on StackOverflow many developers are looking for an iOS implementation. We couldn’t resist choosing it as the subject for our demo app either. The implementation can be found in the DemoVideoCaptureViewController class. This is a subclass of VideoCaptureViewController and, as described above, we’ve added our app-specific processing code by overriding the processFrame:videoRect:videoOrientation: method of the base class. We have also added three UIButton controls in InterfaceBuilder to demonstrate how to extend the user interface. These buttons allow you to turn the iPhone4 torch on and off, switch between the front and back cameras and toggle the frames-per-second display.

Processing the video frames

The VideoCaptureViewController base class handles capturing frames and wrapping them up as cv::Mat instances. Each frame is supplied to our app-specific subclass via the processFrame:videoRect:videoOrientation: method, which is overridden to implement the detection.

The face detection is performed using OpenCV’s CascadeClassifier and the ‘haarcascade_frontalface_alt2′ cascade provided with the OpenCV distribution. The details of the detection are beyond the scope of this article but you can find lots of information about the Viola-Jones method and Haar-like features on Wikipedia.

The first task is to rotate the video frame from the hardware-supplied landscape orientation to portrait orientation. We do this to match the orientation of the video preview layer and also to allow OpenCV’s CascadeClassifier to operate as it will only detect upright features in an image. Using this technique, the app can only detect faces when the device is held in the portrait orientation. Alternatively, we could have rotated the video frame based on the current physical orientation of the device to allow faces to be detected when the device is held in any orientation.

The rotation is performed quickly by combining a cv::transpose, which swaps the x axis and y axis of a matrix, and a cv::flip, which mirrors a matrix about a specified axis. Video frames from the front camera need to be mirrored to match the video preview display so we can perform the rotation with just a transpose and no flip.

Once the video frame is in the correct orientation, it is passed to the CascadeClassifier for detection. Detected faces are returned as an STL vector of rectangles. The classification is run using the CV_HAAR_FIND_BIGGEST_OBJECT flag, which instructs the classifier to look for faces at decreasing size and stop when it finds the first face. You can remove this flag at the start of DemoVideoCaptureViewController.mm, which instructs the classifier to start small, look for faces at increasing size and return all the faces it detects in the frame.

The STL vector of face rectangles (if any) is passed to the displayFaces:forVideoRect:videoOrientation: method for display. We use GCD’s dispatch_sync here to dispatch the call on the main application thread. Remember that processFrame:videoRect:videoOrientation: is called on our private video processing thread but UI updates must be performed on the main application thread. We use dispatch_sync rather than dispatch_async so that the video processing thread is blocked while the UI updates are being performed on the main thread. This will cause AVFoundation to discard video frames automatically while our UI updates are taking place and ensures that we are not processing video frames faster than we can display the results. In practice, processing the frame will take longer than any UI update associated with the frame but its worth bearing in mind if your app is doing simple processing accompanied by lengthy UI updates.

// Dispatch updating of face markers to main queue
    dispatch_sync(dispatch_get_main_queue(), ^{
        [self displayFaces:faces
               forVideoRect:videoRect
         videoOrientation:videOrientation];    
    });

Displaying the face markers

For each detected face, the method creates an empty CALayer of the appropriate size with a 10 pixel red border and adds it into the layer hierarchy above the video preview layer. These ‘FaceLayers’ are re-used from frame to frame and repositioned within a CATransaction block to disable the default layer animation. This technique gives us a high-performance method for adding markers without having to do any drawing.

// Create a new feature marker layer
    featureLayer = [[CALayer alloc] init];
    featureLayer.name = @"FaceLayer";
    featureLayer.borderColor = [[UIColor redColor] CGColor];
    featureLayer.borderWidth = 10.0f;
    [self.view.layer addSublayer:featureLayer];
    [featureLayer release];

The face rectangles passed to this method are in the video frame coordinate space. For them to line up correctly with the video preview they need to be transformed into the view’s coordinate space. To do this we create a CGAffineTransform using the affineTransformForVideoRect:orientation: method of the VideoCaptureViewController class and use this to transform each rectangle in turn.

The displayFaces:forVideoRect:videoOrientation: method supports display of multiple face markers even though, with the current settings, OpenCV’s CascadeClassifier will return the single largest face that it detects. Remove the CV_HAAR_FIND_BIGGEST_OBJECT flag at the start of DemoVideoCaptureViewController.mm to enable detection of multiple faces in a frame.

Performance

On an iPhone 4 using the CV_HAAR_FIND_BIGGEST_OBJECT option the demo app achieves up to 4 fps when a face is in the frame. This drops to around 1.5 fps when no face is present. Without the CV_HAAR_FIND_BIGGEST_OBJECT option multiple faces can be detected in a frame at around 1.8 fps. Note that the live video preview always runs at the full 30 fps irrespective of the processing frame rate and processFrame:videoRect:videoOrientation: is called at 30 fps if you only perform minimal processing.

The face detection could obviously be optimized to achieve a faster effective frame rate and this has been discussed at length elsewhere. However, the purpose of this article is to demonstrate how to efficiently capture live video on iOS devices . What you do with those frames and how you process them is really up to you. We look forward to seeing all your augmented reality apps in the App Store!

Links to demo project source code

Git – https://github.com/aptogo/FaceTracker
Download zip – https://github.com/aptogo/FaceTracker/zipball/master

Share and Enjoy:
  • Print
  • email
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • Reddit
  • Twitter

{ 67 comments… read them below or add one }

Hitesh Vaghasiya October 1, 2011 at 7:42 am

Thanks nice example its help me a lot.

Now, I am working on eye blink detection.Can you please guide me for same?

Thanks & Regards
Hitesh

Reply

Robin Summerhill October 1, 2011 at 9:35 am

For each face detected, perform another detection using the haarcascade_eye_tree_eyeglasses.xml cascade. You can restrict the eye detection to the upper portion of the face rectangle to speed things up. If you detect two eyes then everything is OK. No eyes = a blink, one eye = a wink ;-)

I actually implemented this in the demo but stripped it out because it was getting too complicated and moving away from the original purpose of just showing how to process live video.

Reply

Hitesh Vaghasiya October 4, 2011 at 9:07 am

Thanks for the reply.

If i blink eyes it also detect both eyes.So, I am not able to detect actual blink.

Can you provide me some sample code for it.

Thanks

Reply

Hitesh Vaghasiya October 6, 2011 at 3:28 am

Can you please give me some sample for eye tracking?

Thanks

Reply

Mohit May 19, 2012 at 11:21 am

Im trying to draw both the eyes and the face.
But I can only show both the eyes or the face depending which of the statements i write first :

// Detect faces
std::vector faces;
_faceCascade.detectMultiScale(mat, faces, 1.1, 2, kHaarOptions, cv::Size(60, 60));
//Detect eyes
std::vector eyes;
_eyesCascade.detectMultiScale(mat, eyes, 1.1, 2, kHaarOptions, cv::Size(30, 30));

Here eyes.size() = 0. If i interchange the position of the two statements, i get eyes.size() = 2 and faces.size() = 0

Dan October 1, 2011 at 12:16 pm

Hi great articles!!

1) ‘NSInvalidArgumentException’, reason: ‘videoMirrored cannot be set because it is not supported by this connection. Use -isVideoMirroringSupported’ When I run this.

2) On previous article red/blue colors reversed on video capture, how to fix?

thanks great stuff!!!

Reply

Robin Summerhill October 1, 2011 at 3:16 pm

1) Thanks for pointing that out! The lines setting the videoMirrored property were left over from an experiment. I’m developing on iOS5 where this property is supported so I didn’t see the exception. I’ve updated the Git repo. Look out for articles on new iOS5 features for image processing when the NDA is lifted.

2) OpenCV defaults to BGRA channel order rather than the RGBA order that you might be used to. It is the developer’s responsibility to ensure that cv::Mat objects are ordered correctly before display. Use cv::convertColor if you want to switch between BGRA and RGBA.

Reply

Jim Hubshman October 4, 2011 at 4:54 pm

Thanks for the excellent articles. Can you give me some guidance about how I would modify your code to apply a convolution kernel to the pixels within the face rect and then display the result in the rect?

Thanks in advance!

Jim

Reply

Hitesh Vaghasiya October 10, 2011 at 2:17 am

How to restrict the eye detection to the upper portion of the face rectangle to speed things up? For eye blink detection. Can you please give me some idea?

Thanks

Reply

nofx November 15, 2011 at 6:17 am

Any idea why i can’t add images to a view? I’m trying to convert a cv::Mat back to a UIImage, but it never shows in the view…

This worked with the code from your older post, but in this version it isn’t working for some reason… Any idea why?

- (void)viewDidLoad
{
[super viewDidLoad];

UIImage *testImage = [UIImage imageNamed:@"ie.png"];
cv::Mat tempMat = [testImage CVMat];
cv::cvtColor(tempMat, tpl, cv::COLOR_RGB2GRAY);

myView.image = [UIImage imageWithCVMat:tpl];
}

Reply

Robin Summerhill November 15, 2011 at 8:19 am

I’ve tried your code above and it works as expected. I had to declare ‘tpl’ as follows because it was missing from your code:
cv::Mat tpl;
There is no difference between the UIImage extensions between this project and the previous one so I’m not sure why you are seeing problems. Can you step through with the debugger and check that ‘testImage’ is being loaded (i.e. not nil)?

Reply

Hitesh Vaghasiya November 18, 2011 at 5:09 am

Hello, I am not able to track face in landscape mode.

Can please provide me some solution?

Thanks

Reply

Mark Blackwell December 7, 2011 at 6:11 am

Thanks so much for putting this together… just starting OpenCV and it’s so helpful to have some working code to start with.

One problem: I’m trying to run on an old iPhone 3, so need to use armv6… but xCode tells me that “aptogo-FaceTracker/OpenCV.framework/OpenCV, file is universal but does not contain a(n) armv6 slice for architecture armv6″…

reading your earlier post, it seemed like the build script was doing an armv6 build? perhaps you haven’t tested with an armv6 device?

makesensical?

Reply

Robin Summerhill December 7, 2011 at 7:52 am

Mark,

I recently re-built the framework for svn revision 7017. You are right that the armv6 slice is missing. I tested the original build on an iPhone 3 but didn’t re-test with this new version. My fault, sorry!

I’ll fix this and push a new version to GitHub. Thanks very much for pointing this out.

Reply

Robin Summerhill December 7, 2011 at 9:18 am

I’ve updated the project on GitHub to re-instate the missing armv6 slice in the framework and also to address a compatibility problem with iPhone 3.

Reply

Lukas December 12, 2011 at 12:24 pm

Hi,
I’ve problem with using “some” function from openCV…when I compile your project, everything is OK and work well. But for example if i want use function cvSmooth(), doesn’t matter for what, It doesn’t works.

I don’t understand why? Any suggestion?

PS: Interesting is, that I can use Erode(), Dilate(), but cvSmooth() doesn’t.

Reply

wingnet December 25, 2011 at 10:13 pm

u can use cv::medianBlur , GaussianBlur etc. to do the job

Reply

ck January 28, 2012 at 5:14 pm

The application in Part 1 built and worked fine, but the code from Part 2 gives the following error:
2012-01-29 00:06:37.343 FaceTracker[1596:10703] *** Terminating app due to uncaught exception ‘NSUnknownKeyException’, reason: ‘[ setValue:forUndefinedKey:]: this class is not key value coding-compliant for the key view.’

I have absolutely no experience with iPad or Objective C, so I can’t do much. Judging from the fact that others are able to build this without any problems, unless you have made a change to the code since you submitted the article, it seems there is a problem with my Xcode settings. A compiler issue perhaps?

Thanks

Reply

Robin Summerhill January 30, 2012 at 10:58 am

Hi CK,

This is a problem that I haven’t seen before. What device are you trying to run this on and which version of iOS is it? Do you see any more information in the stack backtrace showing where in the code the exception was thrown?

Robin

Reply

123m February 1, 2012 at 12:22 am

how do I convert this to run with a video stream from the machine? for me, seems it is not doing anything in the simulator?. how do i use this to run in the simulator

Reply

Robin Summerhill February 1, 2012 at 2:48 am

You cannot capture video using the simulator. You must run this on an iOS device with a camera.

Reply

123m February 1, 2012 at 10:20 am

Thank you Robin.

Can you help me run this program by taking video from the machine, instead of taking a video from the camera. Thanks. What are the changes do I have to do?

Reply

abhishek February 20, 2012 at 3:28 pm

Hi,
Is there a way i can get the curve of the face instead of a rectangle. I tried searching for it, but was not able to find anything for ios, can you please guide me through the same. Any help would be great .

Reply

Patrick February 23, 2012 at 7:28 pm

Thank you so much for this very useful article!

I’d like to include a record button, like in the AVCam project you linked to. I’d like to record video to a file, and detect live faces at the same time. I’ve been looking at your FaceTracker project and AVCam for days, but I just can’t seem to get that working. Would you mind giving me some pointers?…. or sample code?

I’d really appreciate it!

Reply

Fenton Stone February 25, 2012 at 2:48 am

Hi,Robin
I’m an iOS developer for 2 years , but know litter about OpenCV . You did me a great favor.
BTW , Do you have a Chinese name?If not , I’d like to give you a Chinese name ,罗夏山 (LuoXiaShan) ,
罗 is traditional Chinese family name , 夏 means summer and 山 means hill . :)

Thanks & Regards
Fenton Stone From Shanghai , China

Reply

Robin Summerhill February 29, 2012 at 4:46 am

Hi Fenton,

Thanks for the name! Glad you liked the article.

Regards,
罗夏山

Reply

Harsh Thaker February 27, 2012 at 9:00 am

Hello,

I need help, I don’t know what can be the code for eye detection after face detection. Please, help me out from it.

Reply

Brian February 28, 2012 at 9:03 pm

Where does the haarcascade_frontalface_alt2.xml file come from, is there so repo with a bunch a detectors? I ask because I need to make an app which does hand detection and want to know if there exists a file for this? Otherwise how would I be able to make one. Thanks!

Reply

Robin Summerhill February 29, 2012 at 4:48 am

The cascade is included in the OpenCV distribution. There are several variants for the face and one for eyes but nothing for hands. You could make one yourself by using training images but be aware that the detection doesn’t handle rotation, which might be a problem for hands.

Reply

Brian February 29, 2012 at 1:20 pm

Hey Robin, for rotations do you mean rotations of individual fingers, if so that doesn’t matter for this app because the user will be instructed to hold their hand in an open position. Also, how would I go about making training images for the hand, I’m pretty new to opencv but willing to learn, can you point me at some good info for this? Also, is this the best way to do this? Is there another way you would recommend?
Thanks

Reply

Robin Summerhill March 1, 2012 at 6:17 am

I mean rotation of the whole hand. For example, if you use training images with the hand upright in the ‘high-five’ orientation then that is all it is going to recognize. If you turn your hand away from vertical then it won’t work. This isn’t usually a problem for faces as people tend to stand upright!

I don’t personally have any experience of creating new cascades – I’m coming from an iPhone dev background rather than a computer vision background so this is a learning curve for me too.

Reply

Thomas Kefauver March 6, 2012 at 11:57 am

Hey Robin, this is a great demo app but I’m new to openCV and having difficulty changing the color space of the cv::Mat being passed to the processFrame method into HSV here is my code:

cv::Mat matHSV = cv::Mat(mat.size(),CV_8UC3);
cv::cvtColor(mat,matHSV, CV_BGR2HSV);

This produces the following error:
OpenCV Error: Assertion failed ((scn == 3 || scn == 4) && (depth == CV_8U || depth == CV_32F)) in cvtColor

Also is there any harm in using the CvMat class as opposed to cv::Mat ?? Thanks for the help!

Reply

Thomas Kefauver March 6, 2012 at 12:37 pm

Sorry I figured it out finally! I had presumed changing _captureGrayScale = NO in initWithNib would pass the full image in but this init must not be called by default. Great article though!

Reply

Kin March 12, 2012 at 12:41 pm

Roblin, Awesome tutorial! i got a bit problem here when starting face tracking demo with my iPhone 4s. Error Starting Executable “faceTracker” pop out, but after that i could get the app working on my phone without connecting to Xcode. Do you have any idea? i need Xcode nslog for further implementation. Thanks

Reply

Rebecca Johnson April 4, 2012 at 9:18 am

Hi,

I wanted to test the other Haar Cascade XMLs that come with the openCV package (e.g. haarcascade_mcs_leftear.xml). Thought I could just replace the contents with the contents of the face xml in your sample, but this doesn’t work. No ear is recognized – same with the other openCV xmls. Only the haarcascade_frontalface_alt2.xml) works. Can you possibly explain this? At the moment I am in a complete proof of concept phase, and am checking out technical potential, so I would like to confirm that the openCV ear detection is accurate – any pointers?

- and thank you very much for a fantastic tutorial!

Reply

Abhijit Sathe April 5, 2012 at 2:30 am

Hi Robin,
This is very helpful and innovative work i am loving it.
Actually , in addition to detecting face i want to detect nose,eyes,lips and create something can you please guide me in that sense like,what changes i have to make in the code or adding some extra xml files. I have worked with ios sdk for some time now,but am new to what i am seeing in your code .. can you please help me or give idea as to is it possible to detect nose,eyes etc?????
Please reply…
Thanks a lot

Reply

Gustavo May 11, 2012 at 5:17 am
Manuel May 11, 2012 at 1:07 pm

Is there a way of loading the CascadeClassifier from a string instead of an XML?

Reply

Mohit May 19, 2012 at 11:18 am

Im trying to draw both the eyes and the face.
But I can only show both the eyes or the face depending which of the statements i write first :

// Detect faces
std::vector faces;
_faceCascade.detectMultiScale(mat, faces, 1.1, 2, kHaarOptions, cv::Size(60, 60));
//Detect eyes
std::vector eyes;
_eyesCascade.detectMultiScale(mat, eyes, 1.1, 2, kHaarOptions, cv::Size(30, 30));

Here eyes.size() = 0. If i interchange the position of the two statements, i get eyes.size() = 2 and faces.size() = 0

Reply

AKS April 25, 2013 at 6:02 am

Hi ,

I am using same example and trying to detect eyes but not able to get detect 2 eye.
Every time i am getting eyes.size() = 1.
Please suggest me what i am missing.

Reply

Tom June 6, 2012 at 3:28 am

Hi,

I’m having some problems with the facetracker app.

I have Xcode v. 4.3.2 and when i hit run – in the console i got this :
“FaceTracker[687:10703] WARNING: -[ setMinFrameDuration:] is deprecated. Please use AVCaptureConnection’s -setVideoMinFrameDuration:”

Although the app is not working …… it starts in the simulator but that’s it….
Cannot run anything (torch,camera,fps )…..

Please help i’m very new to objective-c and i really need your help …….
Ps. I downloaded the zip archive – maybe there is something else i need to do with it or add some source, framework ….

Thanks in advance
Tom

Reply

John Rickman June 17, 2012 at 11:16 am

You have to validate the project settings and perform the necessary changes. Do this by going to the issue navigator panel.

Reply

John Rickman June 17, 2012 at 10:52 am

What would it take to track objects other than faces?

Reply

jean July 2, 2012 at 5:16 am

Dear Robin,
When rebuilding the framework using the buildopencv.sh from the opencv for iphone project, the armv6 slice goes missing again!. Could you make the new updated build script available?
Thanks a lot for the nice demo app!

Reply

Thommaes July 23, 2012 at 6:19 pm

ok – i just used your framework to do my own (marker less) augmented reality app – thanks a lot !!!
see: http://itunes.apple.com/us/app/ar-mayo/id539476292?l=de&ls=1&mt=8

Reply

BlueBeetle August 24, 2012 at 12:59 am

Hi, I was wondering if there was any way to adjust the output image so we can see the image processing. Basically I would like the see the scene as it would look with canny edge detection or something else. How would I update the live image using OpenCV methods? Thanks so much, this is a great help!

Reply

JJ August 29, 2012 at 8:10 am

Amazing article!
I’m totally new to OpenCV and looking for a way to use it to process images captured using the AVFoundation framework I just found a perfect match ;) .

I’m not sure if there is a typo in the article though. When you say:
“..and the back camera orientation is AVCaptureVideoOrientationLandscapeRight (as if you were holding the device in landscape with the Home button on the LEFT).”, shouldn’t be RIGHT there instead?

Thanks again Robin.

Reply

nims September 1, 2012 at 5:34 am

Hi hey how to take photo after we got the red square box .

Reply

kaka September 4, 2012 at 4:02 am

Hi,Thanks nice example.Now i want to know how to achieve digtal zoom?

Reply

radit September 11, 2012 at 6:36 am

My name radit new developers on iOS at around 5 months for face tracking it using videos that are in the gallery or take advantage of the camera device.
a very interesting article

Reply

Arun December 5, 2012 at 4:28 am

I am working on the Nail detection in hand please help me on that..

Reply

Harshita December 10, 2012 at 3:33 pm

Hello Robin,

Thanks for this article. It has been very helpful. I am trying to set up face tracking on the iPad. I am guessing I need to make some changes in the VideoCaptureViewController in the capture output method to set it up for the iPad but I am not sure what changes need to be made. Can you please direct me on this?

Thanks,
Harshita

Reply

Harshita December 10, 2012 at 4:29 pm

I have it working in the upright portrait position. I just changed the .xib file so that the dimensions were 768×1024. But if I rotate my iPad to any other orientation the orientation of the coordinate space does not change accordingly.

Reply

Nick December 31, 2012 at 3:20 pm

I’m trying to convert the code for this to detect someone’s face from a picture taken but I’m having trouble doing so, would you be able to help? The main problem is that it isn’t right where the persons face is, or even close.

Reply

Erin January 11, 2013 at 6:05 am

ClipClock is an amazing social video hub and sharing app! Check it out: http://www.clipclock.com/download/4187

Reply

Dmitry February 13, 2013 at 7:32 am

Thank you very much. This article is what I really needed. Thank you once again.

Reply

Oren February 26, 2013 at 11:51 pm

Hi Robbin,

Thank you for this wonderful article! really helps a lot!

I’ve got a question, My project uses ARC, is there an “Arcified” version to “VideoCaptureViewController” class??

Thank you,
Oren

Reply

Neil April 27, 2013 at 8:31 am

Hi,

Thanks for the great work.

I have come across a problem to which I can’t find a reason for or solution. If the change your code to use colour rather than greyscale the delegate method didOutputSampleBuffer never gets called.

Any ideas?

Reply

Neil April 27, 2013 at 8:34 am

Ok, it is dropping every frame.

Any ideas why?

Reply

NIkhil May 10, 2013 at 8:36 am

Hi Robbin,

Thanks for great tutorial.
I have one question why face detection is not working when iPhone is in LandScape.

Thanks
Nikhil.

Reply

Balraj June 21, 2013 at 3:14 am

Hello ,

I need to detect a shape in live camera like a bar code or any square image. and when i click picture i only able to fetch that image only. Can we do this by help of this tutorial ?

Thanks
Balraj,

Reply

deportejuegos.com December 28, 2013 at 12:11 am

Hello there, You have done a fantastic job. I’ll definitely digg
it and personally suggest to my friends. I’m confident
they’ll be benefited from this website.

Reply

how to get bigger breasts naturally without surgery pills or creams January 12, 2014 at 12:34 am

Hi there, just wanted to say, I loved this post. It was
practical. Keep on posting!

Reply

nye casino pa nett April 16, 2014 at 4:50 am

you’re really a good webmaster. The web site loading speed is amazing.
It kind of feels that you are doing any distinctive trick.

Furthermore, The contents are masterpiece. you’ve performed
a excellent activity in this topic!

Reply

Forest May 15, 2014 at 5:41 pm

I don’t even understand how I ended up right here, but I assumed this
submit used to be great. I don’t recognize who you might be but definitely you’re going to
a famous blogger if you happen to are not already.

Cheers!

Reply

casino pa mobilen August 1, 2014 at 4:11 pm

You actually make it appear really easy along with your
presentation but I to find this topic to be actually something which I think I would by no means understand.

It sort of feels too complicated and extremely extensive for
me. I am taking a look ahead to your subsequent put up, I’ll attempt to get the hold of it!

Reply

breast augmentation August 11, 2014 at 6:24 pm

Wow! At last I got a web site from where I be able to really get helpful facts
regarding my study and knowledge.

Reply

Leave a Comment

{ 5 trackbacks }

Previous post:

Next post: