CVFunhouse, a iOS Framework for OpenCV

Ever since I took the free online Stanford AI class in fall of 2011, I’ve been fascinated by artificial intelligence, and in particular computer vision.

I’ve spent the past year and a half teaching myself computer vision, and in particular the open source computer vision library OpenCV.  OpenCV is a cross-platform library that encapsulates a wide range of computer vision techniques, ranging from simple edge detection, all the way up to 3D scene reconstruction.

But developing primarily for iOS, there was an impedance mismatch. iOS deals with things like  UIImages, CGImages and CVImageBuffers. OpenCV deals with things like IplImages and cv::Mats.

So I wrote a framework that takes care of all the iOS stuff, so you can focus on the computer vision stuff.

I call it CVFunhouse. (With apologies to Robert Smigel).

As an app, CVFunhouse displays a number of different applications of computer vision. Behind the scenes, the framework is taking care of a lot of the work, so you can focus on the vision stuff.

To use CVFunhouse, you create a subclass of CVFImageProcessor. You override a single method, “processIplImage:” (or “processMat:” if you’re working in C++). This method will get called once for every frame of video the camera receives. Your method processes the video frame however you like, and outputs the processed image via a callback to imageReady: (or matReady: for C++).

The callback is important, because you’re getting the video frames on the camera thread, but you probably want to use the image in the main UI thread. The imageReady: and matReady: methods take care of getting you a UIImage on the main thread, and also take care of disposing of the pixels when you’re done with them, so you don’t leak image buffers. And you really don’t want to leak image buffers in an app that’s processing about 30 of them per second!

CVFunhouse is dead easy to use. The source is on GitHub at github.com/jeradesign/CVFunhouse. To get started, just run:

git clone https://github.com/jeradesign/CVFunhouse.git

from the command line. Then open the project in Xcode, build and run.

I’ve now built numerous apps on top of CVFunhouse. It’s the framework I use in my day-to-day work, so it’s constantly getting improved. I hope you enjoy it too.