Project Description

My proposed Rhizome project is the development of an interactive art piece that utilizes cameras, face tracking technology, and original paintings to create a live mixed-media portrait of the audience viewing the piece. The work is motivated by my mixed media portraiture series ‘divine reflections.’ The series uses bright colors and fantastic situations to present ordinary people in a novel light, attempting to awaken in the viewer a sense of the beauty or divinity inherent within us all. The connection is particularly interesting when the viewer is the subject, and is faced with a view of themselves within this context. Unfortunately, the creation of each piece is a time-consuming process. By programming the computer to create works of this nature, I hope to allow many more people to experience themselves in this way.





For more examples please visit divinereflections.ca

Early ‘divine reflections’ were created by mirroring layers of paint, and then aligning them symmetrically overtop of a face. This process is possible to mimic using computer vision techniques; a face tracking algorithm can locate the center of the face, and the positional data can be used to create an overlay layer that, when combined with the original input image, projects the viewer into the painting.

A prototype of this system has been created, and a demo of it is offered below. The demo was developed in the programming language c, using a modified version of Intel’s OpenCV library. It runs at ~6.25 frames per second with a 1-2 second lag on a Dell Inspiron 2Ghz laptop using a standard webcam.



Initial exhibition of the piece highlighted a number of design and interaction challenges that would enhance the experience. Framerate and fidelity of the camera image is extremely important; motion blur from the camera, frame rate, and delay in response time can detract from the interactive experience. A significant portion of the proposed budget invests in hardware that will accelerate the system. Acceleration using graphics cards has become much more accessible through the development of nvidia’s new CUDA software libraries, motivating an exploration of this technology as a platform for improving the immersive qualities of video based interactive art.





Exploration of other, potentially more accurate face tracking algorithms is also important; While OpenCV provides a reasonable solution that is easy to modify and use, other algorithms may prove to be more efficient or accurate, particularly when implemented using CUDA.

Watching individuals using the system reveals other forms of interaction that they would like to explore. Many people found it more natural to move their hands, rather than moving their head, motivating the exploration of hand tracking algorithms. Additionally, groups of people would often want to explore the space together. A redesign of the existing algorithm to allow multiple people to simultaneously explore the space would make it more accessible.





Finally, to provide users with a lasting memento of the experience and to provide additional exposure to the gallery, rhizome, and the piece, an automated means of storing and uploading still images captured from the video online will be developed and promoted.

Timeline and Project Budget

The proposed project is broken into three stages, phase one includes:
  • Researching and purchasing hardware (two weeks)
  • Improving frame rate and response time by implementing face tracking and/or image synthesis using CUDA (three weeks)
  • Exploring other algorithms for improving accuracy and speed of face tracking (three weeks)
Completion of this system will give a good idea of how much additional processing can be done to add interaction. With those limitations in mind, phase two includes:
  • Adding interactivity for multiple people (three weeks)
  • Adding hand tracking to allow for additional interaction (three to five weeks)
  • Creating new paintings to enhance the visual aspects of the piece (three weeks)
At this stage the development of the piece will be complete, offering opportunities to promote and exhibit. Phase three is open ended and concludes by:
  • Promoting the piece, online and elsewhere
  • Exhibiting the piece
Any suggestions or help finding exhibition opportunities would be greatly appreciated.

Budget:

Computer for exhibition$800
Projector for exhibition$1200
Digital Video Camera$1000
Graphics card for acceleration$500
Graphics card for display$200
Monitor for development $300
Cuda Developers Conference$1000
Total$5000


CV

Steve Oldridge
1842 East 16th Ave,
Vancouver, BC Canada
V5N 2G6
even@flowformed.com
http://www.ece.ubc.ca/~steveo

Education:

2003-Present Studying towards completion of a Ph.D. in Electrical Engineering at the University of British Columbia, with a focus on the convergence between art and engineering, and on interactive art. Expected graduation: November, 2008.

2000-2002 M.A.Sc. in Electrical Engineering at the University of British Columbia

Grants / Fellowships:

2006 Precarn Scholar
2005 Precarn Scholar
2004-2007 National Science and Engineering Research Council (NSERC) Full Scholarship.

Solo shows:

November 2007 Divine Inspiration :: deities, fairies, spirits, and fates @ Radha yoga spiritual art gallery, Vancouver, BC
April 2007 Divine Reflections @ a small world gallery, Vancouver, BC
September 2006 Divine Reflections :: visionary portraiture for visionary people @ Radha yoga spiritual art gallery, Vancouver, BC

Group shows:

2007 20 Questions ~ Vancouver, BC
2006 Synergenesis ~ San Francisco, CA
2006 Interdimensional Art Show ~ Seattle, WA
2006 Humandala Art Show ~ Seattle, WA
2005 Synergenesis ~ San Francisco, CA
2005 Interdimensional Art Show ~ Seattle, WA

Technical Abilities:

Programming languages and Libraries: C, C++, Processing, PhP, Boost, OpenCV
Software: Adobe Photoshop, After Effects


References:

Sidney Fels
604.822.5338
ssfels@ece.ubc.ca


Work Samples

The portrait series that inspired this project can be found at divinereflections.ca

Raw paintings used in the project are displayed at flowformed.com

My Ph.D. Thesis project on the development of a multi-focal depth software suite is outlined on my school webpage, and a paper outlining the process will be made available there upon publication.