A photograph may speak more than words, but can pictures help us find more pictures, especially precise replicas of other pictures?
Carnegie Mellon School of Computer Science researchers have found that computers can mimic the human ability to pinpoint and match up visually similar images. And not just any image, but pictures that are fairly complex, such as a photograph of a water fountain taken in two different seasons. Or a photograph and painting of the same cathedral.
Many times computers match images by focusing on similarities in shape, color or composition, says Alexei Efros, associate professor of computer science and robotics. The approach works well in applications such as Google Goggles, a mobile app that allows users to look up more information on a place or product by taking its picture.
But these methods often don't usually work when applied to different domains, such as similar pictures taken in different seasons, or under varying lighting conditions, or in different media. By teaching computers to estimate uniqueness, the researchers are improving automated image searches. The applications are many.
The technique can be applied to computation rephotography, a process by which a historic photograph is matched with a modern day photo taken from the same angle. The technique can also be combined with large GPS-tagged photo collections to determine the location where a painting or landmark was actually painted.
The process can also help to assemble a "visual memex," a data set that explores the similarities and contexts of a set of photos, such as a series of images taken of one place, such as the Medici Fountain in Paris. The researchers were able to download paintings, historic photographs and recent snapshots taken from all angles of the fountain using the technique and assemble them into a YouTube
The research team will present its findings
on “data-driven uniqueness” this month at SIGGRAPH Asia, a computer graphics and interactive techniques conference in Hong Kong.
Source: Carnegie Mellon University