• by David Berkowitz , January 9, 2007

Want to be in pictures? You'll soon have better luck discovering if you're already in them, thanks to companies that are stepping up their efforts to allow people to tag and find images and videos online. We'll look at a few examples with Facebook, Riya, and PodZinger today, and return to others in the future.

Facebook

Let's start with an example from a social network, not a search engine. Facebook's search functionality, however, is integral to its success, as users need to be able to quickly and effectively find all their potential friends and associations. Its goals for search extend far beyond just finding friends' profiles, though.

When I upload a photo, I can tag it to denote anyone else in the picture. If the others are Facebook users, the photos will be associated with their profiles. If the others aren't, I can invite them. Better still, when I find people I know in another user's photo, I can click on their faces, and Facebook will then ask who it is, and again I can invite them to view it if they're not yet on the site.

Facebook has a more manageable challenge than other sites, in that there's a known universe of Facebook users, and it's trying to index their photos, not all images out there. In light of that, it relies on user tagging, a manual process, but one that's open to the community, rather than just the content producer (the one who uploads the photos).

Riya

Riya is shooting for something even bigger--building an image search engine, and one that can keep learning how to improve. Traditionally, when users search images and pictures, they're searching metadata and contextual information such as the image's filename, alt text (the title or description that appears when you mouse over the image), or text surrounding the photo on the page. Riya, however, wants its users' help in discovering who's in a picture, and then will aim to recognize the same faces in the future.

To train Riya, you upload photos, it recognizes faces, and then you note who the faces are. As you upload more photos, it will try to see if it recognizes the faces of anyone you tagged. It will then seek to recognize those people in your friends' photos if you import contacts, and it will then build the list from there.

Riya has some work to do. Uploading photos takes way too long through its downloadable software; it recommends running the program overnight if you have more than 500 photos. Additionally, its visual search engine now in beta works better for some people than others. It returned a number of correct results for Bill Gates, but for Britney Spears, it returned images of Jennifer Aniston and Jessica Biel on the first page (though, oddly, past the first page, all the images were Britney).

Much of the promise for Riya is that it can learn from everyone's contributions to keep improving, so there's more to it than just the tagging; it essentially will learn to tag photos itself. As an aside, for a very different and commercial application of Riya's technology, check out its site Like.com, where you can search for products and then find products related to any image you click, with many ways to refine the search from there.

PodZinger

What about videos, though? In my predictions for 2007, I offered high hopes for hotspotting and video tagging, subjects we'll return to in the near future. Another way to make video search work is by searching the accompanying audio, which PodZinger now does for YouTube videos. PodZinger wrote on its blog, "Now besides simply searching on the metadata of the video files, you can search for terms that are actually mentioned inside the audio, allowing for a greater likelihood you will find relevant material. We're also automatically organizing the videos into channels based on the actual content of the video."

It's tough to say how important this will be for consumers, yet for marketers conducting brand audits--as well as others in academia, knowledge management, healthcare, legal fields, and other professions--this is the type of service that, if it gets good enough, companies would even pay for. As more news and entertainment becomes available as podcasts, on YouTube, and on publishers' sites for free, it will be interesting to see if traditional media monitoring services need to reevaluate their offerings to compete with upstarts like PodZinger.

Beware of Baby Brother

The elephant in the room that merits its own column (or dissertation) is the impact on privacy. Whether the technology searches tags, learns to recognize people, or scours audio feeds, the common bond is that it can find people who have no idea they're being searched.

For me personally, I can take consolation in that I know what to look for, and in my circles, I'm doing most of the uploading and tagging. Yet my oldest brother, for example, won't know if I'm tagging him in the pictures until they come up in a search someday. In this case, big brother, it's baby brother who's tagging you.