Computers are increasingly able to recognise constituent parts of images and videos, and this may signal a change in the way we find and organise media online.
One area set to change is social listening, helping brands, particularly in FMCG, to recognise the context in which their products and logos are seen in shared photos.
Image analysis is relatively new
In May 2015, Facebook and Instagram alone accounted for 2bn photos shared online every single day. Since then, these numbers will have risen, and throwing in other open networks such as Twitter, I’d be surprised if the number wasn’t 2.5bn.
Social listening, as we know it traditionally, somewhat ignores these images. Brands keep a track on keywords and hashtags, and won’t necessarily know when their product or logo features within a photo but isn’t mentioned in the text of a post.
It’s easy to imagine that brands in FMCG, sports apparel, automotive, and many other sectors are missing many shared images of their products or brand.
Whilst some social listening platforms have used image recognition (searching primarily for logos in photos) since late 2015, image analysis is relatively new. The difference is in the ‘understanding’ of a photo, the ability to recognise scenes, which a product or image plays a part in.
Platforms such as Brandwatch, Crimson Hexagon and Talkwalker are testing this functionality, allowing a client, for example, to recognise when their product is pictured at the beach, inside a car or at a sports stadium.
Use cases of visual social listening
The image recognition technology that has been in place for a couple of years now allows brands to find their logos in social media posts. You can see an obvious example taken from Talkwalker’s website below.
Use cases include crisis management, product development, influencer marketing, and proving the value of sponsorship. An event or a sports team might want to show how often a sponsor’s logo appears during the run up to an event.
As the technology evolves towards recognising scenes and more complex images, brands will be able to search not just for combinations of text and logo.
This means brands could potentially search for:
- selfies featuring their product/logo (split into smilers and frowners)
- their product/logo with a cat/dog/baby
- their product/logo outside (e.g. at the beach)
- photographs of OOH advertising
The list is pretty endless for a multinational that’s interested in utilising or responding to user generated content.
Isn’t Pinterest already doing this?
Pinterest does have impressive image analysis baked in to its platform.
“Shop the Look” identifies advertiser’s products in pins and allows the user to buy them on a retailer website. And Pinterest’s stand-alone app Lens is a bit like Shazam for visuals – capture an object or scene and it will return related images and content from the network.
Whilst it’s feasible that other platforms could introduce similar functionality for brand advertisers, as far as social listening goes, this will likely remain the domain of specialist software with access to the firehose.
Image analysis is obviously already a big part of Facebook (facial recogniton) and Snapchat (AR), so it makes sense that listening software catches up on this front.
Natural language processing gets a lot of publicity, because it has always felt like the biggest hurdle in computing (to the layman) – the question of ‘when will computers understand us?’
Arguably, the question of ‘when will computers understand the world?’ is just as important. Technology such as autonomous vehicles is progressing quickly, but the implications of image analysis for search are profound, too.
Google already has a cloud video intelligence API which can be used to recognise when to identify when certain entities appear in a video.
As it becomes more and more obvious that every consumer is a content creator as well as a content consumer, brands will need to get more sophisticated in their approach to harnessing this user-generated content. Visual social listening is just the start.