Humans at Heart
Social Media Scraping

Humans at Heart 2/4: Is your global social media
scraping killing the turtles?


Author: Becky Kinnersley

This article is a deeper dive into the talk ‘Humans at Heart’ given by Becky Kinnersley (EMPOWER Translate) and Ed Wheeler (Join The Dots/InSites Consulting) at The Insight Show 2019.

Social media is the biggest and most dynamic global data set available on human behaviour. As a result, we’ve all been clamouring to uncover the vast amount of insight within.

Cue automated scraping: text analytics technology that allows us to mine that data quickly and inexpensively for a number of market research purposes, including market segmentation, measuring customer sentiment, monitoring online branding and identifying market trends.

There’s just a tiny issue… how would a machine cope with this?

AI Artificial Intelligence

Sayonara, turtles (and context)

Off the back of the machine’s insight, you could end up with this…

Before programming a chatbot, you need to work out how you want it to behave

Now this might be an extreme example, but it does hint at troubled waters.

While rapidly-advancing technology is integral to modern market research methods, automated scraping and machine translation still leaves a lot to be desired when it comes to context and tone. Even if a post seems understandable when taken at face value, there is a chance that there could be more going on. This becomes almost impossible to manage using only automated methods when different languages are involved.

Ultimately, you need human intelligence to avoid making the campaign mistake above.

It’s still necessary to keep a bit of that human touch, to find the true insight hidden in social media and avoid missing or misinterpreting key angles – especially when working across multiple markets and cultures. 

Managing ever-changing language

So, we know that sarcasm is an issue for the machines. Which means it’s a lot easier when we can take social media content at face value… right?

Like this tweet for example:

Before programming a chatbot, you need to work out how you want it to behave

Hats off to you if you’ve understood this sentiment! However, if this was picked up by a machine, how easy would it be for automated text analytics to establish that this person doesn’t think much of the price of on-trend footwear?

Now – you guessed it – let’s imagine this type of slang spanning across 10, 20, 30 or 40+ different languages, all constantly evolving in their own markets and cultures. The complexity of analysis becomes impossible for any affordable machine on the current market to manage.

There’s no time like the present…

The fast-moving nature of social media means that analysis for each market must be backed up with an in-depth knowledge and understanding of current affairs, hot topics and cultural touchpoints. These include celebrities, inferences and local product names, all of which will be in the language of ‘now’ for each specific market.

Without this understanding, and by relying solely on automated analytics, we risk investing in useless market research and producing wildly inaccurate insights.

How we save the turtles

Automated social media scraping is an excellent way to stay on top of the masses of information requiring analysis in order to produce up-to-date, local insights. Yet as with all machine learning around sentiment, culture and nuanced language, there is no escaping that it still requires a human touch. At EMPOWER, for example, we combine our SILKY™ and Cultural Intelligence System™ to ensure that social media data are accurately localised in their current cultural context. This also allows us to classify trends and identify where it would be beneficial to dig deeper – as quickly as possible.

Adding the human touch turns global social media scraping into a rewarding and meaningful process… and it keeps the turtles alive, of course.

Becky Kinnersley

Becky Kinnersley

Becky has an MA in Translation and is certified to MRS and BHBIA standards in Market Research. Anyone who’s met her knows that she lives and breathes translation and technology, often bringing the two together in expert and innovative ways for the Insight Industry. In her spare time, Becky likes to polish her 'Project Manager of the Year' Award and read specialist books on all-things-data.


All data and information provided by Empower Translate is for informational purposes only. makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this site and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis.

facebook twitter  linkedin

This site is run on 100% renewable energy

Empower Translate © 2021 All Rights Reserved