Have you ever seen a slew of answers that skew the results overwhelmingly in one direction? Such responses are known as “trolls”. In some cases, the responses may even be the work of nefarious humans seeking to skew an organization’s data for their own ends. Or have you received a survey response that was clearly not written by a human? These are often “robot” responses, and they generally result from an online survey or poll being manipulated by bots, or computer programs designed to mimic human behaviour.
Survey pros are aware of the challenges posed by sophisticated bots and nefarious respondents. This article explains what these strange and concerning phenomena are, why they matter, and how you can avoid falling victim to them when conducting your own surveys.
Don’t Be Fooled
Bots are automated systems searching the Internet for surveys to answer, interfere with, and complete: especially if there is an incentive prize draw. They are programmed to respond to survey questions as if they were being answered by a real person.
Nefarious respondents or trolls come out to play most commonly when you are researching a difficult issue where a special interest group may try to skew or stuff the results. This can significantly undermine the validity of the results. Not to mention waste the time and resources of the municipality conducting the survey.
Prevent & Recognize Dirty Data
One of the ways to guard against bots is to employ what’s referred to as a honeypot. Honeypots provide hidden questions only bots will see. If one of these questions is answered, then you know a human did not respond.
Once your survey is concluded, cleaning survey data is an essential step in the data analysis process. It helps ensure that the data you're working with is accurate and reliable. A key step is to check the IP addresses and assess and potentially eliminate duplicates. Keeping in mind duplicate IP addresses occur naturally when respondents are using common or shared devices such as in city hall or public facilities. Also consider that IP addresses outside of your survey’s community may be a result of residents using virtual private networks (VPNs). If you are gathering email addresses for prize draws or follow up contact it’s also important to scan for suspicious emails composed of random letters or numbers. Keep malicious responses from creeping into your data if they are not focused on the issue at hand.
Unfortunately, it’s not enough to just get the right respondents, you also need to defend against the wrong ones. At The W Group we protect survey research results through careful data cleaning. We don’t merely rely on reports generated by survey software. Email us at communications@thewgroup.ca for more information and support on your next survey!