Friday, 10 February 2012

Surveying online survey tools (by Angharad Roberts)

Following on from Andrew's blogpost about online surveys, yesterday's session provided an excellent opportunity to discuss people's experiences of different online survey tools. I presented a document (which can be viewed below or here) describing a range of online tools, evaluated according to four criteria which seemed important to me: compliance, compatibility, clarity and cost.

Compliance relates in part to the important issue of data protection, raised at the end of last month's meeting. EU Data Protection laws say data shouldn't be transferred to countries outside the European Economic Area, unless the country it is transferred to has equivalent laws. US laws are not regarded as providing the same level of protection (9 countries which do are listed here) but it does have some of the biggest data storing servers, including Google and SurveyMonkey. There is something called the US-EU safe harbor framework, which enables US-based companies to show that their data protection procedures meet EU standards. Google complies with this, as do many big online survey companies. Another compliance issue is to do with the accessibility of the survey - for example, does it work with screen reader technology which may be used by people with visual impairments?
Compatibility relates to the options a survey tool provides for how data can be exported - can it be downloaded directly into SPSS, or would it be available as an Excel spreadsheet?
Clarity - for me, this mostly relates to question types and particularly so-called skip logic questions. I have a number of different potential target audiences and although I want to ask most of them the same questions, there are some questions I only want to ask one particular group. Skip logic allows for different pathways through the same survey, depending on answers to particular questions.
Cost - there lots of free versions of survey tools but these often have very limited functionality. I've set these out in the limitations column of my document. For example, SurveyMonkey just allows 10 questions and 100 responses in each free survey. Export options may be limited in some free tools as well. So I may need a paid for survey tool, which means it's helpful to know what the range of potential costs could be, including potential discounts for academic / research use (SurveyGizmo offers a free student account, but badges these surveys with a SurveyGizmo student research logo - which may not be the image I want to project). It also raises the question: are there survey tools in use within the department which it might be useful to know about?

This was followed by a very valuable group discussion about some of these issues. In response to a question from Alex Schauer, I clarified that most of these survey tools allowed for surveys in "all languages" or 40+ languages. I'm not sure whether these terms are used interchangably, with no survey tool appearing to list more than 59 supported languages - these figures seem to be related to the Unicode standards for supported language character sets. Bristol Online Surveys only supports 10 languages in addition to English (at extra cost); the free version of QuestionPro has no multi-language support (a point omitted from the version of the document I presented yesterday, but included in the copy linked to from this post). Barbara described problematic experiences with attempting to export data from survey tools, and suggested experimenting with the export process before choosing a tool to run the actual survey. Liz Chapman described the different approach to data protection displayed in one recent US survey ("we can't promise anything about your data...") and some of the limitations of SurveyMonkey, which may be addressed in one of the more expensive subscription versions. Mark Hall talked about his experiences of using Lime Survey, and the exciting prospect of an in-house survey tool, developed in the iSchool, which would potentially give iSchool researchers complete control of their survey data. Paula also described some of the more powerful features offered by QuestionPro.

You can view, download and print the summary document here or view this on Scribd:

Online Survey Tools - summary sheet


  1. I read this article, Online Survey article very informative and interesting..I refer your blog to many of my friends as well.
    I have learned a very good lesson from this article. I really like this website and hope you will write more. Thanks a lot for your information.

    1. Thanks very much for your comment about this post - I'm glad it was helpful and thanks for mentioning SurveyTool article.

      It's great to know that you like the blog and I hope it continues to be useful to you!

  2. I agree with Jonathan - it's a very interesting blog entry. Thanks Angharad. Anyone interested in this entry may be interested in Angharad’s earlier entry on blogging and its role in research (15 April 2011)
    Just one point about Google forms - there is the option to make them publicly available. At Sheffield Uni for example, the default option (as Angharad says) is for forms to be accessible only to people with a Sheffield Uni login. There is a tick box labelled "Require University of Sheffield sign-in to view this form." If this is deselected, anyone can complete the form.

  3. Interesting survey and great list of online surveys websites. You can also try online survey solution provided by dotSurvey, they differ from all other survey tools

    1. Thank you for your comment, I'm glad you found the list helpful.

      Thanks also for suggesting dotSurvey, although I've encountered some problems accessing this (my computer freezes, which is worrying... but this may be a local issue with my machine, rather than a website problem). It's interesting to see another UK based survey tool and it does mention Data Protection Act in its have a privacy policy. However, I'm a bit concerned about the part of this policy that says: "if you provide us with personal information... via our websites, our publications or surveys, we may use that information to send you updates on our latest services... or to contact you so that we may learn more about our customers" although an opt out option is mentioned. Having said that, the privacy policy is derived from the parent group dotDigital, so perhaps it means their own customer surveys, rather than other people's dotSurvey surveys.

      It seems to me that this tool's focus is more on providing a survey tool for marketing use (it comes from the same team as dotMailer, an electronic marketing tool), rather than being more focused on supporting academic research.

      It's a very new tool ("launching in April 2012") and although the free 30 day trial version offers unlimited surveys and questions, the subscription version costs £25 per month plus VAT.

      I'm very aware that the list in this blog post is really only the tip of the iceberg in terms of the number of online survey tools which are available. It is certainly very interesting to hear about additional tools - thanks again for mentioning dotSurvey.


  4. This comment has been removed by a blog administrator.

  5. If you are thinking about conceiving a scholar or alumni review as part of your program evaluation toolkit, you may desire to address utilizing an online review device. There are numerous available.
    Online voting
     will be helpful..

  6. Your article is very interesting .
    thanks for share this information i hope you will post more great article in future .
    i love to reading this type to blogs .

    Sheffield student accommodation | Best place for a student to live close to Sheffield University

  7. This comment has been removed by a blog administrator.