Posted by arlene

Internet Survey Response rates

Researchers often want to maximise response rates. Several studies have reported response rates for Internet surveys. These reports have varied greatly, and the lack of consistency in findings is likely due to the wide variation of factors across these studies. Sheehan and McMillan review some of the studies comparing email survey response rates with those of postal surveys and note email survey response rates ranging from 6 per cent to 75 per cent. Smith compared two types of email survey approaches with a Web-administered survey. The email survey was either emailed directly to participants, or preceded by an emailed participation request. The latter procedure generated an overall response rate of 13.3 per cent, while the former procedure generated a response rate of 8 per cent and received one complaint that accused the researcher of spamming. We suggest that sending an initial participation request is good practice in relation to ethical codes (and ‘netiquette’), as well as having the advantage of increasing response rates. The Web survey participation request was posted to a range of related interest newsgroups and resulted in 99 responses. It is not possible to measure how many people saw the request with this methodology, but Smith estimated that it would have been viewed by approximately 8,000 users. This gives a response rate of about 1.25 per cent.

In general the evidence suggests that email survey participation requests gain higher actual response rates than Web-posted surveys. However, given the sheer number of users who are likely to be exposed to a Web-posted survey, large samples can be obtained even with low response rates. The main concern with the Web-based procedure is non-response bias and lack of a sampling frame. For this reason we recommend the email request approach. Posting to newsgroups may be a more convenient alternative for some projects and does allow some estimation of sampling frame and response rates. Since a newsgroup posting will move down the list of postings fairly quickly, and will eventually expire, re-posting the request is essential in order to increase response rates. However, care should be taken not to bombard newsgroups with persistent repeated postings, and to avoid ’spamming’. Smith noted that in her study, response rate peaked immediately

Subsequent to a newsgroup posting. Coomber suggests that re- posting to newsgroups once a week is appropriate.

Internet Survey Presentation format

There is very little research on the effects of different presentation formats in Internet-mediated survey research. Some evidence has been presented that questionnaire length is an issue that can affect response rates. In our own experience of reviewing a range of Web surveys (through participation) this factor is crucial. Internet surveys are more time-consuming than paper and pencil versions by virtue of the medium. Also, it is plausible that Net users may be less inclined to spend a long time completing a Web survey due to the temptation to go and explore other pages on the Web. We found many Web surveys to be fairly lengthy, for example Likert scales with in excess of 100 items were not uncommon. One survey notified the potential respondent that the survey involved six questionnaires and would take up to two hours to complete! We wondered how many respondents actually went on to complete the survey after reading this. We would certainly expect survey length to be an important factor in influencing Internet survey response rates, and predict that future research will likely identify this as a feature on which Internet and traditional surveys differ (traditional survey research lacks conclusive evidence on survey- length effects). We recommend keeping time demands on participants within a reasonable level in order to maximise data validity.

Further issues that emerge are whether users may respond differently depending on factors such as the use of pull-down menus (which display only one option initially) as opposed to use of radio buttons (where all choices are displayed initially). Similar issues have been considered in relation to traditional survey research, but new features emerge in an IMR context. Our advice to the researcher, while awaiting these empirical clarifications, is to strive for simple, clear layouts that closely resemble paper survey formats, and to pilot and modify the survey instrument in order to test for factors like consistency between different browser formats. Another piece of advice is to make sure the survey appears as professional and well planned, with explicit affiliation with a respected institution (such as a university). The potential threat to well-designed high-quality survey research posed by the mass of data-gathering activities on the Web has been highlighted. In this climate it is particularly important for behavioural and social researchers to present their surveys as well-designed pieces of research carried out by professional academics addressing important issues.

Possibly related posts: (automatically generated)
Internet Survey Response rates & Presentation Format

• How to Design and Implement an Internet / Web / Web Page / Site Survey (Design Issue & Postal Survey)
• How to Design and Implement an Internet / Web / Web Page / Site Survey (Internet User analysis)
• Internet Survey Confidentiality; Access Control & Technical issues
• E-NEWSLETTERS - DEFINING AND REFINING Part 6
• Design for different e-mail reader set-up
• E-newsletter Design
• Software Procedures for Implementing Internet Surveys
• Sender-Warranted E-mail
• A Simple UNIX Server Script
• Write for the Web, Successful Websites