Researching Vaccine Willingness, Part III: Preliminary Data

After learning the basics of using AMT — described in Part II — I released a few surveys into the wild (AMT’s actual digital marketplace) to gather some real data.

In this post, I will report my preliminary findings from ~40 responses combined between Liberals and Conservatives (approximately 20 in each category). A more detailed analysis will be posted later.

Survey and Data Collection Procedure

Identifying data sources. Since I was interested in studying differences in vaccination WTP between self-identified liberals and conservatives, I needed a way to determine who’s who. And furthermore, I needed to filter out bots and automatic responders lurking on the AMT marketplace. Luckily, AMT allows you to do both of these things using its ‘Qualifications’ feature.

Qualifications are divided into two types: basic qualifications and premium qualifications. By setting basic qualifications such as HIT approval rate (pictured below), # of HITs approved, etc. to certain levels, requesters can keep out bots. By adding premium qualifications for a fee, requesters can look at more narrow slices of the population or get more targeted data.

As a result, I was able to create two almost identical surveys: same questions, same basic qualifications, but one of them had a Liberal premium qualification and the other had a Conservative premium qualification.

Survey questions. Each survey had 18 questions; the last one is to detect bot responses. Most questions assessed either the respondent’s willingness to get a vaccine or various aspects of respondents’ attitudes toward the pandemic. Several questions asked for demographic information that could help predict WTP.

The survey in its entirety is shown below:

(A quick note — though vaccination WTP was the focus of this survey, I wanted to avoid putting it as the first question. I decided it would be better to ‘jog the memory’ of responders by asking a few basic questions involving thoughts on the pandemic before asking for WTP).

Results

Unfortunately, 2 Liberal respondents and 3 Conservative respondents gave unclear answers to at least one question so I threw them out. This meant I was working with 18 Liberal responses and 17 Conservative responses, so the sample size was quite small.

Summary statistics for responses to each of the questions are provided below in the same order as the questions themselves.

1. 83.3% of Liberals said they were or are planning to be vaccinated, compared to 41.1% of Conservatives
2. 2.22 was the average Liberal responsibility score, compared to 2.72 for Conservatives (This means Liberals have a slightly stronger sense of personal responsibility for vaccination than Conservatives)
3. 61.1% of Liberals supported a vaccination mandate, compared to 11.8% of Conservatives
4. 3.89 was the average Liberal threat assessment, compared to 2.29 for Conservatives (This means Liberals viewed COVID-19 as a substantially greater threat than did Conservatives)
5. The highest WTP was a Conservative with $1000 WTP. The second highest was a Liberal with $500 WTP. 15.7% of Liberals had a $0 WTP, compared to 38.9% of Conservatives. 5% of Liberals were unwilling to be vaccinated, compared to 28% of Conservatives. Discounting outliers and those unwilling to pay, average Liberal WTP was $45 and average Conservative WTP was $25.
6. 55.5% of Liberals were regularly vaccinated for common illnesses such as the flu, compared to 11.1% of Conservatives.
7. 4.22 was the average Liberal unvaccinated blame level, compared to 2.41 for Conservatives (Liberals blamed the unvaccinated substantially more than did Conservatives)
8. 3.22 was the average Liberal approval of the Biden-Harris administration, compared to 2.18 for Conservatives (Liberals felt ‘so-so’ about the Biden-Harris administration while Conservatives expressed moderate disapproval of it)
9. 1.56 was the average Liberal approval of the Trump-Pence administration, compared to 3.00 for Conservatives (Liberals strongly disapproved of the Trump-Pence administration while Conservatives felt ‘so-so’ about it)
10. 1.94 was the average Liberal concern level about vaccine safety, compared to 3.71 for Conservatives (Liberals generally disagreed that the vaccine had substantial safety concerns while Conservatives expressed moderate concerns about its safety)
11. -10% was the average income change for Liberals, compared to -3% for Conservatives
12. 94.4% of Liberals stated they were employed, compared to 88.2% of Conservatives
13. 4.06 was the average job impact rating for Liberals, compared to 3.94 for Conservatives (Liberals and Conservatives both reported a slight negative impact from COVID-19 on their jobs)
14. 27.8% of Liberals had jobs requiring vaccination, compared to 0.0% of Conservatives
15. 50.00% of Liberals had jobs requiring masks, compared to 11.8% of Conservatives
16. 66.7% of Liberals owned homes, compared to 58.2% of Conservatives
17. 41.2 years was the average age of the Liberals, compared to 42.9 for Conservatives. (However, it must be noted that Liberals had a much narrower age spread concentrated at the 30–40 range; Conservatives were relatively more equal and represented a broader range of ages).

Conclusions and Observations

From this pilot study, it seemed that the responses were fairly indicative of the respondents’ political leaning. Even from a very small sample size, several differences between liberals and conservatives regarding COVID-19 vaccine views were reflected accurately (i.e Questions 1 and 13). This is a promising sign: it shows that AMT’s qualifications of its workers are accurate (enough) and that I seem to be asking the right questions. Thus, with more data (being collected now), we should be able to get more accurate characterization of peoples’ thoughts about the vaccines.