Selecting Panel Options

Once you begin creating a study, you can select the demographic characteristics you would like your sample, or panel, to have.

The first step in setting up your study is selecting the number of participants you'd like and the demographic characteristics they should possess. The sample you eventually collect is referred to as a "Panel" and thus by selecting demographic characteristics, you are selecting panel options.

TurkPrime has profiled MTurk workers on a variety of demographic characteristics. Some of these characteristics are basic demographics such as gender, age, race, level of education, income, marital status, etc. Other demographic characteristics are specific like how often people drink alcohol, which social media platforms they use, whether they have been diagnosed with a medical condition, how often they exercise, and whether they own a pet.

To simplify searching through demographic qualifications, you can select filters that will display only the qualifications that are in a specific category.

MTurk Toolkit Study Setup: Panel Options

If you do not see a qualification you are interested in, you can suggest one. We can screen tens of thousands of Mechanical Turk workers within days and create the qualification for you.

TurkPrime MTurk Toolkit Study Setup: Suggest Options

How Does TurkPrime know Panel Demographic Data? And, How Accurate is the Data?

How TurkPrime Gathers Demographic Data

We determine people’s demographic characteristics by adding a few questions to each HIT submission page. Researchers are given the option to allow or disallow these questions on HITs associated with their studies and workers are free to answer these questions or ignore them. This means all the demographic data TurkPrime collects from workers on MTurk is done through volunteer submission.

When researchers grant permission, we add three questions to each HIT. The questions are randomly selected from a list and are asked in the order of priority that TurkPrime assigns to them. We ask workers demographic questions at various points in time and unassociated with any one HIT. Workers are incentivized to answer these questions because doing so creates more opportunities for them to participate in well-paying studies. An example of how these questions appear to workers is below.

TurkPrime MTurk Toolkit Study Setup: Qualification Questions

The Accuracy of Panel Data

Because we ask worker’s the same questions periodically, we can determine their response consistency. People who consistently respond the same way are granted a qualification that makes them eligible when a specific Panel is selected. When the Panel is selected, workers are targeted directly but the study appears like any other study keeping workers from knowing what criteria made them eligible.

While our process of determining response consistency for demographic data is fairly accurate it is not perfect. Sometimes people slip through who do not actually possess the qualities we’re looking for. If you find participants in your dataset who do not match the demographic criteria of a panel feature you selected, let us know and we will replace the participant.