Why has my study been marked as “Complete” when it is short of the requested number of participants?

Sometimes a study gets marked as complete even though the progress bar shows the desired sample size has not been reached. There are several reasons why this may occur, but the most likely reason is that a worker completed your study but was unable to submit the HIT in a way that MTurk counts as complete.

When this happens the number of participants in your data file (from Qualtrics, SurveyGizmo, or whatever survey platform you use) should still match the number you requested. Please check your data file and contact us if the number of participants in your file does not match your requested total. For extended details on this issue, please see below.

Question:

My study has been marked complete, but has not collected the full amount of participants I requested. Why has my study completed prematurely? What happened to the other participants?

completed

Answer:

The most likely answer is that the worker’s HIT was submitted to TurkPrime, but not to Mechanical Turk. When this happens the TurkPrime interface thinks your study has been completed, but the progress bar does not reflect the full number of participants.

The progress bar on TurkPrime logs participants as they are approved on Mechanical Turk. When a worker submits a HIT, their submission is recorded through TurkPrime first, and once TurkPrime confirms the completion, the information is sent to MTurk. If a worker is on a slow internet connection or experiences an interruption in service while submitting a HIT, the information may be recorded on TurkPrime, but never sent to MTurk. If this occurs, TurkPrime will show that the total number of requested workers have taken your survey and mark the study as Complete. However, since the worker’s HIT was never recorded on MTurk the participant will not be counted on the progress bar. Despite this error, the full number of requested participants should appear in your data file.

Although the worker’s HIT is recorded on TurkPrime in the scenario above, the worker’s HIT never gets submitted on MTurk and therefore is never approved, meaning the worker never gets compensated. In such cases, the participant will usually reach out to you and ask for compensation. To pay this worker, you can setup a Compensation HIT by going to the Manage Workers page on the main menu bar and selecting "Create Compensation HIT". For a comprehensive article on compensating workers, please see our Compensating Mechanical Turk Workers blog.