Brynn Marie Evans

This is a pretty high-level summary (e.g., not that detailed) of how I create and run a new survey on Mechanical Turk. Since people have lots of different ways of using Mechanical Turk, this how-to may or may not be for you. I’ve also noticed that nearly every survey or questionnaire I put on Mechanical Turk requires different hacks

Creating A Mechanical Turk Task

Step 1: Select. You’ll start in the design tab, and either copy a template that exists (there are some default templates in there), or edit one of your saved drafts.

Step 2: Properties. Then you set the properties of your HIT. There are some more details like pay rate and automatic approval time at the bottom of this page.

Step 3: Layout. Next, you’re still under the global tab for Designing the hit, but you switch to a sub-tab for Designing the Layout. The other thing to note here is that you can adjust the “frame height” of your hit (which will determine how long the page’s scroll bar is and what users see “above the fold”).

There are a few things to note for when you do dive into the HTML editor:

1. Mechanical Turk is a full-on HTML enabled. Anything you can do with HTML/CSS, you can do to style your survey here.

(See the sample HTML code below for: an embedded survey and an HTML/CSS-created survey)

2. This includes embedding objects (like images or iframes if you are linking to a Google Form or Survey Monkey survey that’s hosted elsewhere).
3. You cannot embed Javascript directly, but you can use Javascript with your HIT. I did this by creating a Javascript snippet and saving it on an external site, but then linking to the external site from Mechanical Turk. There may be other ways to accomplish this as well that I don’t know of.
4. Last I checked, you have to have some variable and associated variable list (the “Input File”) linked to your HIT. This is typically used if you want to test, say, 100 images, but where each user only tests 10. So you have to cycle through the first 10 for the first user, the second ten for the second user, etc. Even if your HIT doesn’t use this method AT ALL, you have to include some fake variable and link a fake variable list (see Step 6).

Here’s my dummy variable line which I always leave at the end of the HTML script:

<p id="${dummy}">&nbsp;</p>


Step 4: Preview. Finally, you’ll preview what your HIT will look like to Turkers. If things look funny, go back to Design Layout to tweak them.

Step 5: Begin Publishing. When you’re done there, you’re taken to the Publish page. Your latest HIT will be highlighted in yellow.

Step 6: Choose Input File. After you Select your HIT, you have to list your variables file (“Input File”). Recall that I mentioned this before? It’s wacky when you’re not using variables (like placeholder images), and this file has to exactly match what your put in the HTML of your HIT. If you’re just doing simple surveys, feel free to use my dummy variable line in your HTML and attach this file (exactly as is, since the Windows formatting was also required) — you’ll be fine!

Step 7: Final Preview. You’ll have one last chance to preview your HIT:

Step 8: Pay and Confirm. And then confirm all the settings, including your final estimated cost. It’ll tell you if you don’t have enough money, and if you click there to add more funds, it’ll pop you back into your workflow here when you’re done.

Here’s some sample HTML code which I used for two surveys recently:

Embedding a survey
<p>Please complete the embedded survey (within the black box). Be sure to his SUBMIT at the bottom of that form, and then SUBMIT for the Mechanical Turk form!</p>

<p>Enter that <u>CONFIRMATION CODE</u> here to receive payment! Thank you!</p>
<p><textarea rows="1" cols="30" name="Name"></textarea></p>
<h4>If you have any comments or feedback, please leave them below!</h4>
<p><textarea rows="3" cols="80" name="comment"></textarea></p>
<p id="${dummy}">&nbsp;</p>
</div>

Styling the survey with HTML/CSS within Mechanical Turk
See the code snippet on Pastie.

If you want additional samples (like with using Javascript), please email me separately.