Verify Questions based on Images and Boxes

Instructions

In this task, you are asked to verify 50 questions for images by clicking on good and bad buttons. For each question, we will show you six boxes on each image, each containing an object. There are one yellow box and five blue boxes. Each box corresponds to a possible answer to the question (e.g., "Which is the most expensive object on the table?"). Your question must have the yellow box as the correct answer and other blue boxes as the incorrect answers. You need to look at the corresponding image to make sure the answer to the question is not wrong, unclear or ambiguous. For minor typos, as long as they don't affect your understanding, don't make judgment based on them. You may refer to the examples below.

Rule: We have labeled six out of the 50 pairs in each HIT. In order to be accepted, your judgments should match at least four out of six with ours. You won't be able to submit your results until your judgments meet this standard. You can improve your results by going back and update your judgments.

Please contact us if you have any questions about the HITs. Thank you in advance!

Bad Blue boxes must be wrong answers.

The man on the left (in a blue box) is also wearing a shirt.

Bad The yellow box must be the only correct answer.

The yellow box is not the correct answer in this example.

Bad The region itself must answer the actual question.

Pointing to the region doesn't answer the actual question.

Bad Another bad example fo the same reason.

Pointing to the region doesn't answer the actual question.

Bad The question must point to the object tightly enclosed by the yellow box.

The which object must be tightly enclosed by the yellow box, not the small objects inside the yellow box.

Good The question should be precise and clear.

Good Another good example.

Okay, let's start!

Rule of Thumb

Consider when you are shown the image, the question and the six boxes (without color distinction).

Are you able to pick the correct box from the boxes? If you can't, it is a bad question.


Your browser does not support the HTML5 canvas tag.

What is striped?

Hint: Press z on the keyboard for good questions and x for bad ones

20%
{% include "simpleamt.html" %}