Annotation - Standard Protocol

Second Half

Medical Data Track, Session B

Now, you will annotate 40 images using the standard protocol. In this method, simply annotate a single boundary around the relevant node.

[1] Setting up the Annotation Interface

Please locate your assigned image range for standard annotation. The annotation assignments are here. For instance, suppose that you are annotator ID #4. Your assigned image range for standard annotation is 335 - 374.

[2] Preparing to Time Yourself

The time will be projected on the board. You will have been given a physical time recording sheet. When you begin annotating, record the current time (minutes and seconds). After you finish annotating an image, record the current time (minutes and seconds) and move on to the next image as soon as you can.

[3] Annotating with the Standard Protocol

The annotation interface is designed to be simple. You will annotate a series of images using a poly tool. To zoom in, scroll forward; to zoom out, scroll back. To move the image around, right click on the image and drag it into the desired location.

To begin, click on Annotate > Poly. This will automatically set you up to draw a polygon. Move your cursor over the image, and click where you want to make a vertex. When you are finished, right-click. This connects your most recently drawn vertex to the first vertex, enclosing a polygon.

If you are unhappy with an annotation, you can delete it. Make sure that the appropriate annotation is selected (it should have red borders). If you created it recently, it may already be selected. You can select an annotation by first pressing Pick, then clicking on the desired regions. Then, click on the red trash can button.

A more convenient way to draw polygons is to 'free-draw', in which you can freely move your mouse to define a curved region. To do this, click on Annotate > Poly, but hold shift and click while you move your mouse around the desired region. When you are finished, right-click. This will similarly connect the most recently drawn point to the first vertex, enclosing the polygon.

Rather than deleting annotations, you can also add and subtract regions to edit an existing annotation.

You can also change the opacity of the annotations by changing the Label Appearance setting. This can let you toggle quickly between the original image and the annotated image to determine if the annotations fit the image.

For the duration of these images, do not touch the Save Min button.

[4] Examples

Consider the following examples of ways one could annotate an image using the standard protocol. Feel free to refer back to these examples as needed while you are annotating.

Additionally, use the following examples of annotations directly taken from the LIDC-IDRI dataset as guiding references (even though they are not annotated in min/max protocol):

[5] Start Annotating

You may now begin annotating your range. Please ask your session lead if you have any questions or difficulties. You should take at most 30 to 40 seconds for each image, although you may be slower at first as you are acquainting yourself with the annotation interface.

[6] Experience Survey

After you are finished annotating, please fill out this quick halfway experience form.

[7] Finished?

Your participation session is complete! 🥳 Thanks for participating in this study. Check in with your session lead before leaving. Expect to receive a gift card soon. Let's Go Home →