The best machine learning models are built on the highest quality training data, which is why Annotation/labeling is such a crucial step in any successful Project.
To make sure you get the highest quality, we recommend a three-step linear Annotation process, which we've built directly into the platform. We keep all of this organized into what we call Annotation Workflows.
Annotation Workflows
Each media (picture or video) you attach to a Project has an Annotation Workflow created for it. This Workflow is made up of three Phases, each of which corresponds to a specific Annotation Task that must be completed.
Let's take a specific picture we want to annotate as an example. Let's say we uploaded this picture into our account and attached it to a Project.
The picture is assigned the unique ID 6806443.
Now, for Media ID 6806443, you can see its individual Workflow ID in the table below.
This Workflow represents all the Annotation work that must be done for this picture. The three Phases of annotation are represented by the color progress dots.
So, for this picture, the Phase 1 task has been completed, and Phase 2 is waiting for someone to get started. Phase 3 doesn't exist, because Phase 2 hasn't been completed yet!
Progress Dots Key
Here are the different colors possible for the Progress Dots, so you always know the status of any Annotation Workflow.
Annotation Phases
Here's a short description to help you understand each Phase.
Phase 1 - Initial Annotation
This is the first pass anyone is doing on a particular picture or video, so you're really "starting from scratch". Whoever does this Phase will need to closely follow the Annotation Instructions to label specific Categories the way you want them labeled.
They'll draw polygons, boxes, or other shapes, or maybe just apply a classification label to the media itself—however you've set up that particular Project.
(Sometimes, we can use an existing model to automate this Phase.)
Phase 2 - Annotation Review
Phase 2 is the first step in quality control. Ideally, this is a chance for someone else to review the work someone did in Phase 1. It's good to have a second set of eyes to confirm that your Categories were annotated correctly and in-line with your specific definitions.
Phase 3 - Final Review
This Phase is somewhat optional, but highly recommended. It's always a good idea to check yourself one more time before you begin training a machine learning model. Perhaps you want to do one last quality control check before you consider your Annotation Workflow complete, so Phase 3 is perfect for just that.
Note: The number of phases your tasks go through is highly customizable! When selecting the tasks in the annotation tab, you can press the ellipse “…” button and select “change phase”. Here you can fast-forward selected tasks to complete workflow (which means they are ready for training) or you can even rewind tasks to phase 1 and erase or keep the labels, OR you can just simply move tasks from phase 1 to phase 3.
If you only want to do Phase 1 because your use-case is simple, you can move everything from phase 1 to complete workflow and start training!
If your use-case is a little trickier and you want everything to go through 1 more Phase of QA, you can select all and reset phase 3, this will keep all labels from Phase 3 intact and ready for another set of eyes.