Phases of annotation are very customizable on the platform! You may have an easily identifiable object in your use case and each media/task only needs to be labeled once before its ready to train OR you may have a very difficult use case which needs multiple phases of QC/QA. On the platform you can customize how many labeling phases each task goes through.
One thing to note:
You must be a superuser or admin on the account to change phases of tasks
How to Change Phases of tasks:
Navigate into your project and move into the annotate tab at the top
Scroll down to the bottom where you can see all the Annotation Workflows
From here you can filter and select the workflows that you want to change. You can select them all by clicking the box next to workflow ID or you can select individual workflows. You can also use the search column to filter between phases that you want to select.
Once you have your media selected, click on the ellipses on the right-hand side
This will bring up a menu.
Select "Change Phase All" or it will say "Change Phase" if you only have some selected
From here a pop up will open:
Depending on what tasks the workflows you selected were in, different options will appear. (if you only select workflows that are in phase 2, you will not see the phase 1 workflow or phase 3 workflow options)
You will need to choose what to do in each phase.
Workflows that are in Phase 1 Completed, can be reset to phase 1 to re-do any initial labels -- this can be useful if the instructions have changed, and you need to start from scratch. Or you can move these to Phase 2 or 3 completed; or you can FastForward to complete workflow which will mean they have been submitted in phase 3 and ready for model training.
Note: You must keep in mind that technically all workflows/tasks start at Phase 0, meaning nothing has been labeled yet. Once you do an initial label you are submitting Phase 1 -- those are the phase 1 workflows that you can edit here. Then the task will go to phase 2, once you submit in phase 2 it will be phase 2 completed and ready for phase 3. Once you submit in phase 3 it will be completed workflow. So, anything you edit here already has labels.
Then you will choose what to do with Phase 2 and 3:
You can now rewind to the previous phase meaning you are moving it from phase 2 completed, back to phase 2 -- same goes with phase 3. This is how you can get more phases of QC if needed.
If you want a 4th round of labeling, I suggest resetting all phase 3 to "reset current phase" this will give you another round of QC, keeping all the labels that were adjusted in the last phase 3.
The last thing you will need to do before submitting is, choose to keep or discard pre-existing labels.
If you have selected to rewind or rest any workflows but you want to keep the labels intact, then select "yes". If you need to restart fresh, then select "no". If you are unsure-- you most likely want to select "yes", or you will lose all the work that has been done. For example, if you are just resetting Phase 3, for another round of QC, then select "Yes" because you don't want to lose the previous work, you just want to check it over again.
Thats it! It can be somewhat tricky to understand what you want to do, but just keep in mind that everything you can edit in each phase means it has been COMPLEATED in that phase already and you are choosing to rest the phase or move it on to the next one.
Some examples of why I use this feature:
If we have a simple project annotating only 1 object per image and I preform some QC of phase 1 tasks that are completed, and everything looks good -- I may choose to move all phase 1 tasks that are completed to complete workflow and start training the model. No need for phase 2 and 3 annotations here.
If we are working on a more complicated project where we are detecting fires, how annotations are done may differ from one person to the next so i may choose for there to be an initial phase 1 and then a phase 2, so every task is checked by another person. Once phase 2 is done and looking good, I will move all phase 2 tasks to complete workflow to start training.
If we are labeling types of ships and in phase 3 notice that there are still many tasks where the ship type is incorrectly labeled, I may choose to reset phase 3 and get a 4th round of QC on each task. This will ensure that everything is looked at one final time and edited before training.
Sometimes ontology changes right away, and we have already labeled over 100 tasks. if this happens, I will reset everything to phase 1 and choose to not keep the existing labels so the labeler can start from scratch with the new ontology. You could also create a whole new project if this happens but resetting the phases is quicker and you don't have to build the project again.