| layout | title |
|---|---|
page |
Zooniverse Case Study |
Zooniverse is a platform for crowdsourcing citizen science. Volunteers assist professional researchers from all kinds of domains, by collectively classifying data for their projects. Zooniverse developed from the Galaxy Zoo project and is now a collaboration between the University of Oxford, the University of Minnesota - Twin Cities (UMN) and Chicago’s Adler Planetarium. With more than 2.5 million registered users, and almost 700 million classifications as of September 2022, it is one of the biggest online platforms for citizen science.
For projects on Zooniverse, the main research objects are large datasets of images, videos or text in need of human annotation, which are typically provided by professional researchers. Projects span a wide range of topics and users are asked to annotate individual data points such as telescope images of galaxies, create transcriptions of historic diary pages, or identify noises of deep sea animals from acoustic samples. The complete datasets are not visible to volunteer contributors, and project organizers have proprietary rights to the data for a period of about two years, after which they are thought to make the datasets publicly available.
The main task type is the annotation of a data point according to the instructions given for each individual project. Beyond this, users can make collections of data points they classified, and the platform offers forums for each project as well as talk pages for each individual data point (see Figure 2.2. C). These talk pages can be used for leaving comments, tags, adding data points to collections, and for participating in discussions for quality control and learning (see respective paragraphs below). Committed users can ask to become moderators of talk forums for specific projects.
Individual annotation tasks are typically small, but depending on the project, they can differ in complexity and duration, e.g. ranging from selecting from predefined options to transcribing documents. Within one project, the granularity of annotation tasks is of similar size. Modularity is given by the automatic integration of user contributions to the database after annotation.
Users choose which projects they want to contribute to based on their interests, and the quantity of data points they want to process, within the limits of available data. Beyond this, users do not self-select tasks as the data points are automatically assigned to the user one at a time. To engage further, users can take part in discussions on projects or individual data points using the aforementioned talk pages. Users can become talk forum moderators after being granted the rights by the project leads. Creating new projects requesting annotation is usually reserved to professional scientists.
In Zooniverse, quality control consists in several users classifying the same data points, which are then aggregated to a consensus answer (Kosmala et al., 2016). Users cannot see annotations of other users and quality control is done by project leads without further input by the contributors. However, post-classification, the user can access the data point’s talk page to provide feedback or discuss with other users who have annotated the same data point.
For each project, the project organizers offer tutorials to instruct users on how to annotate for that specific project. Additional engagement can occur through activities such as discussions on talk pages, by adding hashtags to data points or creating collections. Users also have the opportunity to take on more advanced roles, such as becoming a moderator in a project talk forum. The assignment of these advanced roles is handled by the project leads.
There is no coordination between users in the main data processing workflow, since annotations of other users are hidden. However, after annotating, users can access a page dedicated to the respective data point, where they can add tags and comments, and see meta-information other users entered. For further communication there is a forum attached to each project.