Related GitHub Issue. How we might track progress (task format).
The tech industry has a well-documented representation gap, and this issue is even more prevalent in open source communities. While there has been a marked emphasis on increasing diversity in these communities in recent years, numbers lag and the ability to foster inclusive environments remains challenging. With no standards or best practices for measuring inclusion, contributors, users and project teams lack the opportunity to make ethical, data-driven decisions.
The CHAOSS D&I Working Group has between January 2018 and December 2018:
- established collaboration spaces, including a mailing list, GitHub repository, and weekly calls.
- defined 7 focus areas which group metrics.
- established a template for resource pages that describe why D&I metrics are helpful and how to measure them.
- filled out an initial 10 resource pages.
- presented at 5 conferences via presentations, panels, and tutorials.
For the D&I Working Group, within the CHAOSS project.
CHAOSS mission from website:
The CHAOSS community is developing metrics, methodologies, and software for expressing open source project health and sustainability. By measuring open source project health and sustainability, CHAOSS seeks to improve the transparency of open source project health and sustainability so that relevant stakeholders can make more informed decisions about open source project engagement.
Community health cannot exist without diversity and inclusion. Diversity and inclusion is thus central to the CHAOSS project goals. See Objective 3 for more details on adding D&I to the CHAOSS mission.
The CHAOSS D&I Working Group establishes and broadly communicates an ethical, peer-validated, research-informed, set of standards and best practices for measuring diversity and inclusion in open source projects, and ensures that CHAOSS project software can implement D&I metrics according to identified goals for inclusion with validation from key industry partnerships in open source.
- Not all of the key CHAOSS project members recognize the D&I Working Group as relevant or in strategic alignment with overall CHAOSS goals.
- The CHAOSS project does not have a mission that states inclusion as central, and is thus without guardrails to ensure that standards developed are not weaponized to actually exclude or exploit people.
- There is currently no standard measurement for 'what is a quality metric' that we can apply as part of a pull request. (e.g., we have no way to ensure that a metric is validated by research, includes consideration for region or cultural interpretation).
- Open source projects (especially D&I specific leaders) are reluctant or simply don't have the time to help validate the outcomes of this project.
- Progress of D&I Working Group may appear slow to outsiders (despite rapid progress), and thus, it's difficult for executives to remain committed to the end goals.
- What we define as a "metric" is perceived as "best practices" or difficult to discuss and evaluate for other terminology barriers.
- Most D&I metrics are not implemented and few projects use these D&I metrics in their work.
The D&I Working Group has documented three high quality and compelling use cases for D&I metrics across multiple focus areas. Each use case demonstrates how in different circumstances the use of D&I metrics is beneficial. Each use case provides at least two metric examples and discusses ethical considerations for data collection, storage, and reporting.
- KR 1 - We have documented and shared for feedback three high quality and compelling use cases for D&I metrics across multiple focus areas.
- KR 2 - We wrote a blog post for each use case and published it on the CHAOSS website.
The D&I Working Group has created 7 high-quality, ethical, peer-validated, research-informed focus areas and shared stories of how 5 open source project partners applied those metrics to establish baseline understandings of their project's diversity and inclusion.
- KR 1 - Focus areas for events, leadership, governance, communication, contribution and one other are released in alpha phase.
- KR 2 - We have a method for validating alpha release of metrics, that includes 3 qualitative and 3 quantitative means.
- KR 3 - We have an agreed set of criteria for partnership cooperations on how to interact, communicate, provide feedback, and provide help for projects that adopt our D&I metrics.
- KR 4 - We have published a blog post about how each project used D&I metrics in their work.
The CHAOSS project website includes a mission statement that clearly articulates inclusion, diversity, and ethics of this work as central to community health, and thus project goals.
- KR 1 - Diversity, inclusion, and ethics is added to a revised CHAOSS mission statement, approved, and posted on the website.
The D&I Working Group has explored key questions of ethics and established ethical guidelines which everyone has to follow before they are allowed to use the brand "CHAOSS D&I Metrics."
- KR 1 - We have crowdsourced and prioritized ethical considerations around data collection, usage, storage, analysis, and reporting.
- KR 2 - We have documented ethical best practices for D&I metrics and embedded them in our project documentation.
- KR 3 - We evaluated solutions for how we could enforce ethical best practices for anyone who wants to use CHAOSS D&I Metrics.
- KR 4 - We have published a blog post about how we developed the ethical guidelines and encourages adoption.
The D&I Working Group has documented a development workflow for metrics development including state (wip, alpha, released) for metrics.
- KR 1 - Working Group workflow is documented, and shared for feedback.
- KR 2 - All existing resources are updated based on standards, and implemented as part of development workflow.