Beginning Evaluation with a Data Justice Lens 

The best time to rethink an evaluation process is at the beginning of a process.

This is the ideal moment to identify specifically why you are collecting information, tweak your line of questioning to avoid bias, and to trim out unnecessary data collection by following data justice principles.

It is also an ideal moment to double check the security of your data collection and storage, and formulate plans to share back with the community and partners. 

These steps are some of the fundamentals of data justice.

Photo Credit: Levi Shand / ACRE

What is data justice? 

The Metropolitan Alliance of Connected Communities defines data justice as, “Data justice is the collection, analysis, sharing, and use of data entirely in service of and with accountability to participants and their communities. Data justice is a lens to evaluate the intended or unintended consequences of researching or collecting data on individuals and communities. Data justice values the dignity, privacy, and humanity of all people. Data justice is active resistance to all oppressive systems within the arena of data as a part of building a better world that promotes dignity, consent, truth, accountability, and learning. This work will be necessary as long as oppressive systems exist.”

Readers can find more detailed information at the MACC website. 

How does data justice apply to the beginning, middle, and end of a project? 

At the beginning of a project, it is important to pause and reflect on your processes.

Start by identifying how the data you plan to collect will be useful to individuals, communities, the organization, and how it will be part of a specific utilization plan. Once you have articulated this, it will be your ‘why’ statement, which answers, “Why collect this data?” Plan to share this why-statement with all partners and participants. 

Additionally, before starting the project it is best to limit the size and scope of your evaluation process as much as possible. Recognize that data has financial costs (wages for staff and technical infrastructure) and human costs (surveys can act as barriers to services, and can pose retraumatizing questions – do not do this).

At this point in the process, eliminate collecting data that do not serve participants and their communities. This could look like collecting data ‘just in case’ or just because it would be ‘nice to have’. Ask yourself, “What would happen if we didn’t collect ___?” 

Once you’ve determined what you will collect, have sensitivity in evaluation design, questions, and purpose. Review materials for biases and potential to cause harm. Elevate diverse voices and perspectives, as well as accessibility and inclusion, at all stages. Prioritize evaluation that contributes to program improvement, equity, access, and organizational learning.

This might also be a moment to revisit your ‘why’ statement In case your reflections have changed any of your answers. 

Photo Credit: Indigenous Roots

Next, double-check the security and privacy plan for storing the data you plan to collect. Once that is confirmed, you can share the evaluation engagement expectations with partners and participants as you start your time together. Be open to feedback and Iteration based on that feedback. Allow people to opt-out without threat of losing funding or other support/activities. 

Towards the middle of a project, It’s a good idea to check-in with your partners and participants to briefly ensure that needs are being met. A simple three question check-In can help people provide concise feedback.

These three questions will help provide an opportunity to iterate and improve before the conclusion of the project whenever possible. 

At the end of a project, there is often a final evaluation. At this time, any narratives and information about communities and individuals should be true and not fabricated or assumed. Cross-check narratives and information with the partners and participants. Additionally, consider the balance of power in situations in which data are shared.

Finally, make a plan to give the data back to the community formatted as a usable tool (potentially for advocacy). This might take the form of a presentation, a 1-sheet summary, or other format appropriate to the context. 

Steps to start off your evaluation journey: 

Resources used in developing this tool: 

Metropolitan Alliance of Connected Communities, “Data Justice.” 

Metropolitan Alliance of Connected Communities “Data Justice for Nonprofits” panel presented by at MCN 2022 conference by Alicia Ranney, Adam Cowing, Emily Barter, Joylenna Garcia, Rebecca Mino, and Lucy Geach.