We envisioned this external evaluation as an opportunity for FRIDA’s community to see itself and learn together. For this reason, we contracted external consultants with extensive experience in participatory research methodologies. The consultants co-designed the methodology along with the FRIDA staff members who have been the most active in building, facilitating and managing FRIDA’s PGM process.
In addition to the team of external research consultants, we contracted as co-researchers nine young feminist activists from the FRIDA grantee partner and advisory community. As part of their engagement, co-researchers contributed to the design of the data collection tools – including defining the objectives of each tool and framing the content of questionnaires.
The 9 co-researchers were recruited through an open call process and selected based on:
- their background in feminist organising and participatory methodologies.
- regional diversity.
- availability to participate in all key stages of the research.
After the selection, co-researchers were trained in conceptual frameworks around grantmaking, participatory grantmaking and data collection tools. We also led specific sessions to train them in informed consent, reducing bias and tackling the challenges they might experience when conducting interviews online.
Data collection methods included:
This entailed reviewing the feedback and voting comments from more than 900 groups collected during the calls for proposals in 2016, 2018, and 2020.
The youth co-researchers conducted semi-structured interviews via Skype or Zoom with both grantee partners and FRIDA staff/advisory group members. The interviews lasted 45 minutes to 1 hour and were carried out in 6 languages. In total, co-researchers carried out 34 interviews with grantee partners, 7 interviews with advisors and 5 interviews with applicants who did not receive funding.
Based on advice and feedback from the FRIDA team and co-researchers, consultants defined questions that sought to capture experiences and feedback on participatory grantmaking from a larger number of respondents through a survey. The survey was open for a period of 3 months, and it was available in 6 languages. It was sent to all collectives that participated in the FRIDA voting process from 2016-2020. We received 158 responses. A separate survey was also created for FRIDA advisors who were part of the peer review process during these cycles. Note that since data was collected in different languages to ensure better reach and participation, some of it had to be translated for further analyses. Data was analysed and triangulated to identify emerging themes, trends and outliers which were then confirmed with the original data.
When the research process was set up in November 2019, it included in-person gatherings for reflection and data interpretation between the co-researchers. Due to the Covid-19 pandemic, we had to modify the methodology, and the data collection process was conducted exclusively online.
FRIDA allowed this evaluation process to take as much time as it needed. Our priority was to approach each aspect of it with care while providing continuous support for the FRIDA community involved. FRIDA is aware that communities might need time, resources and capacity strengthening opportunities to be able to participate in an evaluation process. In some cases, groups were not able to find time to participate in the interview, even though they had voiced their interest. In those cases, we needed to extend the timelines to ensure their participation.
We also need to acknowledge the power dynamics that exist between FRIDA and the participants (i.e. grantee partners) who were interviewed. If they hope to be funded by donors, participants may be reluctant to share their challenging experiences. The evaluation tried to account for power dynamics by making the process participatory and confidential. FRIDA staff members took part in the co-design of the participatory methodology and supported cross communication; however, they did not carry out interviews or engage in any data collection activities.
Note on research ethics and data management
Before starting the research, the methodology development team reflected on and spelled out the potential risks for both co-researchers and other research participants. This included the possibility of feeling uncomfortable answering certain questions and of social risks if any of the sensitive information they revealed were to be disclosed outside of the research.
Co-researchers signed a consent statement which clarified the objectives of the process, a timeline highlighting key deadlines and the key responsibilities of all parties involved. Co-researchers took on the role of reminding other research participants that they were under no obligation to participate. They told interviewees they could choose not to answer any question or terminate the interview if they felt uncomfortable for any reason.
All data collected was securely stored, and the methodology development team protected the confidentiality of all information gathered. Identifying information from participants, including first names and contact details, was gathered only after they consented to participate in this process. Such information has not and will not be disclosed publicly unless otherwise approved by them.
Raw data was fully anonymized for protection. Co-researchers had access to participants’ interviews and transcriptions only. Once the data was processed, care was taken to anonymize any identifying markers to ensure confidentiality and anonymity. Identifiable data (e.g. voice records) was deleted within three months following the completion of the study. Each stage of the process was in line with FRIDA’s Safeguarding Policy.
For the reasons above, the quotes shared in the evaluation are all anonymous.