From 505a2db7932bbebd97a23416f2e9858349b652fd Mon Sep 17 00:00:00 2001 From: Rachel Chase Date: Fri, 31 Jan 2025 17:00:21 +0100 Subject: [PATCH 1/2] update socioeconomic --- .../socioeconomic-sampling.md | 31 ++++++++++++++++++- docs/{09-FAQ => 09-FAQ and resources}/FAQs.md | 0 .../_category_.json | 0 .../resources.md | 0 4 files changed, 30 insertions(+), 1 deletion(-) rename docs/{09-FAQ => 09-FAQ and resources}/FAQs.md (100%) rename docs/{09-FAQ => 09-FAQ and resources}/_category_.json (100%) rename docs/{09-FAQ => 09-FAQ and resources}/resources.md (100%) diff --git a/docs/03-experimental-design/socioeconomic-sampling.md b/docs/03-experimental-design/socioeconomic-sampling.md index 67b65652..61525c39 100644 --- a/docs/03-experimental-design/socioeconomic-sampling.md +++ b/docs/03-experimental-design/socioeconomic-sampling.md @@ -9,12 +9,41 @@ Expertise and social inclusiveness. A guide to choose participants for on-farm t A common weakness in standard participatory variety selections is that farmers are chosen without eye for their experience and the specific work they are doing and to which local social category they belong. Where this is considered usually very broad general categories are used such as age and sex., occupation, leve of education, farm size. Furthermore, when gender is brought in focus, the practice is mainly on having both men and women farmers in equal numbers evaluating the trials, disregarding their specific expertise or experience in farming. Another problem is that often farmers get chosen who feel comfortable talking and interacting within the sphere of a scientific evaluation, which emphasizes experience in reasoning and talking. This often excludes very skilled persons that however are not able or are normatively not allowed to communicate these skills and knowledge through language. But even if the respondent is good at talking it still does not include the tacit knowledge, the embodied skill and knowledge that people have. Breeders are however interested in detailed concrete hands-on information if they want to align with a demand led breeding approach such as the stage gate breeding approach that is now introduced in the CGIAR public sector breeding. Within the current reform to a stage gate breeding approach it is also crucial to get feedback from not only farmers but also processors/prepares and marketers who turn the RTB crop into an edible quality food product. To overcome these issues while choosing the TRICOT participants, we therefore work with a so-called purposive sampling using a task group approach with an explicit gender dimension. This gender dimension is not only important within the light of gender equity but is also very practical and concrete if we want to know the expertise and experience people have with regards to work related to the RTB corps because often tasks related to the RTB crop are gendered: certain tasks are often carried out be a specific sex. Important to note here is that a task group is not necessarily a group that works together but a category of people of the same social segment that carry out similar tasks related to the crop. + A task group approach is also in line with a much more performative way of participation instead of only a deliberative one (Richards, 2007) . In such a performative approach people who are verbally not strong or are not allowed to speak up are included and approached more self-evidently and tacitly, because they have been identified as a specific task group within a locally defined and thus relevant social group or intersection of different groups. From a demand led perspective breeders are interested in good information on the suitability of improved varieties within the livelihoods of the users and therefore want to know the user preferred crop characteristics. To get good information it is therefore important to get this from experienced people that are skilled in farming but also with regards to processing of the crop into the food products and their quality as well as the marketing of the crop. Importantly storability of food products as well as fresh roots tubers and bananas are important in the last instance. E.g if men are are hardly involved into processing cassava into gari , a breeder will not be interested much in men’s preferences with relation to processing because they will not always be able to give correct hands on experience-based knowledge because they do not possess the skill. Men in this case might have indirect knowledge about it (e.g. through their spouses), but certainly not the embodied skills. -A bad example of how participants got chosen is a farce case where someone wanted to work with mango farmers and ended up with only male participants of which non had a mango tree field. When later confronting these participants with this, the village chief and his friends all claimed to have a mango tree in their backyard. It is obvious that a tricot with such participants will not yield the best information. +A bad example of how participants got chosen is a farce case where someone wanted to work with mango farmers and ended up with only male participants of which none had a mango tree field. When later confronting these participants with this, the village chief and his friends all claimed to have a mango tree in their backyard. It is obvious that a tricot with such participants will not yield the best information. + +A task group approach and gender +Sex-disaggregated data collection protocols on variety preferences are problematic as they put upfront sex and gender differences as an explanatory factor. From a political and ethical perspective this approach is also guilty of up-front discrimination based on sex difference. This type of data also renders invisible how gender roles are shaped by the intersections of locally defined identities, such as occupational tasks, immigrant or ‘local’, ethnic/language group, age group and economic status. So instead of initially segregating by sex groups we propose an identification of task groups. Simply: Who does what within each of the locally defined social categories. +This task group approach can capture the intersectionality of local identities by focusing on who does what along the value chain and allows for a closer integration with practices related to participatory trials, post-harvest processing evaluation, food processing and marketing as these connect to specific tasks and thus specific crop characteristics/traits. Identifying who does what in each of the social groups in the community can accurately and concretely tap into the knowledge, skills and working conditions of these groups without referring to gender as a subject, but with the work that different often highly gendered task groups do, as subject. +Taking the task and the work as central entry points avoids the highly political aspects that come into view when explicitly addressing sex and gender. By continuously referring to the work and working conditions related to a specific crop or set of crops, one can much more neutrally gain knowledge about how gendered certain task are, and learn about the constraints and opportunities and preferences of the different task groups. And this is something breeders are interested in if they want to see an increase of adoption of their varieties. In addition, by valuing and showing interest in the work done by different groups one triggers senses of proud and belonging and it facilitates more politically neutral and realistic conversations and cooperation based on the, organization and technicalities of the work and the working conditions and how this work can be done better in the current context. Knorr-Cetina (1999) would call this; building epistemic cultures based on fascination and content of the work. This method communicates focused skilled practice instead of emotion and values as abstract themes. +Therefore, task group based PVS trials and participatory processing evaluation exercises as well as participatory breeding events are perfect social science tools to achieve such a more performative and tacit cooperation based on technological practices and their organization. Such PVS exercises can be done with merchants as a group, the growers as a group as well as processors. Consulting people who are involved in more than one of these activities are an opportunity to catch more task groups at once. And it is this opportunity that we will have to use because choosing someone as a TRICOT participant, assumes that such a person is farming the crop. In many cases however the person does a lot of more tasks related to the crop. +People can simultaneously belong to different task groups and the extent to which they belong to one and not another and the extent to which certain tasks are done by women and men are informative about gender roles and norms and current possibilities and developments within farming, processing and selling. + +How to practically use a task group approach? + +After applying a representative sampling frame for your region: having a certain amount of communities per state or district in the entire area of focus (there where the crop is commonly grown) and after having consulted e.g. government bodies at different geographical levels to identify active regions where the crop is cultivated, it now comes down to identifying participants within communities on ground. + +For this a task group approach distinguishes the following practical stages to follow at village community level: + +1. Identify which crop and which tasks around that crop exist: whose experience and skills and work are of interest to us? e.g. Are people who have experience in cultivation as well as processing relevant to the study? This is useful if you want to practically and directly relate the relative importance of agronomic, processing and food quality characteristics. Usually a breeder should be interested in skills/expertise in all the work related to the crop because it can reveal crucial traits within the value chain that are needed to assure adoption of a variety. This will help the breeder in determining the key traits necessary. + +2. Identify how a community itself defines its different social groups. This can be done through meetings with village heads and discussions with key informants, transect walks and interviews with several people carrying out the work around the crop under study. This also involves making the TRICOT’s scientific inquiry explicit among the leading elite and key informants in the community so that they clearly understand that you want to include respondents/participants from all social groups, also the disadvantaged and the less better off, and explicitly try to avoid leadership effect (e.g. Humphreys et al 2006) and state that the village leading elite is not to determine only who participates: cultivate a democratic and equity focused discussion and debate on studying experienced based crop related expertise that exist among the different social groups. Table 1 can help here in mapping the community. + +3. Within each locally defined and thus relevant social category, identify ‘Who does what’ in relation to the production, processing and sale of the crop and its food products and identify respondents/participants representing the task or combinations of tasks that you are interested in (crop dependent) without using sex as initial criteria of selection. This should be done by verification that the respondent/participant indeed has detailed knowledge on the tasks that she or he has indicated to master: probe with work related jargon and discuss detailed practices related to each skill. N.B.: Make sure you have detailed knowledge of these practices on board in your field team to make good verification possible. The field team should therefore include breeder, anthropological, socio-economic and food science expertise. Food science is particularly important for the verification of the participants skills and obviously because quality food products and their relation to different varieties can be crucial for adoption. + +4. Include participants from each social group if applicable (e.g if Fulani herdsmen do not cultivate or process the RTB crop it would not make sense to include them as participant). + +5. Determine how you organize participants. For cassava in Nigeria we identified one lead farmer per 10 participants and 10 participants per community for whom the data collections is managed by the lead farmer that has or does not have a TRICOT trial her/himself. The 10 participants were representative of the community and included all important local social groups. It was important to build on local authority and respect: the group of 10 farmers chose their own representative! This was found very important to assure providing ownership of the activity to the participants. One could also choose an agricultural extension officer to lead the farmer group, but this can only be done and will only give good data if these officers are respected and almost part of the community, which is often not the case (and it was not the case in most the communities visited in Nigeria). Important to note here is that we chose for a group dynamic (of 10-11 persons in each community, depending on if the lead farmer also was a TRICOT participant or not) to stimulate the sense of belonging to the project as a group member. One could choose individual farmers even far from each other that will each independently communicate the information to you. Theoretically, from a positivistic sampling position, this might seem more appropriate, but we argue that the performative aspect of sharing the same project with a group creates a spirit of dedication essential in a crowdsourcing approach. It also facilitates better reporting back of information to the farmers: they become a community of colleagues rather than mere sole participants. + +*N.B. Remember that trial visits and research station staff costs should be reduced to a minimum, so it is very important to invest in choosing and building the right reliable local management unit and this works best using local respected persons and authorities.* + +6. Determine an appropriate mode of compensation to the participants. We compensated through the lead farmer and then followed up with the participants if they received their payment from the lead farmer. Payment was done through bank transfer. In case leadfarmer with bank accounts cannot be found another system of payment has to be thought of, like paying out during the lead farmer training. A modest sum for the work of the lead farmer moving between participants fields was arranged. The amounts should be low as to assure participants motivation for the project and not just to hang in for the monetary benefits, but it should however be that substantial to motivate participants and make them feel that they are not doing it all for the sake of the research project. These amounts have all to be locally determined based on the context. The agreement should be clearly stated in the agreement made with the lead farmer and signed to avoid disputes. +Table 1. Example of local social groups based on interaction with a village community in the Southwest of Nigeria focusing on cassava. Apparently ‘ethnic group’ was found important to distinguish different people. The groups highlighted in grey are the groups included among the TRICOT participants as the others are not involved in cassava work. | Locally Relevant Social Group* | Share of Local Population (oral or record share) [B1.4a] | Language (record language) [B1.4b] | Associated Livelihood(s) or Crops [B1.4c] | Tasks Related to Cassava (Women) | Tasks Related to Cassava (Men) | Better Off Group(s)? (Yes=1, No=2) [B1.4d] | Politically Active & Influential Group(s)? (Yes=1, No=2) [B1.4e] | |----------------------------------------------------|----------------------------------------------------------|-------------------------------------|------------------------------------------------------------------------------------|------------------------------------------------------|--------------------------------------------------------|------------------------------------------|---------------------------------------------------------------| diff --git a/docs/09-FAQ/FAQs.md b/docs/09-FAQ and resources/FAQs.md similarity index 100% rename from docs/09-FAQ/FAQs.md rename to docs/09-FAQ and resources/FAQs.md diff --git a/docs/09-FAQ/_category_.json b/docs/09-FAQ and resources/_category_.json similarity index 100% rename from docs/09-FAQ/_category_.json rename to docs/09-FAQ and resources/_category_.json diff --git a/docs/09-FAQ/resources.md b/docs/09-FAQ and resources/resources.md similarity index 100% rename from docs/09-FAQ/resources.md rename to docs/09-FAQ and resources/resources.md From 0fc4843a2692f32ed27ac9e7819ae6dd270e37eb Mon Sep 17 00:00:00 2001 From: Rachel Chase Date: Mon, 10 Feb 2025 10:21:28 +0100 Subject: [PATCH 2/2] update from manuals --- .../01-introduction-to-tricot/introduction.md | 60 +++++++++- docs/01-introduction-to-tricot/steps.md | 6 +- docs/02-planning/introduction.md | 58 +++++++++- .../socioeconomic-sampling.md | 56 +++++++++ .../standard-operating-procedures.md | 14 +++ docs/03-experimental-design/tpp.md | 8 ++ .../trial dimensions.Rmd | 29 +++++ docs/04-climmob-suite/climmob.md | 55 +++++++++ docs/04-climmob-suite/data-submission.md | 17 +++ docs/05-implementation/implementation.md | 109 +++++++++++++++++- docs/06-data-analysis/data-analysis.md | 50 +++++++- docs/07-feedback-dissemination/feeback.md | 71 +++++++++++- docs/glossary.Rmd | 74 ++++++++++++ 13 files changed, 600 insertions(+), 7 deletions(-) create mode 100644 docs/03-experimental-design/trial dimensions.Rmd create mode 100644 docs/glossary.Rmd diff --git a/docs/01-introduction-to-tricot/introduction.md b/docs/01-introduction-to-tricot/introduction.md index 7201f514..104906d2 100644 --- a/docs/01-introduction-to-tricot/introduction.md +++ b/docs/01-introduction-to-tricot/introduction.md @@ -30,4 +30,62 @@ Tricot's simplicity allows broad implementation across geographies, crops, and t 6. Outcomes and Impact -Enhances crop diversity and resilience by tailoring recommendations to local needs. Increases adoption rates by aligning product characteristics with farmer and consumer preferences. Supports sustainable and climate-adaptive agriculture by integrating real-world testing with robust scientific analysis. In summary, the tricot approach is a dynamic, end-to-end solution for product use testing in agriculture, integrating farmer trials, consumer insights, and conceptual testing. It drives innovation by prioritizing user needs, ensuring product relevance, and enabling resilient and inclusive agricultural systems. \ No newline at end of file +Enhances crop diversity and resilience by tailoring recommendations to local needs. Increases adoption rates by aligning product characteristics with farmer and consumer preferences. Supports sustainable and climate-adaptive agriculture by integrating real-world testing with robust scientific analysis. In summary, the tricot approach is a dynamic, end-to-end solution for product use testing in agriculture, integrating farmer trials, consumer insights, and conceptual testing. It drives innovation by prioritizing user needs, ensuring product relevance, and enabling resilient and inclusive agricultural systems. + +#Introduction (RTB) + +Tricot (triadic comparisons of technologies, pronounced “try-cot”) is a citizen science approach for testing technology options in their use environments, originally conceived in 2011 (van Etten 2011). The Oxford English Dictionary defines citizen science as “the collection and analysis of data relating to the natural world by members of the general public, typically as part of a collaborative project with professional scientists”. Different definitions are given by others, but our use of it is not far from this one. As a citizen science approach, tricot actively involves non-scientists in experimental data generation and interpretation. This follows a broader movement of applying citizen science and crowdsourcing methods in research on food and agriculture, providing a fresh lease of life to participatory agricultural research (van de Gevel et al., 2020; Ryan et al., 2018; Minet et al., 2017). + +Tricot addresses important challenges that have plagued on-farm testing, which is an often-underrated activity in the agricultural sciences (Kool, Andersson, and Giller, 2020). Also, the approach is increasingly used for areas closely related to on-farm testing of varieties, such as fertilizer testing (AKILIMO scaling project by IITA and CIP in Rwanda) and food product testing for sweetpotato implemented by CIP (Moyo et al., 2020, 2021). Testing technologies in their use environments is important for external validity of experiments, the degree to which the findings have application outside of the experimental setting. To overcome common issues in user testing, the tricot approach streamlines the approach through digital support throughout the experimental cycle, simplifies the experimentation format to make user participation easy, and enhances data analysis by enriching it with data about the user context. + +The method was first implemented and tested in the period between 2013 and 2016 for on-farm testing of varieties, and an earlier article reported about methodological progress in this period (van Etten et al., 2019). Much of this work was part of the Seeds for Needs initiative, aiming at broadening the range of varietal diversity to farmers to adapt to climate change (Fadda et al., 2020). These projects were focused on cereals and grain legumes. Since then, the tricot approach has been used for other trials, by different organizations (including private sector) and for different applications (food products, fertilizers, etc.), and for clonal crops (cassava, sweetpotato, potato), vegetables, and a perennial crop (cocoa). The present article reports on 1) methodological progress; 2) discusses important considerations that implementers of the tricot approach need to consider; and 3) areas open for future research. + +#Description of the tricot approach (RTB) + +The tricot approach has been described in several publications (van Etten et al., 2019; van Etten, 2011; van Etten et al., 2020; Steinke et al., 2017; van Etten et al., 2019; Fadda et al., 2020; Beza et al., 2017). Here, a short synthesis will be provided. + +The word ‘tricot’ is derived from triadic comparisons of technology options. ‘Triadic’ refers to the sets of three technology options that are compared by each participant. Tricot enables many citizen scientists doing a small experiment while contributing to answering a larger question. Researchers and citizen science participants are supported throughout the experiment cycle by digital tools to design, execute, monitor and analyze the trials. As many citizen scientists contribute and do experiments in their typical use environments using their usual practices, it becomes possible to start to understand how variation in environments and practices affects the results. + +The particular way in which tricot works makes these steps possible. The following aspects are key to tricot: +1. the use of incomplete blocks of three items – to make the threshold of participation low in terms of farm size, and reduce resource needs and training required; +2. the use of ranking as the main way to report observations -- to facilitate digital data collection and to make it possible to evaluate a tricot plot with very little training (in contrast, scoring requires calibration and absolute yield measurements require training); +3. the limited control of experimental conditions – following common local technology use practices to maximize external validity; +4. the use of a streamlined digital process from trial design to analysis – to make it manageable, executable with many participants, to reduce errors, to reduce costs, and to quickly deliver feedback to achieve high motivation and impact on subsequent decisions; +5. early feedback of the results to the participants -- to provide ownership to and stimulate engagement of participating “citizen scientists” and to validate results. + +Tricot builds on existing participatory research formats that have been used in the past, as documented by (Van Etten et al., 2019). The novelty of the format is the combination of the different elements in a standardized, widely used approach supported by a corresponding digital platform, ClimMob (https://climmob.net). + +Another innovation behind the tricot approach is the use of the Plackett-Luce model (Luce, 1959; Plackett, 1975). This statistical model is also not new, but an appropriate software implementation was not available. In previous analyses of on-farm data, data were converted to pairwise comparisons, after which the Bradley-Terry model was used (Coe, 2002; van Etten et al., 2019; Steinke et al., 2019; Dittrich et al., 2000). However, this leads to anti-conservative statistical error estimates and the conversion from rankings to pairwise comparisons implies information loss. This was the reason to implement the Plackett-Luce model in R (Turner et al., 2020). Also, a number of other R packages were created to support data management and analysis. These are described in section 12 below. + +The approach is supported by the ClimMob digital platform (https://ClimMob.net). The platform will be described in detail in a forthcoming paper (Quirós et al., forthcoming). It supports the user in designing a trial, randomizing the entries, creating electronic questionnaires, collecting the data, monitoring trial progress, and generating reports. + +There are several other elements that support the users. The different steps of the tricot approach are described in a manual (van Etten et al., 2020). Also, there are online guides and videos available from https://climMob.net. + +#How tricot works (from short guide) + +With the tricot method, large numbers of farmers carry out many small, simple trials on their own farms instead of a few big, complex trials conducted at research stations. A research center provides the participating farmers with material for the on-farm trials. The farmers +provide observations from their trials to the ag-ricultural research center, where the data from all mini-trials is aggregated and analyzed. The +research center then feeds back the findings to the farmers. + +With tricot, research centers can validate and disseminate new agricultural technologies in a participatory way, collaborating with a large number of farmers under diverse conditions. Large-scale tricot experiments, involving many farmers, generate excellent/reliable results about the performance of different technology options (such as different crop varieties or different fertilizer types) in different environments. Farmers evaluate the new technology options on their own farms and under real conditions. + +The tricot trial format is very simple for participating farmers: each executes the mini-task of evaluating only three technology options, out of +a range to be tested. This makes it possible to engage many farmers without expending excessive effort on training or supervising them. But this does not mean we can only evaluate three technology options at once! Even though each farmer only evaluates three options, they evaluate many different combinations of technology options, that partially overlaps with the combination of other farmers. By putting the results of their experiments together, a tricot trial can evaluate how well each the options performs relative to the others. Tricot is like a world sports ranking. These rankings cover all players (or teams) and reflect their relative strength. The scores depend on the matches the players have won from other players. But these calculations can be done even if certain teams never played against each other. + +Tricot is a valid strategy to overcome the ‘bottleneck’ of technology dissemination to users,often faced by research institutes, because it +presents the following advantages: + +• Farmer-led innovation +Being fully executed by farmers on their farm, tricot experiments account for important adoption criteria that could easily not +occur in researcher-managed trials. + +• Specific solutions +Rural households benefit directly and firsthand from discovering new technology options that fit their environmental and socio-economic conditions, with a high probability of improving their farm production. + +• Capturing diversity +Tricot experiments address the challenge of diversity in regions where environmental conditions or socio-cultural preferences vary strongly across the landscape. The tricot approach helps research centers to collaborate with farmer organizations, development organizations or input providers in the organization of large trials with many farmers. + +• Meaningful data +Tricot uses a data-driven approach that can combine farmer-generated experimental and preference data with data about cropping systems and farming households, thus it enables a rich analysis. Without tricot, this could only be achieved by very complex methods (such as crop modelling). The tricot data can be analyzed with existing maps of temperature, rainfall, altitude, and other environmental variables. These analyses can provide recommendations for different environments or strategies to deal with climatic risk. Tricot makes it possible to combine several seasons of data to do in-depth analyses of this kind. + +insert figure from page 3 \ No newline at end of file diff --git a/docs/01-introduction-to-tricot/steps.md b/docs/01-introduction-to-tricot/steps.md index 95bd736c..d3753bdc 100644 --- a/docs/01-introduction-to-tricot/steps.md +++ b/docs/01-introduction-to-tricot/steps.md @@ -6,7 +6,7 @@ sidebar_position: 2 > Rachel Chase -This guide provides a short overview of each of the 10 steps needed to develop and implement a tricot project. +**This guide provides a short overview of each of the 10 steps needed to develop and implement a tricot project.** ![Tricot 10 steps](./img/10StepsTricot.png) @@ -15,7 +15,7 @@ This guide provides a short overview of each of the 10 steps needed to develop a Researchers define a set of comparable technology options to test. For example, they decide to compare crop varieties with each other, or different fertilizer types, or irrigation technologies. They will provide the necessary materials (inputs or other) to project implementers (organizations that will reach farmers). Typically, about 8-12 technology options (comparable items) are included in the trial to be tested. ## Step 2: Design -The implementing organization uses the ClimMob (climmob.net) free online software to design the project. This digital platform has been specifically created to manage tricot projects, from designing the experiment to data collection and analysis. The use of the digital platform streamlines the process. ClimMob offers the following benefits: +The implementing organization uses the ClimMob [(climmob.net)](https://climmob.net/) free online software to design the project. This digital platform has been specifically created to manage tricot projects, from designing the experiment to data collection and analysis. The use of the digital platform streamlines the process. ClimMob offers the following benefits: 1. ClimMob helps to avoid mistakes by introducing QR codes and electronic forms; 2. ClimMob provides a dashboard to monitor progress; 3. ClimMob reduces or eliminates the effort spent on digitalizing data collected on paper; @@ -46,3 +46,5 @@ The implementers provide feedback to every participating farmer: the names of th ## Step 10: Evaluation Tricot is an iterative process: after every project cycle, researchers, implementers and farmers collaboratively evaluate how the process may be improved in the next cycle. + + diff --git a/docs/02-planning/introduction.md b/docs/02-planning/introduction.md index cb6ecf4e..503f522c 100644 --- a/docs/02-planning/introduction.md +++ b/docs/02-planning/introduction.md @@ -16,4 +16,60 @@ sidebar_position: 1 ## Resources and Budgeting - Required materials (e.g., seeds, observation tools, digital platforms). -- Time and financial planning. \ No newline at end of file +- Time and financial planning. + +From Planning (short guide) + +Whether or not the tricot method is suitable for your project should be based on knowledge of local farming. Tricot is a methodology for introducing agronomic innovation. It is most useful in situations where farmers are experiencing agronomic challenges or where they are dissatisfied with product quality of their harvests. Tricot should only be used when it is believed that agronomic innovation can be part of the solution. + +A thorough problem analysis must first be done. By discussing with experienced field agents and members of your target group about their needs and aspirations, you should ask: Is there a pressing problem that can be solved through agronomic innovation? If yes, Which technology should be considered (for example: crop varieties, irrigation technologies, fertilizer dosage, tillage systems)? + +Which technologies will be tested? + +Researchers should be considering and proposing technologies that have the potential to solve local problems and can be easily adopted by farmers. The more you know about the agronomic problems experienced by the target group, the more precisely you can select the technology options. As a start, a total number of 8-12 technology options is recommended. A good way to select these from an even larger pool is by conducting focus group discussions with a core group of local farmers from diverse locations. + +In which area will the project be conducted? + +For practical reasons, it is best to work in a defined region. If the project is spread across an entire country it can be hard to stay in touch with the local field agents and to assemble farmers for the initial training. + +How many farmers will participate? +It advisable to involve as many farmers as possible. The larger the number of trials evaluated, the more useful the information about the technology options becomes. Bear in mind that involving more farmers will also take more work to assist farmers in completing the process. Avoid including more farmers than the local field agents can assist. Each field agent may be responsible for up to 25 farmers. When starting a project and gaining experience with the methodology, it is advisable to include around 100 to 200 farmers, which is enough to obtain good results in most situations. In +future iterations, the tricot experiment can be scaled up to involve more farmers. + +Who should participate? + +It is important to think about the selection of farmers, who should be representative of the broader group of potential users of the technological options. Think about age and gender aspects, but also about different uses that can be given to the technology in different contexts. For example, technology needs can be very different between a household that produces for its own consumption and another that produces for the market. Also, different users may perform different tasks in relation to the technology and may therefore have different knowledge about it. For example, in +the case of crop varieties, it can be relevant to include processors and consumers. Decisions on the groups to include in the trial will influence you planning the recruitment (see Step 3). + +Which criteria will be evaluated? + +Maybe one technology option provides higher yields, but another one is less labor-intensive. Both criteria can be important, and there may be many more aspects that matter. You will need to define the criteria to be evaluated by the farmer-researchers. These can be defined by consultation with experienced field agents and local future users of the new technologies, both women and men of all ages. Many criteria can be evaluated, but it is recommended to pick no more than ten criteria. With more criteria, farmers may be discouraged by the complexity of observation. The key question must be: What really matters to the farmers? Most importantly, farmers should be asked to give their opinion about the overall performance of their technology options. Also, they should be asked why they prefer the best option. This is an open question and it is therefore possibe that farmers mention criteria that had not been considered beforehand. + +How will data be collected? + +Tricot uses the Open Data Kit (ODK) Collect app as the main way to collect data. The ODK Collect app is available free of charge on Google Play Store and can be installed on any Android smartphone or tablet. It allows implementers to register participating farmers, and field agents can collect farmers’ observation data for each of the criteria. When field agents gather the data collected by farmers in the field, the data will be stored on the device until an internet connection is available. All data is then sent to the ClimMob server for storage and analysis. During different steps of +the project, ODK forms will be automatically generated by the ClimMob software or will be available on the ClimMob website for download. Other data collection methods can be made available (interactive voice response, Whatsapp). Contact the ClimMob team (climmob.net) for more information. + +What do you need to know about the participating farmers? + +Tricot research can be used to evaluate how farmers’ adoption preferences for different technology options differ by region, gender, wealth status, or other farmer-specific variables. Understanding these differences can help to generalize by category the results from the experiment and to tailor technology recommendations for further households. Project implementers should define variables they consider important, so these can be collected from the farmer-researchers. Project implementers can formulate their own questions or they can use questions from the ‘Rural Household ulti-Indicator Survey’ (RHoMIS) to gather key household information. RHoMIS is free for download on the ClimMob platform. + +Should participation be rewarded? + +This question requires careful thought. Providing a reward to motivate farmers could increase participation. But some types of rewards can undermine enthusiasm, curiosity, and the desire to learn, which are often the most important reasons for participation. In several tricot projects, farmers received extra seed of the variety they preferred. This kind of reward is closely tied to the goal of the project and motivates farmers not only to contribute, but also to pay attention to the process, andto be sure to pick a good technology option for their farm. + +Which visual materials are needed? + +At www.climmob.net you will find examples and illustrations to help you generate your own visual materials. In order to explain the process to your tricot farmers and to facilitate the data collection, the following materials can support you: + +• Informative leaflet or poster, as an aid to explain the tricot process to the farmers. +• Observation card, for the farmers to collect their observations on the field. It is designed to enable participation with a minimal level of literacy. + +insert observation sheet image from page 2 + +SCALING: COST REDUCTION AND NEW BUSINESS MODELS (RTB) - not sure if this goes on this page? + +An important benefit of tricot is the possible reduction of trial costs, which could drive its adoption across different organizations, apart from the improved insights that it produces. Precise cost comparisons are difficult as there is no gold standard to compare with. Both ‘conventional’ participatory variety selection and tricot can be implemented in different ways. If both are implemented in a very intensive way (organizing farmer groups, meetings, working with RTB seed materials), the cost reduction is estimated to be roughly 40%. At the other extreme, a tenfold cost reduction is possible in the US, where farmers receive seeds by mail and are connected by smartphones. One reason for cost reduction is that some costs are externalized to farmers who volunteer to execute mini-trials using their own labor, land and inputs. This raises the question whether farmers’ motivation to participate is sufficiently enhanced by tricot to justify this investment. Previous studies showed that farmers’ +motivation to participate in tricot is mainly related to access to seeds and information (Beza et al., 2017). Cost and motivation analyses are underway and should be available for Rwanda and Ghana in 2021. + +Further cost reductions are possible if farmer networks are maintained over time, if they are serviced through channels that are also used for other means (credit provision, for example), and if they can reach economies of scale by testing varieties and other options for multiple crops. Tricot would make it possible for breeders and agronomists to ‘outsource’ trials to farmer-facing organizations. Alternative business models have already been introduced in the US context by organizations such as the Farmer Business Network, FIRST (Farmers’ Independent Research of Seed Technologies), and SeedLinked. The latter uses the tricot approach for its trials. The Alliance of Bioversity International and CIAT is exploring alternative business models following this trend focusing on the global South. Research is ongoing within RTB to determine the best scaling strategies for Rwanda and Ghana. \ No newline at end of file diff --git a/docs/03-experimental-design/socioeconomic-sampling.md b/docs/03-experimental-design/socioeconomic-sampling.md index 61525c39..175b6d06 100644 --- a/docs/03-experimental-design/socioeconomic-sampling.md +++ b/docs/03-experimental-design/socioeconomic-sampling.md @@ -17,9 +17,13 @@ From a demand led perspective breeders are interested in good information on the A bad example of how participants got chosen is a farce case where someone wanted to work with mango farmers and ended up with only male participants of which none had a mango tree field. When later confronting these participants with this, the village chief and his friends all claimed to have a mango tree in their backyard. It is obvious that a tricot with such participants will not yield the best information. A task group approach and gender + Sex-disaggregated data collection protocols on variety preferences are problematic as they put upfront sex and gender differences as an explanatory factor. From a political and ethical perspective this approach is also guilty of up-front discrimination based on sex difference. This type of data also renders invisible how gender roles are shaped by the intersections of locally defined identities, such as occupational tasks, immigrant or ‘local’, ethnic/language group, age group and economic status. So instead of initially segregating by sex groups we propose an identification of task groups. Simply: Who does what within each of the locally defined social categories. + This task group approach can capture the intersectionality of local identities by focusing on who does what along the value chain and allows for a closer integration with practices related to participatory trials, post-harvest processing evaluation, food processing and marketing as these connect to specific tasks and thus specific crop characteristics/traits. Identifying who does what in each of the social groups in the community can accurately and concretely tap into the knowledge, skills and working conditions of these groups without referring to gender as a subject, but with the work that different often highly gendered task groups do, as subject. + Taking the task and the work as central entry points avoids the highly political aspects that come into view when explicitly addressing sex and gender. By continuously referring to the work and working conditions related to a specific crop or set of crops, one can much more neutrally gain knowledge about how gendered certain task are, and learn about the constraints and opportunities and preferences of the different task groups. And this is something breeders are interested in if they want to see an increase of adoption of their varieties. In addition, by valuing and showing interest in the work done by different groups one triggers senses of proud and belonging and it facilitates more politically neutral and realistic conversations and cooperation based on the, organization and technicalities of the work and the working conditions and how this work can be done better in the current context. Knorr-Cetina (1999) would call this; building epistemic cultures based on fascination and content of the work. This method communicates focused skilled practice instead of emotion and values as abstract themes. + Therefore, task group based PVS trials and participatory processing evaluation exercises as well as participatory breeding events are perfect social science tools to achieve such a more performative and tacit cooperation based on technological practices and their organization. Such PVS exercises can be done with merchants as a group, the growers as a group as well as processors. Consulting people who are involved in more than one of these activities are an opportunity to catch more task groups at once. And it is this opportunity that we will have to use because choosing someone as a TRICOT participant, assumes that such a person is farming the crop. In many cases however the person does a lot of more tasks related to the crop. People can simultaneously belong to different task groups and the extent to which they belong to one and not another and the extent to which certain tasks are done by women and men are informative about gender roles and norms and current possibilities and developments within farming, processing and selling. @@ -55,3 +59,55 @@ Table 1. Example of local social groups based on interaction with a village comm | vii. Fulani | 150 | Fulfulde | Cattle rearing | | | 2 | 2 | | viii. Yoruba | 18145 | Yoruba | Farming (food and cash crops), trading. Includes cassava | More of processing, some farming (weeding), firm marketing | Farming, marketing of fresh roots | 1 | 1 | + +MAKING TRICOT INCLUSIVE: GENDER AND SOCIAL HETEROGENEITY (RTB) + +External validity of tricot trials has an important social science aspect. As has been indicated above, tricot trials imply sampling a representative range of use contexts, which are characterized not only by environmental variation, but also by gender and social heterogeneity, which will have an effect on variety preferences through various proximate causal factors. Firstly, crop management tends to reflect cultural and socio-economic conditions and identities (Adekambi et al., 2020). For example, the ability to purchase fertilizers or spend sufficient labor on weeding will influence how the trial plots are managed and will influence perceptions of variety performance. Another example is that farmers and processors might favor a particular variety because of its suitability for preparing a food product that is locally important or consumed by a particular social segment of the population. For example, farmers’ orientation towards market production and household consumption can influence how they perceive traits related to marketability, cooking or taste (Adekambi et al., 2020). Thirdly, the degree to which farmers that participate in tricot trials have adequate knowledge of a different aspect of variety performance will depend on their involvement in different agronomic, processing and culinary activities (Teeken et al., 2020). + +Gender is important in all three of these aspects (Weltzien et al., 2019). There may be differences in socio-economic status between men and women, as well as gender-based labor division for crop-related tasks. In the past, many trials have therefore addressed issues of gender by including sex of the participant as an important ovariate. However, so far no tricot data analyses have shown that there are statistically detectable differences between men and women. This contrasts with the finding that trait prioritization exercises often end up with different traits mentioned by men and women, reflecting their tasks and final use of the product (Weltzien et al., 2019). This contrast may have different explanations. + +First of all, tricot data and analysis did not include other social identities that can strongly intersect with gender or gender-related constraints on access to resources, knowledge and opportunities. Statistical interactions between these other social variables and gender could be revealed in aggregate datasets. This will only be possible when such data becomes available (see below). Gendered norms and roles do often not follow generalized stereotypes and can change over time, for example when outmigration of men leads to a feminization of agriculture (Abidin, 2004). Certain tasks are executed by both men and women. Gender and social heterogeneity in study areas may lead to aggregate tricot results in which general variation overwhelms any differences between men and women. + +On the other hand, existing studies prior to tricot may have some limitations as well. Few studies ask participants to rank the importance of traits directly (Weltzien et al., 2019). Most studies rely on free-listing exercises, in which participants mention all the traits that occur to them. Free-listing has methodological limitations if it is used as a comparative approach. If free-listing is done in focus group discussions, they may be influenced by leadership effects (which make more senior members more influential in the results) (Richards, 2005). Also, free-listing exercises measure perceptual saliency and importance in local discourse, which may not always translate to relative importance in a realistic decision-making context in which tacit knowledge comes into play. Relative +weights are often difficult to elicit through deliberation. Another possible factor is the loss of information in translation during data interpretation (for example, overzealous lumping of local concepts into more general categories) and translation from local languages. + +Specific elicitation exercises to put weights on traits and segment user groups have become more prevalent recently as a result of methodological simplification, providing viable alternatives to the usual approaches from economics (conjoint analysis) which were somewhat burdensome (Byrne et al., 2012; Steinke and van Etten, 2017). This could provide important opportunities to avoid the limitations of free-listing. These new approaches use pairwise comparisons and are therefore methodologically very similar to the ranking approach used in the tricot approach. Our comments on the specific limitations of free-listing should not be interpreted as a diatribe against free-listing per se or qualitative methods in general, just as a caveat against the possible overinterpretation of qualitative results in uncontrolled and unrepeated comparisons. We advocate for judicious combinations of different qualitative and quantitative methods. + +Sex disaggregation used in isolation will tend to overlook other issues that may correlate but also intersect with gender, such as income, occupation, marital status, ethnicity, age, or social status. Sex disaggregation alone as a basis for gender analysis will therefore not capture the high heterogeneity within the two resulting segments and give limited insights in causal relationships. This means we need to move to more subtle approaches that address intersectionality. This will require innovating on methods of analysis to analyze social differences and how they come to bear on trait and varietal choices. Innovation in two directions is ongoing. + +The first innovation direction involves the use of RHoMIS (Hammond et al., 2017; van Wijk et al., 2020). This is a standardized household survey method that includes questions about the gendered execution and control of activities and control over the income derived from them. Also, the survey covers questions about household composition, farming system, nutrition, poverty and other indicators. For tricot, a selection of questions and indicators has been made to reduce the length of the questionnaire to the bare minimum to reduce respondent fatigue. The resulting data will be used to analyze the farmer-generated tricot data to determine how gender and socio-economic factors affect trial management, variety performance, and farmer variety preferences. A publication of this “layering” of RHoMIS onto tricot trial data for cassava is forthcoming. The promise of RHoMIS is that it could combine with tricot to a standardized approach that will enable comparisons across studies in variety evaluation. This does not preclude that the precise RHoMIS format as applied in combination with tricot may still need further methodological evolution. + +The second innovation direction is to get a better grip on participant recruitment. Again, often fairly simplistic methods are used to address social/gender inclusion, generally quota recruitment to arrive at balanced numbers of men and women as participants. This was done in tricot trials in India, for example (van Etten et al., 2019). In a way, this puts a small set of variables upfront as explanatory factors, ignoring the importance of intersectionality or the possibility that non-identified variables may be more important to differentiate locally important social segments. For example, differences between people who are long-term residents and recent immigrants in the village may be more important than overall gender differences and can constitute important gender differences, for example, where women immigrants are in a very different position than autochthonous women (Forsythe et al., 2016). This would be impossible to capture through sex-based quota sampling, which may miss out migrants entirely. Also, during recruitment, there may be a bias towards more outspoken, talkative individuals who may not always have the best observation and judgement skills for variety evaluation. Random recruitment from the membership base of collaborating organizations has been used. This can suffice if the resulting participants represent the target population and a widely grown crop is targeted, but often local social segments remain invisible and can therefore be under or over-represented Also, in the analysis a reweighting can be done if recruitment is not representative, however excluding participants reduces statistical power and increases the relative costs of studies. However, for RTB crops generally the volume of planting materials is an important limitation. Also, not all farmers may grow relevant quantities of the target crop. Both these cases call for a better-informed sampling strategy. + +IITA has implemented a purposive sampling strategy with a gender dimension for cassava trials. This sampling strategy starts with qualitative work in communities to define locally relevant social groups. Participants are then selected making sure that each local social group, and gender within them, in which cassava growing and processing expertise is present is proportionally represented. To achieve this potential participants (cassava farmer/processors) in each group are randomly interviewed and evaluated on thorough experience in cassava farming and processing (using enumerators equally having experience in this domain to assure a good check) to also capture feedback from processors that are important additional stakeholders in addition to farmers and are often also marketers and very much informed by market demand and related traits (see determination of stakeholder/value chain actors section below). This approach then makes it possible to perform a better-informed gender analysis by comparing men and women’s preferences with regard to the same expertise and across different relevant social identities. This is even better facilitated as all participants are interviewed using +a RHoMIS questionnaire assuring the availability of standard demographic information next to the locally determined social grouping based on the qualitative research in the communities. This approach therefore focuses on the participation of task groups/segments (Maat, 2018; Richards, 2000; McFeat, 1974). These are segments/groups that are organized around a task (for example, processing cassava into gari) and are internally relatively homogeneous in their work culture (but groups doing the same task may have other differences between them). Task groups develop a focused skilled practice which tends to generate shared language and thinking. Tapping into the expertise of these task groups is therefore an appropriate way to organize participation in order to assure that each participant is skilled and experienced which is an important condition if we want to know about crop related user preferences. It mobilizes participants around a skill set and professional identity in which they tend to take pride. A focus on task groups may also help to avoid micropolitical considerations, make the process transparent, and be more inclusive to less outspoken professionals. Task groups can be identified by tracing who does what task in the crop value chain trajectory from seed to stomach. This is done by considering local identities, including gender, but also other potential factors (e.g., age). If gender is the overriding factor in the constitution of task groups, it would accentuate the need for a nuanced gender analysis that takes into account intersectional identities beyond only sex-disaggregation of data. By using ethnographic observation methods (interviews, transect walks, market visits, etc.) the information to identify these groups can be gathered. IITA has prepared a draft guide to implement this approach (Teeken et al., in preparation). + + +# ETHICS, PRIVACY AND RIGHTS ON TRADITIONAL KNOWLEDGE (RTB) + +Tricot involves human subjects and must therefore observe certain research ethics standards. In general terms, the application of tricot must minimize the possible risks, discomfort, nuisances and costs for participants while maximizing the benefits that they and other farmers may obtain (directly or indirectly) from the trial data obtained through tricot. +Tricot is also subject to privacy issues, and data management needs to conform to General Data Protection Regulation as the Alliance of Bioversity International and CIAT is headquartered in the European Union (Italy). + +In general, this will mean the following for tricot trials: +● Research ethics clearing is obtained from the relevant Institutional Review Board (IRB). +● Research ethics clearance may be also necessary from a national organization. For this purpose, Tricot users must take national laws and guidelines into account. +● Prior informed consent is obtained from all participants, which would allow for data publication after anonymization. +● Participants are given the right to withdraw from the study while it is executed. +● Participants are given the right to withdraw their data from the study while it is in the course of being executed. +● Participants can indicate if they want to be recognized with their name in the publications based on the data. This does not compromise privacy (names cannot be linked to personal identifiable information such as addresses, telephone numbers or coordinates). + +In practice, this means the following for the further development of the tricot approach and the ClimMob platform: +● ClimMob should provide features to make it easy for trial designers to follow the principles and procedures indicated above: +○ Automatically generated document to request IRB clearance; +○ Standardized, short prior informed consent forms and practical ways to implement paper-based signature + photograph of the document, electronic signature, or spoken approval (audio); +○ Names of participants that want to be named in the research publication exported by the platform. +○ Anonymization of data before exporting. This can be automatized through automatic detection of potential personal identifiable information (see https://dataverse.scio.systems:9443/). +● Throughout the design of an experiment, ClimMob should provide cues to prompt users to consider research ethics, privacy and traditional knowledge rights in the design of tricot trials; +● ClimMob needs to be GDPR-compliant to users (cookie policy, explicit notice about usage of data). The version available at the moment of writing already has this implemented. + +A more complex topic that deserves separate discussion is that tricot may be affected by national laws on the access to genetic resources and associated traditional knowledge and the sharing of benefits arising from their use (ABS, for short). There are two aspects in which tricot is affected by ABS, via the use of traditional varieties and via the use of traditional knowledge held by participants. We consider both aspects. + +Firstly, tricot may need to observe ABS rules when using traditional varieties. Tricot is usually applied to test the performance of new, improved varieties. However, in some cases, genetic materials of traditional varieties are to make comparisons. Although the utilization of the check varieties does not fall within the activities that are usually subject to ABS requirements in most countries, whether or not ABS obligations apply will depend on the definition of utilization adopted by the country of provenance of the variety (i.e., the country where the research is implemented). Therefore, tricot users will need to analyze the applicable access rules in the country where they are operating, obtain the access permits and negotiate mutually agreed terms when necessary. If the country where the traditional varieties come from is a party to the International Treaty on Plant Genetic Resources for Food and Agriculture (Plant Treaty), the acquisition of the traditional varieties for their use in tricot may be subject to the terms and conditions of the Plant Treaty’s multilateral system of access and benefit-sharing. In this case, access to the samples would be facilitated by the Standard Material Transfer Agreement. Since the purpose is not to breed the traditional varieties or incorporate them in new, improved lines, the multilateral system’s mandatory monetary benefit-sharing conditions would not apply, and thus the tricot users would not have any benefit-sharing obligation. However, they would have the obligation to transfer the varieties they have obtained with the SMTA under the same terms and conditions as those of the multilateral system, whenever the recipients of such material are going to use it for conservation, research, training and breeding. + +Secondly, tricot may be exposed to ABS laws when using traditional knowledge. Farmers’ ability to perceive crop characteristics is often considered to be part of traditional knowledge related to genetic resources (Mancini et al., 2017). In tricot trials, farmers use their skills to produce new knowledge, which would usually not fall under national ABS laws, but whose use may be anyway subject to rules and protocols related to the interaction with indigenous and local communities, the access to their knowledge and their natural resources. Even if the country has not yet enacted ABS legislation in relation to genetic resources and/or traditional knowledge, or even if the existing laws and regulation do not apply to tricot trials in a particular context, it is wise to observe, the CBD and the Nagoya Protocol principles in the management of farmers’ varieties and knowledge in tricot trials, as ‘best practice’, as recommended by the Guidelines on the Nagoya Protocol for CGIAR Research Centers. This means, among other things, sharing non-monetary benefits back with the participants, in the form of informational results, best performing varieties and other types of technologies. + diff --git a/docs/03-experimental-design/standard-operating-procedures.md b/docs/03-experimental-design/standard-operating-procedures.md index 815d982d..636c6314 100644 --- a/docs/03-experimental-design/standard-operating-procedures.md +++ b/docs/03-experimental-design/standard-operating-procedures.md @@ -8,3 +8,17 @@ sidebar_position: 2 A crop protocol provides a standardised overview of the basic information that will be collected during a crop tricot trial. The protocol provides an overview of the data collection moments during the trials and the variables and traits collected at each data collection moment. Farmers evaluate varieties in their on-farm trials and provide comparative observations by ranking the varieties based on their performance throughout their growth and post-harvest qualities including taste and consumption preferences. +DATA STANDARDIZATION AND ONTOLOGIES (RTB) + +ClimMob will connect with relevant ontologies for agriculture, thus securing its compliance with the Breeding API (BrAPI), for enabling the interoperability of the tricot data with breeding data. This way, ClimMob will extract defined traits and variables for the creation of project-specific questionnaires and storage in the database. + +The Crop Ontology used by breeding databases provides descriptions, URIs (unique identifiers) and relationships of agronomic, morphological, physiological, quality, and stress traits. It follows a conceptual model that defines a phenotypic variable as a combination of a trait, a method and a scale (Shrestha et al, 2012, Arnaud et al, 2020) (Shrestha et al., 2012; Arnaud et al., 2020). Therefore, the tricot ranking method needs to be included into the Crop Ontology for all traits that are relevant to the tricot trials. CO contains today 4,456 traits and 6,292 variables with methods and scales for 31 plant species. The Agronomy Ontology provides descriptions of field management practices (Devare et al, 2016). The conceptual model is centered on the plot or the entire field. It describes planned and unplanned time-bound processes occurring in the plot (e.g., fertilizer application), along with ‘participants’ to the event that can be a tool, a chemical component (e.g., manure spray, limestone). + +To support the connection of breeding product profiles to multiple sources of trait information, the ontology work is being extended to traits described by the social groups or market segments (e.g., sensory traits linked to the food products qualities, food product processing techniques for local processors) and will be completed by an ontology of the social groups and their roles in the value chain. The newly created socio-economic ontology (SEONT; Arnaud et al, 2020) based on the mini version of RHoMIS will support the use of socio-economic data for ClimMob projects. + +Ontologies can also support the management of multilingual trait lists by mapping the concepts across languages. The agricultural thesaurus called AGROVOC, maintained by FAO, contains 38,000 concepts in around 40 languages and will be an important resource for concept translation. + +A closely related effort is to create consensus on the variety traits and the socio-economic variables that should be included in tricot trials. A high degree of consensus about the traits would benefit the combined analysis of different datasets (see section Data analysis), ensure that important traits or variables are not omitted, and reduce the time spent on debating the different options for the design of each project while permitting the flexibility to add traits and variables that are thought to be of importance to a particular trial. + +Trait lists have been developed for a number of crops but not yet published. Table 1 gives an example for cassava varietal traits to be elicited at harvest. These trait lists and questions have been generated through iterated discussion between domain experts. These traits will be available in ClimMob for each crop. Also, drawings were made to illustrate each trait, which are used to develop printed materials for farmers. Figure 3 shows an example. + \ No newline at end of file diff --git a/docs/03-experimental-design/tpp.md b/docs/03-experimental-design/tpp.md index 6f077aba..0ed46b95 100644 --- a/docs/03-experimental-design/tpp.md +++ b/docs/03-experimental-design/tpp.md @@ -6,3 +6,11 @@ sidebar_position: 1 > Ganga Rao Nadigatla, Harish Gandhi +GENETIC GAIN: ON-FARM YIELD ESTIMATION (RTB) + +One of the goals of on-farm testing is to get insights into genetic gain achieved by breeding programmes. Some aspects of genetic gain are related to traits that are highly heritable so that on-farm performance is not different from on-station performance. For example, the color of the product may not be affected by genotype by environment interactions. An aspect of genetic gain that is important as a goal shared by most breeding programmes is the yield. As tricot is based mainly on rankings, generally yield estimations have been provided in that form. This provides an insight into the yield-based reliability, the probability that a new variety will outperform the current market leader, an important indicator for breeders and product managers to make decisions (Eskridge and Mumm, 1992). The CGIAR Excellence in Breeding strategy focuses on product profiles that emphasize cumulative gains towards product replacement, taking over market share from existing varieties (Cobb et al., 2019). Tricot is well suited to address the challenge of providing early indicators of the probability that product replacement happens. + +In many cases, however, breeders need to have absolute estimates of yield levels, for example because this is a requirement for a variety release procedure. In one case, a subset of the fields has been visited to obtain yield estimates (NextGen Cassava), in other cases, all fields were visited for yield measurements (de Sousa et al., 2020). This ‘undermines’ the tricot approach to some degree in the sense that the field visits become an important cost driver. This leads to the question whether farmers themselves can provide reliable yield estimates. + +Ochieng, Ojime, and Otieno (2019) have addressed this question by comparing yield estimates by researchers (taking into account grain moisture) and by farmers (volumetric, using 250 ml tins). They set up an experiment with common bean (P. vulgaris) in Kenya. They obtained a high correlation between the two types of measurements when all seasons and locations were aggregated (r = 0.98). When differences were smaller than 0.5 t/ha, the match between values provided by farmers and researchers decreased. We aim to replicate these studies in other contexts with other crops in order to get a better grip on the accuracy of farmers’ measurements and to use these accuracy estimates in statistical analyses. On the other hand, these studies will provide insights in how to maximize farmers’ accuracy. +It would be ideal to be able to combine yield ranking data and yield measurement data when the measurement data is only available for a part of the trial. It is possible to feed absolute measurements and ordinal (ranking) data into the same statistical model, directly (Böckenholt 2004) or through a Bayesian approach. This has not been implemented in software yet; this is a pending task. \ No newline at end of file diff --git a/docs/03-experimental-design/trial dimensions.Rmd b/docs/03-experimental-design/trial dimensions.Rmd new file mode 100644 index 00000000..7cdd1aef --- /dev/null +++ b/docs/03-experimental-design/trial dimensions.Rmd @@ -0,0 +1,29 @@ +--- +sidebar_position: 1 +--- + +# Trial dimensions (RTB) + +A recurrent issue is deciding about trial dimensions: plot size and the number of plots (replications). In the literature about crop trials, there are several methods to guide decision making. At the same time, this is not just a question of statistics, but also biological considerations are important. For tricot trials, there are a number of additional considerations related to farmers’ capacities. +3.1 Plot size + +Having larger plots and more replications reduces the variance within entries and increases the accuracy of the value estimates for each entry, while it increases the costs. The tricot approach is driven by external validity, the ability to replicate the findings in target environments. For external validity of on-farm trials, it is important to represent the diverse growing conditions in the target environment in the trial as well as the gender and social heterogeneity of end users. This can be done best by having a large number of farms, and to generate data about the conditions of the use environment and the users that can be entered into the analysis as covariates. Small plots can be easily accommodated on both small and large farms, avoiding a bias towards the latter. Also, small plots help to reduce the quantities of planting materials that are needed for the trial, which is often a limiting factor. + +In training courses on tricot, we have repeatedly noticed that agricultural scientists have strong views about trial plot size. Course participants argued that results from small plots are not reproducible on large plots, which tend to give lower yields in all cases, and concluded that large plots are needed for on-farm work. This perception is perhaps at least partly due to the regression fallacy, the mistaken expectation that selecting entries with a high mean performance in a trial will reproduce the same mean performance in a subsequent season. In reality a ‘regression to the mean’ effect is what is to be expected, as not all the entries with a high yield in the first trial are truly superior; an important part of the differences in yield between entries are due to non-genetic factors (error) (Galton, 1886). So systematically lower yields on larger plots is not a valid concern. + +A valid concern is that plot size biases performance results via neighbor effects, and edge or border effects. When the plots are so small that the borders become more important, the results can be biased by the competition between varieties or the resource advantage (or sometimes disadvantage) of border rows. Much depends on the differences in competitive ability between the varieties and their ability to take advantage of the extra resources and light on the border. Rebetzke et al. (2014) provide a review on plot size, focusing mainly on wheat in Australia. Differences in height between cultivars caused neighbor effects. Plots with four rows of wheat showed a 10% bias in one example, reversing the ranking of varieties. They discuss how under drought more competitive varieties outyielded more drought-tolerant varieties. They recommend plots with at least 6 rows for accurate yield assessment in the case of wheat. Omitting border rows can further help to reduce biases +arising from border effects. Border effects due to plant height are less accentuated at lower latitudes as shadowing is less important. Also, competition is less important in areas where resources for water or nutrients are not the main yield limitation, but heat or cold stress, or pests and diseases. Omitting border rows is often a standard practice in on-farm trials. + +Biological understanding of differences between varieties is needed to make judicious decisions about plot size. This may generally be based on the experience of breeders and the literature. For example, for potato competition effects between plots consisting of single ridges seems unimportant for yield, as stolons rarely extend beyond ridges (Connolly et al., 1993). For cassava, interplot competition effects have been found to extend beyond the first row (Elias et al., 2018). For sweetpotato, interplot interaction is thought to be substantial due to above-ground competition (Grüneberg et al., 2019). For sweetpotato trials, 30 m2 plots with 100 vine cuttings have been recommended for on-farm trials (Grüneberg et al., 2019). For a tricot trial in Ghana, however, much smaller 6 m2 plots with 20 vine cuttings were used, which is two thirds of the recommended plot size of preliminary (on-station) trials. + +Statistically, the neighboring effect can be partially dealt with by considering the ranking order effect (the middle position in each incomplete block has two neighbors, whereas the first and third position have one neighbor only). The order effect is not yet available in the PlackettLuce R package yet (Turner et al., 2020). This enhancement is planned for 2021. + +Plot size is closely related to the number of seeds that is provided to farmers. In grain crops, breeders often provide seeds based on the average weight needed for a unit of land. However, this can be problematic when there are seed size differences between varieties. As was pointed out to us by bean breeder Juan Carlos Hernandez (INTA Costa Rica, personal communication), this means that a small-seeded variety would be represented by more seeds. Consequently, the farmer may decide to increase the plant density of a small-seeded variety or add more planting positions. This could bias yield estimates to favor small-seeded varieties. It is therefore recommendable not to provide the same weight of seed for each variety, but the same number of seeds. + +For clonally propagated crops, weight biases may be less of a concern as usually seed quantities are determined in terms of the number of units (cuttings, seedlings, etc.), rather than weight. However, clonal crops have another set of issues, especially related to the perishability of planting materials, which are often also bulky. In our experience, it is important to account for possible losses of planting materials during transport and distribution. Food products provide another set of constraints. For a tricot evaluation in which processing and culinary aspects are evaluated at the same time as agronomic aspects, the plot size would also need to be sufficient to produce the minimum quantity of product necessary for food processing. For example, cassava is elaborated into many food products using batch processing techniques that require fair amounts of product (e.g., 50 kg of product in Nigeria). The minimal quantity needed that can be processed by participants using local customs, expertise and processing equipment should be considered to determine the plot size (Teeken et al., 2020). + +3.2 Number of blocks + +Another decision that needs to be taken is the number of incomplete blocks, which is equal to the number of farmers in tricot for on-farm evaluation. The numbers that are needed depend on farmers’ accuracy in observing differences between varieties, as well as the expected size of the differences. Often, the numbers to do power calculations are lacking as no previous trials have been done. (Steinke et al., 2017) estimated the accuracy of farmers for a bean trial in Central America and provided some calculations to guide trial size decisions. The results suggest that for a trial with around 12 entries (varieties, lines, etc.) typically 100-200 farms would provide solid results to make recommendations. This is the same order of magnitude that was found in previous on-farm trial work with cereals (Atlin, personal communication, 2020). If the trial covers more agro-ecological environments (to which the set of varieties is expected to respond in different ways), the number of farmers +should be proportionally higher. Future studies should provide better guidance regarding optimal trial dimensions. + diff --git a/docs/04-climmob-suite/climmob.md b/docs/04-climmob-suite/climmob.md index c9206bd1..ce72536e 100644 --- a/docs/04-climmob-suite/climmob.md +++ b/docs/04-climmob-suite/climmob.md @@ -7,3 +7,58 @@ sidebar_position: 2 > MrBot Software Solutions Experimental citizen science offers new ways to organize on-farm testing of crop varieties and other agronomic options. Its implementation at scale requires software that streamlines the process of experimental design, data collection and analysis, so that different organizations can support trials. This article considers ClimMob software developed to facilitate implementing experimental citizen science in agriculture. We describe the software design process, including our initial design choices, the architecture and functionality of ClimMob, and the methodology used for incorporating user feedback. Initial design choices were guided by the need to shape a workflow that is feasible for farmers and relevant for farmers, breeders and other decision-makers. Workflow and software concepts were developed concurrently. The resulting approach supported by ClimMob is triadic comparisons of technology options (tricot), which allows farmers to make simple comparisons between crop varieties or other agricultural technologies tested on farms. The software was built using Component-Based Software Engineering (CBSE), to allow for a flexible, modular design of software that is easy to maintain. Source is open-source and built on existing components that generally have a broad user community, to ensure their continuity in the future. Key components include Open Data Kit, ODK Tools, PyUtilib Component Architecture. The design of experiments and data analysis is done through R packages, which are all available on CRAN. Constant user feedback and short communication lines between the development teams and users was crucial in the development process. Development will continue to further improve user experience, expand data collection methods and media channels, ensure integration with other systems, and to further improve the support for data-driven decision-making. + +From Design (page 11) of short guide: + +Once you have chosen which technology options will be evaluated and you have identified which criteria are most important to the farmers, your project can start. + +Create a new project + +As explained earlier, tricot uses ClimMob (climmob.net), a free online software specifi-ally created for tricot projects. ClimMob is the fundamental tool for any tricot project, and is used for the following activities: + +• Designing the experiment +• Generating a randomized list of combinations of three technology options for the individual trial packages +• Project management and data overview +• Input of farmers’ observation data +• Data analysis and automatic generation of the post-trial information sheets for farmers + +The basic steps for setting up an account and developing and adjusting your project are listed below. More detailed information on how to use and make the most of the ClimMob software can be found on the ClimMob website. + +A. Setting up an account +To create a new user account, access the ClimMob software from the main menu on the ClimMob homepage and click on ‘Log in or register an account’. If you already have an account, you can enter your username and password, and click ‘Log in’. If you have not registered, start the process by clicking on ‘Register an account’. + +B. Creating a project +When you first log in after having registered, you will see a ‘Create a new project’ button. After clicking here, you will be asked to fill out +general information, size and location of your tricot experiment. In cases where you have already created a project, you can navigate to ‘Projects’ (upper right corner of your screen) to get an overview of your existing projects, navigate between them and create new projects. + +After selecting an existing project or designing a new one, ClimMob will take you to the Main menu, which is the central hub to design your tricot experiment. You need to specify the information on each of the field agents who will work on this project, the technology options that you want to compare, and the registration questions the farmers will be asked when they register to participate. ClimMob will only move on to the next step after you have provided this information. + +C. Define the evaluation criteria +Depending on the technology options included in your tricot experiment and the needs of your target group, you will define which observations the farmers should make. Each observation corresponds to a question on their observation cards. For example, common criteria for varieties are ‘yield’ or ‘plant height’. The corresponding questions would be ‘Which variety produced highest yield / lowest yield?’ and ‘With which variety did plants grow tallest / least tall?’ See also Step 6 about how farmers will observe crop performance on their trial plots. + +D. Define the time point for evaluation +The intervals at which farmers are expected to make an observation during the trials will vary depending on the technologies being tested. For each evaluation criterion you will need to decide at which point in the tricot project farmers make their observations. For example, if you intend to test different crop varieties, you might want to ask farmers to make observations at the start of the project (day of sowing), again after 30 days, and lastly at the end of the trial (day of harvest). + +E. Assign field agents +All field agents who will work on your project must be added individually to the ClimMob project design. Field agents are the people who will be working on site, communicating with the farmers and later collecting observation data. You need to assign a username and password to each field agent, which they will use when logging collected data into the ODK Collect app. + +F. Select technology options +Here you specify the technology options you will compare in your tricot experiment. We recommend a pool of 8 to 12 options. For example, if you want to test which bean variety is best adapted to the region, you would add the names of all the bean varieties to be tested. If you want to test which fertilizer type makes crops grow best, you would add the names of all the fertilizer types to be tested. + +G. Prepare farmer registration +Once farmers have registered to participate in the trials they will be asked a number of questions by their field agent. Here, you will define which questions should be asked. This information is important for administrative reasons (farmer name, telephone, village, and other relevant details) and for data analysis (e.g. registering the gender and age can help to understand if social indicators influence farmers’ preferences for a certain technology option). You can define the questions yourself here, or you can use the standardized Rural Household Multi-Indicator Survey (RHoMIS) +provided on the ClimMob website. This is a widely validated format used to characterize farming households. Once you define the list of questions, they will become available as an ODK form, which can be downloaded to your ODK Collect app. Field agents can input the information directly into the app on their smartphone or tablet when registering the participating farmers. + +H. Prepare data collection +Throughout their tricot trial, the farmers make comparative observations about their three technology options. Here, you will define which types of observations farmers should make. For example, a common criterion to observe is the total crop yield achieved with each technology option. You must decide which criteria are important for your experiment. Eventually, all these questions will be printed on the observation cards and handed out to farmers at the distribution stage. + +I. Prepare the packages +ClimMob will take you through the steps to execute the randomization. Once the randomization is set up, ClimMob will make a list of the packages and the content of each package (each has three technology options drawn from a larger set). This list is available as a downloadable spreadsheet (available in the Downloads section). Also, ClimMob generates a document with QR codes for each of the packages. The project implementer prints the +codes and pastes it on to each package. These QR codes are used to identify each package during distribution and avoid mistakes. Print these documents and use them to prepare the packages. This process should be done very carefully. Try to follow a procedure that avoids mistakes and allows for checks. + +At the end, each package has a unique number (1, 2, 3, etc.) and contains three different technology options (package 1 has 1A, 1B, and 1C). To get there, organize the work in the following steps: + +• Before starting, keep all small bags of one technology option together, each having their own place on a table or a corner of the room. +• As a next step, mark all the small bags with their respective code (1A, 1B, 1C, 2A, 2B,etc.). +• Only when all the small bags are coded, they are picked up and combined in packages of three. +• When a package is ready, it is handed to a different person who checks its contents before closing it. \ No newline at end of file diff --git a/docs/04-climmob-suite/data-submission.md b/docs/04-climmob-suite/data-submission.md index 6feb57ec..10561d5e 100644 --- a/docs/04-climmob-suite/data-submission.md +++ b/docs/04-climmob-suite/data-submission.md @@ -5,3 +5,20 @@ sidebar_position: 3 # Data Submission Data Submission + +From Compilation page 23 of short guide + +Step 6 has been completed when every participating the data ready, but it needs to be compiled to be analyzed. + +The local field agents will compile the farmer-generated observation data. To do so, they have different options, including using ODK-based forms generated by ClimMob. Data +compilation in the field can be done offline with ODK Collect. If the local field agents cannot upload the data directly from the field, they upload the data from ODK Collect as soon as +they have an internet connection. It is important to upload the data regularly to avoid any inadvertent data loss. To upload the new data to your database, choose ‘Send data’ with the +ODK Collect app on an Android device. If data is collected through physical visits, each field agent can usually cover up to 25 farmers. + +Some alternative options for data collection can make the process more efficient. Some of the different options include: +• Visit farmers, inspect observation cards and transcribe farmers’ observations directly to the ODK Collect App. +• Take photos of the observation cards to copy the data later directly into your database or input the data using ODK Collect App. Remember to write down the farmers’ name +and package ID with the number of each photo. +• Call the farmers on their own or their neighbor’s telephone and fill out the form in ODK based on the information transmitted by the +farmer during the call. +• New data collection formats (WhatsApp, interactive voice response) can be made available. Check with the ClimMob team. \ No newline at end of file diff --git a/docs/05-implementation/implementation.md b/docs/05-implementation/implementation.md index 4a23277f..f383bcb4 100644 --- a/docs/05-implementation/implementation.md +++ b/docs/05-implementation/implementation.md @@ -6,4 +6,111 @@ sidebar_position: 1 > Kauê de Sousa, Rachel Chase -Setting up experiments. For OFT, best practices in planting and maintaining the plots. For consumer testing, best practices in handling the samples. Preparing the packages for distribution. \ No newline at end of file +Setting up experiments. For OFT, best practices in planting and maintaining the plots. For consumer testing, best practices in handling the samples. Preparing the packages for distribution. + +From Recruitment (page 15 of short guide) + +Any farmer who wishes to participate can get involved in a tricot experiment. Recruiting as many motivated farmers as possible is key to the success of the project. The local field agents should help to identify and recruit farmers in their communities. + +Hanging posters in agricultural shops, village halls or corner shops may also help to attract attention. You do not need to know the farmers before they participate. + +However, farmers should be: +• volunteers who are ready to commit time and effort to participation; +• farmers who enjoy experimenting and trying out new methods; +• both women and men, preferably at an even ratio. + +A. Tricot is an iterative process +This means that farmers ideally participate more than once in different experiments and across different seasons. When a tricot project starts and farmers participate for the first time, upfront project investments are required. The local field agents must be trained. Also, setting up and implementing the training workshops for participating farmers takes time. During their first cycle, farmers may have many questions and need assistance from local field agents. As the farmers will learn many things during each iteration of the process, and as they get to know only three randomly chosen technology options per cycle, we would encourage farmers to participate repeatedly. This way, first-time farmers can ask their more +experienced farmer-colleagues when they have doubts, and the farmers get the chance to experiment with new sets of technology options with every cycle. + +B. Local groups can carry out a joint trial +Carrying out a group or joint trial makes the learning process easier and participation more fun. Any existing group, like farmers’ committees, credit cooperatives, or a religious group can receive a trial package and participate together. In this case, a ‘host’ farm is needed, where the technology options can be tested. The host farmer will be the contact person for the local field agent, while all activities – such as planting and making the trial observations – can be performed jointly by the group. In the following season, individual group members may want to plant a trial for themselves, building on the experiences they gained in the group trial. To enhance the participation of women farmers, it can be useful to establish ‘women’s research groups’, who would be in charge of a number of tricot plots. + +From Distribution (page 17 of short guide) + +The tricot process starts with a training and distribution workshop. Here, the farmers receive their trial packages and learn about the tricot methodology and data collection process. + +A. Organization and logistics +The training and distribution workshop should take place about four weeks before the start of the trial, so that farmers can adapt their own +farm planning. It is most effective to invite a maximum of 20 farmers at a time to the workshop. Women and men should be invited in equal numbers, if possible. + +Required workshop material and logistics: +• A meeting place for about 20 persons +• Snacks +• Trial packages. + +Every trial package should contain the following four elements: + +1. A QR code generated for each package that will be used as a unique ID to track all the information collected during the trial. It is important to tell farmers to keep this code throughout the duration of the trial. + +2. Three bags with equal quantities of the technology options (e.g. seeds, fertilizers) or instructions on how to apply the alternative technology options (e.g. tillage systems), according to the randomization that was generated by ClimMob. + +3. An observation card where testers will note their on-farm observations + +4. A brochure that explains the entire process to the farmer. + +The randomization is done in such a way that it ensures that, in all places, the technology options will be made available with the same frequency. In technical jargon, the randomization is ‘balanced’. This avoids, for example, that one of the technology options did not occur in one of the villages in the trial. ‘Balancing’ the trial means that all technology options are spread across all the villages. For this to happen, however, it is crucial that each of the villages receive packages with consecutive numbers (1, 2, 3, 4, 5, etc.) and not random numbers (3, 11, 9, 23, 1, etc.). For example, the first village receives packages 1 to 9, the next village receives packages 10 to 23, etc. If this principle is followed, each of these villages will receive a balanced set. If it is not followed, there is a risk that one or more technology options will be completely absent in some of +the villages, so that you will never know if it is suitable there or not. + +B. Teaching tricot +The project implementers, together with the local field agents, invite interested farmers to a central location. This can be a village meeting hall or an NGO office. They explain the tricot trial, its purpose, its benefits, and the responsibilities the farmers have. It is important to visualize what a tricot trial looks like, so farmers can see what is expected of them. If al nearby beforehand. Otherwise, the trials can be visualized with a video (video 1: available at climmob.net). A small pictorial guide for farmers on tricot should also be handed out at the training workshop. Aformat for a foldable, guide (the size of a credit card) is available from climmob.net. + +At the workshop, farmers are also trained on how to fill out the observation cards. Every farmer receives one observation card for the immediate exercise. It is important to fully explain the design of the card and go through filling out the card to allow the farmers to practice and gain familiarity with the process. Farmers will then be advised on how data will be collected, and whether a project implementer will be calling them or visiting them in person. + +C. Registration of farmers +The ODK Collect app is used to register participating farmers. When the farmers receive their personal trial packages, they are registered by Field Agents using the project-specific registration form.‘Prepare farmer registration’ and will be available when the ODK Collect app is connected to the project on the ClimMob digital platform. The form should be downloaded to all field agents’ devices. + +At a minimum, these basic data are required: +• Trial package QR code +• Name of the tester (participating farmer) + +The trial package code uses an QR code generated by ClimMob as a unique package ID throughout the trial. The QR Code is generated once the technologies are defined and the randomization is set up. The project implementer prints the codes (available in the Downloads section) and pastes it into each package. Note: farmers should keep their package (QR) code for the duration of the project. + +More in-depth information regarding household and farm characteristics can be collected during registration using the pre-developed RHoMIS survey (available on the ClimMob website). + +insert image on page 16 + +From Execution page 19 short guide: + +The farmers plant and manage the trials independently.Every farmer is responsible for his/her own plot. + +• Carrying out an on-farm trial is simple. No special skills are required. Any farmer can participate. +• Farmers are farming experts. The participating farmers deserve full respect as generators of new knowledge. + +Through the training and distribution workshop (Step 4), farmers were trained in tricot methodology, received their individual trial packages, saw a trial plot (on-site or through video), and received a brochure about tricot. Now they need to choose a part of their land on which to conduct their own trial. It is important to understand that the trials must represent regular farming practice for the results to be useful. + +Two principles should be kept in mind. +1. The trial should resemble production conditions that reflect reality, not optimal production conditions. +• To ensure this, the trial plot should be located right next to, or even within, the farmer’s regular production plot. Farmers should neither select the best nor the worst spot, but an average, representative location. +• Also, each trial should be managed by the participating farmer in exactly the same way as they normally manage their crop (unless the technology under analysis is about crop management). For example: If the farmers usually intercrop with another crop, they may also do intercropping with +the trial varieties. The regular plot and trial plots should be treated and maintained equally. Special attention to the trial plots, but also negligence, will distort the results. For example, if the farmers do not irrigate their production plot, they should not irrigate the trial plot either. + +2. The trial should enable a fair comparison between the three options on each plot. +• The three technology options are applied next to each other, in separate sub-plots of the same size, and in the exact same way. In the case of varieties, each variety is planted in the same defined number and length of rows. For example: Six rows of five meters’ length each, or four rows of +eight meters in length. +• In the case of fertilizers or other input trials, amounts or combinations are applied as specified by the implementers. +• Technology option A is used to the left, B in the middle, C to the right. The borders between the technology options may be marked with sticks or a rope. The three technology options should never be mixed with each other. + +Apart from the small plot size, there is really nothing new or special about planting the trials. The farmers should be confident in using their own farming skills and implement the new technologies in the same way as they would normally conduct their work. + +From Observation page 21 of the short guide + +As the crop grows, the farmers observe the technologies and record their observations on the observation card. For many farmers, the questions asked in tricot pose a new way of looking at things. + +Most farmers can tell which of the three technology options they generally like best. But it is not always easy to decide which one is the best +for a specific evaluation criterion. + +The farmers observe and evaluate the technology options in their trials and focus on only one criterion at a time. The observations they make always follow the same structure: the ‘best’ and the ‘worst’ among the three trial technologies need to be identified. Farmers mark their choices on the appropriate page of the observation card. On the card, the question is asked in as few words as possible to make +it easier for farmers. For example, instead of asking ‘Which of the three varieties has developed the best foliage?’ the observation card just asks: ‘Best foliage?’. + +A. Focus on one criterion at a time. +Sometimes it is hard to acknowledge that a technology option was not successful for one criterion, but still performed best for another. For example, imagine a maize variety that was heavily affected by drought and disease and hardly produced yield, but has an excellent growth habit, with many tillers. It could look poor overall but would still be ‘best’ for ‘growth habit’. For best results, it is crucial to really focus on only one criterion at a time and ignore all others. + +B. Choose the right dates for the evaluation. +Appropriate timing is important, and farmers should be told at which point in the process each criterion needs to be evaluated. For fertilizers or varieties, it is common to evaluate the trial in three stages: earlier-developing criteria +(for example, foliage development), later-developing criteria (for example, disease resistance) and post-harvest criteria (for example, yield +or market value). The project implementers should suggest the evaluation steps and dates to the farmers. + +C. Provide follow-up assistance. +Many farmers have a busy life and their tricot trial will be one activity among many others. Through telephone calls, the project implementers or the local field agents may help the farmers to keep track of their evaluations and remind them of upcoming observation steps. The telephone calls will also help to clarify open questions and to let farmers know that their contribution is important and valuable. Within their own capacities, the local field agents may also support farmers directly in the evaluation at the plot. These follow-up calls can also be used to support the data compilation if farmers mention that they have already collected their data. + diff --git a/docs/06-data-analysis/data-analysis.md b/docs/06-data-analysis/data-analysis.md index 88e6f1e2..e81b734f 100644 --- a/docs/06-data-analysis/data-analysis.md +++ b/docs/06-data-analysis/data-analysis.md @@ -6,4 +6,52 @@ sidebar_position: 1 > Kauê de Sousa, Joost van Heerwaarden -Analytical framework. Overview of statistical models (e.g., Plackett-Luce, Bradley-Terry). Tools for analyzing ranking data. Visualization. Integration with other datasets (agroclimatic, soil, and socio-economic data). Case studies." \ No newline at end of file +Analytical framework. Overview of statistical models (e.g., Plackett-Luce, Bradley-Terry). Tools for analyzing ranking data. Visualization. Integration with other datasets (agroclimatic, soil, and socio-economic data). Case studies." + +##Analysis page 24 in short guide + +When all the data is uploaded to your ClimMob database, analysis can start. + +The analysis will give you an automated report with useful results, such as: +• Description of the methodological approach applied +• A rating of how well each technology performed for each pre-defined criterion (see Step 2) +• Information on differing performances (if any) depending on explanatory variables (e.g. the highest yielding crop variety with or without irrigation; or the variety preferred by women, variety preferred by men) +• A rating of how all pre-defined characteristics were correlated with the overall performance. This is useful to assess which characteristic influenced the overall appreciation of the technologies tested. + +The analysis is conducted in six simple steps: + +1. Select the project you will analyze. + +2. Press the button ‘Select variables to analyze’ in the main menu of ClimMob. + +3. Select the explanatory variables you want to include. Explanatory variables can lead to a better understanding of different observations about the tested technologies, and to more useful results. For example, one crop variety may perform best under irrigation, but in drought conditions, a different variety may give best results. + +4. Select the documents you want to generate. +Two types of outputs are possible: +• Analysis report +This is for the implementing organization and the researchers. It is a report presenting all results: it tells you which technologies performed best for every tested criterion, and whether there are any differences due to explanatory variables, for example gender, age, irrigation. +• Infosheets +This is a document that contains a personal information sheet for each participating farmer. These infosheets contain: +• The names of their three specific tested technology options +• The farmer´s own answers +• The most recommended technology options for the farmer’s own farm. + +5. Press OK. + +Depending on the number of farmers, the analysis can take a long time. In some cases, it may take up to half an hour to generate all of the infosheets. You can obtain the infosheets and reports from the Downloads section. + +## Data Analysis (RTB) + +Tricot data consist of rankings, an unusual data type in the agricultural sciences, in spite of some experience with it in participatory research (Coe, 2002). As indicated in section 2, the tricot approach relies much on the Plackett-Luce model. The Plackett-Luce model also allows for the inclusion of covariates using recursive partitioning, which uses binary splits (Strobl, Wickelmaier and Zeileis, 2011). At the moment, the Plackett-Luce package is being expanded to include Plackett-Luce regression, which uses linear covariates (Yildiz et al., 2020). + +Van Etten et al., (2019) showed how the Plackett-Luce model can be used in combination with seasonal climate data and cross-validations to produce robust, locally-specific variety recommendations. Tricot data analysis has recently expanded into two directions. + +Firstly, Brown et al., (2020) have described how the Plackett-Luce model can be used to synthesize trial data from across different trials. Ongoing work is doing this with tricot trials (and other trials, after converting absolute values to ranks to deal with highly heterogeneous data). This is a promising new direction, as it will show in the future that working with a standardized approach like tricot and sharing data openly has strong benefits for science in general and generates recognition for individual scientists who decide to publish their data. Also, it may stimulate data publication from trials that are not worth a peer-reviewed journal article on their own but gain value after being combined with data from other trials. + +In a forthcoming paper, de Sousa et al. (2020) take the Plackett-Luce model in another direction, by adding genomic relatedness data to the model (as a covariance matrix). This increased the predictivity of the model in an important measure, showing that it may be feasible and relevant to use relatedness data to allow for more diverse sets of materials to be tested by farmers. This may require important changes in breeding approaches but it provides an interesting prospect. + +The new Plackett-Luce regression approach will also allow the use of variety traits as covariates (Yildiz et al., 2020). This opens interesting possibilities to analyze the relative influence of known trait values on on-farm performance. It will also possibly provide avenues to link tricot results with trait prioritization exercises, discussed in section 4 above. + +Data analysis has also been increasingly supported by implementing the existing code, which was generated to a large degree for the analyses presented in (van Etten et al., 2019), into R packages. The R packages that have been generated as a result from this research are listed in Table 2. + +insert figure - table of R packages page 20 RTB guide \ No newline at end of file diff --git a/docs/07-feedback-dissemination/feeback.md b/docs/07-feedback-dissemination/feeback.md index b7796c62..0ca1fd18 100644 --- a/docs/07-feedback-dissemination/feeback.md +++ b/docs/07-feedback-dissemination/feeback.md @@ -6,4 +6,73 @@ sidebar_position: 1 > Anna Müller, Charlotte Schumann, Juan Manuel Londoño -Feedback session to participants. Follow up and reporting. \ No newline at end of file +Feedback session to participants. Follow up and reporting. + +##Feedback + +From Feedback page 25 in the short guide + +You have run the analysis using the ClimMob online software. Now the farmer-researchers are eager to know the results of their trials. All farmers are invited to a final workshop to receive and discuss the results. + +Soon after all the data are collected and the analysis is completed, participating farmers are invited to a feedback workshop. Here, they receive information sheets about their trials and will have a chance to discuss the results. The farmers have had different experiences with their trials, so reciprocal sharing of these experiences with other farmers is an important part of the learning process. Plan at least half a day for each feedback workshop. + +The workshop consists of three parts: + +1. The project implementers or the local field agents present the overall results of the technology evaluation. Farmers learn which technology options performed best under which conditions. + +2. The farmers receive their personal infosheets about which technology they have preferred and are given time to discuss +the results with other farmers and implementers. It is recommended to form small groups for this activity (of about 5-7 persons), including a facilitator (a field agent or experienced farmer). Groups can present their conclusions in a plenary session. + +3. Farmers then receive a practical agronomic lesson as another incentive for participation. For example, field agents may use this opportunity to disseminate knowledge about seed storage or seed selection. + +These points should be considered by the field agents: +• Discussion among the farmers is important: everyone can learn from each other. +• It is crucial to make it clear that there is no single best technology option. In fact, optimal technology options can differ across farms and farmers. +• Field agents should also annotate feedback provided by the farmers on their experience with the trials and the project in general. + +Preparations for the final workshops: +• As with the training and distribution workshop, in most cases farmers should be limited to around 20-25 per event, in a central location accessible to all. +• Have the infosheets for all farmers ready for distribution during the workshop. + +##Evaluation + +From Evaluation page 27 of the short guide + +The first tricot cycle has finished. What can be improved? + +Countries, crops, farming systems, and people are diverse, so every tricot project is different. This booklet can only be a guide to assist you +in designing your own local experiment. Tricot is an iterative process and the last step in a project cycle is the evaluation of the project for +further improvement. + +Listening to the farmers’ experiences is most important. It is crucial that the farmers perceive tricot as both simple and beneficial. You should try to identify possible improvements in managing and executing the trials. At the feedback workshop, farmers can express their experiences, recommendations and complaints about the process. Moreover, the local field agents can provide project implementers with many valuable comments and recommendations, since they have constantly been in touch with the farmers and in some cases have followed the trials in person on site. + +After every project cycle, the project implementers, researchers, and local field agents should discuss how to improve the process. Including more farmers with every project cycle should be a constant objective in tricot, so that more households can benefit from the investigation. + +Also, with the results of every cycle, you may identify one or two technology options that were not well accepted by the farmers, or that did not work well in your region. For the next cycle, you can discard those technology options ranked lowest by farmers and replace them with new ones. This way, there is ‘refreshed’ input to the research system, and the farmers’ chances of discovering a suitable technology option for the conditions of their farm remain high. + +Indicators of success + +The success of your tricot project can be measured. You can evaluate five indicators, which will give you an idea about the individual trials’ +impact, and the project’s overall success. + +1. The rate of completed trials +Count the trials that were fully completed, as well as the trials where data was missing. You can evaluate whether the loss of information is due to natural causes (e.g. drought that made it impossible to evaluate certain criteria on farm) or to the farmer’s management of the trial (e.g. a mistake with the package code (QR code), lack of interest in finishing the observations). This way, important knowledge about the specific difficulties can be generated, which will help you find strategies to avoid them being repeated. + +2. Farmers’ gender ratio +Women tend to have less access to the profits of agricultural production and other resources generated by such work. Participation in a tricot experiment can open doors for the empowerment of women. It is recommended that every tricot project strives to achieve a balanced gender ratio among +farmers by specially encouraging the participation of women. + +3. The percentage of farmers who participate again, after the first cycle +Returning farmers are a clear indicator of the farmers’ motivation. If many of the farmers do not want to participate a second time, something about the tricot process design may need to be changed. + +4. Changes in the technology choice +On the observation card, the farmers write whether they will continue using any of the new technology options from their tricot trial. If they choose to use at least one of the three technology options, this shows the impact of the trials. If no or very few farmers want to continue using the newly introduced technology options, then the initial pool of technology options may need to be reconsidered. + +5. Dissemination of technology into the communities and information exchange +Because of their joint experience in the tricot trial, farmers may become more active in experimenting with technologies and exchanging information within their communities. This can be checked by estimating the scale of diffusion of technologies into communities a year after the tricot experiment, by talking to the farmers, as well as to other farmers in the communities. + +DOCUMENTING TRIALS AND PUBLISHING DATA (RTB) + +Open access publication of the data should be a goal of the trial. Tricot has already published a number of sizable datasets from on-farm trials (van Etten et al., 2018; Moyo et al., 2020; de Sousa et al., 2020). These datasets could become important for other research that repurposes these datasets (see section 11 below). Kool et al. (2020) have provided an incisive critique of on-farm testing in agronomy, especially the limited replicability of many trials as authors fail to report contextual factors (crop management) and sampling of locations and participating farmers. Similarly, a study on PVS in RTB crops reveals that on-farm trials are often documented in a very deficient way and that data are hardly published at all (Jose Valle et al., forthcoming). Data publication could become more attractive if it is easy to do and has rewards (citations of datasets repurposed by others). Publishing all data from trials could prevent the so-called file-drawer problem, which means that only certain datasets (for example, novel analyses, striking results) are published, which then lead to biased statistics in meta-analyses. + +The tricot approach should address this issue by facilitating and standardizing the way in which on-farm trials are documented and published. Standardization should be done using the insights of the studies cited above. Specifically, meta-data on the trials could be standardized and some elements on the trial context could become recommended elements that are easily available from within the software. For example, it is becoming more and more clear that plot use histories and fertilization in preceding seasons of plots are highly influential on yields (Njoroge et al., 2019; Zingore et al., 2007). For this, an existing metadata schema for phenotypic experiments could be adapted (Papoutsoglou et al., 2020). Also, the data publication process should be automatized, including the anonymization procedure (removing personal identifiable information such as names, addresses and telephone numbers as well as aggregating geographic data to a sufficient level to prevent identification). diff --git a/docs/glossary.Rmd b/docs/glossary.Rmd new file mode 100644 index 00000000..dbcf286a --- /dev/null +++ b/docs/glossary.Rmd @@ -0,0 +1,74 @@ +--- +sidebar_position: 1 +--- + +# Glossary + +From short guide + +ClimMob +Online software for the design and management of any tricot experiment (www.climmob.net). The database of all tricot projects is stored here. Project implementers also use ClimMob for the analysis of results and the generation of information outputs at the end of the project. + +Balancing a trial +‘Balancing’ the trial means that all technology options are spread across all the participating villages. Each village will receive packages with +consecutive numbers (1, 2, 3, 4, 5, etc.) and not random numbers (3, 11, 9, 23, 1, etc.). If this principle is followed, each of these villages will receive a balanced set and all of the technology options will be tested and evaluated. + +Evaluation criteria +The 5 to 10 criteria that will be evaluated within the tricot experiment. These criteria should be chosen in consultation with all stakeholders. +For example: Plant height, disease resistance, yield, and others. + +Explanatory variables +Information about meteorology and agronomic management of the trials, serves to improve the analysis. The explanatory variables refine the results and help to identify the most suitable variety for the local conditions of every farmer. Examples: Use of irrigation, use of fertilization, +season was rainier or drier than usual, etc. + +Field agents +Lead farmers of rural communities, field workers, or extension agents. They are trained and remunerated by the implementing organization to assist the farmers in the execution and evaluation of their trials. They collect the data from the farmers and pass them on to the project implementers. + +Implementing organization/ project implementers +The organization that is in charge of carrying out and monitoring the project. It can be an NGO, a government service, or a research program, among other options. Implementers have the major responsibilities in the project, for example: +• Training the field agents and farmers +• Distributing the trial packages +• Carrying out the data analysis once all data is collected and compiled +• Feeding back the information to the farmers via the field agents. + +Infosheet +Personalized information output for every farmer. It is generated automatically using ClimMob and includes: +• Names of the three technology options that the farmer received and tested +• Names of the most recommended option for their farm +• Information about where to obtain more material of the preferred technology option (if applicable). + +Observation card +A pictorial form printed on thick paper, on which farmers mark their observations of the technology options being tested on their plots. A generic design can be found for downloading at climmob.net and can be adapted to the local requirements. + +ODK Collect +A free app available for download from Google Play Store to all Android-based mobile devices. ODK Collect is used for farmer registration and +data collection in tricot projects. + +Farmers / participating farmers +Women and men who participate in a tricot experiment by managing their own tricot trial and carrying out the observations, marking the observations on the observation card at the appropriate dates, and eventually reporting the observations to the local field agents. Their recruitment should involve considerations of gender, age and other demographic factors, as well as their task related to the technology under evaluation. In some tricot trials, non-farmers participate, based on their role in food processing, trading, retailing or consumption. + +Randomization +The balanced creation of sets of three varieties from the full pool of varieties. The randomization is generated by the ClimMob software and is required to prepare the trial packages. + +Researchers +Experts studying or using the agricultural technology under evaluation. They select the technology options to be included in the project and supply experimental material for each technology option to the implementing organization. + +Technology +With tricot, many different kinds of farm innovations can be tested. Crop varieties can be one kind of agricultural technology, but irrigation systems, fertilizers, fertilizer dosage, or cropping styles and tillage systems are also ‘technologies’ that can be tested using the tricot approach. Within each technology, there are different variants or options (see next entry ‘Technology options’). + +Technology options +Each tricot experiment focuses on one agricultural technology (for example, ‘fertilizer composition’), but tests several technology options +(fertilizer composition X, fertilizer composition Y, etc.). These technology options should in principle be suitable to local conditions and have the potential to be adopted by some of the farmers. The researchers select the technology options, and they are recommended to begin a first experiment made up of between 8 to 12 options. + +Trial package +A bag given to every farmer at the initial workshop. The large bag is marked with a number and a QR code. It contains: (i) three small bags containing material of the different technology options ( marked with ‘A’, ‘B’, and ‘C’); (ii) an observation card; and (iii) an explanatory brochure about the tricot process. + +Tricot +The word ‘tricot’ is derived from three words: Triadic comparison of technology options. ‘Triadic’ refers to the sets of three technology options that are compared in each trial. In technical jargon, three things define tricot: (1) the use of incomplete blocks of three items (to make the threshold of participation low in terms of farm size and to make it cognitively manageable), (2) the use of ranking as the main way to report observations (to facilitate digital data collection and to make it possible to cultivate a tricot plot with very little training), and (3) the limited control of experimental conditions (following usual technology use practice to maximize external validity). + +Trial plot +A small area within or at the margin of the farmer’s production plot, with representative soil conditions. It is divided into three equal parts, for the testing of the three technology options assigned to the farmer. + +Tricot trial +Field test of different technological options, in sets of three, each grown and observed by a farmer in a small designated area of her/his +own farm. \ No newline at end of file