Language is so fundamental to how we engage with the world and with each other. We mostly choose the words we use to convey an idea, to coordinate our understanding of a thing, or to connect with other human beings. But in the process, we may also either reinforce or challenge existing biases, habits, and power structures.
We at Fourth Quadrant Partners (4QP) have spent a lot of time talking about the definition of “learning” in Emergent Learning — the disciplined attention to data and insights that emerge from our work and the deliberate application of these to improving future results, and the implication that the place where learning actually happens is in the work itself.
The term “Learning Agenda,” however, means something very different to many people in the social sector. Two examples:
- “A learning agenda includes: 1) a set of questions addressing critical knowledge gaps; 2) a set of associated activities to answer them; and 3) products aimed at disseminating findings and designed with usage and application in mind.” — USAID
- “Learning agendas are a set of prioritized research questions and activities that guide an agency’s evidence-building and decision-making practices.” — Urban Institute
We believe that these definitions of “learning” convey two assumptions that can reinforce existing power structures.
The first assumption: Each one starts by listing a set of questions. Who defines the questions that are most important is unstated, but in practice, the questions are commonly defined by the entity funding an initiative. We believe that this fundamental, implicit assumption needs to change: The funder should not be the only one with the agency to decide which questions are important.
That is not to say that a funder has no right to identify questions that matter to them. But it does mean that each set of actors in a system also needs to have the right — and we would say the responsibility — to identify the questions they need to learn about, at the moment in time when they are most relevant to their work.
The idea of any single entity creating an agenda of questions to answer is problematic in many ways. In the EL community, one challenge that gets raised over and over is this: There are just too many questions! This is a natural reflection of the complexity of the challenges we’ve set out to address. How do we avoid creating a laundry list of learning questions to answer? How do we choose the most important questions to focus on?
The very notion that any single group can identify a set of questions at any one point in time has become even more questionable in the face of COVID and the increased attention to racial injustice. A list of questions that seemed important six months ago now feels out-of-date. Yet we may still be accountable to senior staff to answer them. These two disrupting themes have fundamentally upended our thinking and feelings about which questions matter most and how and when to ask them. And, perhaps most importantly, who gets to choose.
The second assumption: By these definitions, the responsibility for learning has been exported either to the learning and evaluation staff or to an external resource. They are responsible for finding “the answer” (including making meaning of any data they gather in conclusions or recommendations) and handing it back to the organization to assist with decision-making or disseminating it to the field. There is no role in this process for the people doing the work to contribute to, or make meaning of, the answer. The fact is that in complex and dynamic work, there is no single answer. We can do all the research we want. But history tells us that unless and until we — the actors in a system — try to apply those findings and recommendations to our own messy, complex environments, full of human beings, they will remain answers-in-theory, not answers-in-practice.
Distinguishing between research, evaluation, and learning questions
Within the frame of Emergent Learning, we would say that these two definitions by USAID and Urban Institute describe a research agenda. To begin to shift our habits and understandings, we think that it is important for the social sector to learn to distinguish between learning questions, evaluation questions, and research questions and where to use which ones and why.
Broadly speaking, a research question is posed by or to a researcher and involves surveying and analyzing what exists outside of our own boundaries, typically to help inform the field’s future actions or our own decisions. Broadly speaking, an evaluation question is posed by or to an evaluator and involves collecting data on the results of a team’s activity to assess performance against expected outcomes and, often, to provide feedback along the path.
A learning question, on the other hand, is posed by and to ourselves — the actual people doing the work. A learning question asks us to look forward — to think about what we’re trying to accomplish, what we know so far, and what it’s going to take to achieve the outcomes we have set for ourselves. And then it asks us to test our thinking along the path, in order to improve our results over time. That’s what we mean by “learning.”
Example: Research, Evaluation and Learning Questions for a Leadership Development Initiative
| Research Questions | What skills and resources do leaders need in order to be able to achieve racial justice in their communities? How do these skills and resources vary, based on the characteristics and needs of different types of communities? |
| Evaluation Questions | To what extent did the curriculum and networking provided by our grantees prepare leaders to achieve racial justice in their communities? What were the contributing factors? |
| Learning Question | What will it take to build a network of leaders who are committed to achieving racial justice in their communities? |
This distinction should not be new to practitioners of Emergent Learning. But this other understanding of “learning” is so fundamentally woven into the way senior leaders, boards, program and evaluation staff think about learning that it’s going to require a very deliberate and visible effort to shift our shared mental models and habits.
To help address the overwhelming number of possible questions to ask; the question of who has the agency to choose the learning question and when; and who does the learning, we created the idea of a layered learning agenda. The notion of layering, or nesting, will sound familiar to EL practitioners — it is an important part of developing line of sight. What will it take to…? And what will it take to do that? And so on.
A sample layered Learning Agenda for a leadership development program
In this simple example of a leadership development initiative, the actors who own pieces of answering this larger question include foundation leadership, foundation program staff — and their work and the questions that matter to them are probably different; grantees who have responsibility to deliver leadership programs, and the fellows who are participating in those programs and who are expected to use what they learn to lead in their communities.
At each layer, the most profound overarching learning question is this: “What will it take to [achieve our part of the larger goal]?” For example, for grantees in this initiative, the overarching question might be simply: “ What will it take to build a network of leaders who are committed to achieving racial justice in their communities?” Beyond that, the questions they ask might change over time, driven by circumstances and by the opportunities in front of them — in this case for example revising the curriculum or welcoming a new cohort or, as is the case for all of us now, figuring out how to do all of this virtually. It keeps their own work at the center.
In practice, each layer of actors could create their own detailed learning agenda at their level that might include participants, indicators, potential data sources, reporting expectations, etc. And, in reality, some layers of learning may remain tacit, but mapping out the whole learning ecosystem can help those of us who are stewarding learning to know where to focus and what questions matter the most at any moment in time.
It also meets the needs of the funder to ask the questions that matter to them, and includes the possibility of asking parallel research or evaluation questions at each layer, without shifting the responsibility for learning away from the people doing the work itself.
Taken together, a layered learning agenda can help to create a learning ecosystem with a shared line of sight — actors at each level addressing the questions they are best able to address, in a way that rolls up to a larger body of knowledge. Senior leaders often ask learning staff to demonstrate how the learning they are doing at the program level rolls up. This is one way to help demonstrate that relationship.
We offer one caveat: In fact, layering a learning agenda like this makes the focus of the agenda much simpler at each level. But to a new audience that is unfamiliar with Emergent Learning, the larger framework may appear complex and labor-intensive, so take that into consideration if you choose to share this framework with your colleagues.
Regardless of what your learning agenda looks like, being thoughtful about who the actors are and finding the overarching learning question that is most relevant to them is consonant with the principles of Emergent Learning. It can create the kind of passionate commitment to learning that extends beyond the bounds of a funded initiative — the questions don’t go away when you leave the building. And authorizing and encouraging those actors to identify the questions most relevant to them is an incredibly more efficient way to figure out which questions to ask, when, and why.