A is something we can measure that informs us about the performance and capability of a process.

An organisational assessment is a systematic process for obtaining valid information about the performance of an organisation and the factors that affect performance.  It differs from other types of evaluations because the assessment focuses on the organisation as the primary unit of analysis. 

Organisations are constantly trying to adapt, survive, perform and influence. However, they are not always successful. To better understand what they can or should change to improve their ability to perform, organisations can conduct organisational assessments.  This diagnostic tool can help organisations obtain useful data on their performance, identify important factors that aid or impede their achievement of results, and situate themselves with respect to competitors. Interestingly, the demand for such evaluations is gaining ground. Donors are increasingly trying to deepen their understanding of the performance of organisations which they fund (e.g., government ministries, International Financial Institutions and other multilateral organisations, NGOs, as well as research institutions) not only to determine the contributions of these organisations to development results, but also to better grasp the capacities these organisations have in place to support the achievement of results.   

Example

Examples of Application of Organisational Assessments

• The Multilateral Organisation Performance Assessment Network (MOPAN) is a group of 16 donor countries that have joined forces to assess the performance of the major multilateral organisations which they fund. MOPAN has developed an assessment approach that draws on perceptions and secondary data (i.e., documents) to assess the performance of organisations with a focus on their systems, behaviours, and practices (or capacities).  The exercise is used to encourage discussion among donors and multilateral organisations about ways to enhance organisational effectiveness.
• In 2011, an evaluative report was disseminated by the International Monetary Fund (IMF) regarding its performance leading up to the global financial and economic crisis. Among the factors that hampered the organisation’s ability to detect important vulnerabilities and risks, the report highlights the pervasiveness of cognitive biases and groupthink as well as the operational structure of the organisation: on the one hand, it was widely believed in the organisation that a financial crisis could not happen in a large advanced economy and on the other, the existence of a silo mentality prevented information from being shared across units and departments to help predict the crisis.  The assessment results are being used by the IMF’s board and executive management to revise how the organisation operates.
• The Center for Effective Philanthropy developed a conceptual framework for assessing the performance of foundations. This framework provides a way for a foundation to infer the social benefit created by its activities relative to the resources it invests, and aims to allow its leaders to understand the performance of their organisation over time and in relation to other foundations. In 2011, the center surveyed CEOs American foundations and found that nearly 50% of respondents conducted organisational assessments, notably to learn and improve their foundation’s future performance, to demonstrate accountability for their foundation’s use of resources, and to understand the impact of their foundation’s work.

Frameworks

A number of models or frameworks for conducting an organisational performance assessment exist. The choice of which framework (or combination of frameworks) to use depends on the nature of the organisation, on the purpose of the assessment, and on the context in which the assessed organisation operates. The Reflect and Learn website presents details on the rationale and particularities of various frameworks. As highlighted in a paper presented on the Impact Alliance website, it is important to note that different frameworks are underpinned by different philosophies and theories of organisational change; an organisation should choose a framework that is congruent with its own management beliefs and culture, to ensure that it fully engages in the process and truly benefits from the assessment (EDITOR: The link for this paper is no longer working. Please bear with us while we find an alternate link for this source. July 5, 2016). 

One of the most comprehensive frameworks for Organisational Performance Assessment (OPA) is the Institutional and Organisational Assessment Model (IOA Model) elaborated by Universalia and the International Development Resource Center (IDRC).  This model views the performance of an organisation as a multidimensional idea, that is, as the balance between the effectiveness, relevance, efficiency, and financial viability of the organisation (see schematic diagram below). The framework also posits that organisational performance should be examined in relation to the organisation’s motivation, capacity and external environment. Indeed, a review of the literature conducted as a preliminary step for developing the framework showed that organisations change: in response to factors in their external environment, because of changes in their internal resources (e.g., financial, technological, human), and as a result of fundamental shifts in values within the organisation, which in turn affect the organisational climate, culture and ways of operating. The book Organisational Assessment: A Framework for Improving Performance by Lusthaus et al. further details the IOA Model and discusses key methodological issues for carrying out an OPA. Meanwhile, Enhancing Organisational Performance: A Toolbox for Self-Assessment by Lusthaus et al. provides tools and tips for organisations wishing to conduct an OPA. Both these resources are publically available (in French and English) on the IDRC website. 

Source: Universalia Institutional and Organisational Assessment Model (IOA Model)

Key resource: As highlighted above, the Reflect and Learn website presents a range of frameworks for conducting organisational performance assessments. The site also introduces the process and management of OPAs and provides a database of concrete tools that organisations can use to carry out assessments.

Important considerations.

A  Self-Assessment or an External Assessment?

A key decision that an organisation needs to make when undertaking an organisational assessment is whether to self-assess its performance, to commission an external assessment, or to use a combination of both approaches. Some advantages of a self-assessment are that it encourages the organisation’s ownership of the assessment, and thereby increases the latter’s acceptance of feedback and commitment to the evaluation’s recommendations. However, drawbacks of the self-assessment approach are that external stakeholders may question the independence or validity of the findings and may fear that hard issues will not be tackled, due to potential sensitivities within the organisation.

What issues to prioritise?

As underlined by Kathleen Immordino in her book on organisational assessment in the public sector, the questions an organisation needs to ask as part of an assessment depend on the specific context of the organisation. “In any complex organisation, there are innumerable ‘things’ that can be measured and studied. An effective assessment process focuses on those things that have the greatest impact on the way the organisation functions.” Practical considerations which may guide the selection or prioritisation of key questions for an organisational assessment are: i) the time required and resources available to answer each question; ii) the organisation’s purpose for conducting an assessment (for example, a desire to strengthen accountability or a desire to inform a new strategic planning cycle); and iii) the need to balance the interests of multiple stakeholders.

Once an organisation has a clear picture of what it wants to measure, it will need to identify what indicators (quantitative and qualitative) to use to assess its performance. This can be one of the more challenging steps in the organisational assessment process as a plethora of potential indicators may appear useful, but weeding out the ones that really matter and that answer the assessment questions can be difficult.

Challenges of selecting indicators as part of the Organisational Assessment process

• Measuring something within an organisation can increase its importance:  for example, a social service NGO that chooses to track the number of people it serves within a community may end up trying to increase the number of people it visits and to reduce the time spent with each person (with potential consequences for the quality of the services rendered).
• Simple indicators may not always fit the bill and may need to be combined: Developing adequate indicators to measure the complex dynamics that exist within an organisation can be quite challenging. Organisations may develop a set of carefully considered indicators but need to modify them over time as they analyse their results.
• Indicators may be interpreted differently amongst stakeholders: For example, an indicator that measures the diversification of funding of an organisation to assess its financial viability can be viewed positively by certain stakeholders, as diversification signifies that the organisation is not overly reliant on a single donor. Meanwhile, other stakeholders may view this measurement in a negative light, as dealing with multiple donors can lead to fragmentation and increased organisational costs in order to manage multiple donor requirements (each donor may have its own priorities, expectations, systems, and evaluation and reporting requirements). 

What to consider in selecting the options?

Organisational assessments follow the tradition of a case study methodology.  A case study requires a research design that focuses on understanding the unit (the organisation) and can use a combination of qualitative and quantitative data.  The choice of options depends on the specific circumstance for the organisation and its stakeholders.    We have found that observation (site visits), document review, interviews and surveys are some of the most common options used. Site visits and observation provide vital information on the facilities, physical artifacts, and interactions between staff of an organisation. Meanwhile, a document review is used to follow the written record of the organisation: meeting minutes, reports, policies, etc. Interviews are a prime source of data for organisational assessments and should be conducted with a wide range of respondents (both male and female). Surveys are particularly useful for gathering data from a large number of people and for obtaining information regarding people’s attitudes, perceptions, opinions, preferences, beliefs, etc. These four data collection options can be used to triangulate information and validate conclusions: using more than one data source can help identify discrepancies between what people say and what people do, as well as between what the organisation is and what it ought to be. 

Factors that support the use of results of the assessment

Organisational assessment results have a wide variety of uses. For instance, they can be used by an organisation to build its capacity, to validate its work, to promote dialogue with funders or partners and to help devise its strategies for the future. However, to ensure that results of the organisational assessment are used, their use must be planned for by the organisation from the onset of the assessment, as well as considered throughout the implementation phase and even once reports have been submitted and disseminated. Some conditions which enhance the utilisation of the results are when:

• The purpose and benefits of the assessment are clear to the organisation’s stakeholders.
• The main focus of the assessment is on learning rather than on accountability.
• Internal leadership is identified to champion the process and results of the assessment.
• The organisational culture is one that supports use of positive and negative feedback in planning and managing change
• Stakeholders are involved in the assessment process (from the negotiation and planning stages).
• Stakeholders see the assessment as relevant, credible, transparent, of high quality, and the findings have face validity.
• The assessment team is able to communicate the intent of the assessment, their approach, and the results to senior staff and board members.
• The report is timely (i.e., produced at an opportune time within the planning cycle of the organisation).
• There is a process in place and resources allocated to implement and follow-up on the assessment’s recommendations.
• Recommendations are realistic and feasible (for example, financially).

Resources

Examples

Guides

Overview

Websites

• Reflect and Learn: this website presents details on the rationale and particularities of various frameworks for conducting an organisational performance assessment

---

Page 2

Approaches (on this site) refer to an integrated package of options (methods or processes). For example, 'Randomized Controlled Trials' (RCTs) use a combination of the options random sampling, control group and standardised indicators and measures.

A strengths-based approach designed to support ongoing learning and adaptation by identifying and investigating outlier examples of good practice and ways of increasing their frequency.

Click an approach on the left to navigate to it

An approach that focuses on assessing the value of an intervention as perceived by the (intended) beneficiaries, thereby aiming to give voice to their priorities and concerns.

Click an approach on the left to navigate to it

A research design that focuses on understanding a unit (person, site or project) in its context, which can use a combination of qualitative and quantitative data.

Click an approach on the left to navigate to it

An approach designed to support ongoing learning and adaptation, which identifies the processes required to achieve desired results, and then observes whether those processes take place, and how.

Click an approach on the left to navigate to it

An impact evaluation approach based on contribution analysis, with the addition of processes for expert review and community review of evidence and conclusions.

Click an approach on the left to navigate to it

An impact evaluation approach that iteratively maps available evidence against a theory of change, then identifies and addresses challenges to causal inference.

Click an approach on the left to navigate to it

An approach used to surface, elaborate, and critically consider the options and implications of boundary judgments, that is, the ways in which people/groups decide what is relevant to what is being evaluated.

Click an approach on the left to navigate to it

Various ways of doing evaluation in ways that support democratic decision making, accountability and/or capacity.

Click an approach on the left to navigate to it

An approach designed to support ongoing learning and adaptation, through iterative, embedded evaluation.

Click an approach on the left to navigate to it

A stakeholder involvement approach designed to provide groups with the tools and knowledge they need to monitor and evaluate their own performance and accomplish their goals.

Click an approach on the left to navigate to it

A particular type of case study used to jointly develop an agreed narrative of how an innovation was developed, including key contributors and processes, to inform future innovation efforts.

Click an approach on the left to navigate to it

A way to jointly develop an agreed narrative of how an innovation was developed, including key contributors and processes, to inform future innovation efforts.

Click an approach on the left to navigate to it

A particular type of case study used  to create a narrative of how institutional arrangements have evolved over time and have created and contributed to more effective ways to achieve project or program goals.

Click an approach on the left to navigate to it

Approach primarily intended to clarify differences in values among stakeholders by collecting and collectively analysing personal accounts of change.

Click an approach on the left to navigate to it

An impact evaluation approach suitable for retrospectively identifying emergent impacts by collecting evidence of what has changed  and, then, working backwards, determining whether and how an intervention has contributed to these changes.

Click an approach on the left to navigate to it

An impact evaluation approach which unpacks  an initiative’s theory of change, provides a framework to collect data on immediate, basic changes that lead to longer, more transformative change, and allows for the plausible assessment of the initiative’s contribution to results via ‘boundary partners’.

Click an approach on the left to navigate to it

A range of approaches that engage stakeholders (especially intended beneficiaries) in conducting the evaluation and/or making decisions about the evaluation​.

Click an approach on the left to navigate to it

A participatory approach which enables  farmers to analyse their own situation and develop a common perspective on natural resource management and agriculture at village level. 

Click an approach on the left to navigate to it

A strengths-based approach to learning and improvement that involves intended evaluation users in identifying ‘outliers’ – those with exceptionally good outcomes - and understanding how they have achieved these.

Click an approach on the left to navigate to it

An impact evaluation approach without a control group that uses narrative causal statements elicited directly from intended project beneficiaries.

Click an approach on the left to navigate to it

An impact evaluation approach that compares results between a randomly assigned control group and experimental group or groups to produce an estimate of the mean net impact of an intervention.

Click an approach on the left to navigate to it

A Rapid Evaluation is an approach that uses multiple evaluation methods and techniques to quickly and systematically collect data when time or resources are limited.

Many terms are used to describe these approaches, including real time evaluations, rapid feedback evaluation, rapid evaluation methods, rapid-cycle evaluation and rapid appraisal.  The common feature of these different models is the expedited implementation timeframes which generally range from 10 days to 6 months.

An approach especially to impact evaluation which examines what works for whom in what circumstances through what causal mechanisms, including changes in the reasoning and resources of participants.

Click an approach on the left to navigate to it

An participatory approach to value-for-money evaluation that identifies a broad range of social outcomes, not only the direct outcomes for the intended beneficiaries of an intervention.

Click an approach on the left to navigate to it

Click an approach on the left to navigate to it

An approach to decision-making in evaluation that involves identifying the primary intended users and uses of an evaluation and then making all decisions in terms of the evaluation design and plan with reference to these.

Click an approach on the left to navigate to it