Viewing Judicial Independence and Accountability through the “ Lens ” of Performance Measurement and Management

This article is a review of the European Network of Councils for the Judiciary’s (ENCJ) framework and vision of the independence, accountability, and transparency of the judiciaries in member states of the European Union. Its purpose is to aid ENCJ in the further development of its indicators of judicial independence and accountability. The focus of the article is on performance measurement and management (PMM) as seen, depending on one’s views, either as an instrument for strengthening judicial independence or, alternatively, an instrument for reigning in the power of the judiciary and threatening its independence. The article begins with a general discussion of judicial independence, accountability, and transparency as seen through the “lens” of PMM. It continues with a critical review and assessment of the conceptual framework of the ENCJ’s 22 indicators and 64 sub-indicators of judicial independence and accountability and identifies several shortcomings that decrease the utility of the framework. It urges a rethinking of the conceptual framework and proposes an alternative model – an input/output/outcome “logic model” --more amenable to understanding and improving indicators of judicial independence and accountability. It makes four recommendations aimed at a better alignment of ENCJ’s framework of indicators with principles and practices of modern PMM. The article concludes with a warning about troubling developments at a higher level of governance and politics that some see as a retreat from democracy and the rule of law in Europe and in many other parts of the world, one that poses an existential threat to the judiciary as a coequal partner in government.


Introduction
The goal of judicial independence is to secure for individual judges, courts, and court systems the independence to resolve disputes according to the law and to shield them from improper interference from the other branches of government, or private or partisan interests.A judge, court, or court system compromise their impartiality or independence, for example, when they merely ratify plea bargains, serve solely as a revenue-producing arm of government, or perfunctorily place their imprimatur on decisions made by others.Our understanding of judicial independence today encompasses not only control and authority over the legal decisions of individual judges but also an array of administrative powers of courts and judiciaries as organizations, including authority over budgeting, information technology, human resources, allocation of judicial services (supply chain management), judicial selection, retentions, and assignment, and the education and training of judges and justice system staff. 2As noted by van Dijk and Vos in their lead article in this special issue of the Journal, "insufficient and arbitrary funding of the judiciary, can make individual independence an empty shell." 3 Well designed and executed, judicial independence instills legitimacy and public trust and confidence in the judicial system of a country.
In 2013, the European Councils of the Judiciary (ENCJ), an organization of national councils of the judiciaries in the member states of the European Union (EU), 4 began an ongoing initiative to evaluate the judicial independence and accountability of the judicial systems in the member states of the European Union (EU) through the "lens" of performance measurement.To date, ENCJ has developed a framework and vision of independence and accountability and a set of indicators to evaluate the state of independence and accountability of EU judicial systems.

Purpose
This article is a review and an assessment of ENCJ's framework and vision of the independence, accountability, and transparency of the judiciaries in member EU states.Its purpose is to aid ENCJ in the further development of its indicators of judicial independence and accountability.The focus of the article is on performance measurement and management (PMM) as seen, depending on one's views, either as an instrument for strengthening judicial independence or, alternatively, an instrument for reigning in the power of the judiciary and threatening its independence.
The article explores how ENCJ "frames" judicial independence by its identification, classification, definition, and use of the 22 indicators and 64 sub-indicators5 of judicial independence and accountability as described in its 2017 report. 6Because the focus of the article is on the conceptual framework of the indicators, which is applied consistently across the all the indicators and subindicators developed by ENCJ, and not on the detailed substance of the indicators, the exploration is limited to the 22 indicators and 64 indicators of independence and accountability, and does not cover the 21 new indicators and 82 sub-indicators of the quality of justice that are still in the early development stages in ENJC's ongoing initiative. 7For the same reason, the article does not delve into the substance of the sub-indicators.Key questions and related issues addressed by this article are posed in italics in this and the next paragraph.Do the demands of rigorous PMM as part of the administration of justice -including the regular and continuous monitoring of performance in a transparent and accountable manner -threaten judicial independence?It is inevitable that there will always be defenders and critics of the performance of courts (e.g., about delays, costs, and reliability) and arguments over how to improve court organization and operations, but how can PMM foster in the judiciary the strength to resist encroachments by the other branches of government, private or partisan interests into its domain?
What is the appropriate scope of judicial independence at the level of the individual judge (decisional independence) and the organizational level of a court or a national court system (institutional independence)?Where should the lines be drawn that separate the powers of the judicial, executive, and legislative powers of government and that protect judicial independence?Does judicial independence mean complete autonomy and self-government, and if not, how much comity, coordination, and cooperation with the other branches of Government and other public and private interests does it require?Most observers would agree that judicial independence does not demand absolute autonomy or isolation of judges and courts.Do the demands for measurement and management of the performance from within (e.g., emanating from presiding judges and court administrators) and outside the judiciary constitute an imposition that threaten judicial independence?These are fraught questions faced by judiciaries in Europe and around the globe.
Much like the relationship between doctors and hospital administrators, the relationship between judges and court administrators (including judges in administrative positions) --who generally are more likely to advocate for rigorous performance measurement --are strained and fraught, more so than they seem to be in Europe than in the United States.Both doctors and judges are semiautonomous professionals who often chafe at attempts by administrators to limit their freedom to do as they please (e.g., doctors order medical tests that are not needed and judges postpone trials too many times contributing to court delay).
In the United States, a recognized world leader judicial administration, the field of court administration is more mature and more stable than in Europe.All the 50 states in the United States have state court administrators who have varying degrees of management control over the courts in their states.Almost all courts at all levels employ court administrators or court executives, many with considerable influence over judicial administration.At the national level, professional organizations such as the Conference of Chief Judges, the Conference of Court Administrators, the National Center for State Courts, and the National Association for Court Management promote the role of court administration in governance of the judicial branch.
While it is common for judges and judicial groups in the United States to complain about an infringement of the independence of the judiciary and a blurring of the separation of powers, such as when court administration pressures courts to adopt rigorous, transparent, and accountable performance measurement and management, or to cut costs, the complaints have been more muted and have not created the level of tension and rift between judges and court administration that we see in Europe.

Contents
Following this Introduction, this article is organized into four sections.Section 2 is a discussion of judicial independence and judicial accountability as seen through the "lens" of performance measurement and management (PMM).Section 3 is a critical review and assessment of the conceptual framework of the ENCJ's 22 indicators and 64 sub-indicators of judicial independence and accountability.It identifies several shortcomings that decrease the utility of the conceptual framework.It urges a rethinking of the framework of the indicators and proposes an alternative model -an input/output/outcome "logic model" --more amenable to understanding and improving indicators of judicial independence and accountability.Section 4 makes four recommendations, with supporting commentary, aimed at a better alignment of ENCJ's framework of indicators with principles and practices of modern PMM.In addition to a rethinking of the conceptual framework of the indicators urged in Section 3, the recommendations in Section bear scrutiny by ENCJ as it takes the next steps in the development of its indicators over the next two years. 8The article concludes in Section 5 with a warning about troubling transnational developments at a higher level of governance and politics that some see as a retreat from democracy and the rule of law in Europe and in many other parts of the world, one that poses an existential threat to the judiciary as a coequal partner in Government.
In addition to the basic source, i.e., the 2016 -2017 report by ENCJ, 9 this article is based on the following sources of information: (a) a draft report by Professors van Dijk and Vos, which appears in its final form in this special issue of the Journal, and which along with the ENJC 2016 -2017 report was sent to participants in advance of a scientific conference in April 12-13, 2018 in Utrecht, Netherlands, sponsored by ENCJ and by Utrecht University's Montaigne Centre for Judicial Administration and Conflict Resolution; this conference brought together 26 international researchers and scholars to review ENCJ's framework for the measurement of judicial independence and accountability; (b) presentations and discussions at the scientific conference in Utrecht in April 2018, which subjected the indicators of independence and accountability to external review by the scientific community and by partners of the ENCJ (both within and outside the Judiciary); (c) two other papers by ENJC describing its ongoing initiative on judicial accountability , one preceding its 2016 -2017 report 10 and its most recent in written 2018 shortly before this article was written 11 ; and, finally, (d) my own experience with justice PMM around the world. 12

Performance measurement, accountability, and transparency
Beginning with a definition of PMM, this section is a discussion of judicial independence and judicial accountability as seen through the "lens" of PMM.

Definition of performance measurement and management
There is no precise agreed-upon meaning of "performance measurement."For this article, performance measurement and management (PMM) in courts and justice systems is defined as it is in the second edition of the International Consortium of Court Excellence's International Framework of Court Excellence: Court performance measurement and management (PMM) is the discipline and the process of monitoring, analyzing, and using organizational performance data on a regular (ideally in real or near-real time) and continuous basis for the purposes of improvements in organizational efficiency and effectiveness, in transparency and accountability, and in public trust and confidence in the courts and the justice system. 13veral part of this definition merit highlighting.First, performance measurement is defined as a management discipline encompassing a system of concepts, methods, and techniques, as well as a process.The definition of PMM as a discipline addressing the key question "How are we performing?" emphasizes that PMM is a rigorous way of thinking about solving problems, discovering opportunities, and identifying possible solutions (whatever moves the measurement "needle").Second, by recognizing that performance measurement data are of no use if not used, measurement is explicitly paired with performance management to emphasize that to be used effectively PMM should be infused into the very DNA of governance and management of operations such as budgeting, resource management, supply chain management, and strategic planning.This pairing, which firmly anchors the discipline of PMM in the tradition of management and business, is a relatively new development widely seen as a major step in transforming measurement into management for real organizational change.
Third, the discipline of PMM should be practiced on a regular and continuing basis, ideally in real time or near real time, as performance occurs and as the needs of users of specific measures dictate.Actionable performance measures are sensitive to interventions and, therefore, must be taken and used in tight timeframes, not months or years after measurement occurred.PMM is analogous to the "measurements" we take as we monitor the performance of our car using the car's dashboard of indicators.It would be nonsensical and unsafe if our car's speedometer only registered speeds once every hour.Finally, the definition aims PMM toward specific purposes including efficiency and effectiveness, transparency and accountability, and public trust and confidence in the judicial branch.

Truth, accountability, and transparency
In his inspiring 2017 book, Principles, Ray Dalio urges us to trust in "radical truth and radical transparency" in our work in organizations. 14Understanding what is true is essential for success, and being radically transparent about mistakes and weaknesses, helps to create the understanding that leads to improvements," he writes.15 To its credit, the ENJC has approached its work on judicial independence, accountability, and transparency through the "lens" of PMM.ENCJ has an understanding that superior performance is the product of accountability.It recognizes that judicial independence and performance accountability and transparency go hand in hand, the latter being a necessary condition of the former.In its 2017 report, ENCJ asserts that a "[j]udiciary that does not want to be accountable to society and has no eye for societal needs will not gain the trust of society and will endanger its independence in the short or long run."Conversely, it notes that "accountability without independence" reduces a judiciary to an agency of the executive or legislative branches.16 This position is reinforced by van Dijk and Vos, noting that justice systems "that are accountable to no one are likely to have weaker incentives to improve themselves than systems that open up to the outside world and tackle criticism seriously.By emphasizing and strengthening independence solely judiciaries risk insulating themselves from society and becoming irresponsive to justified demands of society."17 Because of their "independent" status, the judiciaries' rules and procedures are not based on the same analysis and subject to the same level of scrutiny as those of agencies of the executive and legislative branches.Many, if not most, of the judges in the Netherlands and elsewhere in Europe and beyond generally embrace --or at least acquiesce to --the need for accountability and transparency in theory.They recognize that judicial reform aimed at making courts and other judicia institutions more efficient and effective nearly always involves trade-offs between independence and accountability. Peps, as a nod toward comity, coordination, and cooperation with other branches of government, and other public and private interests, they acknowledge that the greater degree of freedom from scrutiny and interference traditionally enjoyed by courts compared to other public institutions (at least in much of the Western world) must be balanced by the quid pro quo of transparency and accountability.Some see their accountability and transparency as a source of pride reflecting their commitment to serve the people which in turn, they believe, ensures the public trust and confidence.
However, to many judges the words "accountability" and "transparency" conjure up something to be feared.They have grave concerns about judiciaries' acceptance of rigorous measurement and management of performance enabling such accountability and transparency. 18They feel pressured by executive branch representatives (including ministries of justice) and court management to balance independence with accountability implemented by performance measurement focused on, for example: (a) increasing the productivity of judges and courts (i.e., resolving more cases more quickly); (b) reducing costs by closing or consolidating courts; (c) rationalizing and justifying the allocation and distribution of judicial resources including judges, administrative staff, and courts; and, generally, (d) improving supply chain management (i.e., balancing the supply of judicial services delivered to people with the demand for those services).Courts may be ill-equipped with limited staff and technical resources to translate concepts and roomy objectives and definitions of independence, accountability, and transparency into precisely defined performance measures and metrics.Judges perceive these as constraints or threat to their independence.
On the other hand, advocates of performance accountability and transparency -both within and outside of judicial institutions --argue that, done well, performance measurement shines a light on real progress in solving entrenched problems that matter to ordinary citizens.The promise of organizational performance measurement is suggested by maxims that have gained much currency in recent years: You can '  19 In an essay in the Wall Street Journal in 2013, accompanied by a large picture of a globe wrapped in a tape measure, Bill Gates, the former head of Microsoft and co-chair of the Bill and Melinda Gates Foundation, wrote that his plan to solve the world's biggest problems was, plain and simple, to measure them."In the past year," Gates wrote, "I have been struck by how important measurement is to improving the human condition.You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal.This may seem basic, but it is amazing how often it is not done and how hard it is to get right." 20In a similar vein, John Doerr, in his 2018 book, Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs (OKRs are objectives and key results) echoes Gates' view.He exhorts private and public organizations to embrace performance measurement and management as a "sharp-edged tool for world-class execution." 21

The conceptual framework of ENCJ's indicators of judicial independence and accountability
This section presents a critical review of the conceptual framework of ENCJ's indicators of judicial independence and accountability.As noted in the Introduction, the focus is mainly on the conceptual framework and not the substance of the 22 indicators and 64 sub-indicators of independence and accountability.Following the critical review, which finds the conceptual framework problematic, the section offers an alternative model -a logic model --that is simpler, aligns with the principles and practices of contemporary PMM, and promises a more productive platform to launch further development of ENCJ's indicators of judicial independence and accountability.and, (d) the independence of the judicial system as a whole.This classification and categorization can be depicted as 2 X 2 matrixes as shown Table 1 and Table 2 below The indicators are described as "key aspects" of independence and accountability "generally consisting of sub indicators to capture the diverse elements that are relevant for a key aspect." 22For example, Indicator No. 1, in the upper left quadrant of Table 1, is identified with the key aspect of the "legal basis of independence," consisting of five sub-indicators including: (i) formal guarantees of the independence of the Judiciary; (ii) formal assurances that judges are bound only by the law; (iii) formal methods for the determination of judges' salaries; (iv) formal mechanisms for the adjustment of judges' salaries; and, (v) formal guarantees for involvement of judges in the development of legal and judicial reform.formal mechanisms for the adjustment of judges' salaries; and, (v) formal guarantees for involvement of judges in the development of legal and judicial reform.

Table 2. Nine ENCJ Indicators and 31 Sub-indicators of Accountability in Two Categories
Indicators of objective or formal independence and accountability, in the upper-left quadrants (for the judiciary as a whole) and in the lower-left quadrants (for individual judges) in Table 1 and Table 2, are conceived as "formal, legal safeguards," i.e., "actual" resources, processes, and formal arrangements that can be directly observed and, presumably, counted and measured.
In public sector administration, objective data or, more accurately, "quantitative data," is viewed as the gold standard because it is based on measurable facts that are presumed to reflect the "real" world that can be observed, counted, and measured.As is explained in Section 4, the ENCJ indicators and sub-indicators do not meet this standard.An example consistent with this gold standard of quantitative data might include measures such as the number of complaints about judges registered, or the proportion of judges disciplined or dismissed.Objective and quantitative measures such as these can be aggregated to reflect the judiciary as a whole (e.g., the proportion of all dismissed or involuntarily relocated judges in a country, region, or court dismissed) and disaggregated to show the performance of individual judges (e.g., the number of complaints made against them).
Subjective independence or perceived independence in ENCJ's scheme (the term "subjective" is used interchangeably with the term "perceived"), depicted in the left halves of Table 1 and Table 2, is conceived as perceptions of independence and accountability in society, including those of the general public, users of courts, private companies, experts of various stripes, and judges themselves.The data is collected through participant observation and understood from the respondent's perspective.For reasons not altogether clear, the five indicators of judicial independence in the left column of Table 1 make no distinction between indicators for the judiciary as a whole and individual judges.There are no subjective/perceived indicators for judicial accountability in Table 2.
Subjective data are based on personal opinions, assumptions, interpretations, and beliefs gathered by surveys or reports of groups of experts.For example, the Eurobarometer Flash survey of judicial independence conducted by the European Commission, between 25 and 26 January 2017, the results of which are part of Indicator No. 9 in ENCJ's scheme, relies on survey methodology including questions such as: "Could you tell me to what extent each of the following reasons [e.g., no interference or pressure from government and politicians] explains your rating of the independence of the justice system [in your country]?"Respondents are asked to rate their agreement along a five-point scale including "very much," "somewhat," "not really," "not at all," and "don't know." 23

Shortcomings of the ENJC's scheme
The ENCJ's classification and categorization scheme has shortcomings that diminish its understanding and usefulness.

Objective and factual versus subjective and perceived
First, the distinction between objective and factual, on the one hand, and subjective and perceived, on the other, is blurry.Most of the indicators and sub-indicators are based on perceptions as measured by survey responses.However, as categorized in ENCJ's scheme, subjective independence is measured by only five indicators (Indicators Nos. 9 through 13) of perceived independence in response to surveys of both the judiciary as a whole and individual judges.No distinction is made between subjective/perceived indicators of independence (see 2).The difficulty is that the indicators of objective/formal independence also are generated from the opinions and perception of surveyed experts. 24A similar situation holds for judicial accountability.Moreover, the ENCJ describes the indicators in the European Union's Justice Scoreboard and the other third-party surveys it relies on as "objective," 25 when in fact those indicators also are based on independence as perceived, i.e., "subjective," using ENCP's definition of this term.Putting the indicators in the objective category does not make them so.
In their lead article in this special issue of the Journal, van Dijk and Vos note that the "scores for the indicators of objective independence and accountability were derived from the questionnaire, whilst those for subjective independence were based on international opinion surveys," suggesting that the former are somehow based on objective data. 26In fact, the former also are derived from surveys based on the perceptions and opinions of judges, members of the councils for the judiciary, and other individuals in the governing bodies such as ministries of justice.Van Dijk and Vos do recognize that this may lead to self-serving bias.While many of the indicators are observable things (e.g., the existence of laws, regulations, and rules for the adjustment of judges' salaries), and can be readily verified by any knowledgeable observer, most are subject to considerable variations of opinions and possible bias (e.g., the sufficiency of budgets).As further explained below, the circumscribed use of survey data based on perception is valuable; a complete reliance on opinions is not.
A long-standing problem with survey and polling data is that in professional circles the term "subjective" is often couched in pejorative terms meant to characterize subjective measures as "soft," mere opinion, not empirical, and not based on the application of the scientific method.This is unfortunate if by "subjective" we mean simply that the data is collected through participant observation and understood from the respondents' perspectives. 27Court user satisfaction measures such as the percent of court users who believe that the court they have just exited provides procedural justice (i.e., accessible, fair, accurate, timely, knowledgeable, and courteous judicial services), are among the most robust and widely used measures by justice systems around the globe. 28Because perception data reflect personal ideology, intuition, and even personal bias, experts in public sector performance generally prefer objective or quantitative data.However, this preference is determinative only if no good objective data are available or (and this is an important distinction) when the beliefs, opinions, and feelings, including the preferences and biases of respondents, are precisely what is sought, as is the case in measuring "procedural justice" or "procedural fairness" as experienced by users of courts. 29

Organizational versus individual judges' performance
Another shortcoming might be better characterized as a problem with the placement of two separate processes and disciplines into the same conceptual framework -organizational performance measurement of the judiciary as whole and judicial performance evaluation (JPE) of individual judges.The U.S.-based National Center for State Courts has developed extensive separate resource guides for organizational (court) performance measurement 30 and JPE 31 that reflect the differences between these two disciplines and processes.Business leaders have learned, painfully so according to John Doerr, that organizations and individuals should be measured and managed quite differently. 32fective court performance measurement, anchored by the right performance measures, indicates organizational performance, and not necessarily the performance of individual judges or court staff.There is evidence to suggest that highly competent judges and court staff do not necessarily guarantee a high-performing court.And a court with the best practices supported by good supporting infrastructure and technology can operate at high levels with less than highly competent judges and staff, though not for the long-term.Organizational performance measurement and individual performance evaluation differ in purpose, methodology, interpretation, and use.The evaluation of individual judges with personal problems such as disabilities, health issues, personality disorders, marital problems, alcoholism, or drug addiction simply have no parallel in organizational performance management.
Reporting the performance of individual judges as part of organizational court-wide or justice system-wide performance is likely to cause animosity and divisiveness.It can carry the danger of derailing an otherwise promising performance measurement effort.Disaggregations or breakouts of organizational court performance to the individual judge level figure prominently in the reasons raised in opposition to organizational performance measurement. 33There is today simply insufficient knowledge and experience to support a formal linking of the two.For example, a court that is "speedy" -a court timely and expeditious in its case processing -is generally considered a "good" court; a court with a large backlog of cases awaiting resolution wherein people experience considerable court delay is generally viewed as a court in need of improvement.No such consensus exists about a "speedy" judge.Many so called "slow" judges, are considered deliberative, thoughtful, and thorough, and are held in high esteem by their colleagues for their integrity, opinion writing, mentoring of junior judges, and management and leadership skills.
Consistent with the literature on JPE, it might be most prudent that organizational performance data be limited to providing only a background and broad context for the discussion of individual performances of judges and staff, not to evaluate, to set individual goals, to apply sanctions, and not to rate and rank judges against their peers on measures such as clearance rate.For example, a judge could be made aware that he or she is resolving cases at a clearance rate lower than the court and lower than the overall average of the courts in a jurisdiction.

Cohesion and Logical Connection
A final shortcoming of ENCJ's conceptual framework of indicators and sub-indicators, which is related to the above, is that the classification and categorizations of indicators and sub-categories lack coherence and logical connections to principles and concepts of performance measurement.Why, for example, is a distinction between objective and subjective data even made?What purpose does it serve?Is it helpful in identifying further development of the indicators and sub-indicators?Users of the framework may be prone to be confused by its complexity, especially in the context of the sheer number of indicators -a total of 43 indicators and 146 sub-indicators when one includes the 21 new indicators and 82 sub-indicators of the quality of justice that are still in the development stages.The issues that this shortcoming raises are touched upon in the following subsection and directly in Section 4, "Principles and practices of modern performance measurement and management."

A logic model of measures of judicial independence and accountability
The figure below depicts a logic model (also called an outcome-sequence chart 34 ) adapted here to identify and to prioritize indicators of judicial independence and accountability.It is offered here as an alternative to ENCJ's conceptual framework, one that promises to make up the shortcomings of the framework identified in the previous section.The model is much simpler, easier to understand, and to apply.In common sense terms, the elements or aspects of independence of accountability are categorized simply as the things we have at our disposal (inputs), the things we do (outputs), and the things we accomplish (outcomes).
The logic model, especially when depicted as in the accompanying figure, provides a visualization of what an indicator is intended to measure.It prompts fundamental questions such as: What is it that is being measured?Why does it matter and to whom?What critical outcomes and achievements do the things we have (inputs) and the things we do (outputs) lead to?Using the inputs of formal safeguards identified in the EMCJ indicators -i.e., one of the set of the things we have in the logic model -to highlight the fundamental questions that the logic model prompts us ask and think about, consider what we might count if we want to measure the success of doctors, hospitals, and the health care system, or the success of teachers, schools, and educational systems of the countries in the EU.What should we count?What should we measure that truly matters?What is the logic to support it?Few health care and education leaders and managers in these sectors, as well as patients and students, would be satisfied, for example, with an outcome indicator such as formal assurances that doctors and teachers are bound by law modeled on one of the sub-indicators of Indicator No. 1, Legal Basis of Independence, in the objective independence of the judiciary category in the ENCJ framework. 35nally, the logic model has coherence and logical connections to principles, concepts, and sound practices of PMM.The descriptions of PMM in international models such as the Global Measures of Court Performance 36 and the CourTools 37 include a concise operational definition, statements of purpose, and rationales for the prescribed performance measures including their alignment with values and critical success factors.This supporting logic is a fundamental requirement of the scientific method.No logic, no science.I strongly believe that such logic needs to be included in the ENJC framework.

Externally oriented versus internally oriented measures
Judicial independence and accountability are not ends in-and-of themselves, but rather means to ends that matter to the public.Justice systems do not exist for the benefit of judges but rather for the benefit of the citizens they serve.Indicators of internal inputs (such as available human and fiscal resources) and outputs (such as activities and procedures for disciplining judges) focused on justice system "insiders" running the courts may prompt outside observers to see the judiciary as a "closed shop" operating solely for its own benefit.On the other hand, outcome measures show how well or how poorly a court system is making things better for people by improvements in their status or condition, instead of how much effort (outputs) has been expended or by with what resources (inputs).
The European Commission and the Council of Europe, as well as other groups such as the International Consortium for Court Excellence and the National Center for State Courts in the United States, make distinctions between two types of justice system performance measures. 38The measures of the first type are seen as core judicial service performance measures, defined as externally oriented outcome measures that register changes in the status or conditions of the people served by the courts.In justice systems that have PMM systems, and in various models of PMM, the former type tends to be limited to a vital few aligned with the handful of mission-critical success factors of courts such as access to justice, expedition and timeliness, fairness, and public trust and confidence in the courts.
Performance measures of the second type are internally oriented input and output measures of court administration, structures, organization, and operations.These measures can be quite valid and useful organizational, operational, structural (internal) measures -judiciaries need to know how many courts and judges are in their jurisdictions --but they have limited value as core judicial service performance measures.Furthermore, we simply do not know if the inputs and outputs measured are directly correlated with outcomes.In fact, there is research by one of the authors of an article in this special issue of the Journal, showing that they may be negatively correlated. 39These internally oriented measures also can expand without limits to cover a myriad of internal input and output measures of court administration, structures, and operations that may be of special interest limited to only a few agents of the justice system or a special issue at a particular time.
All but five of the 22 ENCJ's indicators are input or output measures of formal arrangements, not outcome measures, and all five are measures of perceptions.Again, simply put, inputs are the resources we have; outputs are the things we do; and outcomes are what we accomplish.This input-output-outcome scheme, highlights problems in the ENCJ framework of indicators recognized by van Dijk and Vos, i.e., the gap between de jure and de facto independence and accountability, the difference between what we would like to see happen and what is actually happening.They write: "This allows for the possibility that a legal system may possess all the formal requirements of independence, but nonetheless have judges who do not act independently.It allows also for the possibility that systems are not transparent, such that the public cannot determine whether the judiciary is independent or not

Externally oriented versus internally oriented measures
Judicial independence and accountability are not ends in-and-of themselves, but rather means to ends that matter to the public.Justice systems do not exist for the benefit of judges but rather for the benefit of the citizens they serve.Indicators of internal inputs (such as available human and fiscal resources) and outputs (such as activities and procedures for disciplining judges) focused on justice system "insiders" running the courts may prompt outside observers to see the judiciary as a "closed shop" operating solely for its own benefit.On the other hand, outcome measures show how well or how poorly a court system is making things better for people by improvements in their status or condition, instead of how much effort (outputs) has been expended or by with what resources (inputs).
The European Commission and the Council of Europe, as well as other groups such as the International Consortium for Court Excellence and the National Center for State Courts in the United States, make distinctions between two types of justice system performance measures. 38he measures of the first type are seen as core judicial service performance measures, defined as externally oriented outcome measures that register changes in the status or conditions of the people served by the courts.In justice systems that have PMM systems, and in various models of PMM, the former type tends to be limited to a vital few aligned with the handful of missioncritical success factors of courts such as access to justice, expedition and timeliness, fairness, and public trust and confidence in the courts.
As it continues its development of indicators of judicial independence and accountability, I urge ENCJ to adopt and use the logic model along with the principles of modern PMM described in the next section.

Principles and practices of modern performance measurement and management
This section consists of four recommendations and supporting commentary aimed at a better alignment of ENCJ's framework of indicators with principles and practices of modern performance measurement and management (PMM) including: (i) the emphasis on outcomes and measures of judicial independence and accountability that matter to the people that are served by the judicial system instead of the "insiders" who run the system; (ii) the prioritization and streamlining of the indicators and sub-indicators to a vital (and manageable) few: (iii) definitions of the indicators and sub-indicators that meet five "SMART" criteria -specific, measurable, achievable, relevant, and time-based; and, (iv) the management, not just the measurement, of judicial independence and accountability.

In their continuing efforts to improve judicial independence and accountability through the "lens" of performance measurement, ENCJ and its member judicial councils and their partners, should count what counts, measure what matters, and emphasize outcome indicators that matter to those who are served by the judiciary, instead of those who "run" the judiciary. They should maintain an external orientation and look outward instead of inward.
This recommendation has been alluded to in several places earlier in this article.It's importance merits making it explicit here.Figuratively and literally, performance does not count unless it is related to the things that really matter, that are critical to the success -the efficiency, effectiveness, and accountability of a justice system.The connection between well-known health indicators like blood pressure, cholesterol level, and blood glucose, and the outcomes for our health and well-being is self-evident to most people.We know that these measures mean something vital and something very important to us.Such key success factors for judiciaries as access to justice, fairness, and timeliness have been referred to in the literature of justice system performance measurement as major performance areas, high level goals and objectives, standards of success, perspectives, domains, performance criteria, key results factors, and key outcomes.Whatever they are called, they form the framework of a judiciary's accountability and transparency to the public and other stakeholders.
Increasingly, court systems' stakeholders -ordinary citizens, politicians, litigants, legislators, jurors, witnesses, and court employees -are calling for clear evidence that the resources court systems expend bear a causal relationship to, or are at least are clearly associated with, benefits for people.Core outcome measures emphasize the condition or status of the recipients of judicial services or the participants in court programs (outcomes) rather than the resources (inputs) or the activities and processes, programs and activities (outputs).That is, they measure results and accomplishments, not merely resources, level of effort, and work performed.They measure ends rather than the means to achieve them.It is by this outcome orientation, for example, that the Global Measures41 help link the International Consortium of Court Excellence's values and areas of court excellence referred to "drivers" (e.g., court management and leadership) and "systems and enablers" (e.g., court policies; human, material, and financial resources; and court proceedings) to results and outcomes.

A vital few indicators ENJC's indicators and sub-indicators should be streamlined to a vital few instead of a "trivial" many, a vital few that constitute a "balanced scorecard" of measures of judicial independence and accountability.
The ENCJ should streamline the 22 indicators and 64 sub-indicators to a vital few core measures of judicial independence and accountability.This recommendation becomes more cogent in view of ENCJ's plans to add 20 indicators and 82 sub-indicators of quality of justice -an overwhelming total of 82 indicators and 126 sub-indicators.(Moses made do with just ten commandments.Heaven help the poor Christians if they had to contend with 82 commandments and 126 sub-commandments.)Because most people can remember no more than three to seven things at any one time, too many measures scatter their attention and make them less efficient and effective.The key to a successful PMM is paring down the number of measures to a vital few that are clearly linked to critical success factors.
The question of the ideal number of performance measures that should be part of a dashboard of a car (speedometer, odometer, tachometer, and other instrumentation for monitoring temperature, gas, and oil pressure), for example, is informed by the idea of "a vital few instead of a trivial many."42There is only so much that a driver of a car can pay attention to while driving.The indicators should be the critical few gauges, instrument panel focused on a safe, efficient, and timely journey without unnecessary distraction.
As it moves forward with its important work, I urge the ENCJ to look at other prominent models of performance measurement and management, in particular: (a)  45 of the United Nations Sustainable Development Goals (SDGs). 46These three models provide many insights into methodology of measuring independence and accountability including the need to limit the number of measures.In a 2016 paper, 47 I joined a chorus of critics pointing out that Goal 16 and its associated 12 performance targets and 21 proposed provisional indicators of the United Nations SDGs, as well as the total SDGs package adopted by the United Nations in September 2015, with 17 goals and 160 sub-goals, was simply too much to handle.The Global Measures and the CourTools both have a streamlined number of measures, emphasize outcome measures, and are clearly linked to fundamental values and principles of justice administration. 48 effective PMM system should include only a handful of core measures 49 that gauge important outcomes focused on key performance areas.It is this linkage with key performance areas that also helps to limit the number of core measures to a vital few insofar as broad strategic goals tied to citizens' needs and wants tend to be relatively few in number.Another way of focusing measurement on a vital few performance measures is to build hierarchies (some call these "families" or "cascades") of measures with the few core measures sitting at the top of a hierarchy of related subordinate measures.In this way users can "drill down" into ever more complexity only if they choose to do so.Creating measurement hierarchies keep things simple, allowing many measures in the system yet maintaining the focus on the vital few measures at the top of the hierarchy.Still another way is to develop a simple index of several measures, variables or aspects of a single area of performance, for example, expedition and timeliness of court case processing. 50

Make ENCJ's indicators SMART
The ENCJ should operationally define the indicators of judicial independence, which in their present form are no more than statements of goals or issues, and make the definitions concise, measurable, and actionable; i.e., make them SMART -specific, measurable, attainable, relevant, and time-bound.
Conceptual clarity and focus are requirements of good definitions of goals that make measurement of success possible.In this section, I argue that many, if not most, of the ENCJ indicators and sub-indicators are not measurable and manageable as currently formulated.
The concept of measurement means various things to different people across professions, fields, and disciplines.Generally, however, the scientific community sees measurement as a quantitatively expressed reduction of uncertainty based on one or more observations.Well formulated measures and indicators are SMART.That is, they are specific, measurable, attainable (actionable), relevant, and time-based.Conceived in this way, precise definition and effective measurement are necessary processes to reduce uncertainty.
In its papers, ENJC reflects some apprehensions about the challenge of measuring judicial independence and accountability.Is it even possible to measure something as complex and multifaceted as judicial independence?As pointed out by Douglas W. Hubbard in his book, How to Measure Anything, 51 if we incorrectly believe that measurement means that we must meet some unachievable standard of certainty, then I would agree that judicial independence is immeasurable.But when scientists perform measurements they seem to be using a different definition in terms of observations that quantitively reduce uncertainty.To them measurement is a quantitively expressed reduction of uncertainty based on one or more observations.Anything including judicial independence and accountability can be measured.If it matters at all, it is detectable and measurable, and it reduces uncertainty.
That said, the 22 indicators and 64 sub-indicators need to be developed into concise, measurable, and actionable performance measures.The ENJC surely recognizes this.The popular mnemonic acronym "SMART" has been used to guide the setting of goals and objectives, and the formulation of performance targets, in the fields of management for more than 50 years. 52The following are the most common requirements of the SMART criteria: Relevant: links to what matters, and counts what counts, in a meaningful and worthwhile way (i.e., consistent with visions, values, and strategic goals).
Time-bound: includes a time-frame for accomplishment (e.g., six months or one year).
The final selection of measures needs to meet these SMART criteria.They are not met by any of the ENCJ indicators in their current iteration.Most are merely a restatement of the aspects, elements, or issues of judicial independence and accountability.Similarly, in many cases, the sub-indicators are little more than a restatement of the indicators (e.g., code of judicial ethics, an indicator, is made into the sub-indicator, availability of code of ethics, with only the words "existence of" or "availability" added).

Management, not just measurement
ENCJ and its Members should aspire to manage judicial independence, accountability, and transparency, not just measure performance in these areas.Paying attention to the management of performance --such as the delivery of the "right" indicators to the "right" people, at the "right" time and the "right way" -will pay great dividends even in the early stages of the development of indicators and measures.
Successful leaders and managers see performance measures and indicators as powerful tools to communicate strategy and to change the behavior of individuals and groups.The best performance measures are actionable.Focusing on the management of performance, even in the early stages of identifying the right measures, by ensuring that identified measures are actionable and will lead to better performance measurement as well as better performance management.A surefire way of undermining the building of a successful PMM system is to create measures that are too abstract and poorly defined.Users of measures cannot improve performance if they do not understand what a measure means and what specific steps they may need to take to make use of the measure.

Key requirements and phases of development of indicators
Developing the right performance measures for an individual justice institution or an entire country's justice system, and making sure that they are used effectively, can be translated operationally into three overlapping and interdependent key requirements and corresponding phases of development outlined below.These three requirements and phases of development are described in sequential fashion.In practice, they are iterative.The third requirement and phase, the "right" use, for example, is likely to need addressing early in the development process of performance measures to ensure that judicial leaders and top management are engaged early and remain so throughout the development of the PMM.While it may be frustrating for the developers, it is inevitable that the early question of the "right" measures will need to be answered in the context of satisfactory answers to the questions of the right use.
• Right Measures -Identifying and developing the right performance measures is much more than simply identifying and naming them.They must be painstakingly designed and developed.As described in preceding section, the right measures are SMART --specific, measurable, attainable, relevant, and time-bound.
• Right Delivery and Distribution of Performance Data -Ensuring that the right measures are delivered to the right people, at the right time, and in the right way.Increasingly, this is done by information technology -including performance dashboards, business intelligence, and data visualization applications -that let users view critical performance information at a glance, and move easily through successive layers of strategic, tactical and operational information on a self-help, on-demand basis, allowing them to spot patterns, anomalies, proportions, and relationships that they would otherwise miss. 53 • Right Use -Adopting, implementing, and integrating the measures of performance, as well as the delivery system and distribution system (e.g., performance "scorecards" and "dashboards"), with key management processes and operations, including budgeting and finance, resource and workload allocation, strategic planning, organizational management, and staff development.

Numbers should support but not replace judgement
The performance measurement process uses numbers to provide an understandable and comparative result, but it is ultimately not about the numbers.It is about perception, understanding, and insight to inform better decision making.Even strong advocates of numbers and mathematical computations aver that numbers should support but never replace our judgement. 54Ultimately, it is not the measure itself that is important, but rather the questions outlined below that it compels us to confront.This part of the recommendation may appear nonsensical and at odds with the essence of PMM.It is not.If you are asking the wrong questions or don't act in response to the right questions, the answers really don't matter.• Baseline Performance.What is the state of judicial independence and accountability today?Where is the judiciary, a court system, or an individual judicial institution starting from?What is the current performance level compared to established upper and lower "controls" or boundaries (e.g., performance targets, objectives, benchmarks and tolerance levels)?
• Trends.How well is the judiciary, a court system, or an individual judicial institution performing -trending --over time?Is performance better, worse, or flat?
• Variability.How much variance in performance is there over time and across various dimensions?Does performance depend on the location or division of the judiciary, a court system, or an individual judicial institution, case type, or some other variable?Are there any positive deviants or "bright spots" that might serve as examples or models?Where are the "trouble spots" that demand immediate attention?
• Performance Patterns.Are there any apparent predictable regularities?
• Analysis and Problem Diagnoses.Why are changes or variations in performance occurring?What may have happened that coincides with performance decline or improvement?Where is the problem area?What are some credible explanations?
• Planning.Given past performance, what can one predict and plan?What is the judiciary, a court system, or an individual judicial institution doing to improve or maintain performance levels (planning and strategy)?What actions, strategies, or initiatives should be started, continued, or stopped altogether as a result of what the measure reveals?What should be done to improve poor performance, reverse a declining trend, or recognize good performance?
• Goals and Targets.What broad and specific performance goals, objectives, and specific performance targets should be set or reset?

Conclusion -the politicization of the judiciary
The more one knows, the more one will be able to control events.

Sir Francis Bacon, 1597
The preceding sections explored risks to judicial independence and accountability at two levels: decisional independence (e.g., when a judge perfunctorily places a court's imprimatur on decisions made by the other branches of government) and institutional independence (e.g., when the executive branch meddles improperly in judicial selection, retentions, and assignments).Today, there are transnational developments of risks at a third level -governance independence --that are more troubling than risks to decisional independence and institutional independence.Some see these developments as a retreat from democracy and the rule of law55 and a movement toward "illiberal democracy" around the word that pose an existential threat to the judiciary as a coequal partner in government?In many countries, lip service is paid to democratic principles in constitutions that are ignored in practice.
Strongmen who rule in authoritarian Russian-style oligarchies amid corruption and political chaos in countries that once looked like they were democratizing today seem to be moving in opposite directions.This is occurring not only in Russia, Egypt, and in Turkey, but also in my country, the United States, long thought to be the engine room of democracy. 56As they cling to power, embattled strongmen elsewhere, like Daniel Ortega in Nicaragua and Nichola ś Maduro in Venezuela, show no inclination towards sharing power with others, never mind their own judiciaries.
Closer to home for the councils of the ENCJ is Hungary's Prime Minister Viktor Orban who touts the virtues of illiberal democracy and who, no doubt, is willing to dispense with such constitutional niceties as an independent judiciary and the separation of powers because, he might argue, judicial independence impedes economic growth by making it harder for his government to respond quickly and flexibly to changing circumstances and national crises.Other leaders in Poland, the Czech Republic, and Austria, and further afield in Eastern Europe in the Ukraine and Moldova, all seem to be following Orban's playbook.
Individual judges are, of course, constrained in their positions as judicial officers to respond to such existential threats to democracy.
Entering the political arena in their judicial capacity risks violating the very principles they pledged to uphold.However, this is an area where ENCJ and individual councils can do much good.For example, they can assume an invaluable role by leveraging ENJC's influence as a multilateral organization and by, for example, referencing treaties that promote judicial independence and the separation of powers.ENJC and its partners can give voice and agency to judges in support of judicial independence, separation of powers, and the rule of law in their roles and capacities as representatives of councils of the judiciary that otherwise might not be possible.
ENCJ and national councils could take several approaches to advocacy of judicial independence that judges in their official capacity as judges cannot easily take.Such efforts can establish norms and set up obstacles to authoritarian forces that flaunt judicial independence.For example, they could urge their countries to redouble commitments to be subject to the jurisdiction of international tribunals like the International Criminal Court at the Hague.They could encourage international cooperation and exchange of information through various international treaties that promise to curtail strongmen in their exercise of power at the expense of judicial independence.Finally, their support of emerging technologies like blockchain and artificial intelligence that decentralize fundamental rights away from autocratic politician also have the potential to protect judicial independence. 57ese approaches can be anchored in PMM, specifically ENCJ's development of indicators of judicial independence and judicial accountability.PMM is not just a grand diagnostic or accounting exercise but also an instrument of power and control.There is today no longer much doubt whether good measures and indicators of performance can help achieve worthy ends.The question is whose values and vision of justice and court excellence they advance. 58Being able to "control events" (in the words of Sir Francis Bacon quoted at the beginning of this Conclusion), by determining the single version of the "truth" about judicial independence and accountability, by determining patterns and trends, and by measuring successes and failures, is as much an exertion of authority, power, and control, as it is a diagnostic exercise.
ENCJ classifies a total of 22 indicators and 64 sub-indicators --13 indicators and 33 sub-indicators of judicial independence and nine indicators and 31 sub-indicators of judicial accountability --into four categories: (a) "objective" or "formal" independence and accountability; (b) "subjective" or perceived formal independence and accountability; (c) individual judge's judicial independence; . (The 22 indicators in in the tables are identified only by name and only by the number of subindicators associated with each of the indicators that appears in parenthesis after each indicator.) 20 W. Gates (2013), My Plan to Fix the World's Biggest Problems, Wall Street Journal, January 25, 2013, http://www.wsj.com/articles/SB10001424127887323539804578261780648285770 [accessed 26 July 2018].21 J. Doerr (2018).Measure What Matters: How Google, Bono, and the Gates Foundation Rock the World with OKRs ["Objectives and Key Results"], New York: Penguin Random House, 2018, p. 215.22 van Dijk & Vos, supra note 2, p. 9-12 devote a large portion of their article to the identification and explanation of the sub-indicators.With the exception of noting the numbers of sub-indicators associated with the classification and categorization of the 22 indicators in Section 3, this article does not delve into the substance of the sub-indicators.
." 40 38 See, for example, European Commission for the Efficiency of Justice, (CEPEJ).Measuring the Quality of Justice: https://rm.coe.int/16807477e4accessed August 1, 2018].39 S. Voigt, S. & N. El-Bialy (2014), Identifying the Determinants of Judicial Performance: Taxpayers' Money Well Spent?European Journal of Law and Economics, 11 December 2014, p, 41, finding that rates of case resolution are not a function of court budgets (i.e., higher budgets will not necessarily "buy" increased efficiency) and that resolution rates are negatively correlated with the existence of judicial councils https://link.springer.com/article/10.1007/s10657-014-9474-8[accessed 2 August 2018].40 Van Dijk and Vos, supra note 3, page citation needed 15 the International Consortium for Court Excellence's Global Measures of Court Performance, 43 which is part of the International Framework of Court Excellence of the International Consortium of Court Excellence; (b) the CourTools and Appellate CourTools 44 of the National Center for State Courts; and (c) the 12 sub-goals and 21 indicators of Goal 16, the justice goal,