ANNOUNCEMENTS: OEB YARDSTICK TASK FORCE REPORT

REPORT OF THE
ONTARIO ENERGY BOARD
PERFORMANCE BASED REGULATION
YARDSTICK TASK FORCE

MAY 18, 1999

Prologue

The Ontario Energy Board (OEB) is proposing performance-based regulation (PBR) for the electricity distributors in Ontario. The OEB’s approach in developing a PBR framework for electricity distribution is to involve the stakeholders through task force efforts. As such, the OEB set up four PBR task forces consisting of volunteer stakeholders to examine the following: cap mechanisms, yardstick grouping, implementation issues, and distribution rates. The task forces had a total of 83 members representing various electricity distributors, gas utilities, customer groups, and special interest groups.

The Task forces were formed in mid-January and worked on the assigned tasks for approximately 3 months. The task force meetings were co-managed by OEB consultants Michael King and Frank Cronin of PHB Hagler Bailly, who also provided the task forces with technical expertise on PBR and restructuring issues in general.

To address the diversity of scope and the large number of emerging issues, working groups were formed within the task forces. Each working group produced reports which Board staff has collated into the task force reports.

All four task forces ran into concerns that led to the common proposal that the OEB should allow for a regulatory transition period. The regulatory transition period would allow utilities the opportunity to meet restructuring requirements without rigorous regulatory impositions, and allows for the collection of consistent and robust baseline data for PBR. The task forces agreed that a three-year first generation PBR plan should apply for the transition period to avoid gaming opportunities, in anticipation of PBR, during the transition period.

The first generation plan will have sophisticated incentive parameters (i.e. industry specific price indexes and productivity factors) developed from data collected from the electricity distributors and will also have risk mitigation terms (i.e. earnings-sharing). However, inconsistencies in data and utility practices precluded the implementation of yardstick groupings and a complete set of comprehensive performance standards applied to all distributors for the first generation plan.

The OEB would like to express its sincere appreciation for the conscientiousness of the task forces members and the time expended on the task force efforts, as well as its admiration for the collaborative attitude demonstrated by each of the task forces. Board staff and their consultants are confident that the outcomes of the discussions by the task forces will facilitate the production of a draft Board PBR Rate Handbook and result in a fair and practical PBR framework for the electricity distributors in Ontario.

OUTLINE

INTRODUCTION

YARDSTICK AND ADJUSTMENT MECHANISM

INITIAL RECOMMENDATIONS
APPENDIX A - Yardstick mechanism Data Requirement Survey

Return to Yardstick Announcements

PARTICIPANTS

NAME	TITLE	ASSOCIATION

Richard Archdekin	General Manager	Wasaga Beach HEC
Margaret Brown	Administrative Services Supervisor	Pickering Hydro
Chris Buckler	Executive Director, Marketing	Mississauga Hydro
Stephen Cartwright	Manager, Business Development	Enbridge Consumers Energy
Guy Cuff	General Manager/Chief Engineer	Kanata Hydro
Fred Druyf	Manager, Finance	Waterloo North Hydro
Arthur Emmet	General Manager	Nepean HEC
Tim Fryer	Director of Finance	Collingwood PUC
Tim Lavoie	Coordinator, Management Information	Great Lakes Power Ltd.
Jerry McIntyre	Director	Municipal Electric Association
Brian McKerlie	Director, Strategic Utility Planning	Municipality of Chatham-Kent
Ed Robertson	Consultant	Econalysis Consulting Services
David Simmons	Director of Operations & Engineering	Sarnia Hydro
Colleen Walwyn	Advisor, Regulatory Affairs	Toronto Hydro
David Wills	General Manager	North Bay HEC/UCEA
Ted Wojcinski	Manager Operations	Vaughan Hydro
Larry Yanchuk	Supervisor, Engineering Projects	Sudbury Hydro

TASK FORCE MANAGEMENT TEAM

Co-Manager	Michael King	Vice-President, PHB Hagler Bailly Consulting, Inc.
Co-Manager	Frank Cronin	Senior Advisor, PHB, Hagler Bailly Consulting, Inc.
Coordinator	Judy Kwik	OEB, Research & Policy Analyst
Co-coordinator	Edward Sweet	OEB, Regulatory Officer

INTRODUCTION
The objective of the Yardstick Grouping Task Force is to identify, quantify and assess the characteristics determining the number of yardstick groups and assignment of utilities to a group.

In attempting to meet this objective the task force discussed the following issues:
- Identification of significant but uncontrollable factors affecting utility performance;
- Qualitative and quantitative determinants;
- Sufficiency of existing data on determinants for grouping purpose;
- Annual rate adjustment; and
- Yardstick methodologies.
The intent is for Board Staff to draw on the task force’s discussions and recommendations in preparing an Ontario Energy Board draft rate handbook for the electricity distributors. The Board will hold a public consultation on the draft rate handbook in the summer of 1999, with issue of the rate handbook expected in the fall of 1999. The distributors will then be in a position to file evidence according to the guidelines contained in the rate handbook for a rate order establishing unbundled PBR rates prior to the introduction of open access expected in the fall of 2000.

YARDSTICK AND ADJUSTMENT MECHANISM

The terms of reference of the Yardstick Task Force were to:

identify and evaluate yardstick characteristics,
determine the number of yardstick groupings and the assignment of utilities to groups,
assess the specific benchmarks and their implications, and
recommend policies and procedures for providing periodic updates to the OEB.

In principle, Yardstick PBR is particularly well suited to electrical distribution in Ontario, with its large number of local distribution companies (LDCs). Under yardstick regulation, utilities are placed in peer groups that exhibit similar characteristics. The object of Yardstick PBR is to provide incentives to LDCs to seek economies of scale and scope, and to adopt management practices that will drive costs lower, using the peers within their grouping as a standard. Yardstick PBR avoids the need for extensive and expensive cost of service studies and evidentiary hearings associated with both price and rate cap regulation. Refer to the section, PBR Benchmarking and Efficiency Methodology, below for a discussion of the application of yardstick methodologies.

The Task Force met on a number of occasions to determine appropriate benchmarks and utility groupings for the implementation of yardstick regulation to LDCs. Initial efforts focused on developing a data base to allow consistent analyses of peer groupings and rate adjustment mechanisms. The survey instrument is attached in Appendix A. It considered the adoption of a number of specific benchmarks which would be calculated on a per customer basis, including:

operations and maintenance costs,
billing and collection costs,
administration costs,
capital expenditures,
losses, and
ROE.

However, the Task Force identified several problems and impediments to the application of yardstick regulation at this time. Determining those factors that are within the control of management is a challenge. External factors such as customer density, topography, prevailing weather patterns, historical distribution voltages and age of plant all significantly affect cost of service. Furthermore, baseline data that would support analysis is inconsistent or non-existent among utilities. A lack of consistent accounting standards and practices also makes grouping and comparisons of utilities difficult. The Task Force has concluded that, until consistent accounting practices are adopted and supporting baseline data is collected, it will be very difficult to adopt truly effective Yardstick PBR, or even to determine appropriate yardstick groupings (See Section 2).

The Task Force agreed that Yardstick PBR is ideally suited to all but the largest LDCs in the long term. The largest LDCs should probably be regulated using price or revenue caps, but the measures of efficiency and incentives for improvement should be applied in a similar manner to all LDCs.

However, in the near term, the Task Force recommends that a form of price cap PBR be adopted as a first generation PBR for all LDCs, and that appropriate accounting and data collection standards be established to support the adoption of comprehensive yardstick PBR in the second generation. Preliminary recommendations are documented in Section 3. With consistent baseline information, the application of sophisticated analysis techniques such as data envelopment analysis (DEA) can be applied to establish the metrics required to support the adoption of Yardstick PBR (see Section 4).

The Task Force then turned its attention to the question of the form of adjustment mechanism to apply in first generation PBR.

The basic PBR incentive formula is:

P =

PI - X
Where:

P = Change in price from base year to year 1

PI = Change in the input price index from base year to year 1
X = productivity factor

Under yardstick regulation, utilities are grouped according to similar and comparable characteristics, and the average cost structure of the group is taken as the starting point for incentive regulation. As indicated above, we do not have sufficient data to group utilities at this time.

Two approaches to incentive regulation were considered in addition to the application of a single X-factor to all utilities. The first approach, which the Task Force termed the Total Cost Differential approach, makes use of the cost distribution among utilities. The range of costs per customer among Ontario LDCs is very broad and approaches a normal distribution. If one assumes that cost per customer is a direct measure of utility efficiency, then those utilities with costs below the mean would be more productive than those with costs higher than the mean. It can be argued that the higher cost utilities should be required to achieve a higher X-factor in order to earn the same approved rate of return than lower cost utilities that have already achieved efficiencies. Otherwise, the more efficient utilities will be penalized.

The second approach to incentive regulation that the Task Force termed the Earnings Differential approach, would tie a utility’s productivity factor (X) to its approved rate of return. If it accepted a higher X-factor, a utility would be permitted to earn a higher rate of return, in essence encouraging increased efficiency and sharing the cost savings with the customer. The Implementation Task Force has been examining the issue of earnings-sharing. For example, a 2% X-factor might correspond to a 10% ROR, and a 3% X factor to a 12% ROR. For each combination of X-factor and approved ROR, there would be a deadband within which the LDC could earn a variable rate of return. Care would have to be exercised to ensure that appropriate signals are incorporated into the scheme for utilities whose rates of return rise above the deadband. While Earnings Differential would encourage increased efficiencies, it may also favour inefficient firms that have greater room for improvement.

The Task Force also recognized that there is an additional incentive inherent in all PBR schemes. The price index (PI) is based on averages. The cap mechanism task force has recommended the use of input price index specific to the electricity distribution industry in Ontario. Any company that can “beat the averages”, by lowering its input costs (labour, materials etc.) will benefit through higher returns.

The following table summarizes the alternatives the Task Force considered:

Adjustment Mechanism	Option 1	Option 2	Option 3	Option 4
Single X factor for all utilities	Yes	No	No	No
TCD – Higher X for higher cost utilities	No	Yes	No	Yes
ED – Higher X for higher return	No	No	Yes	Yes

All three approaches have advantages and inherent problems, at least under current circumstances where no baseline data exists to define the relationships between productivity and cost. A single X-factor is simple to apply but it provides no additional incentive for utilities to improve relative to their peers, nor does it recognize that some utilities are already more efficient than the average.

Total Cost Differential PBR rewards low cost utilities, some of which may be inefficient, while penalizing higher cost utilities that may be determined to be very efficient, once all factors are accounted for. For example, a utility that reduced its operating costs by allowing its plant to deteriorate ought not be rewarded. Without an appropriate grouping mechanism, all we can conclude at this time is that highly productive firms will have cost structures that are lower than otherwise equivalent unproductive firms.

Earnings Differential PBR rewards those firms that are able to achieve the greatest increases in productivity, and inefficient firms have the greatest room for improvement. On the other hand, this form of PBR gives an effective signal to maximize efficiency improvements to the benefit of the customer and experience may demonstrate that more productive LDCs have greater capability to achieve even higher efficiencies.

The level of acceptance by LDCs of the three forms of adjustment mechanism can be predicted to some degree, as follows:

Single X-Factor

The single X-factor would likely be more acceptable to utilities with higher than average cost structures than those with lower cost structures. Low cost utilities will argue that they are already efficient and should not be required to achieve the same percentage improvement as high cost utilities. The single X-factor would also be attractive to many smaller utilities that will prefer its simplicity.
Total Cost Differential

The total cost differential approach would be favoured by low cost utilities, and likely opposed by higher cost utilities that could take exception to its inherent assumption that cost is the sole measure of productivity. They would argue that it is premature to differentiate X factors solely on a cost basis, and that other important and statistically significant measures of efficiency should be taken into account.
Earnings Differential

The earnings differential approach will find its greatest favour among utilities that believe they can increase productivity to the greatest extent. While some of these utilities may be among the most inefficient, it may be equally true that the greatest increases in productivity will be achieved by those utilities that have already demonstrated their ability to be productive. The initial reaction from many low cost utilities to Earnings Differential could be somewhat negative because they may conclude that the methodology will penalize them for their relatively lower cost structures.

The Task Force originally supported the Total Cost Differential approach to PBR, on the basis that it would recognize the results that lower cost utilities have achieved. After lengthy discussion at its last meeting however, the Task Force expressed greater preference for the Earnings Differential approach, for the following reasons:

Unquestionably, cost is not the sole measure of efficiency. Total Cost would tend to penalize efficient firms that have justifiably high costs.
Earnings Differential is applied to all LDCs in the same manner.
Earnings Differential also provides options to all utilities, including by default a lower X, lower ROR alternative, which may be attractive to many smaller LDCs.
Earnings Differential provides an incentive to strive for greater productivity and offers a means of sharing the resulting gains between the customer and the shareholder.

Recommendations

The ideal form of Yardstick PBR would be based on statistically significant baseline data, would account for external factors beyond the control of the firm, and would combine the Total Cost Differential and the Earnings Differential approaches to reward highly efficient (low cost) firms, while also providing appropriate incentives for all firms to maximize efficiencies. That should be the goal of the second generation of PBR, but it is impossible to achieve at this time. Consequently, the Task Force recommends:

That a Uniform System of Accounts be implemented and that data to support specific yardstick metrics be collected on a consistent basis for a sufficient period of time to provide accurate comparison of efficiency and effectiveness levels among LDCs.
That a form of price cap PBR be adopted as a first generation PBR for all LDCs.
That an Earnings Differential adjustment mechanism be considered for application in the first generation of PBR.
That the OEB continue to work with LDCs to develop the application of more sophisticated techniques such as Data Envelopment Analysis to Yardstick PBR in Ontario.
That Yardstick PBR be adopted for all but the largest LDCs in the second generation of PBR, as soon as there is sufficient supporting data available to make meaningful comparisons among LDCs.

INITIAL RECOMMENDATIONS

The Task Force split into two work groups: one on rate adjustment mechanisms and the other on yardstick groupings. The final outcome of their deliberations was presented above. The discussions leading to this approach are presented in Sections 3.1 and 3.2, while benchmarking and efficiency methodologies are described in Section 3.3.

3.1 Initial Rate Adjustment Mechanism Recommendations

These recommendations and considerations about midway through the Task Force process helped redirect the direction toward a first generation rate cap mechanism for all utilities.

The PBR Rate Adjustment Mechanism Work Group of the PBR Task Force considered the following:

Standards of service and quality of data vary widely between medium MEUs and between medium MEUs and large MEUs. Therefore, the new USofA must be implemented and in place to gather accurate activity based costing data on MEU practices, before a comprehensive yardstick regime can be implemented.
The new USoA must be implemented and in place to gather accurate activity based costing data on MEU practices, before a comprehensive yardstick regime can be implemented.
The OEB must work with MEUs to determine minimum and acceptable service levels and what can be included in LDC rates. In particular a level playing field between what larger utilities like OHSC and what MEUs include in rates must be addressed.
Once USoA accounting has been in place for enough time to collect data, say 3 years, the OEB should specify activities required to meet the standards set by the applicable codes and a regulatory method that addresses combinations of inputs and outputs that can vary but still reflect efficiency.
In the interim, MEU rates could be regulated based on yardstick groupings. For example, these recommendations and considerations about midway through the Task Force process, helped redirect the direction toward a first generation rate cap mechanism for all utilities.
Yardsticks could be set to include Operations and Maintenance, Billing and Collecting, Administration, Capital Expenditures, Losses and Return on Shareholder Equity. The result will be a bundled Allowable Income for the MEU.
MEUs could be grouped by size based on number of customers served: 0 to 5,000, 5,001 to 20,000, 20,001 to 50,000, over 50,001 to Mississauga Hydro, Toronto Hydro, OHSC.

MEUs could be permitted to increase Allowable Income based on the following formula:
AI = AI Yr N -1* (1 + (RPI Yr N –YPG Yr N-1* ( MEU AI Yr N-1 /Yardstick AI Yr N-1)))

AI = Allowable Income
RPI = Regulation Price Index
YPG = Yardstick Productivity of Group
Yr N = Rate Year
Yr N-1 = Previous Year

An example is as follows :

Yardstick O&M + B&C + A	= $165 per customer per year
Yardstick Capital Expenditures	= $150 per customer per year
Yardstick Losses = 4%	= $55 per customer per year
Yardstick ROE = 10% on $1500 after tax	= $150 per customer per year
Permissible income = Total of above	= $520 per customer per year

Assume RPI = 1%, YPG = 2%

If Utility A is at $460 and Utility B is at $660 then for the next year

Utility A’s target = $420 * (1+(.01-.02(420/520))) = $420 * (1+(.01-.016)) = $420 * 0.994 = $417.50
Utility B’s target = $620 * (1+(.01-.02(620/520))) = $620 * (1+(.01-.024)) = $620 * 0.986 = $611.32

This formula could be used for the first 3 Years of OEB Regulation and considered a 1st Generation PBR to be superceded by a more appropriate model. Exemptions could be considered only for MEUs who can demonstrate bona fide special circumstances.
The OEB should provide rate caps or guidelines for transition costs, such as incorporation costs, and new services, such as retail billing systems, to assist MEUs in determining reasonable levels.

An Example of Allowable/Permissible Income Model based on depreciation. It shows the inadequacy of any yardstick model we consider with today’s data.

# of Utilities in $10 bands	*OMBCA - OR + Depreciation + 10% (Eq-CC) +**	# of Utilities in $10 bands	*OMBCA - OR + Depreciation + 10% (Eq-CC) +**
210	1	350	2
220	2	360	4
230	2	370	2
240	3	380	2
250	4	390	1
260	2	400	2
270	6	410	1
280	4	420	1
290	9	430	1
300	3	440
310	4	450
320	4	460
330	3	470
340	1	480	1

OMBCA = Operations, Maintenance, Billing, Collecting and Administration
OR = Other Operating Revenue
Eq = Utility Equity
CC = Total Capital Contributions

3.2 Initial Discussions of the Yardstick Grouping Work Group

The group initially discussed their objective, which at this point in time is to investigate the groupings of those utilities within the medium sized utility category, 5000 to 50000 customers. They referred to the data that has been supplied by the OEB, which unfortunately is not complete as only 42 of the 69 utilities have responded. The group agreed that although it was not important to have all of the data right now, it was very important to establish guidelines for the formation of the groupings, that would allow for easy update as the other utilities finally responded.

After discussing the data further the group agreed to choose some key factors that would be useful in starting the process of relating the utilities to each other. These factors were:

Density Related
1. Number of metered customers
2. Number of metered customers per km of Overhead Lines
  Number of metered customers per km of Underground Lines
3. Number of metered customers per km of Serviced Area
Revenue Related
1. Total Revenue per total number of metered customers
  Total Kilowatthours per total number of metered customers
2. Load Factor
3. Distribution Losses.

Factor 1 was judged to be the best starting point for ranking the utilities. Number of Customers was deemed to be the most important factor, at this stage of the process. We related to the fact that the MEA had grouped utilities on this basis over the years, identifying them as large, medium and small. Essentially the task force has done the same by establishing and small and large category and then deciding to try to group the medium size of 5,000 to 50,000 customers. There was some discussion about using customers per square kilometer but we finally settled on using Factor 1.

Then one of two different methods could be used to continue the process. Out of the five remaining factors another one would be used to provide a ranking for each of the utilities. Based on the rankings either you could look for relationships immediately or you could continue with the rest of the factors and determine a ranking for each of these categories. It may be best to do all rankings and then look for relationships because it would be hard at this point in time to determine if the remaining five factors are of the same importance.

Inevitably the group believes that using this process would allow some breakout into groups. It is our belief that for now the 69 utilities should be broken down into only a small number of groups, maybe anywhere between two or five. This small number will allow for an average of say 15 to 20 utilities in the same group. Hopefully this will reduce the possible biases that may be created by one or two utilities being in the wrong grouping.

Also factors such as geographic location, which would open up a myriad of related issues like soil conditions etc. that can affect the cost of operation, have not even yet been considered, so hopefully the larger grouping size will mitigate the adverse affect of this.

Another reason for large groupings is that this process is eventually only to apply to the Wires Company or Local Distribution Company. We recognize that we are working with bundled data in trying to establish guidelines of an unbundled environment. Hopefully the larger groupings will again mitigate any possible effects of this.

Finally the group believes that this will be a starting point. The groupings can be put together and the rate mechanism sub-committee makes their recommendations. The group believes that once this is completed then we can examine the affect of being in one group or in another. If there is dramatic affect between groupings then further study will be necessary to insure that the groupings are accurate.

3.3 Benchmarking and Efficiency Methodology

Yardstick Regulation Theory

Yardstick regulation allows utilities to adjust their rates so that they are comparable to a group of other utilities. Intuitively, this has some appeal since a competitive market would make comparisons to competing firms. In an intensely competitive market, the prices and quality of service should gravitate to the standards set by the very "best" firms. Utilities that exceed the performance of the best utilities would be rewarded while utilities that fell short of exemplary performance would not realize any benefits and may be required to compensate their customers for sub-standard performance.

The key to a successful yardstick regulatory program is consistent and accurate statistical performance data from utilities to drive a benchmarking process. The benchmarking process determines which are the ”best” firms and which are the “best” practices. The yardstick process suggested for Ontario LDCs would benchmark utility customer costs within specified groupings of utilities.

Problem with Yardstick as Applied to Ontario LDCs

The yardstick process, as proposed for Ontario LDCs, generally produces a “bell curve” of costs, for each defined group, with the assumption that the “average” is the best or acceptable practice. This is not necessarily a correct assumption. With this yardstick process, each utility’s productivity will be measured against a single output (cost/customer). This is known as a “partial productivity” measure and if viewed in isolation these indicators can be misleading. To determine an organisation’s “overall productivity” requires a more comprehensive productivity measure.

A Comprehensive Benchmarking Solution to Efficiency Analysis

It is necessary to examine the benchmarking process that is used to determine productivity and drive the efficiency factor in the PBR process.

Numerous regulatory bodies throughout the world have had to deal with benchmarking and productivity issues. A number of studies have been undertaken in relation to electricity distribution activities, specifically examining the distribution production function. A common modelling methodology used in these studies is Data Envelopment Analysis (DEA). This method can be used to characterize the complete Ontario LDC production environment including operations, maintenance, billing , collecting and administration(OMBCA), losses, capital expenditures or any other chosen parameter.

Data Envelopment Analysis

DEA is a mathematical programming technique that calculates the efficiency of an organisation relative to the firms that are assessed as the most efficient in their use of “inputs”. These “best practice” firms define what is called an efficiency frontier. They use the minimum quantity of inputs to produce a given level of output. Firms that use different combinations of inputs to produce different combinations of outputs can be frontier efficient, provided they represent the best practice for that particular combinations of outputs.

DEA:

Is capable of handling multiple outputs and inputs, allowing efficiency measures to be calculated that capture all aspects of performance (superior to partial measures of performance). Inputs and outputs comprising the production function should be:
- quantifiable
- consistent across organizations
- relate to key objectives of the distributor
- capture as many aspects of the distributor’s production relationship as possible
Can readily produce a range of efficiency scores reflecting different sources of efficiency, including pure technical efficiency and scale efficiency;
Measures of efficiency are against best practice, not average practice;
Relevant performance peers are identified; and,
Explicitly accounts for a range of the operating environment characteristics of the distributors that may impact on efficiency, but are outside the control of managers (ie. limitations on the ability of a firm to reach the “efficient” frontier).