Investigation of Contemporary Performance Measurement Systems for Production Management of Renovation Projects

Renovation projects exhibit complex characteristics due to the presence of constraints that lead to cost and schedule overruns. Numerous researchers have concluded that the performance of renovation projects is typically lower than that of new construction projects.This paper discusses the initial phases of a research conducted atMichigan State University, which focused on developing a framework for production management of renovation projects.The emphasis of this paper is on the findings from literature review and interviews, pertinent to performancemeasurement in renovation projects that led to the framework development.However, the framework development and the framework itself have not been discussed. This paper primarily addresses two questions: (1) what are the complexities of renovation projects that lead to underperformance in cost, time, and quality? and (2) what are the limitations of state-of-the-art construction performance measurement systems for managing production in renovation projects? Interviews of 10 construction companies were conducted to identify current practices of production management in renovation projects. This research observed a lack of a formal production management process in renovation projects, with a limited use of performance measurement systems. The research identified essential attributes for avoiding schedule and cost overruns on renovation projects.


Introduction
Renovation projects represent about one-third of the total expenditure spent on construction projects [1,2].In the late 1990s, a study conducted in the United States concluded that renovation projects including remodeling, reutilization, and rehabilitation constituted up to 50% of the total construction budget expended [3].In the commercial and institutional sectors, the annual expenditure on renovation projects has increased from close to $20 billion in 1992 to approximately $120 billion in 2007 [4][5][6][7].This increasing investment in renovation projects has been attributed to the growing needs of public owners and governmental institutions, which represent a major clientele for renovation projects, in order to maintain and upgrade their infrastructure and built facilities [8].Moreover, renovation projects mostly offer an economically viable alternative for public owners in comparison to new construction, as multiphase renovation projects tend to eliminate the need to close owner's operations during construction [2,3].
While renovation projects are a growing sector of the construction industry, its performance is typically lower than that of new construction projects in terms of time, cost, and quality [3,8,9].Previous research has concluded that renovation projects, unlike new construction projects, involve considerable risks and uncertainty in existing conditions that adversely impact the project performance [1,8,10].Renovation of an operational facility poses additional constraints on its construction process, which if not considered during project planning processes could lead to project underperformance [1].
However, this issue of project underperformance that plagues renovation projects is extensively investigated for new construction, as illustrated by the Sixth Annual Owner Survey conducted by the Construction Management Association of America [11].According to this survey, building trust and integrity in construction processes was the prime concern of the surveyed owners.The owners stated that trust and integrity are necessary for procuring successful projects that meet the desired performance levels of cost, time, and quality.In addition, one of the greatest challenges for delivering successful projects lies in controlling the inefficiency in construction processes and not the cost of labor, materials, and equipment [11].
In 1998, as a vital contribution to the U.K. Construction Best Practice Program, the Egan Report titled "Rethinking Construction" emphasized that contractors need to reduce high levels of inefficiency and waste in construction processes.According to the Egan Report, this was the principal concern that owners had about the U.K. construction industry.This report underscored the need for development of performance measurement of both the quality and efficiency of construction processes for building strong relationships between owners and contractors [12][13][14].
Both CMAA's owner survey and the Egan Report emphasize the significance of construction performance measurement, yet, with the high levels of complexity, uncertainty, and speed in renovation projects, the investigation of "what is to be measured" becomes extremely difficult [1,8,10].Moreover, previous studies have concluded that there are numerous constraints in executing renovation projects which make traditional project planning and monitoring techniques inappropriate for measuring and controlling the performance [1].Therefore, this study investigated the application of stateof-the-art performance measurement systems in identifying the causes of inefficiencies in renovation projects and assessing and improving their performance.This investigation was the initial phase of a research conducted at Michigan State University in which the overall goal was to develop a framework for production management of renovation projects to improve schedule development and execution.

Research Method
The overall goal of this research was to develop a framework for production management of renovation projects.To achieve this goal, the authors first investigated the constraints of renovation projects that lead to cost and schedule overruns, the contemporary practices of performance measurement, and the role of production management in integrating performance measurement with the constraints.As shown in Figure 1, this investigation was done through literature review and interviews.This was followed by a thorough analysis of contemporary performance measurement systems to understand where they fail to assess the impact of constraints on production assignments of renovation projects.This impact could be in terms of the cost, schedule, or quality of production assignments, which eventually lead to overruns at the project level.In addition, the analysis resulted in essential attributes that a performance measurement system should possess in production management of renovation projects.

Performance Measurement in Construction
In general, performance measurement has been recognized as a vital aspect of project controls for providing the basis for monitoring and controlling construction activities by bringing to surface their inefficiencies [15].The adoption of performance measurement in current construction practices has been also driven by the widespread dissemination of total quality management (TQM) principles and ISO9001: 2000 standards.TQM was considered a potential solution to increasing complexities of construction processes, competitiveness, and wasted efforts in reworks.Consequently, construction firms began investigating "what is to be measured" and "how is it to be measured" for achieving desired results [16].
A performance measurement system, whether in construction or any other industry, begins with the identification of a balanced set of indicators through which performance can be measured [15].Numerous studies have investigated adequate set of performance indicators for measuring the financial and nonfinancial aspects of construction processes that define their true performance.These studies have also focused on reviewing some of the state-of-the-art performance measurement systems being developed by collaborative benchmarking initiatives among construction companies across the globe.These state-of-the-art performance measurement systems were developed for the purpose of providing guidance in performance measurement of construction projects and establishing benchmarks for identifying best practices [17].
However, previous research has concluded that these globally developed performance measurement systems have limited implementation in construction firms due to their incompatibility with organizational capabilities.This has also been attributed to the complexity of construction projects being comprised of different participants in a temporary organization [18].Some of the systems also still focus on financial and contractual measures, ignoring those which are important to competitive success [13,14].In addition, past research efforts have not reached a consensus on the most appropriate set of performance indicators suggested by different systems [19].It has also been observed that these performance measurement systems find limited application in construction projects which are more complex, uncertain, and quick (CUQ) [17,20].Consequently, contractors still suffered from inefficiencies and the resulting inability to deliver projects to owners on time, at budget, and desired quality and value.The lean construction movement has revolutionized performance measurement by emphasizing the need to focus on the production system performance, as well as the project level [21].

Traditional Performance Measurement in Renovation Projects
Traditionally, renovation projects, and new construction projects, are evaluated by assessing achievements against project objectives of budget and schedule, with the measurement of cost variances (CV) and schedule variances (SV) as success indicators through the technique of earned value (EV) analysis [13,14].In the traditional project management approach, a corrective action is planned and executed only when the CV and/or SV return a negative value, which would mean that the project is over-budget and/or behind schedule, respectively [22].Thus, the traditional project control model is based on a reactive approach, which has been concluded to be a deficient approach of controlling projects [20,23].
The earned-value performance indicators, CV and SV, have been extensively criticized for their inability to indicate sources of problems as well as their low predictive value with respect to performance during the construction process [13,14].Being aggregate outcomes of a simple algebraic formula, these customary performance indicators of cost and time provide information about the results of a process and not the process itself.In other words, these performance indicators are incapable of establishing the cause-effect relationships between the process input and output [15].At best, the variance values provide a construction manager what a stock market price index provides a mutual fund manager: an overall sense of how the market (site) is doing but no indication of how an individual company (activity) is doing.Moreover, these lagging indicators do not support rapid decision making necessary for corrective actions, as the information retrieval usually gets delayed [18].
Additionally, in the case of renovation projects, reporting SV and CV are particularly not informative due to the presence of a number of renovation-specific constraints that make the establishment of cause-effect relationships even more difficult.As shown in Figure 2, it becomes an extremely complex task of investigating the negative deviations of CV and SV for a renovation project, as the source could be a single constraint or multiple constraints [10].One of the major renovation constraints identified is the uncertainty of unforeseen and varying conditions.This uncertainty relates to unknowns in preexisting conditions, which can adversely impact the performance of construction activities, especially if not accounted for in production planning.There are other constraints in renovation projects that add to this difficulty including space limitations, design coordination with existing conditions, traffic limitations, owner's operations limitations, pollution control, safety constraint, and limited capacity of existing utilities [10].
From a lean production perspective, schedule controls have been criticized for their inability to control and stabilize the workflow.Kim and Ballard [24] state that an EV analysis does not consider the work sequence, which results in managerial actions that typically increase the variability of workflow.In addition, an EV analysis does not provide any indication as to whether upcoming planned work can be performed as planned; hence, limiting the ability to control the project progress and the ability to identify appropriate corrective action before problems are encountered [2].Therefore, the trend in current research on performance measurement is toward identifying process-oriented indicators that provide relevant information for improving the ongoing processes to achieve desired performance levels [13,14].

Production Management in Construction
Ballard [20], one of founders of Lean Construction, stated that project control in the construction industry functions on a reactive model being focused on detecting cost and schedule variances rather than proactively dealing with the management of production at the site level.The emphasis has been on only the transformation aspect of construction processes assuming the workflow and value generation as inherent parts.Mitropoulos [25] stated that traditional project control methods in the construction industry are based on the thermostat model that identifies negative deviations from target levels and applies corrective actions to bring the project back on track.
Researchers in the Lean Construction arena have been focusing on developing appropriate production control systems that would conform to the integrated transformation, flow and value aspect for improving performance, and reducing waste.A major milestone was established by the introduction of the Last Planner System (LPS) of production control, designed by Glen Ballard in 2000.This system stressed the importance of formation of sound production assignments through communication with construction crews or those individuals or groups that implement the assignments.
The constraint analysis in the look-ahead process of the LPS represents a key process in establishing sound and feasible production assignments.The constraint analysis screens the assignments for the upcoming 1 or 2 weeks under possible constraints and makes the work ready by pulling necessary resources.The constraint analysis ensures that the production assignments are released for execution only when all the applicable constraints have been satisfied or removed.
There is agreement that production planning and control that considers constraints removal a priority and establishes upfront communication with construction crews allows for planning quality production assignments [20,25,26].By identifying and analyzing the potential constraints of upcoming work, realistic production assignments are planned that "can" be achieved by the crew instead of what "should" be achieved according to the original project plan.The constraint screening and analysis makes the planned production ready by pulling required resources rather than pushing them on to the original plan [23].
Once the production is planned and executed, the application of performance measurement at the production level provides quantitative information that could be utilized to reduce production variability between different tasks and to identify the actual causes of poor performance [20].
Therefore, in the context of renovation projects, constraint analysis could prove to be a proactive approach of planning production assignments which is in contrast with the traditional "throw-it-over-the-wall" approach and assuming the executing party will figure it out, and when needed react to disruptions.By studying the constraints of renovation projects at the production level, it is possible to better plan for these constraints and to minimize their impacts on the production of construction activities and overall performance of the project.
Moreover, previous studies have concluded that these constraints make traditional project planning and monitoring techniques inappropriate for measuring and controlling performance in renovation projects [1].Therefore, in order to assess the impact of constraints on production level and measure the performance, it is essential to identify an appropriate set of performance indicators for improving the construction throughput in renovation projects.Measuring performance at the production level of renovation projects also requires an understanding of the complex nature of unforeseen conditions, which is one of the biggest constraints contributing to the uncertainty in construction planning processes [10].

Step 1: Literature Review and Interviews-Brief Overview
In order to develop a framework for production management of renovation projects, it became essential to investigate the literature of three main facets, namely, (i) state-of-theart construction performance measurement systems; (ii) the complexities of renovation projects that affect production performance; and (iii) production management of construction processes.

State-of-the-Art Construction Performance Measurement
Systems.The limitations of traditional performance measurement systems led numerous researchers to focus on developing what have been termed as state-of-the-art performance measurement systems for the construction industry.
In fact, a study conducted between 1994 and 1996 found that a performance measurement article was published every five hours of every working day [13,14].Although not all of these systems were related to construction, they illustrated that performance measurement has been an issue of concern in all the industries.These state-of-the-art performance measurement systems have investigated different models for measuring and analyzing construction performance but the overall focus was on identifying appropriate indicators that could measure the performance of construction processes.Some of these systems also developed the measurement methods of performance indicators at the project level, trade level, and activity level [19].
Reviewing the literature on state-of-the art construction performance measurement systems revealed its focus on identifying key performance indicators for general construction.Consequently, there is a need to investigate their application or adaptation to production management in renovation projects.This could result in either revising the current methods to plan and assess production performance in renovation projects or developing new methods.In addition, as past researches have not collectively concluded the adequate set of performance indicators for general construction [19], it becomes essential to investigate the feasibility of additional performance indicators for assessing the impacts of constraints on the production cost, time, and quality in renovation projects.This would lead to better production planning, execution, and assessment methods for renovation projects in the presence of constraints.
From a systems perspective, this investigation necessitates development of a framework for production management that integrates various renovation-specific constraints with current production planning and performance assessment methods and defines appropriate performance indicators to be measured at the production level of renovation projects.

Renovation Projects.
The literature of renovation projects was reviewed to understand the complex nature and scope of additional requirements for managing the construction process, which resulted in a listing of external constraints that affect the production in renovation projects and lead to schedule and cost overruns.These constraints are generated from those project conditions in which typical renovation projects are undertaken and are not controllable by the contractor for the majority of cases [1,10].Through the identified constraints, and as will be discussed later, this research captured some critical activities that have significant contribution in performance of renovation projects.These activities are critical in nature as they involve constraints during their execution and, if their production is not planned appropriately with the nature of the constraints, they could result in underperformance in time, cost, quality, and/or safety.
Numerous researchers in renovation projects and construction performance measurement systems underscored the significance of managing construction processes under the constraints of renovation projects.Therefore, importance of constraints has been emphasized because of their major impacts on the project success [10].This requires considerable preconstruction planning effort for minimizing the impacts of constraints on the estimated cost and time [1].
In addition, the literature also explains the conditions or factors that generate these constraints and contribute to project cost and time overruns.In other words, the constraints are composed of various conditions or factors, which may or may not be the part of a project's scope but are present in the project location.Successful assessment of these conditions or factors would lead to a better constraint assessment and definition of relevant performance indicators, and, hence, leads to better production management of renovation projects.
The constraints identified from the literature primarily impact specific activities of renovation projects where the occurrence frequency of constraints' conditions becomes highest.For this research, these specific activities that involve maximum frequency of the constraints' conditions are termed critical activities.These activities become critical in nature as their execution requires additional planning and control of constraints to finish renovation projects on time, within budget, and at the desired quality.Therefore, consideration of constraints in the critical activities' planning could provide a possible solution for minimizing their impacts.For instance, demolition processes have been extensively discussed in the literature and found to require considerable attention due to a confluence of constrains, namely, pollution constraints, uncertainty constraints, and physical constraints.Therefore, selective demolition becomes one of the critical activities in renovation projects that impact the performance of the whole.Similarly, other critical activities in a renovation project include the following: The overall literature review provided an overview of some of the essential attributes that a production management system should possess in renovation projects.The essential attributes are discussed later in this paper.

Production Management in Construction.
Review of construction production management systems provided insights into how inefficiencies of construction process operations at the production level were previously addressed.Production management practices such as Last Planner System provided an understanding of operational shielding from various constraints of general and renovation projects by effective planning and assessment of performance at the production level [20,21,27].In addition, studies that focused on the impact of variability on crew production rates and reliability of hand-offs were reviewed due to the importance of stabilizing the workflow against the impact of constraints in a renovation project.

Interviews of Constructors. Ten construction companies
were interviewed regarding their production management systems for renovation projects.The essential attributes obtained from the literature review assisted in drafting the interview questions.
The interviewees were selected from a sample set, which formed a part of another research project titled "Vendors' Performance Assessment" (VPA) underway at the School of Planning, Design & Construction, Michigan State University (MSU).The sample set was comprised of 15 constructors that were suggested by the Physical Plant division of MSU for conducting the VPA project mentioned above [9].Typically, for one construction company, the research team met with a group of interviewees comprised of construction managers, site superintendents, project engineers, and field engineers.So, in total, the individuals interviewed totaled 46 individuals.

Interview Findings.
From the interviews of constructors, it was observed that the state-of-the-art performance measurement systems in practice do not involve major changes in their application between new construction and renovation projects.This is exemplified by the critical success factors considered and the adopted measurement methods which mostly were stated to be the same for new construction and renovation projects.In addition, the critical success factors reported by the interviewees were on a macrolevel of a project such as quality, repeat business, and client satisfaction.There was no success factor reported that dealt with the production level of construction activities such as percent plan complete (PPC) and quality of work delivered to the next trade.As the complexity level increases in renovation projects due to the presence of multiple constraints, interviewees agreed that use of production level indicators becomes imperative for managing construction on a daily basis and assessing the production performance of a construction crew.
Most of the project conditions and constraints stated in the literature were also reported in the interviews to be a normal encounter in practice.These constraints include uncertainty, dynamic nature, irregularity in plans and specifications, coordination with owner's operations, and traffic conditions.These stated constraints adversely impact the time, cost, and quality performance of renovation projects.
Although the interviewees agreed on the complexities of renovation projects and the constraints involved, it was concluded that there was an overall lack of a formalized documented procedure for assessing project conditions and constraints of renovation projects for establishing crew assignments.This is exemplified by informal processes of production planning in practice and the similarities in performance assessment systems for renovation and new construction projects, as reported by most of the respondents.In fact, an implicit process between the superintendents and foremen for establishing crew assignments was par for the course.
As renovation projects experience lower performance levels in terms of cost, time, and quality, the interview data underscores the need to develop a production management system for renovation projects that could be employed to deliver better performance level.In addition, production management of renovation projects should be viewed from a perspective that involves thorough analysis of constraints during production planning of crew assignments.This would deliver quality production assignments that the crews can actually carry out and also provide a basis for performance assessment of construction crews with regard to their ability to cope with the analyzed constraints.

Step 2: Essential Attributes of a Production Management System for Renovation Projects
Notwithstanding that the literature does not explicitly state the essential attributes of a production management system in the context of renovation projects, this research has assimilated them from the principles of performance measurement systems, complexities of renovation projects that impact the production of construction activities, and practices of production management advanced in Lean Construction.Some of these attributes have also been echoed during the contractor interviews.These essential attributes are as follows.
(i) Identifying project conditions that could generate constraints.(ii) Determining the type of constraints that are expected for the project.(iii) Identifying critical activities to be managed: these are not activities with a total float equal to zero as in CPM; rather these are activities that have high numbers of constraints associated with their execution.(iv) Assessing the relative impact of each constraint in impacting the production of critical activities.These identified essential attributes were applied to the development of a production management framework for renovation projects which will be discussed in a subsequent paper.

Interface between Renovation Projects and Contemporary Construction Performance Measurement Systems
Based on the insights gained from the interviews and literature review, this research has developed a graphical illustration, shown in Figure 3, of the complex state of renovation projects interfacing with the reviewed contemporary construction performance assessment systems.In general, Figure 3 demonstrates the intricacies involved in production assessment of a construction activity in renovation projects.Figure 3 illustrates some of the essential attributes of a production management system for renovation projects, as mentioned earlier.These essential attributes are: project conditions, constraints, critical activities, performance indicators at the production level, and broad performance categories.
Figure 3 displays also how the production of a construction activity relates to the macrolevel performance indicators (time, cost, quality, and safety).The impact of multiple constraints and various project conditions on these macrolevel indicators can be quite significant in a renovation project.Typically, renovation project management, as well as general construction project management, has only considered labor, material, and equipment availability to be the primary constraints impacting project level performance.
The literature and interviews indicate that there are numerous project conditions that generate constraints in a renovation project [1,10,28].A thorough investigation of these project conditions would lead to better identification and assessment of constraints and, therefore, a better production management of renovation projects.In addition, these project conditions need to be investigated in order to assess the impact of constraints on the production of critical activities.Therefore, this research assumes that these project conditions could define the performance indicators for production management of renovation projects because these project conditions would relate directly to the ability of a construction crew for completing a production assignment within budget, on time and at desired quality.Thus, Figure 3 shows these project conditions as performance indicators for each constraint.
Figure 3 clearly shows that a performance indicator of any constraint could have multiple impacts on the production of various critical activities with respect to their estimated cost, time, quality, and safety performance.In other words, any critical activity's production could be impacted by multiple performance indicators of numerous constraints.This impact can only be quantified and planned when all these essential attributes are linked in a systematic process of production planning, execution, and performance assessment.
For a renovation project, failing to systematically plan for constraints could lead to budget and schedule failures and could incapacitate project teams in identifying real causes of underperformance.If a project were running over budget or behind schedule, it would become extremely difficult to ascertain which constraints have caused these impacts in which critical activities.
From the interviews of constructors and the literature review of contemporary construction performance measurement systems, this research concluded that existing practices and systems do not incorporate the constraints, project conditions, and critical activities of renovation projects to measure production performance.Previous researchers have also concluded that contemporary performance measurement systems have not been able to identify a definitive set of performance indicators which assess the true performance of construction projects [10,17].
Therefore, contemporary performance measurement systems are not sufficient for use in production management of renovation projects.This research has also observed that contemporary performance measurement systems do not account for the constraints of renovation projects as shown in Figure 3, especially with respect to the performance indicators proposed by these systems.In addition, the interviews of constructors revealed that current practice only encourages macrolevel performance assessment of renovation projects.
Performance measurement of renovation projects should be handled at the activity level where the constraints generate their most impacts.Performance planning and assessment at the activity level will identify the causes of inefficiencies at their sources and improve the performance at project level.Therefore, production management should be undertaken by analyzing all constraints and their associated project conditions in establishing suitable crew assignments and the performance should be assessed by using operational level indicators.

Conclusions
Despite the complex, uncertain, and dynamic nature of renovation projects, the interviews of constructors revealed that there are no perceived differences in performance assessment methods for new construction and renovation projects.None of the respondents reported a success factor or performance indicator for renovation projects that dealt with the production level.The respondents did not state constraint management as an important process for renovation projects.
The reviewed performance measurement systems proposed different sets of performance indicators for assessing the performance in the three categories of cost, time, and quality, but a consensus has not been found as to the most appropriate set of performance indicators that define the actual performance of construction projects and that can assist in identifying the sources of inefficiencies.In addition, the reviewed contemporary performance measurement systems do not incorporate assessment methods for constraints and/or project conditions encountered in renovation projects.Consequently, contemporary performance measurement systems find limited application in managing the performance of renovation projects at the activity level.
As the contemporary construction performance measurement systems mainly focus on assessing performance of construction projects in three categories of cost, time, and quality, it becomes imperative to investigate the impacts of constraints on the cost, time, and quality performance of renovation projects.The complexities of renovation projects contribute to the need for developing a simplified and methodical process of production management that takes into consideration the essential attributes discussed in this paper.This research proposed a set of performance indicators, constraints, and project conditions to be systematically considered during the planning of renovation projects, and established their connection to the macrolevel constraints on cost, time, quality, and safety.
This initial research phase concluded that current analysis of project conditions and constraints of renovation projects is performed on an informal basis.There is a lack of formalized and documented procedure for assessing the constraints in production planning of renovation projects.Essential attributes of a production management framework were identified from literature reviews and interviews.A framework that incorporates these essential attributes has been developed and is expected to provide a valuable template to constructors to incorporate their subjective assessment of constraints as they relate to specific project conditions.The development and validation of the mentioned production management framework for renovation projects will be outlined in a future paper.

Figure 3 :
Figure 3: Interface between renovation projects and contemporary construction performance measurement systems.