post-occupancy evaluation

Building Assessment

Once the building is completed and the commissioning agent has verified that building systems meet design intent, live loads for heating and cooling (i.e., occupants and office equipment) become an important part of the building system. To measure building performance, especially the effectiveness of building systems, feedback must be obtained by the occupants. POEs seek to systematically evaluate the performance of a building after it has been built and occupied for a period of time (Federal Facilities Council 2001).

Practices for Assessing Buildings

The POE seeks to learn how a building performs and to learn the level of users’ satisfaction with the built environment. When used appropriately, the POE communicates the effectiveness of building systems between the facility’s users and the facility’s management. The rigorous process of evaluating a building in use provides an otherwise unavailable insight to the design team. POEs take different forms and survey different user groups, providing a wide array of potential outcomes. POEs are divided into three main categories; indicative, investigative and diagnostic, each with an increasing level of detail (Federal Facilities Council 2001). While a
diagnostic POE will show qualitative data in the form of occupant feedback as well as quantitative data in the form of data from building controls, there still exists an issue with communicating this information to those able to make the necessary improvements.

The Center for the Built Environment at University of California at Berkeley developed a web-based occupant survey to quantify building performance from the perspective of the occupant. The survey focuses on indoor air quality and its implications on comfort and productivity. The CBE survey has been widely applied to many buildings to establish a database of occupant perceptions for building performance. The survey uses basic performance questions to gather relevant information from occupants, with branching questions to gain more information about user satisfaction (Huizenga 2002).

Three case studies using the CBE survey are reported by Zagreus et al (2004) that compare:
• occupant satisfaction pre- vs. post-move,
• occupant perceptions with actual building performance,
• a new building with respect to design objectives.

These comparisons provide excellent insight into occupant satisfaction for the project team. For example, it was found that respondents were more willing to accept thermal variations if they felt they had control over the space. However, these three case studies were performed on separate buildings, which prevents this information from being synthesized to generate a comprehensive understanding of user-building interaction. Workplace 20-20 is a program used by the General Services Administration (GSA) to evaluate federal buildings. Kempschroer & Heerwagen (2005) reports on GSA’s Workplace 20-20 program and compare it to POE methods. Workplace 20-20 is applied for the whole organization, whereas a POE is applied to a specific building. Workplace 20-20 has 4 components; organizational analysis, design concept development, design development and implementation, and project evaluation. The program uses the balanced scorecard approach to link physical space to organizational goals, evaluate design alternatives and measure performance. There are three goals for Workplace 20-20:
• Provide performance feedback to GSA about the building,
• Assist the design of future projects by capturing lessons learned on completed projects, and
• Analyze multiple projects to identify overarching themes and issues to modify policy.

Unique components of GSA’s Workplace 20-20 program are the development and execution of the program from within the organization and the focus on the building with respect to organizational objectives. In-house administration of the program and an evaluation of the building as a part of a larger organization enable the Workplace 20-20 to provide functional and practical insight on building performance.

Kempschroer & Heerwagen (2005) discuss environmental and social factors on productivity in the office and identify organizational context as a key factor in selecting an appropriate office design for the Workplace 20-20 program. The Balanced Scorecard approach was used to link physical space to organizational goals, evaluate design alternatives, and measure performance. The POE had three goals; provide performance feedback to GSA about the building, assist the design of future projects by capturing lessons learned on completed projects,
analyze multiple projects to identify overarching themes and issues to modify policy.

McDougall et al (2002) reviewed leading performance measurement tools for assessing buildings in the UK, concluding that POEs are useful tools for gathering data on building performance, but the question remains as to what to do with the information gathered. The information has limited value since the design team has moved on to another project with different functional requirements. Three evaluation tools with established track records in the UK are outlined: Building Quality Assessment (BQA), Serviceability Tools and Methods (STM) and PROBE. BQA assesses buildings based on what amenities are provided for use (e.g., ‘backup power’ or ‘elevator’) however it does not document the occupants’ satisfaction with the performance of the amenity. STM has two parts; setting up the occupant requirements and rating the building. STM, though accounting for the occupants’ requirements still has issues with comprehensiveness for buildings under evaluation. The PROBE study makes use of a questionnaire for end-users and includes technical performance and energy performance indicators, however even the PROBE studies had problems with statistical validity and proving causal links – addressing the variable within the respondent that they term occupant forgiveness.STM and BQA are good tools for benchmarking, whereas PROBE differentiates itself by collecting additional sources of data.

Bordass and Leaman (2005) argue that the POE as part of a feedback process can present a viable tool to design teams and facility managers. The industry has placed great emphasis on the completion of the project according to the contract, not on ensuring customer satisfaction. However, with owners increasingly seeking sustainable facilities that maintain healthy environments for occupants, ensuring customer satisfaction will become a major driver for companies to acquire repeat business. In a series of studies, the Post-occupancy Review Of Buildings and their Engineering (PROBE) process sought to investigate the technical and energy
performance with, as well as management and occupant satisfaction of, the building (Cohen, Standeven, Bordass 2001). Two tools were used; an occupant survey to gauge satisfaction and an energy assessment report to generate and analyze building performance. The survey collected information on individuals:
• background,
• perception of the building’s design,
• personal control over heating, cooling and lighting ,
• perception of the speed and effectiveness of management response after complaints were reported,
• perception of temperature,
• perception of air movement,
• perception of air quality,
• perception of lighting,
• perception of noise,
• perception of overall comfort,
• perception of health, and
• perception of productivity at work.

The energy assessment report involved a three staged process to gather increasing levels of details on the most critical areas of energy use. Stage 1 was very general, showing resource consumption for each type of energy compared to total energy use. Stage 2 incorporated extenuating factors into the analysis, which might include weather, occupancy or end-use patterns that vary from design intent. Stage 3 identifies energy consumption by each system, subsystem and component. After the third stage, the researchers were able to pinpoint areas with greatest potential for improvement.

Results from the PROBE studies illustrated the impact of common design concerns on occupants, such as noise problems in open-plan office settings. While the authors stated that PROBE creates a “self-fulfilling loop”, PROBE was not seen as an approach that can be used by any individual involved in the process. Experienced assessors were deemed necessary to achieve the results of the PROBE studies (Cohen, Standeven, Bordass & Leaman, 2001). To implement a feedback loop in an industry struggling with knowledge management, the sources of data would ideally come from existing databases and surveys requiring minimal resources to distribute and
analyze.

A Building Performance Evaluation (BPE) Protocol has been developed by Canadian based EcoSmart and recently released into the public domain. The BPE Protocol is provided on the EcoSmart website and is based in multiple worksheets in Microsoft Excel. The BPE purports to structure an evaluation to “trace the implementation of major strategies through design and construction into their effectiveness in the occupied building” (EcoSmart 2007).

The worksheets in the BPE Protocol compile a thorough list of questions to pose to the project team, which is identified to include the owner, architect, mechanical engineer, electrical engineer, and the BPE team (consisting of IAQ, lighting, acoustics and controls consultants). The questions touch on many important aspects of the delivery process and the current building operations, however to generate the level of detail from the responses requires a level of expertise on the part of the questioner. The questions posed are very general, and may not yield the answers required to reveal the true level of performance and user satisfaction.

Upon rigorous completion of all worksheets in the BPE Protocol spreadsheet, the building will have a comprehensive documentation of the level of performance and satisfaction (pending data from the results of a CBE survey are included). However, no further guidance is provided to analyze the performance and satisfaction data gathered. If further analysis is not completed, the information was only collected to be lost in a file, and no long-term benefits.

As part of Getting to Fifty, a recent initiative by the New Buildings Institute, a list of current industry needs was developed to accelerate the penetration of energy efficient design and construction in the commercial construction sector. Included in this 7-point list are (NBI 2007): Early design process improvements to improve choices and commitments related to building performance,
• Improvement in feedback protocols from measured building performance, and
• Building case studies and POEs to reduce real and perceived risks of low-energy buildings.

These three elements help to validate the core goal of this research; to develop a rigorous building evaluation methodology that accounts for multiple stakeholders in the operation of the facility while evaluating with respect to the original project requirements. Most recently, the NBI (Owens et al 2007) concluded that “Energy modeling predicts program impacts well, but the accuracy is highly variable for individual projects. There is a need to develop a better feedback loop from actual building performance to design intent and modeling.” The report explains that “energy use of high energy building types is not well understood by designers. Neither the LEED program nor the modeling protocol address these projects well.” And, “although on average, energy modeling predicts actual program performance well, there is a great deal of variability in the accuracy of individual project results. This can have significant adverse impacts on Life Cycle Cost Evaluation for individual projects.” This further validates the stated need for a building performance evaluation methodology to comprehensively account for actual building conditions and the end-users’ satisfaction with respect to the original design intent.

Principles for Providing Feedback

The proposed research will be based on two key methods for providing feedback to the delivery of future projects. To identify and document user perspectives, the POE has proved to be an effective tool. However, to communicate this feedback to the project team, the Design Quality Indicator instrument is better suited. The process of generating user feedback for the project team can benefit from Design Quality Indicator guidelines developed from Whyte, Gann and Salter (2004). The instrument to collect information from the end-user must:
• Be clear and concise,
• Take no more than one hour to complete,
• Combine subjective and objective indicators,
• Include as many different perspectives as possible,
• Be test piloted widely and intensively,
• Be flexible to address different objectives of different projects, and
• Communicate results to all participants in the project.

Similarly, Chew et al (2003) calls for “a quality chain and tracking system composed of designers, contractors, operators to verify completed building performance.” These guidelines align with the principles of lean, and will be used extensively in the development and execution of the user feedback loop.

Structuring the feedback to effectively communicate with the project team will be another obstacle to the research. Pulaski and Horman’s (2005) Conceptual Product/Process Matrix Model documented and organized the main strategies of constructability in design based on the timing of design decisions and the levels of details. Adapting this model to focus on user requirements will produce a tool for effective communication to the design team on a broader scale.

Practices for Providing Feedback

The Federal Facilities Council (2001) identifies the POE as having six goals:
• To measure facility performance,
• To fine tune the facility,
• To adjust repetitive programs,
• To research the effects of the building on occupants,
• To test application of new technology, and
• To justify actions and expenditures.
However, the techniques used to fulfill these goals vary greatly due to building usage variations. The POE only represents a tool to gather the data to meet defined goals. The user must act on the results to achieve the benefits of a POE.

Bordass and Leaman (2005) outline a process to make facility feedback a routine process in the delivery of buildings. Five steps are created which include the building audit, discussion with users, questionnaire administration, processing of data and finally the packaging of data. In their PROBE study (Cohen, Standeven, Bordass & Leaman 2001), the authors found the obstacles to a widespread adoption of POEs were a lack of reproducibility on projects. The survey needs to be written around the primary concerns for a given building. This customization makes the comparison of two independent buildings extremely difficult, much less benchmarking a building with respect to the industry as a whole. This lack of comparability holds true for the proposed research as well. However, the goals of the proposed research vary from the PROBE study in that the research intends to provide a project team with ideas in the form of continuous improvement.

In “Closing the loop between design and use” Whyte (2001) identifies the POE simply as a knowledge source, but a process implemented by an owner or project team to continually evaluate a facility to ensure building quality is maintained and the changing needs of the occupants are accommodated. The proposed user feedback loop allows for repeated cycles of data collection therefore building management could execute on a regular basis for the life of the building to maintain quality levels.

The unique situation of the renovation of the wedges in the Pentagon also raises the possibility for user groups to be analyzed in great detail. Identifying user preferences from a large sample population and rigid data collection practices transforms this research from continuous improvement to a quasi-sociological study – identifying the features and correlated strategies that are most preferred by the user groups. The broad spectrum of individuals will allow for intense data analysis in determining strategies that lead to successful design solutions, such as modular design for building systems and integrated design for building finishes.

Design Quality Indicators (DQIs) can be used “as a tool for thinking” (Gann et al 2003). After completion, the project is evaluated qualitatively and quantitatively to incorporate lessons learned into future projects. While the proposed research is not intended to evaluate the quality of the building, the authors raise some important issues regarding the survey of the building’s user groups. Each group performs different tasks which result in differing opinions on the overall building quality. This difficulty is incorporated into the proposed research through the development of two separate surveys for occupants and operators. To further identify differences within these two groups, the focus groups will gather tenant groups to concentrate on extreme data points in the survey.