A group of Marina del Rey (Calif.) Hospital staff members review design plans for a replacement hospital with Cedars-Sinai facility planning, design and construction team members and a consulting team.

Photo courtesy of Cedars-Sinai

As health systems expand their footprints to better serve their communities, it can be increasingly challenging to ensure that all patient care environments reflect lessons learned and best practices across the enterprise. The issues become even more complicated if the system’s building portfolio includes a combination of owned and leased assets, all of which have distinct areas of specialty and are designed and constructed by different teams.

That’s the scenario at Cedars-Sinai, a Los Angeles-based health system that serves more than 1 million people annually in more than 70 patient care locations. Each year, Cedars-Sinai initiates hundreds of facility capital improvements. These projects include a range of new construction and renovation programs throughout its primary medical campuses and expansive outpatient medical network. Each assignment is typically overseen by an external architect and contractor, working under the direction of an internal team of architects, planners and project managers. 

Until recently, Cedars-Sinai relied on a culture of oral tradition for sharing best practices and lacked a formal process for gathering and sharing lessons learned from its historical projects with internal and external teams. “Over the years, we’ve always shared lessons informally, but as our team has grown and taken on a larger volume of projects, our ability to pass on lessons manually and share war stories over coffee was no longer possible,” says Ian Grant, manager of design and construction at Cedars-Sinai. 

The consistent sharing of best practices is also essential because of the stringent inspection and construction requirements of California’s Office of Statewide Health Planning and Development, which oversees construction of the state’s health care facilities. “Working with the state of California is very challenging and extremely regulated, so it’s really important that our teams learn from prior projects and don’t make the same mistake twice,” says Grant. “Any percent we can shave off our change-order rates allows us to put more money into delivering health care.” 

Because these collective insights weren’t easily accessible, individual project teams risked repeating the same “mistakes” and oversights. Too often, this led to inefficiency, frustration, schedule delays and costly rework.

Building a database

To address the issue, Cedars-Sinai’s facilities planning, design and construction (FPDC) department began to consider opportunities for developing a simple digital resource that teams could access for guidance and best practices from previous design and construction projects and pass on knowledge for future projects.

Cedars-Sinai team members involved in the development of the lessons-learned database include Project Manager Chitose Freeman (seated), Facilities Data Administrator Armen Haroutoonian, and Executive Director Alicia Wachtel.

Photo courtesy of Cedars-Sinai

The first version of the platform debuted as a basic Excel spreadsheet, with fields for standard project details, observations, learnings and keywords. It was initially populated by about 200 lessons gleaned from post-occupancy evaluations conducted on a collection of recently completed and occupied projects. 

As the spreadsheet continued to grow with additional lessons from FPDC team members, it became progressively more cumbersome. Everyone agreed it was unlikely to serve as a useful resource unless it became easier for team members to access and navigate. 

Ideally, team members needed access to a centralized searchable database of lessons that enabled them to easily filter across various project attributes and view the issues that are most relevant and helpful to the execution of a specific project. This resource would also benefit from a clear, simple interface so team members were motivated to enter their own lessons into the system on an ongoing basis and could easily search through the repository of collected lessons to plan for risks on in-flight projects. 

To progress toward that goal, Cedars-Sinai tapped into the machine-learning expertise of Enstoa Inc., a consulting firm engaged to oversee the health system’s digital transformation. Machine learning — a form of artificial intelligence (AI) — involves building computer algorithms that learn from data and automatically improve the accuracy of the results over time. 

“I approached our business intelligence group and asked if there was a way we could use machine learning to distill the growing population of data into something that was more manageable and useful,” says Enstoa Director Ruchira Banerjee, who had been collaborating with Cedars-Sinai on the lessons-learned database project. 

She and her colleagues examined various options for leveraging natural language processing (NLP), a type of machine learning that’s used to help computers understand, interpret and manipulate human language. They focused the inquiry on the observations field of the database, which included details of the specific issue identified and how it was resolved.

“Since most machine learning algorithms are developed to look for patterns in much larger datasets, we knew that we needed to consider an NLP method that would be effective on a smaller dataset,” says Banerjee.

After considering several potential solutions, the Enstoa team opted to use the RAKE (rapid automatic keyword extraction) method for text summarization and TF-IDF (term frequency-inverse document frequency) with K-means clustering for text categorization. Both are known to deliver accurate results on smaller datasets.

“We were able to distill long descriptive paragraphs into clusters of just a few words that could be grouped together to visualize commonalities across the dataset,” says Banerjee.

The resulting bar graph provides a simple visual depiction of the highest-frequency issues that impacted recently completed projects — from acoustical concerns to flooring durability to Americans with Disabilities Act compliance.

“Instead of putting the onus on the employee to find lessons that are relevant to them, the machine-learning algorithm identifies the most common issues that they may face,” says Banerjee. “It also allows leadership to quickly identify problem areas and urgently communicate to their teams about them.”

Each month, Enstoa runs the entire dataset through the machine learning algorithm to incorporate the most recently added observations in the results. In most cases, new entries naturally fit within one of the existing clusters.

Ultimately, the identification of high-frequency learnings will allow FPDC to be more strategic in its approach to avoiding pitfalls on current projects, planning for risks on future projects and making modifications to existing departmental standards. By strengthening the group’s overall capabilities, it will also open the door to greater opportunities to innovate and take on more risk. 

Triage team engaged 

An internal “triage team” is helping to ensure the lessons-learned database is as accessible and valuable as possible to fellow team members.

The cross-disciplinary group, which includes an architect, construction manager, project manager, administrator and financial manager, reviews all data entries to ensure they’re recorded in a consistent language, align with department standards and reflect industry best practices.

A post-occupancy evaluation of the Cedars-Sinai Family Medicine clinic in Culver City, Calif., revealed that casework was positioned too high for most staff to access the upper shelves, so the system is currently studying the optimal height for installation and configuration of storage cabinets for future clinics.

Photo courtesy of NBBJ

“As a triage team, our role is to validate the lessons and make sure they’re categorized so that they’re easy to find in the future,” says Grant. 

During a series of virtual learning events, all FPDC team members had opportunities to learn more about the developing database and contribute their ideas and suggestions. Because these knowledge-sharing sessions took place during the pandemic, team members appreciated the ability to connect during this period of isolation. Several of them also expressed their appreciation for the opportunity to bring practical value to Cedars-Sinai through their contributions and take ownership of the tool. 

The cross-disciplinary triage team also worked closely with Enstoa to develop and refine the user interface for a lessons-learned module that’s now part of the department’s centralized project management system. 

The interface links to a digital form where users can document and upload their own lessons that are connected to an individual project or general learnings. Users also can upload photos or drawings that help illustrate the issue and share advice they would pass along to future project teams.

From the dashboard, users can view high-frequency issues and drill down into any of those categories to access additional information on root causes and solutions. A search tool within the module enables users to quickly locate issues related to a specific topic area.

Lessons learned also has become a formal component of Cedars-Sinai’s gated project delivery, a series of approval checkpoints — or “gates” — that all FPDC projects must pass through before they’re given the green light for construction. At each step of the project life cycle, a project team must complete a review process with FPDC leadership to ensure project alignment in terms of scope, quality, budget and schedule. 

These checkpoints serve as an opportune time to ensure that the team is integrating all lessons and best practices from similar projects. 

Applying the lessons 

Since debuting in early 2021, the lessons-learned module has already started to influence the ongoing design and construction of Cedars-Sinai facilities. 

Newly adopted standards include best practices for controlling sound transmission between exam rooms in leased clinic environments. Because of the prevalence of these issues across the Cedars-Sinai portfolio, FPDC now requires its design teams to study the base construction and perimeters of existing buildings, and to apply a specific window detail that mitigates sound transmission. A new standard also specifies a single sliding-door product that may be used in clinics. 

Insufficient generator capacity — another common issue at clinic sites — is being addressed by broadening the site-selection process to include an assessment of existing generator capacity as well as identifying potential locations for expanding this capacity. 

And the initiative also has revealed some highly practical design considerations, such as the importance of locating storage shelving at heights that are within reach of the staff who will be using them. 

Another actionable lesson prompted a ban of large-format ceiling tiles in clinics and office spaces because they’re challenging for maintenance staff to lift without damaging them when they have to access plenum space to conduct repairs. Owners of leased space have refused to lift these large tiles, resulting in additional damage while Cedars-Sinai maintenance staff was deployed to the remote sites.

As the database expands with real-time lessons from FPDC team members, it will continue to inform and improve the overall planning, design and construction process. 

Culture of learning

The primary goal of the lessons-learned initiative is to advance Cedars-Sinai’s culture of learning and continuous improvement. 

According to Mary Clare Lingel, vice president of strategic integration at Cedars-Sinai, it’s a valuable tool for elevating the standards at outpatient facilities and creating a more consistent patient experience across the system’s entire portfolio. 

“If a patient visits or is admitted to the hospital at Cedars-Sinai, they are cared for by exceptional physicians and staff in a welcoming, elevated and healing environment,” she says. “They also should have this experience with their outpatient physician, who they’re arguably spending more time with. My mission has always been to narrow the gap between those two environments.”

Lingel acknowledges the challenges of establishing consistent standards across an outpatient network of mostly leased spaces that span over 30 specialties. “A primary care practice is going to look and feel and have different needs from an orthopedic practice,” she says. “I think it’s all about scaling what you can scale and capturing the characteristics of each particular site to maximize the strengths of that site.”

Large-format ceiling tiles contribute to the open, contemporary aesthetic at the Ventana Cancer Clinic in Tarzana, Calif., but they can be easily damaged when maintenance crews lift them to conduct repairs above the ceiling, so Cedars-Sinai no longer uses them in its leased clinics and office spaces.

Photo courtesy of ZGF Architects

Mark Kruse, executive director of facilities and purchasing for the Cedars-Sinai medical network, hopes the focus on sharing lessons learned will lead to greater standardization of the materials palette — from ceiling tiles to lighting and plumbing fixtures to paint. “We have limited storage space available, so the fewer materials and supplies we have to hold onto in the warehouse, the better,” he says. 

Kruse also thinks the process will help Cedars-Sinai establish more standardized best practices for the design of common elements such as equipment reprocessing rooms, where medical instruments are cleaned and decontaminated for reuse. “Because every space is unique, standards need to flex, but there are specific design elements that need to be incorporated into like spaces, and these standards will help our design teams going forward,” he says.

Guidance and accountability

Ultimately, the process of collecting and sharing lessons learned will provide both guidance and accountability to the planning, design, construction and operational teams that work together to bring Cedars-Sinai building and renovation projects to life.

“Not only has it helped us to crystallize what architects and planners need to know, but it’s also forcing our operational teams to be very clear and specific early on in the process about what they need to have,” says Lingel. “Because it moves the deep thinking about a space earlier in the process, it holds people accountable on both sides.”

Developing a process for sharing lessons from new building projects

By establishing a formal process for gathering and sharing lessons learned, health facilities teams can save money, improve efficiency and reduce frustration on future construction and renovation projects. Recommendations to ensure success include:

1. Understand where lessons reside. Understanding the sources of lessons and how to effectively capture them is key to ensuring the effort will be effective.

Post-occupancy evaluations and project milestones such as project closeout and the start of construction are opportune times to collect observations. To help organize observations, consider grouping projects by phase, such as those that are under construction, recently completed and occupied for six months.

It’s also wise to cast a wide net of potential contributors, including facility users, project managers, department managers and outside sources.

2. Enlist an oversight team. Select a group of representative team members who will guide the initiative and help ensure that it continues to thrive. This group will also review contributions to ensure they’re tagged appropriately and align with system standards.

3. Make the process simple and intuitive. The simpler the system, the more likely that team members will use it. Ideally, the entire lessons-learned process should become a natural extension of someone’s daily workflow. 

The platform or interface should be flexible enough to capture lessons of all types and at all scales — everything from structural best practices to details about where to locate storage areas. 

4. Make the process fun. Consider creating personal or team competitions to motivate team members to identify and share lessons learned with the group.  

5. Keep an open mind. The findings might be surprising, so it’s important to keep an open mind and maintain a focus on continuous improvement.  

Alicia Wachtel is executive director of the facilities planning, design and construction group at Cedars-Sinai, Los Angeles. She can be contacted via email at alicia.wachtelosin@cshs.org.