Benchmarking Pharmaceutical Quality Control Labs Holistic assessment of operational excellence in pharmaceutical companies

BENCHMARKING PHARMACEUTICAL QUALITY CONTROL LABS

Holistic assessment of operational excellence in pharmaceutical companies

Through the years pharmaceutical companies have managed to apply Operational Excellence (OPEX) and Lean-thinking to their operations. Nevertheless, the majority still lacks behind when incorporating an end-to-end perspective. A first step to holistically assess the performance of pharmaceutical companies necessarily incorporates quality control (QC) in addition to manufacturing. Therefore, the St.Gallen QC Lab Benchmarking helps to position pharmaceutical QC labs from a broad range of pharmaceutical companies against each other.

Gian-Andri Steiger, Lorenzo Pirrone, Marten Ritz, Thomas Friedli

Institute of Technology Management The importance of Operational Excellence (OPEX) to drive business performance and sustainable growth has been recognised across industries (Mitchell, 2016). As demonstrated in our previous article St.Gallen OPEX Benchmarking for Pharmaceutical Manufacturing Sites – Measure Yourself Against the Best but Do It Right, pharmaceutical companies have embarked on their OPEX journey through the years. Nevertheless, it can be observed that the industry’s major focus is limited to the manufacturing function (Friedli et al., 2013). However, to advance the company’s overall OPEX performance, an end-to-end perspective on the value chain needs to be incorporated (Bajaj & Reffell, 2018). In the recent past, OPEX in quality control labs has gained increased interest. This is due to the fact that quality control (QC) can become a key bottleneck of a pharmaceutical value chain (Barbarite & Maslaton, 2008; Friedli et al., 2018). Acknowledging this critical position, the US Food and Drug Administration (FDA) integrated one specific QC lab quality metric next to two manufacturing quality metrics to identify companies at higher risk of compliance and quality failures (FDA, 2015, 2016). Furthermore, quality control labs do not solely fulfil their role as safeguards to ensure product conformity and compliance but can contribute to the overall delivery performance by efficient release resulting in competitive advantages (Ritz, 2022).

Specifics of Quality Control Labs

Pharmaceutical companies have been exposed to strong external economic forces that led them to organise themselves into more complex supply chain structures. Another driver for change is an increased interest in achieving operational excellence, in order to maintain their competitive advantages. Quality control takes thereby a critical role in the overall process structures, since its high degree of complexity in operations requires an effective strategy for laboratory management (Costigliola et al., 2017). Köhler (2019) outlined that

even though there is a consensus among practitioners with respect to the high importance of operational excellence in quality control laboratories, only limited research has been conducted in that field.

Testing processes in quality control labs can be divided into various test types, each with specific requirements. This, along with a wide range of products and materials to process, leads to a high degree of complexity in QC labs (RuizTorres et al., 2012). In the meantime, Laboratory Information Management Systems (LIMS) lack crucial functional solutions in the field of scheduling and planning (Costigliola et al., 2017).

In order to cope with the increased complexity levels and the perceived cost pressure, while fulfilling regulatory requirements, it is crucial for quality control labs to manage performance and strive for continuous improvement. By doing so, based on the findings of Ferdows & De Meyer (1990), it is assumed that only if the basic quality levels are met (process robustness), the foundation for subsequent improvements in cost efficiency is given. Only stable processes that show a mature quality performance will provide a company with sustainable and long-lasting efficiency benefits. In case that process stability is not reached, efficiency gains would be offset in a short time by deteriorations of the process effectiveness, the quality performance for example. Consequently, this provides one with a sequential prioritisation of focus topics for OPEX initiatives. One should first work on process stability and quality, before focusing on increasing efficiency. Improvement initiatives in laboratories target at process quality, meaning the quality of the tests to be performed in the lab, rather than conformance of the product. Competitive advantages of quality control labs stem from stable test quality levels that enable a pharmaceutical company to realise efficiency and overall cost improvements building on this.

Similar to the area of manufacturing, the outcome performance of quality control labs shows a strong correlation with OPEX enablers (S. Köhler et al., 2020). This in turn means that for an effective lab management approach two dimensions need to be considered. Firstly, a holistic assessment needs to provide decision makers with the status-quo performance of a quality control lab. Thereby, the efficiency and effectiveness of a quality control lab have to be both considered. Secondly, since the lab performance correlates strongly with OPEX enabler implementation levels, those can be seen as decisive management tools for further performance improvements.

In order to perform this two-dimensional assessment of quality control lab performance, the St. Gallen QC Lab Benchmarking Model was developed, which is introduced in the following.

St.Gallen QC Lab Benchmarking Model

To drive the improvement of the organisation, performance measures have always been of major interest in academic research (Kennerley and Neely, 2003). According to the St.Gallen understanding of OPEX, QC labs have to pursue a balanced management of cost, time and quality, while simultaneously focusing on the customer needs and addressing structural and behavioural changes. To holistically measure the performance of pharmaceutical QC labs, the St.Gallen QC Lab Benchmarking can be understood as supplementary assessment to the St.Gallen OPEX Benchmarking, which was presented in our previous article St.Gallen OPEX Benchmarking for Pharmaceutical Manufacturing Sites – Measure Yourself Against the Best but Do It Right. Thus, the St.Gallen QC

Overall Quality Control Lab Performance

Metrics Productivity

Enabier Maintenance & Quality System Quality

Planning & Steering System

Cost Structural Factors Service

Management System

Overall QC Lab OPEX Performance Score

Productivity

Handled Samples/QC FTES Batches processed/QC FTEs Overall Tests/Direct QC FTEs1 QC Costs/QC FTES QC Costs?Total no. of Tests Average Lead Time1 Average Cycle Time1 Average Release Time1 Average Adherence to Schedule1 Average Adherence to Lead Time1 Lab Deviations/1'000 Test Recurring Deviations Lab CAPAs Overdue Overall RFT1 Overall Investigations/1'000 Tests1 Overall Inv. OOS/100'000 Tests1 Product Re-Tests due to Complaints

Efficiency

Cost Efficiency

Figure 2: QC Lab OPEX Performance Score

Effiectiveness

Service Quality

Lab Benchmarking Model (Figure 1) represents the transfer of an industrytested holistic approach into the environment of QC labs. With more than 130 pharmaceutical QC labs represented, the St.Gallen QC Lab Benchmarking database is the world’s largest academic database to assess OPEX-performance in the pharmaceutical laboratory environment. Figure 1

First introduced in 2016 and continuously growing ever since, the St.Gallen QC Lab Benchmarking database enables a holistic assessment of a QC Lab’s performance. As performance cannot be measured without the consideration of specific approaches and tools, nor without the understanding of the specific situation and role, the QC Lab Benchmarking incorporates the specific context of production facilities. To allow a meaningful comparison to the peer group, the St.Gallen team defines a dedicated peer group which fits best to the respective lab. By considering the so-called structural factors, St.Gallen is able to identify those labs from the database, that have the highest comparable fit to the benchmarked lab. Examples for structural factors are the regional distribution of labs, the lab type (Drug substance and Drug Product Type),the lab volume, FTE structure, or the separation of testing. When setting up the peer group, the St.Gallen team has to consider the interplay between the right size of a statistically relevant peer group and the best possible match of QC labs. The St.Gallen QC Lab Benchmarking model also addresses the cost efficiency of quality control. Costs play thereby a twofold role. On the one hand costs represent a context factor, as they are depending on various factors such as the test types being conducted, the complexity of processes being performed or the country the QC Lab is operating in and the overall regional cost levels. On the other hand, cost efficiency already demonstrates one out of four performance categories, which are used in the St.Gallen QC Lab Benchmarking to assess the QC Lab's performance. A large number of key performance indicators (KPIs) allow the overall assessment of the lab’s performance. The overall performance score can thereby, as depicted in figure 2, be divided into two main sub-scores, the efficiency and effectiveness score. Efficiency is analysed in two aspects: The cost efficiency and productivity of the respective lab. The latter is assessed through KPIs which consider e.g., the handled samples, the processed batches and the number of tests being performed in relation to the number of QC FTEs. The effectiveness score is also divided into two sub-scores: The quality and service score. The service category considers KPIs such as Lead, Cycle and Release time. The quality section contains KPIs like lab deviations, CAPAs overdue or recurring deviations. Figure 2

Besides being calculated for the QC lab which is benchmarked, the specific QC lab performance score is also calculated for each QC lab of the peer group. Within the peer group all metrics are normalised to define the high performer group, which provides the lab with an achievable target for subsequent improvement actions that aim at becoming an industry-leader. The high-performer group includes those labs, which belong to the top 25 per cent regarding overall performance. When comparing the specific KPIs of the QC Lab, a comparison is then made to the high performer group, which always stays the same. That way a comparison to the respective KPI value is made for the average value of those KPIs of the high performer group. Following this approach, it is possible to be compared to a group which reflects respective tradeoffs and not to an artificial virtual best practice value for each single KPI, the combination of which is hardly attainable in the real world.

Besides the status-quo performance assessment based on the abovedescribed set of KPIs, the St.Gallen QC Benchmarking model also incorporates a maturity assessment of the management approaches taken, using a dedicated set of questions evaluating the use of enabling improvement practices. Those enabler questions are raised in the

AUTHOR BIO

Gian-Andri Steiger is a research associate at the University of St.Gallen. At the Institute of Technology Management, he works in the Operational Excellence team with a special focus on the pharmaceutical industry. Currently, his academic research focuses on quality assurance excellence in the pharmaceutical industry. Lorenzo Pirrone is a research associate at the University of St.Gallen. At the Institute of Technology Management, he works in the Operational Excellence team with a special focus on the pharmaceutical industry. His research focus lies in the field of maturity development and performance measurement. Marten Ritz is a postdoctoral researcher at the University of St.Gallen. At the Institute of Technology Management, he leads global collaboration projects to support pharmaceutical companies in developing KPI strategies and implementing quality metrics programs. His academic research focuses on aligning OPEX initiatives across manufacturing and QC lab in pharmaceutical production. Thomas Friedli is a director at the Institute of Technology Management. He leads a division of 14 PhD students and two post-docs. His research areas include strategic management of production companies, management of industrial services, and operational excellence. He is editor, author and co-author of numerous books and articles.

format of a self-assessment and cover various topic fields for which a direct cause-and-effect link to the performance dimensions is given. The enablers are thereby structured in three main categories, which contain each specific sub-categories. The maintenance and quality system is the first category of enablers, which focuses on practices that ensure process stability. For example, the question is asked whether housekeeping checklists are used in the lab. The lab representatives have then five statements available for their multiplechoice answer. Those statements reach from “No housekeeping checklists are used”, over “Housekeeping checklists exist, but are adhered to unevenly”, up to “Regularly updated checklists exist and are consistently adhered to”. By choosing the statement that best fits the situation of the respective lab, an initial evaluation of the maturity of the implemented enablers is possible. The second category planning & steering system is focusing on elements that enable efficient operational processes such as the reduction of setup times or the layout optimisation of labs, as well as planning related instruments. The overall objective of this enabler category is to ensure that available resources are optimally used. Lastly, the management system incorporates culture related elements in the model that support an effective implementation of improvement actions and that effective management structures are in place. Based on the outcome of the precedent performance assessment, the enabler maturity analysis allows a systematic identification of improvement fields and the definition of corresponding improvement actions.

Since the St. Gallen OPEX research team has also long-lasting experience in benchmarking pharmaceutical production, several interconnections between the QC and manufacturing OPEX performance became apparent. With respect to the field of QC, the planning accuracy measured by the adherenceto-schedule (ATS) highly depends on the information flow along the entire value-chain, including manufacturing. Only if sufficient information on production planning is provided, a reliant QC operations planning will become possible. This is one of many examples that shows the importance of cross-departmental process management. Pushing OPEX activities in both fields in an aligned and holistic manner is most promising to achieve lasting improvements.

Outlook

Constantly growing competitive pressure forces pharmaceutical companies to continuously work on improvement of their operational processes. This holds for both, the field of manufacturing and QC. Following an integrated management approach, the objective in the future will be to further improve the companies’ processes end-to-end. Thereby, a major challenge is how to align individual processes across various departments and how to coordinate activities across the fields of QC and manufacturing.

Literature is available at www.pharmafocusasia.com