1. Purpose
The purpose of this Guideline is to provide guidance for the investigation and response to Out of Specification (OOS) laboratory test results.
2. Scope and Applicability
This Guideline applies whenever a laboratory test result is out of specification. It may apply to any laboratory, its joint ventures and licensees who operate to cGMP standards.
Contract laboratories, performing GMP work, would be expected to follow equivalent procedures which should be confirmed as part of the contractor selection process. It is applicable to excipient, component, raw material, intermediate, active pharmaceutical ingredient and finished product release testing, stability testing and any other testing where the material has regulatory, compendial or internal specification limits associated with it. It applies to chemical and physical tests. The guideline does not apply to stressed stability testing which is normally conducted in the development phase within R&D, where OOS results may be anticipated and accepted.
However, it is recommended that for all time points during a stability program where OOS results are initially reported, they are documented as an OOS although the investigation as described in this guideline may not be applicable. Further guidance is given in the body of this document. It does not apply to biological tests where procedures for retesting are described in the relevant pharmacopoeias nor to in-process tests, which are used to trigger real time process/system adjustments during manufacture to prevent process drift.
For certain tests, including content uniformity, weight uniformity and dissolution, specific retesting procedures should be applied as defined in relevant pharmacopoeias. However, it is recommended that any initial OOS result is documented as described in this guideline and an initial laboratory assessment conducted. Specific guidance for tests on multi dose inhalation products is also provided Some of the principles, including the recording, assessment, reporting and trending, described in the guideline may be applicable to Out of Trend (OOT) results.
3. Definitions
3.1 Out of Specification (OOS) Test Result
A laboratory test that is outside its regulatory or compendial limits. In some cases, there may be additional tests and/or limits that are used to assess the quality of a material, but are not included in registrations or compendia. In these cases, the general principles described here are useful, but more latitude is allowable in the disposition of the material as long as it meets its legal requirements.
3.2 Accelerated Stability
Testing Studies designed to increase the rate of chemical degradation or physical change of a drug substance or drug product by using exaggerated storage conditions as part of formal stability studies.
3.3 In-Process Test
Test performed during production which may trigger real time equipment/system adjustments to prevent process drift and to ensure conformance with the required quality/specifications.
3.4 Laboratory Error
Incorrect performance of one or more of the steps of an analytical procedure and/or any identifiable laboratory related problem such as malfunctioning equipment etc.
3.5 Original Sample
The sample that was originally collected for the purpose of laboratory testing. This can be either the original test solution, a test preparation such as a tablet grind, or the group of tablets, capsules, vials or other dosage forms which were originally designated as laboratory samples.
3.6 Outlier Result
A result which is statistically different to the other results for a batch when assessed using a suitable statistical tool, e.g, Dixon’s outlier test.
3.7 Out Of Trend (OOT)
Test Result A laboratory test that is within its regulatory or compendial limit but is atypical of previous results for the test over a number of batches or earlier time points in a stability study and may provide early indication of a potential OOS result. The importance of OOT results increases as the knowledge of the product increases and more latitude in interpretation of, and response to, an OOT result may be appropriate in early development compared to commercial product.
3.8 Process Related Error
An error caused by incorrect performance of the manufacturing steps, malfunctioning process equipment, or issues associated with changes of equipment, scale-up or other problems associated with the process. 3.9 Stressed Stability Testing Studies designed to elucidate intrinsic stability of drug substance or assess the effect of severe conditions of the drug product. Such testing is normally part of the development activities and is carried out under more severe conditions than accelerated stability testing.
4. Responsibilities
4.1 Department / Function
It is the responsibility of each department or function to put in place procedures for the handling of OOS and OOT test results.
4.2 Analyst
The analyst is responsible for:
– Being aware of potential problems that could occur during the testing and watching for problems that could create inaccurate results.
– Ensuring that all equipment used are suitable for their use and properly calibrated when appropriate.
– Ensuring that all system suitability requirements are met.
– Reporting an OOS or OOT result to Line Management.
– Investigating the result together with Line Management.
4.3 Line Management
Line Management is responsible for:
– Conducting the initial laboratory assessment and extended investigation, where appropriate.
– Preparing the report with the analyst.
– Determining the protocol for retesting where required.
4.4 Quality Assurance (QA)
QA is responsible for:
– Ensuring that the extended investigation is carried out, where required.
– Determining the disposition of batches subject to a confirmed OOS result, for batches intended for commercial distribution, clinical studies and for stability studies supporting MAA/NDA.
– Ensuring that trending of OOS and OOT results is carried out.
4.5 Project Team within R&D
The Project team within R&D is responsible for:
– Assessing the implications for an OOS test result obtained during stability studies, including those from accelerated and stressed studies, with respect to defining and supporting specifications and/or product shelf lives.
– Using such OOS data to support proposed changes in regulatory specifications during development, or where appropriate, post NDA/MAA approval.
– Defining the strategy for continuing the stability study when OOS results have been obtained.
5. Guideline
5.1 General
A result is considered to be reportable and available for compliance with specifications, compendial limits or trends when it has been checked by a second analyst. When a laboratory test produces an unexpected OOS result, it is not sufficient to simply repeat the test and ignore the original result. An investigation must be conducted whenever an unexpected OOS test result is obtained. An investigation must be done to determine whether the test result is due to a laboratory error, a true measure of the batch or a sampling error. In some cases, however, the OOS result may be expected and accepted (stressed and/or accelerated stability samples) and no investigation or retesting may be required.
In such cases, the OOS should be documented and a review and justification should be undertaken. It is recommended, however, that the first occurrence of an OOS during these studies is registered and investigated using the OOS procedure and subsequent results referenced to it. However, if the OOS result is unexpected, the purpose of the investigation is to determine the root cause of the OOS.
An initial assessment should be based within the laboratory and, where appropriate, should be referenced in an extended investigation, should this be required. A flow chart outlining the guideline is attached (Appendix A). The presence of an OOS result does not necessarily require the batch to be rejected but indicates that an immediate investigation should be conducted for batch disposition purposes. Even if the batch is rejected based on an OOS result, the investigation is necessary to determine the cause of OOS result and assess the potential impact on other batches of the same drug product or other products. Batch rejection does not negate the need to perform the investigation.
The investigation is also required to determine if there are any underlying root causes that need to be addressed. OOS results should be rare occurrences and investigation reports should be reviewed regularly by Line Management and checked for common errors. Frequent OOS results may be an indication of a problem and must be brought to the attention of management for appropriate action. Where a process related error has occurred the OOS result may be accepted and the laboratory phase of the extended investigation, i.e. extensive retesting, may be unnecessary. Certain analytical methods have system suitability and calibration requirements and analyses not meeting these requirements should not be used. Any data collected during the suspect time period should be identified and not be used.
System suitability failures as well as failure of standard injections during a run must be documented. A pattern of such failures requires investigation. In these cases, the investigations need not be documented using the “OOS procedure”. Additionally, if a specific analytical method and/or a standard operating procedure (SOP) for a given technique has defined acceptance criteria for replication of injections and the replication for the sample is outside these criteria, these data should not be used and re-analysis as per original test should be undertaken. These data must be documented and any pattern of poor replication investigated. If errors are obvious such as spilling of a sample the analyst shall not continue with an analysis that would be invalidated at a later time for this assignable cause. In this instance an OOS investigation is not required provided the analyst immediately documents what happened in the analytical records.
If the OOS result is for a test such as dissolution or content uniformity, where individual dosage units are assessed, a retest at the same stage is only appropriate if an identifiable cause was found during the initial assessment. To be meaningful, the investigation must be thorough, timely, unbiased, well documented and scientifically defensible. The first phase of such an investigation should include an initial assessment of the accuracy of the laboratory’s data, before test solutions are discarded, whenever possible. A written record of the investigation must be made including the conclusion of the investigation and follow-up. All OOS results shall be trended.
This may be done by means of a log, which should be reviewed periodically in order to detect developing trends. Quality Assurance should ensure that trending is carried out but it may be undertaken by the laboratory function. The frequency of trending may depend on the frequency of OOS results; for R&D where analyses may be less frequent than in Operations’ laboratories, an annual trend is considered adequate. For Operations, where large throughput of analyses may be undertaken, a quarterly trend may be more appropriate.
It is recommended that some of the principles described here are applicable to observed OOT results. For example, OOT results should be documented and there should be an initial laboratory assessment to see if there is evidence of a laboratory error. Retesting may be appropriate if a confirmed laboratory error is found otherwise the result is accepted. The assessment and implications should be documented. OOT results shall be trended and confirmed trends reported to For commercial products this may be done via the Annual Product Review process.
5.2 Initial Laboratory Assessment
As soon as an analyst determines that a test result is OOS and before any further testing is carried out, the Line Manager must be notified. Together, they conduct a laboratory investigation to determine if a laboratory error can be identified as the cause of the OOS result. The Line Manager’s assessment should be objective and timely. The following areas should be reviewed and documented, (the use of a pro-forma may be helpful):
– Test method and analyst’s knowledge and ability to use it including changes to automatic calculation methods
– Performance and calibration of the instruments
– Review of quality of reference standards, reagents and solvents
– Method validation data, where applicable
– Check of sampling procedure
This phase of the laboratory investigation must include an intensive assessment of possible causes. The goal is to determine if a laboratory error occurred or if there is reason to suspect that an error occurred. A simple checklist approach is not sufficient and will only reveal a laboratory error in the most obvious circumstances. If this phase of the investigation is not comprehensive, valuable information concerning root cause may be lost. The initial assessment can be greatly facilitated if all laboratory glassware, pipettes, reagents, and solutions are kept together until after the test results are calculated. It can be helpful during the course of the assessment to re-inject or otherwise re-read the original sample solution in order to identify instrument malfunctions or incorrect dilutions. Each step of the assessment must be fully documented and the outcome reviewed prior to batch release or reporting of stability analyses. The result from the initial laboratory assessment phase can be either of the two following:
– Laboratory or sampling error confirmed
– Laboratory or sampling error not confirmed
5.2.1 Laboratory or Sampling Error Confirmed
If the error is due to a written or calculation error, the error may be corrected without further re-analysis. If the error is due to incorrect experimental condition being used, it is appropriate to re-analyze the original sample. If a sampling error is confirmed, the batch may be re-sampled and normal analysis undertaken. These errors must be clearly demonstrable, not just a supposition. The re-analysis is equal in number to that used for the normal test and the re-analysis results replace the original results.
In addition, the investigation should extend to other samples, which may have been tested using the same equipment or by the same analyst and actions should be taken to prevent recurrence of the laboratory error. Laboratory errors should be relatively rare. Frequent errors suggest there may be a problem that might be due to inadequate training of analysts, poorly maintained improperly calibrated equipment or careless work. These frequent errors must be investigated to determine the source of the error and correct and preventative actions implemented.
5.2.2 Laboratory or Sampling Error Not Confirmed
When the initial laboratory assessment does not determine that a laboratory or sampling error caused the initial OOS result, two possibilities remain – that the result reflects the actual batch condition or that the result is due to an unidentified laboratory error. Both possibilities must be investigated using an extended investigation in accordance with predefined procedures.
5.3 Extended Investigation
This investigation may comprise of two phases, a laboratory phase and a product/manufacture phase and may be conducted sequentially. Either phase may be commenced first. (Usually for Operations the laboratory phase of the investigation will be conducted first.) The OOS investigation, including both laboratory and product/manufacture phases, if required, must be completed in 30 working days of the initial checked result. Any extension beyond this timeframe should be rare and must be justified and documented.
For example, for R&D during the manufacture of an active pharmaceutical ingredient (API), and where an OOS result is obtained on a batch of raw material or intermediate, it may be appropriate to conduct the product/manufacturing phase of the investigation prior to any retesting. If this investigation confirms that the OOS result is process related, the result may be accepted without the need for retesting. The batch may be rejected, reworked/reprocessed or subjected to user trial processing without retesting.
The user trials may be undertaken to confirm that the OOS result does not have an adverse effect on the processing and/or quality of the next stage of manufacture. In these cases an OOS result during development can be accepted and used to justify changes in the specification for the raw material and intermediate. If OOS result is accepted the batch may be rejected and subjected to reprocessing or reworking. Where the investigation is inconclusive, extensive retesting should be carried out as part of the laboratory phase of the extended investigation.
5.3.1 Laboratory Phase of the Extended Investigation
Where the initial laboratory assessment has not found a laboratory error or the product/manufacture phase of the extended investigation is inconclusive, an extended retest may be appropriate to determine whether the OOS result was due to an unidentified laboratory error.
However, before commencing the laboratory phase of the extended investigation a preliminary investigation into the product/manufacture of the batch in question should be undertaken. Manufacturing records should be reviewed to determine if there is a possible cause of the initial OOS result and this assessment documented prior to any further analytical testing. The number of retests should normally be at least five.
In some instances the amount of retesting may be determined by the nature of the product, the precision of the test, the history of the product, and any other appropriate scientific reason and may require more than five retests. This protocol must have a clearly defined end point and must be signed by the Line Management prior to any retesting being started. Where retesting is performed to verify an OOS result, the number of tests may be limited (i.e. less than five) but where testing is performed to provide additional data for consequent use in assessing the final disposition of the batch, at least five tests must be performed. When retesting is performed, it shall be done using the original sample container or composite sample that yielded the initial OOS test.
Any further OOS results, which are observed during the retests, unless invalidated by an identified laboratory error, make this a multiple OOS and necessitate the product/manufacture phase of the extended investigation. Retests, when appropriate, may be performed by the original or a second analyst depending on the circumstances After the retesting is complete, a decision should be made as to whether the retest data are sufficient to conclude that the original OOS was in error. All test results, both passing and suspect, must be reported and considered in the batch disposition decisions. This decision is based upon the results of the investigation, the results of any retesting, the variability of the data, the history of the product, precision of the test method and other scientific factors.
An initial OOS result will not be averaged with extensive retest results for the purpose of generating a mean value to justify a passing disposition result for the batch. The results of the retesting, along with the retesting protocol and any conclusions, shall be documented in a report/completed worksheet and reviewed prior to batch disposition. See section 5.3.1.3 “Averaging” below for further guidance on generating a single result reportable result for the batch. If further OOS results are obtained, then a full- scale product/manufacture investigation should be conducted. (5.3.2).
5.3.1.1 Inhalation Products
The following additional guidance should be considered for multi dose inhalation products e.g. Metered Dose Inhaler products (MDIs) and Dry Powder Inhaler products (DPIs). An OOS result obtained during a delivered dose uniformity test or particle size distribution test, where specific dose regimens are defined, shall only be discarded if there is unequivocal evidence that a laboratory error has occurred in the analysis.
Typically for such tests, numerous containers from the batch (or stability Time point) are taken and samples from individual containers taken, representing the delivered dose throughout the container life (e.g. beginning, middle and end) may be analyzed. This may provide a large data set of individual results for the initial analysis.
Where appropriate, retesting on the same container (which had the OOS) may be done to facilitate the investigation, it is recommended that a minimum of 2 full analyses be carried out where an individual OOS result is obtained to gather additional data for reporting and/or batch disposition assessment purposes. All results must be reported for regulatory submission and batch disposition purposes.
The results of retesting, along with the retesting protocol and any conclusions, shall be documented in a report/completed worksheet and reviewed prior to batch disposition. For other parameters, e.g. assay and moisture, a number of determinations from a specified number of individual containers may be defined in the method and the result for the batch may be the mean of all determinations. Where an OOS result is obtained from one of the containers an extensive retest from the same container is allowed, where this is sufficient product.
This testing, plus any additional testing of new containers, may be used provide additional data for consequent use in assessing the final disposition of the batch. If all determinations meet the requirements and although all results must be considered for batch disposition decisions and following an extensive investigation, it may be concluded that the initial OOS result did not reflect the true quality of the batch and that the batch may be considered suitable for release.
Where the repeat determinations from the same container indicates a further OOS result(s), a minimum of 5 full analyses as defined in the method are recommended and a product/manufacture phase of the extended investigation conducted as described below in 5.3.2 is required. All results must be considered in batch disposition decisions.
5.3.1.2 Outlier Tests
Outlier tests should generally be used in the context of an extended investigation, may be used infrequently and based on scientific judgment Outlier tests shall not be used for tests such as content uniformity or dissolution where the variability of individual units within the batch is being assessed. The use of outlier tests for validated chemical tests with a small variance where the sample may be considered homogeneous, are appropriate to identify extreme values but they do not identify the cause and should not be used to invalidate results. Their use should be determined in advance, e.g in the SOP for data evaluation and should state in advance the minimum number of results required to obtain a statistically significant assessment from the specified test.
In very rare occasions and only after a full investigation has failed to reveal the cause of the OOS result, a statistical analysis may be valuable as one assessment of the probability of the OOS result as discordance, and for providing perspective on the result of the overall evaluation of the quality of the batch. For example, an outlier test may be used to identify a single extreme low response from a standard injection among several replicate injections during an HPLC run but the result should only be rejected based on scientific judgment considering factors such as the precision of the test and any possible equipment malfunctions.
5.3.1.3 Averaging
Averaging can be a valid approach but its use depends on the sample and purpose. Averaging shall not be used to hide OOS test results. It can provide a more accurate result for preparations from a homogeneous sample but unless averaging is specified by the test method, or related procedures, all individual results must be reported. Where averaging of replicate injections to produce a single result is defined by the method, and the acceptance criteria for the replication have been met, it is acceptable to average one replicate injection that apparently appears OOS with a replicate(s) which appear within the specification to produce a single test result, as defined by the method.
If the test result is OOS, the requirements of this guideline should be followed. If the reported test result is within specification then no further investigation is considered necessary although all individual results should be documented. Where acceptance criteria for replication of injections, which should be specified in the method or associated procedure, have not been met, these data should not be used and re-analysis as per original test should be undertaken. Unexpected variation in replicate determinations should trigger remedial action and test results from this analysis should not be used.
In the context of additional extended testing performed during an OOS investigation, averaging of the OOS result from the original test that prompted the investigation with retest results is not appropriate. However, it is critical that the laboratory provide all individual results for evaluation and consideration in the final assessment for batch disposition purposes. Where it is considered necessary to report a “single result” which is representative of the batch, e.g. on a Certificate of Analysis or when individual result are averaged in a laboratory data reporting system (LIMS), the following guidance should be followed:
– The laboratory record or database should include all individual results, including initial OOS results, either confirmed or for which no assignable cause has been established but which cannot be repeated
– The average should not include any initial OOS results.
– The average should include the satisfactory initial results and satisfactory retest results.
– Consider the following examples:
The initial analysis was a duplicate weighing and one result was OOS, triggering the investigation according to this guideline. There was no evidence of laboratory error, manufacturing or processing error and a further five sample weighing were taken, and shown to be satisfactory. The reportable result for purpose of the Certificate of Analysis or LIMS reportable result is considered to be the mean of the initial satisfactory result and the five retest results.
The initial analysis was a duplicate weighing and one result was OOS. Where the investigation, according to this procedure, concluded that a laboratory error had occurred, resulting in a normal retest (duplicate weighings), the reportable result would be mean of the duplicate retest results.
5.3.1.4 Re-sampling
Under normal circumstances data obtained from re-sampling cannot be used to release product unless it has been shown that there was a sampling error or that the sample was stored improperly and/or that the original sample was not representative of the batch. Where a sampling error has been found, an assessment should be made as to the implications of the error on other batches sampled in the same manner. Re-sampling is also permissible where there is insufficient quantity of the original sample to undertake the extensive retesting. Re-sampling should be performed by the same qualified methods that were used for the initial sample. However, if the investigation determines that the initial sampling method was in error, a new accurate sampling method shall be developed, qualified and documented.
5.3.2 Product/Manufacture Phase of the Extended Investigation
When the laboratory phase of the investigations does not determine that laboratory error caused the OOS result and testing results appear to be accurate, a full-scale failure investigation using a predefined procedure shall be conducted. The objective of such an investigation shall be to identify the source of the OOS result. Varying test results could indicate problems in the manufacturing process, or result from sampling problems. The product/manufacture phase of the investigation shall involve QA and personnel from other departments as required. The purpose of the formal investigation is to determine whether a process error has occurred. The following general items should be included in the investigation as appropriate:
– The reason for the investigation.
– Responsibilities for actions including preparation of the final report.
– A review of the manufacture and a summary of the process steps that may have contributed to or caused the problem.
– The results of a review to determine if the problem has occurred previously
– Corrective actions required for the batch under investigation and its final disposition.
– Preventive actions required to prevent a recurrence of the problem.
– Other batches or products that may be implicated by the same employee(s), equipment, or process. Also, any corrective action that may be necessary for these batches.
– A review of the original laboratory assessment and/or reference to it.
– Results of any additional testing that may have been performed as part of the investigation.
Signatures and comments of QA and other personnel who conducted and approved the investigation. It may be appropriate to sample and/or retest specific parts of the batch to facilitate understanding as to which parts may be affected and which parts may not. The investigation must justify any considerations in treating parts of the batch differently than others.
If the investigation concludes that the OOS result was due to a process related error, then the OOS shall be considered valid and appropriate actions shall be taken. Where a laboratory phase extended investigation had been conducted prior to the product/manufacture investigation, no further testing is acceptable.
If the investigation is inconclusive, then an examination shall be made of all of the available data in the context of the product’s history, in-process test results, and any other pertinent factors. Based on the above, a scientific judgment shall be made as to its disposition.
5.4 Concluding the Investigation
To conclude the investigation, the results shall be evaluated, the batch quality shall be determined, and a release decision shall be made. Where the investigation concludes with part rejection/part release, this must be clearly justified and documented.
Once a batch has been rejected, there is no limit to further testing to determine the cause of the failure so that a corrective action can be taken. In those instances where an investigation has revealed a cause and the suspected result is invalidated, the results should not be used to evaluate the quality of the batch. In those instances where an OOS is confirmed, the result should be used in evaluating the quality of the batch.
A confirmed OOS result means that the batch does not meet its established standards and should result in batch rejection. For instances where there is an inconclusive investigation, those in which the cause of the OOS is not determined and the OOS result cannot be confirmed, the OOS results and all other results should be given full consideration in the batch disposition decision. This assessment should be fully documented.
5.5 Cautions
Where a series of assay results (to produce a single reportable result) are required by the test procedure and some of the individual results are OOS, some are within the specification and all are within the known variability of the method, the passing results are no more likely to represent the true result than the OOS results. For this reason, it is recommended that the reportable average is treated a OOS and investigation undertaken. Similarly, an assay result which is low but within specification and which may be supported by other tests, e.g. dissolution and/or content uniformity should be cause for concern, and caution exercised in release or reject decisions.
5.6 Stability Testing
5.6.1 Reporting of OOS Results in Development Stability Programs
An OOS result in development stability programs, including up to MAA/NDA may provide data to support the establishment of specifications and/or product shelf lives or in some cases, changes to previously agreed specifications/shelf lives.
These results may be obtained from long term, accelerated or stressed conditions. In all cases, the OOS should be documented as described in this guideline. For the first occurrence of an OOS in a given stability program, an investigation as described shall be undertaken.
If the result is accepted and considered to be a true measure of the batch, investigations of OOS results for the same test at subsequent time points may be less extensive but documented in the laboratory work sheets/notebook, referencing the initial OOS investigation and results accepted.
The project team within R&D shall assess the implications of the OOS test result obtained, with respect to specifications and/or product shelf lives and define the strategy for continuing the stability program when OOS results have been obtained.