Guidance 004 – Analytical Test Method Validation – Precision and Accuracy

Reproducibility:

Reproducibility is not normally performed during method validation. Reproducibility expresses the precision between laboratories and is assessed by means of an inter-laboratory trial.

Reproducibility should be considered when an analytical procedure is to be used in multiple laboratories or sites.

Reproducibility can be assessed in a formal collaborative validation study using multiple laboratories or by comparison of results reported for the same sample by multiple laboratories (e.g. in registration documentation). Execution of Analytical Technology Transfers provides a means of assessing inter-laboratory equivalence using an analytical transfer study.

 Repeatability:

Repeatability expresses the precision under the same operating conditions over a short interval of time.

Repeatability is also termed intra-assay or within run precision.

Repeatability should be assessed using six separate solutions of the test substance at the target method concentration for major component type assays. Alternatively repeatability can be demonstrated by performing 3 replicates each of three separate sample concentrations (9 determinations) covering the specified range of these procedures.

– Recommended Repeatability Data:

Calculate the result for each replicate. The % RSD for each level should meet the recommended criteria.

– Suggested Repeatability Criteria:

Several factors should be considered when selecting criteria: The intended purpose of the test and the expected specification range are important parameters. It is recommended that acceptance criteria be established as recommended in Table 1 and Table 2.

 Intermediate Precision:

Intermediate precision expresses “within laboratories” variations (e.g., different days, different analysts and different equipment).

The extent to which intermediate precision may be established depends on the circumstances under which the procedure is intended to be used. It is suggested that sites establish the effects of critical random events on the precision of the analytical test procedure.

Typical variations to be studied include days, analysts, equipment, etc. ICH does not consider it necessary to study these effects individually and this is endorsed by this guideline.

The use of an experimental design (matrix) is considered useful. Certain markets (i.e. Japan) have more specific requirements for intermediate precision. To meet intermediate precision requirements for Japan for assay and quantitative impurity procedures, it is recommended that the analyses be carried out as prescribed by the method over a minimum of six occasions with at least three analyses per occasion.

An example of such a matrix for Japanese markets is provided in the following table:

Analyses can be carried out using either samples spiked at a suitable level(s) and/or representative lots containing a representative amount of impurities. If the representative lots do not contain specified impurities/degradation products, spike studies should be performed. In cases where specified impurities/degradation products are not available a surrogate material such as a compound with similar structure or API may be used to demonstrate precision. In these cases, a rationale for the use of a surrogate should be given.

Recommended Intermediate Precision Data:

Intermediate Precision: Calculate overall % RSD of the multiple occasions. The overall SD or RSD of the multiple occasions should meet the recommended criteria.

Recommended Intermediate Precision Criteria:

Several factors should be considered when selecting criteria: The intended purpose of the test and the expected specification range are important parameters. The acceptance criteria should be established with respect to the internal release limit. For impurities, precision should be established at the release limits and shown to conform to the limits in Table 1. Greater uncertainty is acceptable at lower levels, as reflected in the table. The criteria in the table apply to % RSD based on reportable value.

Accuracy

Accuracy may be evaluated by comparing the results of another assay for the same impurity or component. This may be especially useful for tests such as Karl Fischer or LOD testing. The need for accuracy for these tests depends on the use of the test. For example, when LOD is used to adjust the potency of a material, accuracy should be assessed. Assurance of lack of side reactions and complete material dissolution is often the focus of KF testing for accuracy.

For reaction mixture IPCs, accuracy may be less significant since in many cases one is determining the point at which a stable value is obtained. If the method is used to ensure consistency from one lot to another (i.e. that the result compares favorably to previous results), then accuracy may not be significant.

For example, when area% methods are used with no specific Relative Response Factor (RRF) or external standard assays with an assumed RRF of 1.0 are utilized. When accuracy is not evaluated, other validation elements such as precision may become more important. If excluding accuracy from the validation, it is recommended that exclusion of this element should be justified and documented.

The accuracy may be inferred once precision (repeatability and intermediate), linearity and specificity have been established. Accuracy may be concluded if no significant level of interference has been observed.

Accuracy is important when a specific value is needed. For example, in the case when a method is used to monitor an impurity that is not reduced in downstream processing or if a minimum titer or amount is required (e.g. specific molar ratio) or if the assay is used to calculate the amount of catalyst needed to drive a reaction to completion.

Accuracy may be established across the specified range of the analytical procedure. Accuracy may be assessed using a minimum of 9 determinations over a minimum of 3 concentration levels covering the specified range (e.g. 3 concentrations /3 replicates each of the total analytical procedure). Accuracy can be reported as percent recovery by the assay of known added amount of analyte in the sample or as the difference between the mean and the accepted true value together with the confidence intervals.

Accuracy may be established by one of the following:

Application of the analytical procedure to an analyte of known purity (e.g., a reference material or stock standard) and demonstration that the expected true value is obtained. Accuracy should be determined across the range. This can be accomplished by spiking the analyte of interest with a known amount of concentration of the analyte material.

In cases where specified impurities/degradation products are not available a surrogate material such as a compound with similar structure or API may be used to demonstrate accuracy. In these cases, a rationale for the use of a surrogate should be given.

Known amounts of impurities or degradation products may be added to the process solution. The spiking procedure should include the high and low extremes of the range plus an intermediate value.

Recommended Accuracy Data:

Percent recovery is calculated for each reportable value as defined in the method. The average percent recovery may be calculated at each level and compared to the acceptance criteria.

Recommended Accuracy Criteria:

Several factors can be considered when selecting criteria: The intended purpose of the test and the

expected specification range are important parameters. See Tables below for recommended acceptance criteria.

Statistical Basis for Acceptance Criteria for both Accuracy and Precision:

The following recommended criteria in the Tables are derived to insure that the method can support its intended purpose, which is release of product against specifications. For impurity methods, the accuracy of the impurity determination can be either determined concurrently with the method precision recovery of spiked impurities or by spiking the impurity into a sample at approximately the Quantitation Limit (QL), 100% and 120% of the specification limit. The overall % RSD of results from multiple occasions should meet the recommended criteria.

For impurities, the following tables may be used as guidance for setting acceptance criteria that is also based on the specification. These recommended acceptance criteria are based on what can typically be achieved by an impurities method, including those that are Area% methods.

Table 1: Recommended Criteria for Precision and Accuracy – Higher Risk Test Method Impurity Determinations.

Impurity Spike Precision Accuracy

The table below summarizes the recommended target criteria for accuracy and precision for major component assays. The recommended precision criteria for Assay methods depends upon the intended use of the method. Higher risk methods are recommended to utilize more conservative acceptance criteria (+/-3 standard deviations) as outlined in Table 2. Medium risk methods are recommended to utilize +/- 2.5 standard deviations and Lower risk methods to use +/-2 standard deviations.

The criteria are categorized by actual or anticipated specification limits of the product.

Methods that meet these criteria are considered sufficiently accurate and precise to support the product specification limits. More exact criteria for accuracy and precision, based upon process capability indices, can also be used. These criteria define a method as suitable if its mean recovery +/-3 standard deviations (typically intermediate precision) for higher risk methods fall within the specification limits.

*If degradation of the material is known to be a concern and/or if this is considered a critical test method, then more conservative criteria are recommended.

** For % Recovery, the amount present in the unspiked sample (if any) should also be taken into account.

To use the tables, first determine the Risk category of the method to determine which Precision column to use.

Next, determine which ‘specification driver’ is relevant to the material being assayed in order to select the category in the correct row in the above table. Typically, for main analytes, the relevant specification driver corresponds to the internal specification limit of the product. For in-process intermediates one can use the 90-110, 80-120, or 60-140% range, depending upon the needs of the process. The target criteria for precision are given in terms of a relative standard deviation (%RSD) for both repeatability and intermediate precision (the criteria for both is the same since the most relevant consideration is the day to day variability). The criteria apply to the RSDs of reportable values, which may themselves be based on averages. These criteria assume that

the limits are symmetrical about the target value. If the limits are not symmetrical then the more restrictive of the two limit ranges could be applied.

For accuracy, the target criteria apply to the overall mean recovery at each level tested. If each individual result is required to meet the criteria, then the range may need to be broadened to avoid a statistically high incidence of failure. It should be noted that if the range is broadened to accommodate variation due to method precision, then there is the possibility that an inaccurate (but precise) method would meet the acceptance criteria.

As an example of the target criteria using mean recovery, consider a typical major component method supporting product specifications of 95-105. The target criteria for precision for a higher risk method would be RSD<1.7% for both repeatability and intermediate precision. Reportable value definitions that met these criteria would be considered to have acceptable precision. The accuracy criteria would be overall mean recovery within 97.5-102.5%.

Background Rationales

The precision criteria are based upon the use of standard deviation prediction limits derived from the overall average and intermediate RSD. The prediction interval for higher risk methods (using 3 standard deviation) defines a range that captures nearly all (>99.7%) future results from the method. The prediction limits are compared to the product specification range. If the prediction limits fall within the specification range then the method is sufficiently accurate and precise to support its intended use (releasing product against specifications). If either limit falls outside of the specifications, the method may not be either accurate enough or precise enough to support the specs. Note that because the prediction limits are a combination of the accuracy and precision, either (or both) could be far enough removed from desirable values to cause the prediction limits to fall outside of specifications.

The choice of the 3 standard deviation range for precision for higher risk methods is driven by two factors. The first is to keep the computation simple. The second is to keep the risk low of failing to meet specifications because of method variation alone. Methods that meet the prediction limit criteria have less than a 0.3% risk of failing to meet specifications due to method noise alone. The choice of this risk level is somewhat conservative, but the choice is driven in part because the specification range must allow for process variation as well as analytical method variation.

The target criteria provided in Table 2 are an attempt to develop separate criteria for precision that are consistent with the set standard deviation prediction limit approach. For higher risk methods the precision criteria represent the maximum RSD (rounded to 1 decimal place) that still allow 3 standard deviation prediction limits to fall within specification limits if the assay is unbiased – essentially the criteria are equal to 1/6 of the specification range. The accuracy criteria allow the method bias to be half as large as the specification range. The tabled values are somewhat riskier than the actual prediction interval since one could be at the maximum of both the accuracy and precision limits and still be acceptable. The total risk approaches 7% in the worst case. When a method has the potential for or is known to have bias then the prediction limit approach is preferred.

For medium risk methods a 2.5 standard deviation precision criteria was chosen, and for lower risk methods a 2 standard deviation precision criteria was chosen. This was due to the inherent lower risk of these types of methods.

The accuracy (%Recovery) criteria are established such that the deviation of recovery from 100% is no greater than 25% of the limit range to which the assay is applied.