Examination of audit trail entries is a key part of the second-person review. What do regulatory documents say about this topic, and how can we speed up review?
Audit trail review is a key component of the second person review of chromatographic analysis for compliance with regulations, procedures, and analytical science. We focus on what the review of audit trail entries means, and how to review by exception if the CDS has appropriate technical controls.
Examination of audit trail entries for an analysis is a key part of the second person review. It is over 20 years since EU GMP Annex 11 and 21
11 required audit trails in regulated applications, including the chromatography data system (CDS). What do the regulations say about review of audit trails? How can we speed up the review process? What is review by exception? How can we use this to save review time?
Where It All Began
All those lucky chromatographers who must review CDS audit trails have Able Laboratories to thank for the drudge of ensuring that the analysis has been performed correctly. As citation 1 of the Able Laboratories 483 Observation states:
...The Quality Unit failed to: review electronic data as part of batch release, review computer audit trails in the <Redacted> Data Acquisition System, and provide adequate training to analytical chemists (1).
Here we have the first regulatory citation for failure to review audit trails. However, audit trails, however rudimentary, have been included in major laboratory informatics applications such as laboratory information management system (LIMS) since the 1980s. The problem was that there was little agreement on what was required from a regulatory perspective.
Four Eyes Principle and Second Person Review
The generation of sound analytical results is based on the established “four eyes” principle; one person to perform the chromatographic analysis, and a second person to review the data to show that the work has been performed correctly and that no mistakes have been made. The involvement of a second person is to look with a fresh pair of eyes for anything that the analyst may be overlooked. Now, with the emphasis on data integrity, the second person review has been expanded to check that work has not been falsified, and must include review of the CDS audit trail entries.
The scope of the review must cover the whole of the analytical process: from sampling to the calculation of the reportable result, however we will only consider the audit trail review here. The terms performer and reviewer are stated in 21 CFR 211.194(a) (2). I will use the term “reviewer” or “second person reviewer” to indicate the one individual who conducts the checks to ensure that work has been performed correctly, and all data and records have been collected. The reviewer will use a general second person review standard operating procedure (SOP) to control their work, and will, in all probability, have a linked work instruction for each different CDS audit trail to be reviewed (unless you have standardized on a single CDS). Ideally, the CDS can support the audit trail review process with software functions (technical controls) to make it quick and efficient, as we shall discuss later.
What is an Audit Trail?
Before we can review audit trail entries, we need to define what and audit trail is, and then understand the regulations surrounding it and the review process. The simplest definition is found in the 2018 FDA guidance on Data Integrity and cGMP, where question 1c asks, “What is an audit trail?”
...audit trail means a secure, computer-generated, time-stamped electronic record that allows for reconstruction of the course of events relating to the creation, modification, or deletion of an electronic record. For example, the audit trail for an HPLC run should include the user name, date/time of the run, the integration parameters used, and details of a reprocessing, if any. Documentation should include change justification for the reprocessing.
Audit trails include those that track creation, modification, or deletion of data (such as processing parameters and results) and those that track actions at the record or system level (such as attempts to access the system or rename or delete a file)...(3).
How did we get here?
Audit Trail Regulations
21 CFR 11 (Electronic Records and Electronic Signatures) regulations has clause 11.10(e) that requires:
Use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records.
Record changes shall not obscure previously recorded information.
Such audit trail documentation shall be retained for a period at least as long as that required for the subject electronic records and shall be available for agency review and copying (4)
See where the definition of audit trail comes from in the FDA data integrity guidance? Straight out of the Part 11 regulation. However, life is not always simple, and Part 11 is no exception.
Interpretation of Part 11 by the Predicate Rule
Part 11 only defines the requirements for electronic records and electronic signatures, as that is the role of the predicate file. In our case, the applicable FDA predicate rule is either Good Laboratory Practice (GLP or 21 CFR 58) or Good Manufacturing Practice (GMP or 21 CFR 211), and you have to interpret these regulations for a complete understanding of FDA regulations for audit trails. For example, there are differences between the two predicate rules. GLP requires a reason for data change in 21 CFR 58.130(e) (5), but GMP does not (2). However, it would be a foolish quality control (QC) laboratory that did not implement a reason for change in today’s data integrity environment.
This now brings up to 2005, and the Able Laboratories case. How did the FDA cite Able for failure to review audit trail entries? Enter the GMP predicate rule, and specifically 21 CFR 211.194(a) for laboratory records. There are two specific requirements:
Laboratory records shall include complete data derived from all tests necessary to assure compliance with established specifications and standards, including examinations and assays, as follows:
8) The initials or signature of a second person showing that the original records have been reviewed for accuracy, completeness, and compliance with established standards (2).
Although this regulation has been effective since 1978, only since 2005 has the FDA interpreted it to include review of audit trails since the Able Laboratories fraud case. Now we see that when a CDS is involved, a key component of the second person review is to review audit trail entries.
For a networked CDS there will be thousands of audit trail entries – must I review all of them? To give a rational answer to this, we have to move to Europe, and the update of EU GMP Annex 11 in 2011, to see how regulators have coped post Able with data integrity and any possible data falsification.
Update of EU GMP Annex 11
In 2011, the update of Annex 11 was issued (6). Before we discuss the specific requirements for audit trail in the new version, it is important to understand that the full interpretation of Annex 11 requires an understanding of EU GMP Chapter 4 on Documentation (7). The updates of both these regulations were issued at the same time. Figure 1 shows the key sections of Annex 11 and Chapter 4 that are pertinent to our audit trail discussion.
Let us work through Figure 1 to understand the most recent regulations for audit trails and their review. First, there is the Chapter 4 requirement for Good Documentation Practices (GDocP) in section 4.7–4.9 (8), this was discussed in one of last year’s Data Integrity Focus articles (9), and will not be repeated here. However, the interpretation of these GDocP requirements on audit trail entries is very relevant, therefore all audit trail entries must be:
This interpretation comes only from the requirements of GDocP and records retention sections in Chapter 4 (7). We now turn to see what the audit trail requirements are in Annex 11.
Annex 11 Requirements for Audit Trail
The updated version of Annex 11 has the following requirements for audit trail documented in clause 9:
Consideration should be given, based on a risk assessment, to building into the system the creation of a record of all GMP-relevant changes and deletions (a system generated “audit trail”).
For change or deletion of GMP-relevant data the reason should be documented.
Audit trails need to be available and convertible to a generally intelligible form and regularly reviewed (6).
Normally, the requirements are presented as a single paragraph, but, for the purposes of this discussion, I have broken these requirements into sentences and clauses. First and foremost, an audit trail is not mandatory, as the phrasing is “consideration … based on a risk assessment…”. Before you all rush to turn your CDS audit trails off, please consider the following issues:
Therefore, we will not be conducting a risk assessment to justify not having an audit trail, as you will have to comply with clause 12.4. The next is that the audit trail focuses on GMP-relevant changes and deletions, unlike a Part 11 audit trail that also includes creation of the records in 11.10(e) (4). You have to interpret what are GMP-relevant changes and deletions.
Any change to data requires a reason for change, as this is consistent with the GDocP requirements of Chapter 4 (7). This can be implemented either by a drop-down list of context sensitive options to save typing, or by free text. Personally, I prefer the default reasons for change, as this ensures consistency. This option can take time to implement, but the basic functionality must be validated. However, when the CDS is operational, it is best controlled by procedure for adding reasons for change (10).
There is the requirement for audit trails to be available in a generally intelligible form, this refers back to the requirements for GDocP discussed above, but also the need for audit trail entries to be easy to understand and follow.
Finally, the three words that come directly from Able Laboratories: “And regularly reviewed.” This is the first, and currently only, explicit regulatory requirement to review audit trail entries, but interpretation always causes much debate and discussion. We shall discuss frequency of review later in this article.
What is An Audit Trail?
Now we have presented and discussed the three main regulations issued over the past 20–30 years for computerized system audit trails, which should be straightforward to interpret in practice.
But, no. We still lack adequate audit trails, or even the existence of an audit trail in many laboratory informatics systems as evidenced in a review of infrared spectroscopy software citations by Smith and McDowall in Spectroscopy (11). Therefore, we need to briefly discuss what is an audit trail, and this is presented in Table I.
From all of the regulations, we can show what an audit trail is and is not in Table I:
The key requirement is that an audit trail is an integral function in any CDS or any laboratory informatics application. As such, it cannot be bolted on as an afterthought of system design. To be encompassing and effective, the foundation of any audit trail in any application must be a database. The debate between a single audit trail containing all entries versus one for system related entries and one associated with data is outside the scope of this article.
What Are GMP-Relevant Changes?
For an effective and efficient review of audit trail entries, it is essential to understand what the phrase “GMP relevant changes and deletions” in Annex 11 means in practice. As the laboratory has configured and validated all laboratory user roles without deletion privileges, a reviewer will not be looking for any deletion entries, will they? That leaves us with just GMP relevant changes, Table II lists some audit trail entries that could be found within the audit trails of most, if not all, CDS applications. They are divided into general entries in the right-hand column and GMP relevant changes in the left one.
Let us take the entries in the right-hand column and discuss them first. Here we have logon, logoff, and failed logons, as well as creating an account and unlocking an account by an administrator. Are these GMP relevant changes? Your answer should be an unequivocal “no,” as no data have been changed. Equally so, are the archiving of an analyst’s project and changing a configuration setting of the application? At this point, there are probably wails of anguish coming from the direction of the quality assurance (QA) department. Let me be very clear here: These last entries are not part of a second person review process. But they will be covered by a QA led data integrity audit to ensure that these actions have the correct authorization, and have followed the appropriate procedure. The entries in the left-hand column of Table II are GMP relevant, and must be reviewed during a second person review of any analysis.
However, if you only have a single audit trail covering the whole system, this can present problems, as entries for all analyses, all user logon and logoffs,or any configuration changes can be found in one huge dustbin. To identify GMP relevant changes for the specific analysis that you are going to review needs good search routines.
Risk Management of Audit Trail Review
Next to discuss is a very important clause in Annex 11 that has a major impact on our discussion. Clause 1 states that risk management should be applied throughout the lifecycle…taking into account patient safety, data integrity and product quality (6). Risk management applies not just in the validation of the system, but also during operation of the CDS. Unsurprisingly, this should include audit trail review, but it often does not. Therefore, we need to consider how we can use risk management to reduce our work reviewing audit trail entries. Of necessity, this approach includes utilizing any technical controls that can be implemented in the CDS application to reduce the amount and number of entries to review.
Who Should Review Audit Trail Entries?
One of the main discussion points in training courses that I have participated in is who should be responsible for reviewing audit trail entries. Answers have varied from analytical development/QC, QA, or even (horror of horrors) IT! How can people and organizations get this so wrong? Audit trail review as part of second person review is a laboratory function. The rationale for my view? Its in the regulations, specifically 21 CFR 211.194(a) (2), and EU GMP Chapter 6.17 vii (12).
Recent guidance documents have reinforced this. The FDA’s approach in Question 7 of their data integrity guidance (3) is that the people responsible for record review under CGMP should review the audit trails that capture changes to data associated with the record as they review the rest of the record (for example, 211.194(a)(8) ). Similarly, PIC/S PI-041 guidance in section 9.5 states that audit trails for each batch should be independently reviewed with all other records related to the batch and prior to the batch’s release, so as to ensure that critical data and changes to it are acceptable .. and performed by the originating department, which is the laboratory (13). QA, please note! Your department can verify the effectiveness of the review during data integrity audits or investigations (13–15).
How Regular is a Regular Review?
This is another question with a multitude of wrong answers! Let’s see what is discussed in the data integrity guidance documents. FDA’s view, stated in question 8 of the 2018 guidance, (3) is that if audit trail review is mandated in 21 CFR 211, then this is the review frequency. If the interval is not specified, then determine this according to a risk assessment (a lovely get out of jail excuse!) based on knowledge of the process and the functions of the CDS application, and also include evaluation of data criticality, control mechanisms, and impact on product quality, to ensure that CGMP requirements are met, appropriate controls are implemented, and the reliability of the review is proven (3).
To help you let you understand this, let us move from the vague to the specific, and give laboratory examples. What audit trail review would you conduct for:
Method development: As method development is seen as outside of GMP, there is no need for a review of audit trail entries. This may be so as there is no mention of method development in ICH Q2(R1) (16). However, this is the critical foundation of a robust analytical procedure, and times are changing, as the analytical world is going to a lifecycle methodology with the publication of a draft USP <1220>, a revision of ICH Q2(R1) in the works. This will be the subject of the next “Data Integrity Focus” article.
Performing the Audit Trail Review
Now we can get down to describing how to perform an actual review of audit trail entries. Remember that an audit trail review is only a portion of the overall second person review that starts with sampling and finishes with the calculation of the reportable result. In this section, we will focus on the CDS audit trail review for one analytical run. In this discussion, the technical controls presented earlier are in place and validated to make the review process easier, and there is a procedure in place for review by exception.
First up, do we review audit trail entries on screen, or do we print them out? For those readers that selected the latter option, please write a letter of resignation immediately and reserve a bed at the local lunatic asylum, as you don’t understand how to make a process easier. We will be reviewing on screen. The reason for this is that an audit trail can contain much more information that fits on a screen, and printing can generate much, much more paper than anticipated. When it comes to on screen review, it is important that the reviewer has one and preferably two large high-resolution screens. Perhaps it is time to ask your boss for
a 55 inch 8K TV, I mean monitor, for the review?
Seriously though, this is an important task and the reviewer needs the right tools. Having an expanded chromatogram on one screen and the pertinent audit trail events on another can help understand and recreate activities more easily than on one small monitor and save switching between views of the data. Comparison and correlation are much easier on two screens, and faster.
Next, let us look at where in the CDS application that there could be GMP-relevant modifications (remember, deletions are not configured for the users). Here are some changes that should trigger audit trail entries:
These are some of the areas where there might be audit trail entries containing GMP-relevant changes.
Review by Exception
Review by exception is a term used to review only the exceptions in any analytical run, rather than each and every audit trail entry. In the discussion above Annex 11 required an audit trail for GMP-relevant changes and deletions (6). If no deletions are allowed in the CDS, then all you need to look for are the modifications or the exceptions to normal working of the system.
Consider an analysis: if all peaks are integrated automatically, do you need to look at the audit trail entries for manual integration? No. Each peak integration shows if a baseline has been placed automatically or manually with the integration codes BB or Bb for example. In the former, both the start and end baselines have been determined by the system; in the latter case, the trailing baseline has been positioned by a chromatographer. This is a slow process of looking at each chromatogram individually, and is only marginally more interesting than watching paint dry.
Does the CDS Aid Review by Exception?
A much better approach is if the CDS can highlight that there is no manual integration at the injection level AND this function has been validated, if there are no exceptions (manual integration) for the run, and you don’t need to review the pertinent audit trail entries.
The exceptions the reviewers should be identifying are the GMP relevant changes such as those listed in Table II. This is where the supplier of your CDS can be a great help or a hinderance to the second person review process. The technical controls built into the application are enabled and validated to highlight changes to data so that a reviewer can focus their attention on the key items. This is risk assessment in practice. The ways that an application can identify changes to data are color coding (for example, a traffic light approach with green indicating no changes, yellow to highlight any data modifications, and red for deletions), or by flagging data changes. This functionality is important to avoid the reviewer from drowning in data.
One area that will NOT be subject to review by exception and MUST be performed in all the second person reviews, even if there is no indication manual integration, is viewing all chromatograms on screen. This can be either singly or overlaid to ensure peak shape and resolution are as expected and consistent throughout the run. This is good analytical science, and must be performed for all analyses to ensure the integrity and quality of the results. There are
Documenting the Review
Few laboratory informatics applications have the ability to document if an audit trail has been reviewed, by whom, and when. Ideally, this is the best approach. However, in many cases, the review must be done procedurally, and the SOP must state that the meaning of the signature of the reviewer includes review of applicable audit trail entries. Sometimes, auditors insist that laboratories print out audit trail entries and sign them to provide documented evidence of the review. This approach should be resisted, as it is unrealistic and untenable, and laboratories should ask where is the specific regulation for this approach. This is an area where users require an electronic audit trail review function. In the absence of this, data integrity audits should focus on the effectiveness of a procedural audit trail review, and be conducted more frequently.
AT Review: System vs. Process
Up to now, we have just considered audit trail review in a CDS that is not interfaced to any other application. However, there are situations where a CDS is interfaced to another informatics application, such as a laboratory information management system (LIMS); how should an audit trail review be conducted in this situation? We must consider process taking precedence over system when considering audit trail review.
Figure 2 depicts a LIMS interfaced with a networked CDS. As I mentioned above, we have to consider process, not system, otherwise potential issues will fall into the interfaces and not be identified in any review.
Compliance Features to Consider When Purchasing a CDS
Apart from the chromatographic and instrument control functionality, one of the key requirements when selecting a new CDS are the technical control available to help protect electronic records, implement electronic signatures, and audit trail functionality. Ensuring regulatory compliance and data integrity are essential criteria for system selection now. These functions are often overlooked in selection of a CDS. Here are some of my compliance criteria that you should include when selecting a new system:
We have looked at the regulations and regulatory guidance for audit trail review, which is a key component for second person review of chromatographic analysis. To facilitate an effective review by exception, technical controls need to be included in the CDS application to identify data changed during an analytical run, including a function to document the review itself.
R.D. McDowall is the director of R.D. McDowall Limited in the UK. Direct correspondence to: firstname.lastname@example.org