In our recent webinar “GxP data integrity: What you don't know may make you non-compliant” we had over 480 registrants. With so much interest in data integrity for GxP monitoring applications, we received many questions, both during and after the webinar. Vaisala Senior GxP Regulations expert Paul Daniel has been busy answering each question by email, but we are happy to share those questions and answers in blog form. 

In his blog, we focus on the questions around how to time periodic reviews of monitoring system audit trails.  Feel free to leave your comments below. 

Question: What is the recommended timing for Periodic Review [of audit trails] in Computerized systems?

Answer:
The timing of periodic review depends on the type of system.  In our experience with customers who use the viewLinc Continuous Monitoring System, many do an audit trail review at the same time they perform normal data review.  So, for a manufacturing system, this would be review with every batch.  For a monitoring system used in a storage warehouse, this might be weekly or monthly.  

Ideally, a computer system will have a way for you to filter the audit trail, so you only review data for the relevant sensors and over the relevant time period, to limit the size of the review task.  Without the ability to filter for sensors and time period, this becomes more time consuming.  A time-consuming task is often put off. Then, if errors are found, it will require much more effort to evaluate the impact.  Audit trail reviews are a normal part of any data review on a computerized system.  If there is a report, you should review the audit trail for all the data relevant to that report.

Question: What if the sensor was recalibrated and found to be out of tolerance? Do we need to scrap all the material, or is there another solution? 

Answer:  
There is usually another way out. This scenario happens: The sensor goes to the calibration lab, and the As-Found values are out of tolerance.  It’s not the end of the world, just yet.  Of course, you must open an investigation, but the result of an investigation could be that everything is okay.  Here are some basic steps: 

1.    Have the sensor manufacturer examine the sensor and find out why it was out of tolerance.  If the sensor is just broken, then we may have a problem.  But the manufacturer may be able to give you some information as to why the sensor is out of tolerance, and most importantly which direction would this change the data, and by how much.  

For instance, let’s say the temperature sensor that was found to be out of tolerance (±0.5C), and it was reading 2°C low. The manufacturer was able to show that this was a result of physical damage to the sensor that made the result come in low all the time.  At the very least, we can now analyze our historical data to see if would still be in tolerance when we adjust the actual data by 2°C.  If we have confidence that the sensor error was in one direction only, and our adjusted data is in specification, then we don’t have a problem. 

2.    Check for backup sensors. For instance, there maybe there was an uncalibrated HVAC sensor in the area that you can crosscheck and compare as a reference standard. That sensor may be used as a backup measurement.  If the data is clearly good, this maybe a pathway to resolution.  

3.    Analyze the temperature data from the failed sensor and see if there was a noticeable event or time where the reading changed suddenly.  This could provide a possible date of failure. With that known date, you may be able to reduce the amount of material scrapped.  

4.    Perform quality tests on the material and see if it meets known quality attributes despite having been stored at the wrong temperatures for an undetermined amount of time. 

All of these steps take time, of course, but they can give options in an investigation.  In the end, after your investigation, you are unable to mitigate and must scrap material, you at least now have a potential risk that will drive future actions to avoid the problem re-occurring.  We have many customers who, as a practice, have redundant back-ups and/or calibrate more often in order to ensure against this possibility. 

Question: Did you suggest an Audit Trail Review after each batch? We have three daily batches! We would need an extra employee for this…

Answer:  
Sorry, I should have been clearer - An audit trail needs periodic review.  That is a requirement direct from the regulations. For example, Annex 11 of the EU GMP Guidelines: 9 Audit Trails states:  "Consideration should be given, based on a risk assessment, to building into the system the creation of a record of all GMP-relevant changes and deletions (a system generated "audit trail"). For change or deletion of GMP-relevant data the reason should be documented. Audit trails need to be available and convertible to a generally intelligible form and regularly reviewed".  

The questions then are: 
•    When do we perform this review?
•    How do we discern a problem? 
•    What actions do we take when we find a problem? 

From a process perspective, it makes sense to do an audit trail review at the same time as a data review.  If the monitored area is a warehouse or a storage area in which data is reviewed weekly, then an audit trail review at that time makes sense. If the monitored area is a manufacturing suite, we would have to check and see when and where monitoring data is reviewed for each batch.  

In my experience, monitoring data is often verified as a step during manufacturing. This is done to verify that the monitoring data was within specification during the time the batch was processed.  Therefore, a data review step occurs naturally.  It makes sense to include a review of the audit trail as part of the normal data review.  

In the case of the viewLinc Continuous Monitoring System, this is quite easy.  Simply filter the audit trail for the time period and sensors involved in process and look for any unexpected changes.  The most likely outcome is that in the few hours of manufacturing the batch, no changes were made to affect the way the data loggers collect data or how the alarms function.  This may take a small amount of extra time, but the alternative is to only review the audit trail periodically.  

Let’s say we review the audit trail for the manufacturing suite once a month.  If we a problem, for example, someone changed the sample rate on a data logger 21 days ago, there is now a potentially big problem: 20 days of manufacturing, with three batches per day, and we must investigate if the change to the data logger affected any of those sixty batches.  

The key is to make your audit trail reviews a small incremental task, rather than a large periodic task. The audit trail review that focusses only on the sensors and time period involved in a given process will take less time, especially when review can be easily searched and filtered for the sensor(s) and time period in question.