It is important that digital systems collect all raw (original) data generated during the conduct of the assay, so a reviewer can determine if all decisions made by the analyst(s) can be scientifically justified. There are two actions that add complexity which must be included in the data set.
First, permitting an original (raw) plate count to be manually adjusted by a qualified person. And second, permitting a plate (or image of a plate) to be re-processed. The regulatory expectation is set forth in FDA Data Integrity Guidance: FDA expects processes to be designed so that data required to be created and maintained cannot be modified without a record of the modification.
Original data: first plate image capture
For automated (digital) plate counting systems, the original data required to reconstruct the would original plate images, plus workflow configuration, plus workflow configuration, plus manual adjustments made by a scientist. It is the combination of these three that determines a reportable result value. For the first image capture of a plate, the workflow settings must be captured and linked to the image, along with the automated result counts. The image and workflow settings permit the result counts to be reconstructed at any point in the future.
Original data: time series image capture
Advanced systems can capture plate images over time and use algorithms to determine if an original “colony” was something inanimate such as a particle or a viable, growing colony. This capability permits more accurate colony counts. For this technology two options exist: The data is extracted from each image then compiled to generate a final cfu result but the images are not displayed. The images are deleted after data extraction. As images are kept and the operator has no access to the data used to calculate the final validated answer and no recourse to recalculate the result the final cfu generated is the reported value. The accuracy of the final result has been validated through the PQ/MQ phase of the system validation.
The images are stored and at the end of the analysis the operator must check the result against the image series and correct any errors. Any such additional time series images must be preserved requiring large data storage capacity and the date/time of such an image must become part of the record in such a manner that a data reviewer would would know that additional time images exist for a given plate identity.
Original data: manual adjustments (overrides)
In addition to a time series of images, it is possible for an analyst to examine the images (or the plate) and determine that the reported result must be adjusted, based on their observation. From a data integrity perspective, this manual adjustment (override) is a high-risk activity. As a result, the system must preserve the original (automated) values and the new values entered by an authorised scientist. Of course, this override capability must be restricted to very few individuals. The system must require a change reason to be manually entered by the authorised scientist, and the change reason (along with the original and override values, date and time stamp, and the scientist’s digital identity) must be preserved. Ideally, the system would set a flag to indicate a manual override has been recorded. This flag is an aide to the second person reviewer.
Raw data file size considerations
Early automated plate count systems often captured a plate image, processed the image to produce a colony count, then deleted the image. Images were not retained due to storage limitations – thousands of plate images required storage space beyond the capabilities of most firms. This was considered as equivalent to the manual process since microbial plates were discarded several days after viewing. Cloud storage vendors now make it feasible to store all captured images online. This is a large step forward, as it permits review and re-calculation of results months after a plate has been physically discarded. This has important implications when sterile operations and their risks to patient safety are implicated in manufacturing issues.
Security and control
It is important to design security roles into the software to limit the ability of analysts to modify important configuration settings. For routine tests, analysts ideally should not be able to select from a range of processing methods as errors can occur. Ideally no method selection is required to define the method appropriate for the type of plate/media under test. This assures consistent conduct of testing and simplifies second person review—if the correct process method/version was in use, the equipment configuration is correct.
Digital controls enable enforcement of actions. Security roles determine the actions of each and preserve a record of roles held by personnel so actions can be limited to the minimum number of personnel needed to perform an action, especially enhanced actions that merit closer review such as modifications to original test results or re- processing plate images to generate new counts.
Another advantage of digital systems is control. Analysts cannot ignore or bypass technical controls enforced by the system, while manual controls such as procedures can be ignored or bypassed. With manual plate counts, analysts can record plate counts on unofficial documents and keep test results they wish to preserve, discarding other values or adjusting them at will. In contrast, automated plate readers always capture an audit trail with a change reason.
FDA recommends that you restrict the ability to alter specifications, process parameters, data, or manufacturing or testing methods by technical means where possible (e.g., by limiting permissions to change settings or data).
High risk activities: manual overrides (adjustments) and re-processing Automated cell counters bring consistency to plate counting, and allows changes to counting algorithms to be compared and optimised in a scientific manner prior to implementation. However, microbiologists recognise the periodic need to correct an automated count due to such things as spreading colonies, colonies growing inside other colonies, or filamentous organisms.
The preservation of plate images at the time of testing opens up a new possibility: re- processing a plate weeks or months after the plate has been discarded. This permits a qualified microbiologist the opportunity to override (adjust) the automated value by providing a new count and a reason to justify the change. Optionally, the microbiologist could choose to re-process the original plate image using a processing method of their choice. In either situation, the audit trail will capture additional data documentation.
Along with the technical data captured by the system, it is important to implement a second person review process that gives a detailed look to these atypical activities, as they represent high risk actions to regulators. Reviewers need to assess the changes carefully to assure that not only is the data record complete but is also scientifically justified.
Metadata is gold
Data captured by the system can be extracted to perform searches that were not practical in a manual test environment. Date and time stamps represent a treasure of information: for example, start and end times of samples can compute total use time of the system, total analyst time, numbers of batches, or system use by time of day. The number of manual overrides, re-processed plates, results for a specific room, product, sample point, or anything captured about a sample, media or person can be plotted vs plate counts.
Timestamps can be used to verify that activities were performed on correct time intervals consistent with the written procedure, or can prove that a procedure is being followed as written.
Automated trending
Results stored in a searchable database provide the ability to automate trending and the reporting of potential issues in trending or individual values (OOS). Thus, it is possible to notify management of issues that require a response without dependence on an analyst for notification. Timely notification enables an investigation to reduce the potential impact to product.
Effective use of electronic systems
In addition to validation of the system after configuring it for use, it is equally important to apply data governance to assure that the system and its data remain in a state of control, and the data continues to be accurate and trustworthy. Good data governance includes the following:
- Data governance policy
- Procedure for system administration l Procedure for system calibration and maintenance
- Procedure for use of system – includes manual overrides and re-processed plates
- Procedure for review of electronic records, including critical audit trails (such as overrides and re-processed plates)
Due to their high risk to data integrity and high regulatory exposure, it is recommended that manual overrides and re-processing be reviewed by a senior microbiologist as routine practice. Firms should reasonably expect that these records will be reviewed by inspectors once they learn the system will permit them to be done. Regulators look at these records to assure that these actions were not carried out to accept products that fail to meet specifications (informally known as “testing into compliance”).
Archive
During the installation and implementation phases, the need for data archiving should be considered, especially when all plate images and audit trail records must be retained to have a complete record that permits reconstruction of testing. Data should be initially preserved in original format so it can be accessed using the active vendor software/hardware. However, original data intended to be kept for long term (especially 10 or more years) will probably require conversion to an alternate format that can be retrieved by third party software/hardware systems, as analytical instruments typically have lifecycles of less than 10 years.
Such conversions may result in loss of some original data, and the risks to the complete (original) test records and the firm’s ability to defend test results that are decades old must be considered and approved prior to any data conversion process.