How proficient is your microbiology laboratory?
21 Aug 2012 by Evoluted New Media
Microbiology laboratories have a vital role to play in the food and drink industry, protecting consumers’ health by helping to ensure that products are safe to eat and drink. But how can laboratories ensure they are performing adequately and benchmark their performance? Fiona Cawkell explains how participation in proficiency schemes is the answer
Erroneous laboratory results can lead to incorrect decisions, which in turn could have significant commercial and legal consequences. At the very least it might jeopardise the commercial relationships between the laboratory, its customer, and their customers (e.g. retailers). At worst it could trigger product recalls or a food safety incident.
Proficiency testing (PT) offers a highly effective way for laboratories to check how they are performing. In fact, participating in a robust proficiency testing scheme not only gives laboratory managers confidence in their laboratory equipment, methodologies and laboratory staff, but also provides assurance that the laboratory is delivering the quality of results demanded by its customers. Indeed, the ability to prove to customers that your laboratory has quality systems in place and a commitment to providing accurate, reliable results is an important part of building long-term relationships as well as securing future business.
PT provides a laboratory with an independent and confidential assessment of its performance in relation to other participating laboratories and is an important element of its quality assurance systems.
Microbiological analyses have particular issues associated with them that make it difficult to know in isolation how accurate they are. Different techniques can and do provide very different results, which in turn could lead to erroneous conclusions, if there isn’t a full understanding of what analysis was carried out and why. Simply put, laboratory analysts must know which tests to apply and must be able to perform the analyses competently. The best way to accurately judge performance is to participate in a good, comprehensive microbiology proficiency scheme.
There are a number of reasons why a microbiology laboratory might report incorrect results and why inaccuracies or uncertainties could creep into analyses. These include:
- Sample handling and storage errors;
- Inadequate staff training;
- Incorrect or inappropriate methods;
- Equipment and culture media failures;
- Calculation errors;
- Reporting errors.
These potential reasons cover what would be considered ‘normal’ uncertainty factors and margins of error. However, the primary concern is that when these reasons are all added together, they result in an incorrect representation of what the true, right answer is.
For laboratories to be confident that the methods they are using are appropriate, that they are being carried out effectively and in the correct way, and that all equipment is being operated adequately, it is essential to test results regularly against a known standard. It is therefore not surprising that laboratory accreditation schemes generally require laboratories to participate in a PT scheme as a prerequisite for accreditation.
In principle, all proficiency schemes, be they chemical, physical or microbiological analyses typically work by providing participating laboratories with standard samples for the laboratory to analyse ‘blind’. The laboratory does the analysis and submits the results to the organiser, who then distributes the processed and anonymised results to all participants. Each laboratory can then see how its results compare with those of other laboratories – without compromising the confidentiality of those laboratories.
Participating in a robust proficiency testing scheme not only gives laboratory managers confidence in their laboratory equipment, methodologies and laboratory staff, but also provides assurance that the laboratory is delivering the quality of results demanded by its customers
In microbiological analysis schemes, the providers know the level of the microorganisms (and the nature of the cocktail of mixtures) provided, but the test laboratories do not. Each participating laboratory analyses the sample(s) using its usual methods (which may differ from one laboratory to the next) and submits its results to the PT scheme provider.
PT results provide an effective insight into the sorts of results routinely obtained by the laboratory if the PT samples are treated in the same way as routine samples. This means that each stage of testing from sample receipt, through all stages of the testing process, to reporting of results, is undertaken by the same laboratory staff that would normally do those types of tests and using the same procedures. It is important to point out that if PT samples are treated differently from routine samples even though the PT results may be excellent, nothing will be learnt about the quality of the routine service.
The PT provider will compile and statistically analyse the results (e.g. results can be plotted as a histogram) and will include them in a report that also contains the ‘true’ result and, for each laboratory, a score (known as the z-score) reflecting how close it was to the ‘true’ value. Figure 1 depicts a simplified plot of how laboratories performed in a typical proficiency test.
As can be seen from Figure 1, the laboratories are not named. This means that individual laboratories can compare their result with the ‘true’ value and with the results obtained by other laboratories. They can also compare their score with other laboratories without confidentiality being breached. This gives laboratories good, independent feedback of their performance for particular tests.
While the principles of microbiological proficiency testing are the same, it goes without saying that not all PT providers operate in exactly the same way. Different schemes will have different features allowing laboratories to evaluate their performance in detecting/identifying and/or enumerating specific organisms or groups of organisms.
For example, Campden BRI operates two types of scheme, primarily for laboratories associated the food and drinks industry. Which scheme is best for a laboratory depends on its requirements, and each includes two choices of test material – depending on whether the laboratory is set up for handling pathogens or not.
The first test material sample contains mixtures of non-pathogenic organisms, for enumerating total viable count (TVC), coliforms, Enterobacteriaceae, lactic acid bacteria, yeasts and moulds, as well as pathogens that are usually quantitatively analysed.
The second test material sample might additionally contain Salmonella, Listeria, Vibrio and Campylobacter and is designed for companies who have suitable laboratory facilities and experience with handling pathogens.
In addition, extra tests have been incorporated into the scheme. For example, some laboratories may want to assess their proficiency in testing for E.coli 0157 – this test is not available in many of the other PT schemes available. As well as non-toxigenic E.coli 0157, thermophilic spores and osmophilic yeasts and moulds have recently been added to Campden BRI’s samples.
In practical terms, a freeze-dried sample is sent to all laboratories in the scheme with the simple instruction to detect and/or enumerate the microorganisms it contains. The laboratory can use whatever approaches and methods it generally uses and be as specific or generic as it wishes in what results it reports.
The participating laboratory then sends the results back to be collated and statistically analysed. Following this analysis, the collated results are then sent to the individual laboratories taking part, so that they can see all the results anonymously. In this way, each laboratory can compare and benchmark its performance with all other participants in a completely confidential and anonymous manner.
The food and drink industry is constantly changing – driven by evolving consumer demands, the legislative framework, accreditation and quality assurance developments, as well as by food and drink manufacturers and retailers themselves. Microbiological proficiency testing is no exception and PT providers must constantly strive to ensure they are meeting their laboratory clients’ needs.
Following a major review of its Campden Microbiology Proficiency Scheme (CMPS), based on feedback sought from participating laboratories, Campden BRI has recently re-launched the scheme. New features of CMPS include: the introduction of new, easy-to-open, user-friendly glass vials to dispatch test material samples; z-scores for all count results and extra statistical information upon request including trending analysis for clients over a minimum 12-month period. Additional organisms have also been included: Non-toxigenic E. coli O157, thermophilic spores and osmophilic yeasts and moulds. In addition, bespoke, tailored schemes are available for example, for businesses seeking to compare the performance of different laboratories within their organisation. Furthermore, a structured programme can be developed for laboratories looking to deploy their own tailored scheme using their own products and samples for analysis at a frequency determined by themselves.
PT is not a substitute for other components of the laboratory quality system and cannot replace a laboratory’s own quality control programme. In fact, it is of limited value without the other quality components such as adequate documentation, internal quality control and a robust staff training programme.
While PT can effectively identify problems or issues with a laboratory’s testing, it will not solve them. Microbiological laboratories need to be mindful that they should never amend their procedures and methods based solely on PT results alone. If PT participation highlights aberrant results and poor laboratory performance, the laboratory manager should seek to identify the root cause to put things right. Continued participation in the PT scheme will help identify whether changes are delivering benefits in terms of laboratory performance.
In other words, PT can help to confirm that particular problems have been eliminated. When used properly, PT helps laboratories to report accurate, reproducible results, demonstrates the effectiveness of the quality system and provides a reliable independent assessment of quality of service. It enables laboratories to measure performance against that of other laboratories and a known standard and ensures laboratory managers can have confidence in the reliability of their analytical data. Combined with the ability PT gives laboratories to facilitate continuous improvement of their performance, it is clear that proficiency testing is, and always should be a ‘must do’ rather than a ‘nice to do’.
The author: Fiona Cawkell, Campden BRI
Contact:
