US20100005342A1 - Redundant Error Detection in a Clinical Diagnostic Analyzer - Google Patents

Redundant Error Detection in a Clinical Diagnostic Analyzer Download PDF

Info

Publication number
US20100005342A1
US20100005342A1 US12/493,689 US49368909A US2010005342A1 US 20100005342 A1 US20100005342 A1 US 20100005342A1 US 49368909 A US49368909 A US 49368909A US 2010005342 A1 US2010005342 A1 US 2010005342A1
Authority
US
United States
Prior art keywords
replicate
red
events
fingerprint
command
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/493,689
Inventor
Joseph J. Dambra
Jean-Christophe Hurpeau
Mark D. Reed
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ortho Clinical Diagnostics Inc
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US12/493,689 priority Critical patent/US20100005342A1/en
Assigned to ORTHO-CLINICAL DIAGNOSTICS, INC. reassignment ORTHO-CLINICAL DIAGNOSTICS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Hurpeau, Jean-Christophe, DAMBRA, JOSEPH J., REED, MARK D.
Publication of US20100005342A1 publication Critical patent/US20100005342A1/en
Assigned to BARCLAYS BANK PLC, AS COLLATERAL AGENT reassignment BARCLAYS BANK PLC, AS COLLATERAL AGENT SECURITY INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: CRIMSON INTERNATIONAL ASSETS LLC, CRIMSON U.S. ASSETS LLC, ORTHO-CLINICAL DIAGNOSTICS, INC
Priority to US15/045,852 priority patent/US10002678B2/en
Assigned to ORTHO-CLINICAL DIAGNOSTICS, INC., CRIMSON INTERNATIONAL ASSETS LLC, CRIMSON U.S. ASSETS LLC reassignment ORTHO-CLINICAL DIAGNOSTICS, INC. RELEASE BY SECURED PARTY (SEE DOCUMENT FOR DETAILS). Assignors: BANK OF AMERICA, N.A.
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/36Preventing errors by testing or debugging software
    • G06F11/3668Software testing
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/40ICT specially adapted for the handling or processing of patient-related medical or healthcare data for data related to laboratory analysis, e.g. patient specimen analysis
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/40ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H70/00ICT specially adapted for the handling or processing of medical references
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H70/00ICT specially adapted for the handling or processing of medical references
    • G16H70/60ICT specially adapted for the handling or processing of medical references relating to pathologies
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16ZINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS, NOT OTHERWISE PROVIDED FOR
    • G16Z99/00Subject matter not provided for in other main groups of this subclass

Definitions

  • a clinical diagnostic analyzer contains extensive software monitoring and controlling its operation. Testing and examining every possible state of the analyzer is often not possible in view of the large number of states possible for the combination of software and hardware over the expected life and use of the analyzer. Testing the software in a clinical diagnostic analyzer is expensive, time consuming and difficult. For instance, if a subsystem in the analyzer fails, it is reduced, which is declared to be inactive, and needs to be restarted and initialized before use. Asynchronous messages, generated, for instance due to an unexpected opening of the cover of the incubation chamber, may require the affected replicates to be treated as potentially tampered with. Further, recovery from error conditions may affect multiple other tests due to exceeded limits, missed steps and many other undetected errors.
  • Errors may also cause additional errors to ripple through multiple other tests. Further, such errors or their effect may not be detected in time by the routine process for controlling and regulating the operation of a clinical diagnostic analyzer because different tests may be affected to different degrees. The alternative often is to discard valuable patient samples in various stages of processing and restart the entire instrument, which results in reduced throughput and efficiency.
  • Laboratories use a wide range of techniques to help prevent incorrect or even potentially incorrect results from being released. These include but are not limited to assay control charts, equipment maintenance processes/logs, periodic analyzer maintenance, comparison to patient normal performance for the particular assay, reviewing changes since last result for a patient, grouping of results to calculate a physiologically stable quantity. All of these approaches are designed to react to a performance issue after results are released. Releasing a result typically means not only providing the result, but also indicating reliability of the result. Many checks are performed as infrequently as once a day and are suitable only for detecting long-term trends or frequent outliers or highly degraded performance. This puts the laboratory and patient at risk due to release of assay data with degraded performance for hours or even days before it is detected using these methods. In a busy lab, this can result in hundreds of patient samples being retested with a heightened risk of other than the desired treatment while laboratory throughput decreases.
  • Clinical diagnostic analyzers like other complex systems comprising scheduler like controllers, control sub-systems using message traffic comprising commands, responses and asynchronous messages, interrupts and the like.
  • the scheduler usually implemented as a software module, controls the operation of the clinical diagnostic analyzer by allocating resources, scheduling the desired tests and tracking the performance of the various commands.
  • the scheduler issues commands to various subsystems, which send back a response indicating the result of carrying out the command.
  • the subsystems may, in turn, in response to receiving a command from the scheduler, send one or more commands to other software modules and receive messages from them indicating the result of executing the corresponding command.
  • All Subsystem Commands, Command Responses, and Asynchronous messages may be considered to be a sub-system input/output.
  • the software module a subsystem manager, writes these messages into Emulator Files.
  • the Emulator Files while voluminous, may be reviewed in course of trouble-shooting. This task, however, tends to be slow and impractical to flag and prevent questionable results from being released without inordinately delaying the release of results.
  • clinical diagnostic analyzers typically support testing of STAT samples requiring expedited processing.
  • U.S. Pat. No. 7,254,601 discloses a method for managing remotely deployed intelligent equipment, but, does not teach timely prevention of release of questionable laboratory results.
  • U.S. Pat. No. 6,757,714 discloses obtaining and communicating an error condition of an apparatus via email to a remote server.
  • a predefined template is used to generate the e-mail message by obtaining and inserting one or more variables into the template.
  • the error condition may be included as part of a body of the e-mail message or as part of an attachment to the e-mail message.
  • the error condition is reported using a self-describing computer language, such as eXtensible Markup Language (XML). In general, the error condition is determined with the aid of an embedded controller.
  • the remote server passes the error condition to a customer relationship management system.
  • the '714 patent does not address avoiding the release of erroneous results by preventing the machine from releasing questionable results due to unexpected scheduler or software errors.
  • a preferred embodiment provides a system and method for detecting or predicting errors to prevent release of questionable laboratory test results.
  • the preferred embodiment employs a redundant error detection capability to further examine the message traffic for possible errors or questionable results by comparing the actual parameters against an expected fingerprint generated for each assay using an assay database and a configuration file or specification.
  • This testing does not rely on inputs from the software module being tested, and hence is an independent test. For instance, if the scheduler software performs in unexpected ways, it is desirable to have an independent method to flag such errors before any results affected by such errors or issues are released. Further, a testing mechanism is provided to test the Redundant Error Detection (“RED”) capability itself.
  • RED Redundant Error Detection
  • the fingerprint is generated, preferably dynamically, from (i) an assay database capable of providing protocol information for each assay, and (ii) hard-coded information in the form of a configuration file for the assay, replicate or device of interest.
  • the fingerprint is a set of expected events required for successful completion of the test in question.
  • the number of events in a fingerprint typically is less than the exhaustive listing of events that are actually executed for the test.
  • the reduction in the number of events in the protocol database to generate the fingerprint is guided by the configuration specification.
  • the configuration specification is merged with information from the assay database to generate the expected fingerprint.
  • RED reviews the message traffic and identifies discrepancies in the form of events required by the fingerprint that are absent from the message traffic to flag questionable results. RED can also review emulator files or data logging files to gain access to the message data and identify discrepancies after results have been processed. This ability is useful in validating and testing software including validating newer versions of RED itself in an economical manner.
  • RED is integrated into existing analyzers easily because it requires few modifications and leaves the subsystem input-output traffic largely unchanged.
  • a preferred embodiment comprises a module executable on a processor in a clinical diagnostic analyzer for detecting questionable laboratory results in the clinical diagnostic analyzer.
  • the module includes a mechanism for receiving a copy of messages comprising Subsystem Commands, Command Responses, and Asynchronous messages.
  • the module also includes a fingerprint of expected events to be tested for a replicate. The fingerprint is generated, in response to a preview replicate command, from at least a configuration file and an assay database, wherein the assay database contains details for an assay required by the replicate.
  • the module includes instructions for completing a comparison, in response to a replicate check command, between the received messages and the fingerprint.
  • the module includes instructions for reporting a result of the comparison; and releases resources required for the redundant checking of the replicate, in response to receiving a replicate complete command.
  • the RED functionality may be readily turned off by an operator dynamically. This may be useful if an error in RED is suspected. If a failure or questionable results are detected, the RED functionality logs corresponding conditions. Such logging notifies the operator of the detected condition and is useful in debugging and improving software. However, the decision to not release results may be turned off until suspected problems in RED itself are resolved.
  • This disclosure includes a method for testing a software module in a clinical diagnostic analyzer.
  • the method comprises the steps of generating subsystem commands, command responses, and asynchronous messages suitable for delivery to a redundant error detector.
  • the redundant error detector reviews the generated subsystem commands, command responses, and asynchronous messages for inconsistencies. In the event such inconsistencies are identified, the software module is debugged to ensure reliable and consistent performance.
  • the software module being tested could be the redundant error detector software. Further, part of or an entire system comprising software modules may be tested in this manner.
  • the subsystem commands, command responses, and asynchronous messages are advantageously generated from a script file containing a record of one or more prior runs of the analyzer. Then the performance of the clinical diagnostic analyzer with alternative versions of the software can be compared using the redundant error detector to detect inconsistencies in any of the required events in a fingerprint.
  • the script file can include commands in addition to data from the example operation of the clinical diagnostic analyzer.
  • a preferred embodiment is a method for preventing release of a questionable result by a clinical diagnostic analyzer.
  • the method comprises facilitating the steps of receiving a preview replicate command for a replicate; allocating resources for redundant error checking for the replicate; generating a fingerprint of required events for the replicate using at least a configuration file and an assay database, wherein the fingerprint has fewer events than the events that could be generated from the assay database for the replicate; comparing messages corresponding to processing out the replicate with the fingerprint; sending a result of the comparison to prevent the release of a result corresponding to the replicate by the clinical diagnostic analyzer; and logging conditions corresponding to the replicate.
  • the configuration file is coded in eXtensible Markup Language for implementation flexibility, which makes it readable by humans and easy to send as text to remote analyzers.
  • a module (and a sub-module) reflects functional partitioning.
  • a module or sub-module may use shared resources to perform its function. Resources typically are processor time slices, database data, data in memory shared with other modules, sub modules, shared library functions and the like as is well known to one having ordinary skill in the art.
  • a developer having ordinary skill in the art, knows how to create a module. Mapping the functional partitioning into executable code is familiar to one having ordinary skill in the art. The figures are briefly described next.
  • FIG. 1 illustrates a schematic of the processing modules in a clinical diagnostic analyzer coupled to a redundant error detector.
  • FIG. 2 illustrates a schematic of a software testing arrangement using a redundant error detector.
  • FIG. 3 illustrates the redundant error detection functionality and various components within it or closely interacting with it, which may further integrated.
  • FIG. 4 illustrates a method for checking for errors using a redundant error detector.
  • FIG. 5 illustrates a method for evaluating and debugging software using a redundant error detector.
  • FIG. 6 illustrates the redundant error detection functionality and various components within it or closely interacting with it, which may be further integrated.
  • FIG. 7 illustrates an example redundant error detection functionality checking a message stream for device and replicate events while updating device state maintained by the redundant error detection functionality.
  • a ‘basic event’ in a clinical diagnostic analyzer typically, is the detection of one or more parameters or a change therein.
  • the parameters are values or descriptors of a state of the clinical diagnostic analyzer.
  • An event occurs upon fulfillment of one or more specified conditions.
  • a large number of such basic events are typically predefined for clinical diagnostic analyzers.
  • the message traffic comprising commands, responses and asynchronous messages is an example of a subset of events of interest.
  • Redundant Error Detector (“RED”) module, a software feature, detects discrepancies between sequences of expected events and actually observed events in the course of operating an analyzer or another sophisticated processing system. Sequences of expected events are formulated to provide a fingerprint for each test type run to avoid repeated exhaustive review of all events. RED provides not only software verification but also regression testing of replicate runs.
  • an event denotes detection of one or more parameters or a change therein. Some examples of events include issuing of or receiving a command, making a measurement, issuing or receiving a response to confirm execution of a command, an interrupt and the like.
  • RED is compatible with OCD's clinical diagnostic analyzer VITROS 5600TM and VITROS 3600TM, which provide a preferred platform for implementing RED. RED is expected to be useful to other clinical diagnostic analyzers as well.
  • VITROS 5600TM and VITROS 3600TM use functions provided by a Master Scheduler, which can support multiple and different types of chemistry platforms.
  • OCD's Master Scheduler can also control a robotic arm and pipettors shared between the various platforms to provide random access to input patient samples, which improves throughput. The random access to the input samples substantially eliminates the typical requirement to process samples in the order they are input.
  • OCD's Master Scheduler performs scheduling functions, for instance, to allocate resources to samples input in any order. Other not so versatile schedulers also benefit from the addition of the RED functionality. OCD's Master Scheduler determines a schedule to minimize the Time to Complete for the tests as a whole thereby maintaining high throughput while honoring applicable protocol restrictions.
  • OCD's Master Scheduler include the synergistic effects of two-dimensional random access to the input samples while providing access to resources including, for example, multiple platforms, supply of consumables including thin film slides, reaction vessels and cuvettes, along with a plurality of sensiometric devices such as electrometers, reflectometers, luminescence, light transmissivity, photon detection, an incubator for heating the samples, a supply of reagents, and a plurality of reagent delivery subsystems, all of which can be accessed and used readily.
  • resources including, for example, multiple platforms, supply of consumables including thin film slides, reaction vessels and cuvettes, along with a plurality of sensiometric devices such as electrometers, reflectometers, luminescence, light transmissivity, photon detection, an incubator for heating the samples, a supply of reagents, and a plurality of reagent delivery subsystems, all of which can be accessed and used readily.
  • a manufacturer of a clinical diagnostic analyzer provides a master list of details of supported assay protocols with its analyzers. This master list of protocol details may be edited to various degrees to add or modify the supported tests.
  • the protocol details are stored in an assay database, or acquired from another information source, which is consulted for each supported assay by a preprocessing module to generate the parameters and resources required by a test.
  • protocol details may not be in the form of events expected on the corresponding analyzer, such events can be generated using knowledge of a particular analyzer.
  • the Scheduler uses the protocol details to allocate resources and schedule various steps to improve throughput and ensure quality.
  • the Scheduler module issues multiple commands to the subsystem modules.
  • various measurements are made, gears may turn or other operations take place and a response message is sent back to the scheduler to confirm the command was carried out successfully to complete the loop. If a subsystem detected unsuccessful execution of the command, then the corresponding response sent to the Scheduler typically includes this information.
  • the failure to perform the step is a rare occurrence, it may not be detected readily during testing. Some such errors in the Scheduler software may be exacerbated by unplanned events such as reduction of various sub-systems, initialization of these sub-systems, and other interruptions.
  • a preferred embodiment provides a system and method for detecting or predicting errors to prevent release of questionable laboratory test results.
  • a list of required events, as opposed to all possible events, for each supported test is prepared in advance in the form of a configuration file.
  • This configuration specification is preferably defined in an eXtensible Markup Language (“XML”) format.
  • XML is useful for defining events, including those that are combinations or variations of basic events.
  • XML encoded instructions and descriptions can be implemented and debugged rapidly in real time using an interpreter.
  • XML based definitions are readily communicated by treating them as text.
  • XML tags although readable by humans, often describe very different functions and data using similar or even the same tag names. Nevertheless, in a context XML provides a description of the functions performed by executing XML compliant instructions. Therefore, given a proper context, XML provides a convenient method for communicating data, instructions and combinations thereof.
  • a specific instance of XML designed to facilitate a particular set of tasks at a target creates a “new machine” capable of performing particular functions.
  • a circuit performing function(s) specified by one or more XML tags in response to interpreting XML encoded instructions is a structure performing the indicated function(s).
  • RED STEP a “RED STEP” tag
  • multiple events are typically describe in configuration specifications, which may be in the form of configuration files.
  • a set of criteria describing tags provide background and contextual information followed by multiple “RED ACTION” tag sets, within each of which may be nested “RED LIST” tags, within each of which may be listed “RED VALUE” tags describing individual requirements defining conditions for satisfying the event. All events so defined must be satisfied in the preferred embodiment by the message stream corresponding to a test of interest for the results of the test to be validated. Further, parameters to be passed in the event of failure or success may also be included along with comments and other commands or instructions. As will be apparent, this is not the only scheme for specifying events in a fingerprint.
  • Alternative tag sets may be devised with no loss of generality or the interpretation of particular tags modified without departing from the sprit of the disclosed preferred embodiment.
  • a fingerprint, for each test is generated using the assay database in combination with the configuration specification. This approach not only conserves required memory resources, but also provides flexibility because updates or changes to the assay database or the configuration specification are automatically reflected in the generated fingerprint.
  • Scheduler 100 communicates with various subsystems including Scripting 105 , which in turn, has a two-way communications with the Subsystem Manager 110 .
  • Subsystem Manager 110 issues a command to Subsystem 115 and receives a response confirming execution of the command as is shown.
  • the Subsystem Manager 110 typically sends the message traffic comprising commands and responses as well as asynchronous messages to an Emulator File 160 and to RED Buffer 120 , which is a mechanism for transferring messages and information to Redundant Error Detector 130 .
  • a preferred RED Buffer 120 is implemented as a queue.
  • Scheduler 100 also sends commands to Postprocessor 155 by sending commands to a Postprocessing Buffer 150 .
  • Postprocessor 155 sends commands to a RED Buffer 120 .
  • RED Buffer 120 also receives a copy of the message traffic being recorded in Emulator File 160 .
  • Redundant Error Detector 130 receives as input the message traffic comprising commands and responses as well as asynchronous messages and commands from Postprocessor 155 . Further, Redundant Error Detector 130 has access to Configuration File 145 .
  • Control 180 controls Scheduler 100 through a two-way linkage. Similar linkages connect Sample Handler 170 to Control 180 and the Subsystem Manager 110 . Under the direction of Control 180 , required test instructions for an input set of patient samples are converted by a preprocessor functionality, using Assay Database 140 , to commands for Scheduler 100 . Scheduler 100 then allocates resources and issues commands to carryout the instructions. As noted, the expected execution order of events by Scheduler 100 may not always hold true.
  • FIG. 3 illustrates the role of Redundant Error Detector 130 in providing an independent test on the results to identify questionable results without increasing the complexity of Scheduler 100 or other components.
  • Postprocessor 155 sends a Preview Replicate command to RED Buffer 120 .
  • Redundant Error Detector 130 allocates resources during step 310 and generates a fingerprint by using Configuration File 145 and Assay Database 140 to determine the parameters and nature of events to be checked for in the message traffic, also received at RED Buffer 120 .
  • the fingerprint is generated, preferably dynamically, from (i) an assay database capable of providing protocol information for each assay or replicate, and (ii) hard-coded information in the form of a configuration file for the assay, replicate or device of interest.
  • the fingerprint is a set of expected events required for successful completion of the test in question. At the same time, the number of events in a fingerprint typically is less than the exhaustive listing of events required to execute the test.
  • Configuration File 145 defines a list of expected events for a given step in a protocol. The reduction in events reduces the effort required of RED as well as the complexity of RED.
  • a preferred fingerprint includes critical events, whose performance is necessary to generate a reliable result and events that improve the accuracy of the results.
  • Critical events' successful completion may itself indicate successful completion of many events that are not expressly checked as part of the fingerprint.
  • parameters configurable per assay are read from the Assay Database 140 at run-time.
  • the expected events of MicroSlide and SI replicates (run on two types of platforms) are completely defined by a single configuration file in Configuration File 145 while the expected events corresponding to a MicroWell and MicroTip replicate (run on two additional types of platforms) are defined by merging multiple configuration files together.
  • each configuration file defines one protocol step and Assay Database 140 is queried to determine the protocol steps required to run each replicate.
  • Redundant Error Detector 130 generates the list of expected events by merging Configuration Files 145 corresponding to each protocol step of the given assay type.
  • Each command event within a replicate step contains an action that causes Redundant Error Detector 130 to check that the event occurred within an allowed time range.
  • the allowed time ranges are modified based upon step durations and incubation durations.
  • Redundant Error Detector 130 preferably processes each event for all devices prior to processing the event for all pending replicates.
  • Each event received by RED is compared to each expected event within each Device Sequence in the order in which the expected events are defined. If an event matches one set of criteria defined for that expected event then the actions defined within that expected event are executed. Once a received event matches one expected event within a Device Sequence (which is the definition of the events, criteria, actions, and parameters that correspond to one device) no other expected events within that Device Sequence will be checked or executed until the next event is received from RED Buffer 120 .
  • FIG. 6 further describes one such exemplary preferred method for testing using RED.
  • an event is selected from a generated fingerprint.
  • a Device Event corresponding to a selected fingerprint event during step 610 .
  • Device Events are preferably specified in the configuration specification. They, however, are not merged with the assay protocol information as is the case with generating a fingerprint. Instead, they are used to independently evaluate device state.
  • a Device Event verifies the state of a device at the time an event of interest takes place.
  • a corresponding device event may be to check if the optical reader was tested against a control within a prescribed period of time prior to the test reading, and possibly an event after the test reading was taken to ensure the optical reader was fully functional. Failure in one or both of these device events, checked by comparing to the device state immediately preceding and following the test reading may mark a replicate as suspect. Thus, failure in a Device Event, checked by comparing to the device state immediately preceding and following the test reading, may identify a suspect replicate.
  • step 605 if there are no further device events corresponding to the fingerprint event selected in step 605 , the selected event is evaluated in step 635 . If the selected event is satisfied, then, control flows to step 640 to determine if there are additional events in the fingerprint. Another event is selected, then, during step 645 , if possible, and control returns to step 610 . Else, control flows to step 650 to mark successful matching of the fingerprint with the message stream or event data being examined. The method then concludes. As is readily appreciated, this method is suitable for analyzing a message stream—including message streams not analyzed in real-time or those that are generated by a simulation.
  • Redundant Error Detector 130 primarily evaluates two types of events: (i) Replicate Events; and (ii) Device Events. This is illustrated in the preferred exemplary embodiment illustrated in FIG. 7 .
  • a message in received as part of the monitored message traffic is used to update a description of Device State information, if applicable, maintained by RED.
  • RED maintains state for each device of interest, which states are based on the message traffic monitored by RED. It should be noted that not all messages resulting in updating of device state may be part of the relevant fingerprint.
  • the received message is next employed to test for device events.
  • a Device Event is selected during step 710 .
  • Device Events are preferably specified in the configuration specification. They, however, are not merged with the assay protocol information as is the case with generating a fingerprint. Instead, they are used to independently evaluate device state.
  • a Device Event verifies the state of a device at the time an event of interest takes place. Failure in a Device Event, checked by comparing to the device state immediately preceding and following the test reading, may identify a suspect replicate.
  • an event is selected from the corresponding fingerprint during step 735 .
  • the selected event is evaluated in step 740 in view of the message. If the selected event is satisfied, then, control flows to step 745 to determine if there are additional events in the fingerprint. Another event is selected, then, during step 750 , if possible, and control returns to step 740 . Else, control flows to step 755 to mark successful matching of the fingerprint with the message stream or event data being examined. The method then concludes.
  • this method is suitable for analyzing a message stream—including message streams not analyzed in real-time or those that are generated by a simulation.
  • MicroSlide PM Replicate is a set of critical events required to process a Potentimetric MicroSlide replicate
  • An example MicroSlide PM Replicate preferably includes multiple events such as:
  • Event # 1 Aspirate command
  • Event # 2 Aspirate response
  • Event # 3 Plunge Slide Supply Cart command
  • Event # 4 Plunge Slide Supply Cart response
  • Event # 5 Dispense slide from cart command
  • Event # 6 Dispense slide from cart response
  • Event # 7 Dispense sample to slide command
  • Event # 8 Dispense sample to slide response
  • Event # 9 Dispense to slide command
  • Event # 10 Dispense to slide response
  • Event # 11 Push slide to Ring command
  • Event # 12 Push slide to Ring response
  • Event # 13 Electrometer Read command
  • Event # 14 Electrometer Read response.
  • Event # 11 Push slide to PM Ring command
  • Event # 10 requires just ‘success’ to be tested to pass.
  • Additional non-exhaustive examples of Replicate Events include
  • MicroSlide CM Replicate Incubating in the Outer Ring is a set of critical events critical events required to process a Colormetric or Rate MicroSlide replicate that performs is major incubation period within the outer ring of the CM Rate Incubator.
  • MicroSlide CM Replicate Incubating in the Inner Ring is a set of critical events required to process a Colormetric or Rate MicroSlide replicate that performs is major incubation period within the inner ring of the CM Rate Incubator.
  • MicroSlide IR Replicate is a set of critical events required to process a ImmunoRate MicroSlide replicate.
  • MicroTip Reagent Addition Step is a set of critical events required to process the step where reagent is added to a cuvette within a MicroTip replicate.
  • MicroTip Sample Addition Step is a set of critical events required to process the step where neat sample is added to a cuvette within a MicroTip replicate.
  • MicroTip Diluted Sample Addition Step is a set of critical events required to process the step where diluted sample is added to a cuvette within a MicroTip replicate.
  • MicroTip Read Step is a set of critical events required to process the step where the cuvette is read using the Photometer within a MicroTip replicate.
  • MicroWell 1st Well Pretreatment Step is a set of critical events required to process the step where neat sample is added to a pretreatment well within a MicroWell replicate.
  • MicroWell Not 1st Well Pretreatment Step is a set of critical events required to process the step where diluted sample is added to a pretreatment well within a MicroWell replicate.
  • MicroWell 1st Well Dilution Step is a set of critical events required to process the step where neat sample is added to a dilution well within a MicroWell replicate.
  • MicroWell Not 1st Well Pretreatment Step is a set of critical events required to process the step where diluted sample is added to a dilution well within a MicroWell replicate.
  • MicroWell Reagent Addition Step To Middle Ring Well is a set of critical events required to process the step where reagent is added to a pretreatment well or a dilution well within a MicroWell replicate.
  • MicroWell Reagent Addition Step To Outer Ring Well is a set of critical events required to process the step where reaction well within a MicroWell replicate.
  • MicroWell 1st Well Sample Step is a set of critical events required to process the step where neat sample is added to a reaction well within a MicroWell replicate.
  • MicroWell Not 1st Well Sample Step is a set of critical events required to process the step where diluted or pretreated sample is added to a reaction well within a MicroWell replicate.
  • MicroWell Preliminary Wash Step is a set of critical events required to process the step where a reaction well receives its preliminary well wash within a MicroWell replicate.
  • MicroWell Final Wash Step is a set of critical events required to process the step where a reaction well receives its final well wash and is read using the Luminometer within a MicroWell replicate.
  • Sample Integrity Step is a set of critical events required to process Sample Integrity readings of a sample.
  • Some preferred examples of device events include those for MicroSlide Electrometer. Independent of the sample slide, the electrometer reads a reference slide to ensure at least one successful read of a reference slide within a prescribed time period. Specifying multiple critical events included in a fingerprint captures this requirement. Other devices with critical events specified may include incubation rings or areas, conveyor or ring transport mechanisms, shuttles, metering devices, controls on metering, reflectometers, slide supplies, buffer rings, movement arms, STAT lanes, reagent supplies, washing stations, blade inserters, blade dumpers, photometers and the like. This list is not exhaustive.
  • Redundant Error Detector 130 reviews the message traffic and identifies discrepancies in the form of events required by the fingerprint that are absent from the message traffic. For each received message, RED preferably matches Replicate and Device Events in a fingerprint as illustrated in FIG. 7 to perform associated actions.
  • Redundant Error Detector 130 completes the discrepancy identification survey by reviewing messages with a time stamp between that of the Preview Replicate command and the Check Replicate command. Results corresponding to such discrepant events are flagged as questionable.
  • Redundant Error Detector 130 sends the result of identifying discrepancies to Postprocessing Buffer 150 , which is a mechanism for transferring messages and information to Postprocessor 155 . If the result is determined to be questionable in step 360 , then, Redundant Error Detector 130 also identifies the test result as suspect with logging of the corresponding conditions during step 370 . Finally, during step 380 a Complete Replicate command is received. Questionable results are preferably not released by Postprocessor 155 , and naturally by the clinical diagnostic analyzer. If needed, sequestered resources are released during step 390 .
  • Redundant Error Detector can also review emulator files or data logging files to gain access to the message data and identify discrepancies in a non-real-time mode. This ability is useful in validating and testing software including validating newer versions of RED itself in an economical manner.
  • FIG. 2 shows a preferred configuration for such testing while FIG. 4 illustrates some of the steps in the preferred embodiment.
  • RED Script File 200 is used by Redundant Error Detector Driver 210 to generate ordered entries for RED Buffer 220 .
  • RED Script File 200 is a record of messages in an example operation of the clinical diagnostic analyzer coupled with some commands typically to assist in the testing.
  • Redundant Error Detector 230 processes RED Buffer 220 to match a dynamically generated fingerprint using configuration files and Assay Database 240 .
  • the result of the processing by Redundant Error Detector 230 is sent to PostProc Buffer 250 , which in turn leads back to Redundant Error Detector Driver 210 , which evaluates the performance of the test software—and may lead to debugging the same.
  • the test software preferably is Redundant Error Detector 230 .
  • the test software may be a module used in the example operation to generate data reflected in RED Script File 200 .
  • the data reflected in RED Script File 200 in alternative embodiments is from an emulator file generated as depicted in FIG. 1 or even a datalogger file being mined to identify errors in an analyzer under investigation.
  • Redundant Error Detector Driver 210 is responsible for converting all these potentially different formats to one suitable for action by Redundant Error Detector 230 .
  • RED is integrated into existing analyzers easily because it requires few modifications and leaves the subsystem input-output traffic largely unchanged. This is readily seen from FIGS. 1 and 2 , wherein Redundant Error Detector 230 is connected without significantly affecting the inner message traffic of the clinical diagnostic analyzer.
  • RED can be configure dynamically, for instance, to release all results. Thus, preferably it is possible to control or configure the penalty imposed by RED when an anomaly is detected.
  • Goals for an efficient error detection system include timely detecting errors to prevent release of erroneous results.
  • the configuration files define the checks performed by RED.
  • RED preferably checks for events critical to result quality; events detrimental to result quality; Pass/Fail/Warn criteria, if any, for each event; and required sample consistency checks.
  • the events checked for by RED are also checked for by the routine command-response design of the clinical diagnostic analyzer or other system of interest. In view of the small impact of RED on analyzer performance, implementing RED does not significantly delay the release of acceptable results.
  • RED preventing the prediction of good results In an effort to reduce the likelihood of RED preventing the prediction of good results helpful guidelines include limiting the number of events checked to only the required events; correctly setting the valid ranges of parameters to account for all scenarios; and providing configurability to prevent the RED module from being able to fail results even when it is suspected to be malfunctioning. Further, the RED module itself is tested to verify all checks are executing correctly in the foreseen scenarios. This may be performed by, for instance, the simulation strategy presented in FIG. 2 .
  • RED detects and prevents potentially misreported results. It is customary to primarily rely upon testing and code reviews as the preferred approaches for achieving a desired quality level. However, RED greatly reduces the need for code reviews, which are necessarily difficult and resource intensive exercises when testing complicated modules. Closed Loop Checking is another strategy that incurs significant development, integration, and verification expenses. Further, closed loop checking performed within the module in question may not be reliable in all situations. Closed loop checking performed within a module is often not independent of the module being tested. The RED architecture can accept many different types of inputs. The two obvious choices are Subsystem I/O (composed of Subsystem Commands, Responses, and Asynchronous messages) and DataLogger events.
  • RED performs the requested checks and passes back a response indicating the status (pass, fail, or optionally—a warning level).
  • RED recognizes that each assay model will have different protocol steps and therefore different integrity checks.
  • the categories of checks associated with a replicate are listed. Each category correlates with a table of expected events that may be dynamically altered depending upon replicate protocol information.
  • FIG. 5 illustrates the RED functionality.
  • RED Buffer 500 which provides commands and data to Redundant Error Detector Functionality 510 .
  • Redundant Error Detector Functionality 510 includes, in a preferred embodiment, a configuration file, fingerprint and its generation capability, capability to compare fingerprint to messages from RED Buffer 500 , result reporting capability and resource allocation and release functions.
  • the Assay Database 520 is accessible to Redundant Error Detector Functionality 510 , which access is useful for generating the fingerprints.

Abstract

Disclosed is a clinical diagnostic analyzer employing a redundant error detection capability to further examine the internal message traffic for possible errors or questionable results by comparing the actual parameters against a fingerprint generated for each assay using an assay database and a configuration file. This testing does not rely on inputs from the software module being tested, and hence is an independent test. Further, a testing mechanism is provided to test the Redundant Error Detection (“RED”) capability itself.

Description

    BACKGROUND
  • Repairing or correcting degraded assay performance in clinical analyzer systems has proven to be more difficult than correcting electrical or mechanical issues. Estimating the expected performance and monitoring the actual performance of clinical laboratory testing and diagnostic systems requires detailed and current information. Design and manufacturing clinical diagnostic systems requires testing and integration of thousands of components and subsystems in a complex software coordinated combination for carrying out accurate assays while maintaining a high throughput. An example is provided by the VITROS 4,3™ and ECi line of clinical Diagnostic Analyzers manufactured by Ortho Clinical Diagnostics (“OCD”) of Raritan, N.J.
  • A clinical diagnostic analyzer contains extensive software monitoring and controlling its operation. Testing and examining every possible state of the analyzer is often not possible in view of the large number of states possible for the combination of software and hardware over the expected life and use of the analyzer. Testing the software in a clinical diagnostic analyzer is expensive, time consuming and difficult. For instance, if a subsystem in the analyzer fails, it is reduced, which is declared to be inactive, and needs to be restarted and initialized before use. Asynchronous messages, generated, for instance due to an unexpected opening of the cover of the incubation chamber, may require the affected replicates to be treated as potentially tampered with. Further, recovery from error conditions may affect multiple other tests due to exceeded limits, missed steps and many other undetected errors. Errors may also cause additional errors to ripple through multiple other tests. Further, such errors or their effect may not be detected in time by the routine process for controlling and regulating the operation of a clinical diagnostic analyzer because different tests may be affected to different degrees. The alternative often is to discard valuable patient samples in various stages of processing and restart the entire instrument, which results in reduced throughput and efficiency.
  • One of the goals in designing clinical diagnostic analyzers is to prevent the release of questionable laboratory results in view of the harm that may result to a patient from treatment or lack thereof based on erroneous results. Thus, timely error detection is of considerable importance in clinical diagnostic analyzers. Indeed, it is desirable to go beyond ensuring assay and equipment performance to merely satisfy health and safety guidelines in order to reduce the cost of assays while improving the reliability of assay results. At the same time, it is imperative to not discard valid results based on an over-inclusive criterion.
  • Laboratories use a wide range of techniques to help prevent incorrect or even potentially incorrect results from being released. These include but are not limited to assay control charts, equipment maintenance processes/logs, periodic analyzer maintenance, comparison to patient normal performance for the particular assay, reviewing changes since last result for a patient, grouping of results to calculate a physiologically stable quantity. All of these approaches are designed to react to a performance issue after results are released. Releasing a result typically means not only providing the result, but also indicating reliability of the result. Many checks are performed as infrequently as once a day and are suitable only for detecting long-term trends or frequent outliers or highly degraded performance. This puts the laboratory and patient at risk due to release of assay data with degraded performance for hours or even days before it is detected using these methods. In a busy lab, this can result in hundreds of patient samples being retested with a heightened risk of other than the desired treatment while laboratory throughput decreases.
  • Clinical diagnostic analyzers, like other complex systems comprising scheduler like controllers, control sub-systems using message traffic comprising commands, responses and asynchronous messages, interrupts and the like. Typically, the scheduler, usually implemented as a software module, controls the operation of the clinical diagnostic analyzer by allocating resources, scheduling the desired tests and tracking the performance of the various commands. The scheduler issues commands to various subsystems, which send back a response indicating the result of carrying out the command. The subsystems may, in turn, in response to receiving a command from the scheduler, send one or more commands to other software modules and receive messages from them indicating the result of executing the corresponding command.
  • All Subsystem Commands, Command Responses, and Asynchronous messages may be considered to be a sub-system input/output. The software module, a subsystem manager, writes these messages into Emulator Files. The Emulator Files, while voluminous, may be reviewed in course of trouble-shooting. This task, however, tends to be slow and impractical to flag and prevent questionable results from being released without inordinately delaying the release of results. Notably, clinical diagnostic analyzers typically support testing of STAT samples requiring expedited processing. Some additional strategies to track or estimate the performance of complex systems that are also not suitable for handling clinical diagnostic analyzers are describe next.
  • U.S. Pat. No. 7,254,601 (the “'601 patent”) discloses a method for managing remotely deployed intelligent equipment, but, does not teach timely prevention of release of questionable laboratory results.
  • U.S. Pat. No. 6,757,714 (the “'714 patent”) and patents and patent applications related thereto disclose obtaining and communicating an error condition of an apparatus via email to a remote server. A predefined template is used to generate the e-mail message by obtaining and inserting one or more variables into the template. The error condition may be included as part of a body of the e-mail message or as part of an attachment to the e-mail message. The error condition is reported using a self-describing computer language, such as eXtensible Markup Language (XML). In general, the error condition is determined with the aid of an embedded controller. The remote server passes the error condition to a customer relationship management system. The '714 patent does not address avoiding the release of erroneous results by preventing the machine from releasing questionable results due to unexpected scheduler or software errors.
  • What is needed is a reduction in the need to exhaustively review and test every possible state of a clinical diagnostic analyzer in order to ensure robust performance. Instead, better error detection strategies are needed to ensure reliability of results released by a laboratory.
  • SUMMARY
  • A preferred embodiment provides a system and method for detecting or predicting errors to prevent release of questionable laboratory test results. Briefly, the preferred embodiment employs a redundant error detection capability to further examine the message traffic for possible errors or questionable results by comparing the actual parameters against an expected fingerprint generated for each assay using an assay database and a configuration file or specification. This testing does not rely on inputs from the software module being tested, and hence is an independent test. For instance, if the scheduler software performs in unexpected ways, it is desirable to have an independent method to flag such errors before any results affected by such errors or issues are released. Further, a testing mechanism is provided to test the Redundant Error Detection (“RED”) capability itself.
  • The fingerprint is generated, preferably dynamically, from (i) an assay database capable of providing protocol information for each assay, and (ii) hard-coded information in the form of a configuration file for the assay, replicate or device of interest. The fingerprint is a set of expected events required for successful completion of the test in question. At the same time, the number of events in a fingerprint typically is less than the exhaustive listing of events that are actually executed for the test. The reduction in the number of events in the protocol database to generate the fingerprint is guided by the configuration specification. Preferably, the configuration specification is merged with information from the assay database to generate the expected fingerprint.
  • With a generated fingerprint, RED reviews the message traffic and identifies discrepancies in the form of events required by the fingerprint that are absent from the message traffic to flag questionable results. RED can also review emulator files or data logging files to gain access to the message data and identify discrepancies after results have been processed. This ability is useful in validating and testing software including validating newer versions of RED itself in an economical manner.
  • In another aspect, RED is integrated into existing analyzers easily because it requires few modifications and leaves the subsystem input-output traffic largely unchanged.
  • A preferred embodiment comprises a module executable on a processor in a clinical diagnostic analyzer for detecting questionable laboratory results in the clinical diagnostic analyzer. The module includes a mechanism for receiving a copy of messages comprising Subsystem Commands, Command Responses, and Asynchronous messages. The module also includes a fingerprint of expected events to be tested for a replicate. The fingerprint is generated, in response to a preview replicate command, from at least a configuration file and an assay database, wherein the assay database contains details for an assay required by the replicate. The module includes instructions for completing a comparison, in response to a replicate check command, between the received messages and the fingerprint. The module includes instructions for reporting a result of the comparison; and releases resources required for the redundant checking of the replicate, in response to receiving a replicate complete command.
  • J In the preferred embodiment, the RED functionality may be readily turned off by an operator dynamically. This may be useful if an error in RED is suspected. If a failure or questionable results are detected, the RED functionality logs corresponding conditions. Such logging notifies the operator of the detected condition and is useful in debugging and improving software. However, the decision to not release results may be turned off until suspected problems in RED itself are resolved.
  • This disclosure includes a method for testing a software module in a clinical diagnostic analyzer. The method comprises the steps of generating subsystem commands, command responses, and asynchronous messages suitable for delivery to a redundant error detector. The redundant error detector reviews the generated subsystem commands, command responses, and asynchronous messages for inconsistencies. In the event such inconsistencies are identified, the software module is debugged to ensure reliable and consistent performance. Notably, the software module being tested could be the redundant error detector software. Further, part of or an entire system comprising software modules may be tested in this manner.
  • The subsystem commands, command responses, and asynchronous messages are advantageously generated from a script file containing a record of one or more prior runs of the analyzer. Then the performance of the clinical diagnostic analyzer with alternative versions of the software can be compared using the redundant error detector to detect inconsistencies in any of the required events in a fingerprint. The script file can include commands in addition to data from the example operation of the clinical diagnostic analyzer.
  • In another aspect, a preferred embodiment is a method for preventing release of a questionable result by a clinical diagnostic analyzer. The method comprises facilitating the steps of receiving a preview replicate command for a replicate; allocating resources for redundant error checking for the replicate; generating a fingerprint of required events for the replicate using at least a configuration file and an assay database, wherein the fingerprint has fewer events than the events that could be generated from the assay database for the replicate; comparing messages corresponding to processing out the replicate with the fingerprint; sending a result of the comparison to prevent the release of a result corresponding to the replicate by the clinical diagnostic analyzer; and logging conditions corresponding to the replicate. It should be noted that allocation and release of resources required for redundant error checking need not be express and instead may be handled by the operating system or other functionality automatically depending on the system or the underlying programming language. Advantageously, the configuration file is coded in eXtensible Markup Language for implementation flexibility, which makes it readable by humans and easy to send as text to remote analyzers.
  • These and other features in some preferred embodiments are described below in greater detail with the aid of illustrative figures. These and other features in some preferred embodiments are described below in greater detail with the aid of illustrative figures and modules or submodules. In a preferred embodiment, a module (and a sub-module) reflects functional partitioning. A module or sub-module may use shared resources to perform its function. Resources typically are processor time slices, database data, data in memory shared with other modules, sub modules, shared library functions and the like as is well known to one having ordinary skill in the art. A developer, having ordinary skill in the art, knows how to create a module. Mapping the functional partitioning into executable code is familiar to one having ordinary skill in the art. The figures are briefly described next.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 illustrates a schematic of the processing modules in a clinical diagnostic analyzer coupled to a redundant error detector.
  • FIG. 2 illustrates a schematic of a software testing arrangement using a redundant error detector.
  • FIG. 3 illustrates the redundant error detection functionality and various components within it or closely interacting with it, which may further integrated.
  • FIG. 4 illustrates a method for checking for errors using a redundant error detector.
  • FIG. 5 illustrates a method for evaluating and debugging software using a redundant error detector.
  • FIG. 6 illustrates the redundant error detection functionality and various components within it or closely interacting with it, which may be further integrated.
  • FIG. 7 illustrates an example redundant error detection functionality checking a message stream for device and replicate events while updating device state maintained by the redundant error detection functionality.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • A ‘basic event’ in a clinical diagnostic analyzer, typically, is the detection of one or more parameters or a change therein. The parameters are values or descriptors of a state of the clinical diagnostic analyzer. An event occurs upon fulfillment of one or more specified conditions. A large number of such basic events are typically predefined for clinical diagnostic analyzers. The message traffic comprising commands, responses and asynchronous messages is an example of a subset of events of interest.
  • Redundant Error Detector (“RED”) module, a software feature, detects discrepancies between sequences of expected events and actually observed events in the course of operating an analyzer or another sophisticated processing system. Sequences of expected events are formulated to provide a fingerprint for each test type run to avoid repeated exhaustive review of all events. RED provides not only software verification but also regression testing of replicate runs. In this context, an event denotes detection of one or more parameters or a change therein. Some examples of events include issuing of or receiving a command, making a measurement, issuing or receiving a response to confirm execution of a command, an interrupt and the like.
  • RED is compatible with OCD's clinical diagnostic analyzer VITROS 5600™ and VITROS 3600™, which provide a preferred platform for implementing RED. RED is expected to be useful to other clinical diagnostic analyzers as well. Each of VITROS 5600™ and VITROS 3600™ use functions provided by a Master Scheduler, which can support multiple and different types of chemistry platforms. OCD's Master Scheduler can also control a robotic arm and pipettors shared between the various platforms to provide random access to input patient samples, which improves throughput. The random access to the input samples substantially eliminates the typical requirement to process samples in the order they are input.
  • OCD's Master Scheduler performs scheduling functions, for instance, to allocate resources to samples input in any order. Other not so versatile schedulers also benefit from the addition of the RED functionality. OCD's Master Scheduler determines a schedule to minimize the Time to Complete for the tests as a whole thereby maintaining high throughput while honoring applicable protocol restrictions.
  • The benefits of OCD's Master Scheduler include the synergistic effects of two-dimensional random access to the input samples while providing access to resources including, for example, multiple platforms, supply of consumables including thin film slides, reaction vessels and cuvettes, along with a plurality of sensiometric devices such as electrometers, reflectometers, luminescence, light transmissivity, photon detection, an incubator for heating the samples, a supply of reagents, and a plurality of reagent delivery subsystems, all of which can be accessed and used readily.
  • This functionality naturally entails additional complexity, which, in turn, requires testing to ensure reliability. RED provides additional assurances against the release of questionable results. Typically, a manufacturer of a clinical diagnostic analyzer provides a master list of details of supported assay protocols with its analyzers. This master list of protocol details may be edited to various degrees to add or modify the supported tests. The protocol details are stored in an assay database, or acquired from another information source, which is consulted for each supported assay by a preprocessing module to generate the parameters and resources required by a test. Although protocol details may not be in the form of events expected on the corresponding analyzer, such events can be generated using knowledge of a particular analyzer. The Scheduler uses the protocol details to allocate resources and schedule various steps to improve throughput and ensure quality.
  • To run each test, the Scheduler module issues multiple commands to the subsystem modules. When a command is carried out, various measurements are made, gears may turn or other operations take place and a response message is sent back to the scheduler to confirm the command was carried out successfully to complete the loop. If a subsystem detected unsuccessful execution of the command, then the corresponding response sent to the Scheduler typically includes this information.
  • Nevertheless, undetected errors in the operation of clinical analyzers are an expensive and undesirable reality. For instance, if a sub-system fails, it is ‘reduced,’ that is made unavailable and all replicates requiring the use of the reduced sub-system stall. However, other tests continue to progress to best use the analyzer resources. To ensure reliability, the reduced sub-system is restarted prior to processing samples, which procedure makes the sub-system performance predictable, but may cause unanticipated delays and perturbations in the scheduling. Similar uncertainty may result due to unplanned events, such as an operator opening the cover of an incubation chamber and potentially contaminating all samples contained therein. Scheduler software may have anomalies resulting in a particular step not being followed. If the failure to perform the step is a rare occurrence, it may not be detected readily during testing. Some such errors in the Scheduler software may be exacerbated by unplanned events such as reduction of various sub-systems, initialization of these sub-systems, and other interruptions.
  • A preferred embodiment provides a system and method for detecting or predicting errors to prevent release of questionable laboratory test results. A list of required events, as opposed to all possible events, for each supported test is prepared in advance in the form of a configuration file. This configuration specification is preferably defined in an eXtensible Markup Language (“XML”) format.
  • There are many advantages in using XML. XML is useful for defining events, including those that are combinations or variations of basic events. XML encoded instructions and descriptions can be implemented and debugged rapidly in real time using an interpreter. Further, XML based definitions are readily communicated by treating them as text. However, XML tags, although readable by humans, often describe very different functions and data using similar or even the same tag names. Nevertheless, in a context XML provides a description of the functions performed by executing XML compliant instructions. Therefore, given a proper context, XML provides a convenient method for communicating data, instructions and combinations thereof.
  • In effect, a specific instance of XML designed to facilitate a particular set of tasks at a target creates a “new machine” capable of performing particular functions. Thus, a circuit performing function(s) specified by one or more XML tags in response to interpreting XML encoded instructions is a structure performing the indicated function(s).
  • An example event described using XML in a configuration specification is shown below:
  •  <?xml version=“1.0” ?>
     − <!-- $Id: RED_MsPmRepSeq.xml,v 1.8 2008/03/31 14:04:05 mreed7 Exp
    $
      -->
     − <RED_STEP RED_STEP_NAME=“RED_MsPmRepSeq”
    RED_STEP_DESC=“MicroSlide PM Replicate”>
     − <RED_EVENT RED_EVENT_NAME=“Aspirate ERF command”
    RED_EVENT_DESC=“Aspirate ERF command”>
     − <RED_CRITERIA_SET>
      <RED_CRITERIA RED_EVENT_TOKEN_NAME=“EVENT_TYPE”
    RED_CRITERIA_VALUE=“CMD” />
      <RED_CRITERIA RED_EVENT_TOKEN_NAME=“DEV”
    RED_CRITERIA_VALUE=“DRY_REF_MET_PUMP_DEV” />
      <RED_CRITERIA RED_EVENT_TOKEN_NAME=“CMD”
    RED_CRITERIA_VALUE=“ASPIRATE” />
      </RED_CRITERIA_SET>
     − <RED_ACTION
    RED_ACTION_NAME=“RED_SET_SEQ_START_TIME_IF_NOT_SET”
    RED_ACTION_COMMENT=“Set the start time of the sequence to equal the event
    time plus the offset” RED_EVENT_TOKEN_NAME=“TIME”
    RED_ERROR_ID=“1”>
     − <RED_LIST RED_LIST_TYPE_NAME=“PRIM”
    SC_SCRIPT_DATA_NAME=“RED_EVENT_MS_TIME_INTERNAL”>
      <RED_VALUE RED_VALUE_TYPE_NAME=“RED_STATIC_VALUE”
    RED_VALUE_VALUE=“−1800” />
      </RED_LIST>
      </RED_ACTION>
     − <RED_ACTION
    RED_ACTION_NAME=“RED_CHECK_EVENT_EQ_CFG_TIME_RANGE”
    RED_ACTION_COMMENT=“Check that the event starts within the specified time
    range from the start of the step.” RED_EVENT_TOKEN_NAME=“TIME”
    RED_ERROR_ID=“2”>
     − <RED_LIST RED_LIST_TYPE_NAME=“PRIM”
    SC_SCRIPT_DATA_NAME=“RED_EVENT_MS_TIME_INTERNAL”>
      <RED_VALUE
    RED_VALUE_TYPE_NAME=“RED_PREVIEW_MSEC_TIME_RANGE_VALUE”
    RED_VALUE_VALUE=“0” />
      <RED_VALUE
    RED_VALUE_TYPE_NAME=“RED_PREVIEW_MSEC_TIME_RANGE_VALUE”
    RED_VALUE_VALUE=“6550” />
      </RED_LIST>
      </RED_ACTION>
     − <RED_ACTION
    RED_ACTION_NAME=“RED_CHECK_DEV_EQ_CFG_PARAM”
    RED_ACTION_COMMENT=“RefMet CAM positioned for aspirate”
    RED_DEVICE_NAME=“RED_MS_REF_MET_DEV” RED_ERROR_ID=“7”>
     − <RED_LIST RED_LIST_TYPE_NAME=“PRIM”
    SC_SCRIPT_DATA_NAME=“REF_MET_CAM_POS”>
      <RED_VALUE RED_VALUE_TYPE_NAME=“RED_STATIC_VALUE”
    RED_VALUE_VALUE=“RESERVOIR” />
      </RED_LIST>
     − <RED_FAIL_CMDS>
      <RED_TEST_CMD RED_TEST_CMD_NUMBER=“69” />
      </RED_FAIL_CMDS>
      </RED_ACTION>
     − <RED_ACTION
    RED_ACTION_NAME=“RED_CHECK_EVENT_EQ_DEV_PARAM_BY_TIME_RANGE”
    RED_ACTION_COMMENT=“Check that RefMet has been used in last
    15 Minutes” RED_DEVICE_NAME=“RED_MS_REF_MET_DEV”
    RED_ERROR_ID=“8”>
     − <RED_LIST RED_LIST_TYPE_NAME=“PRIM”
    SC_SCRIPT_DATA_NAME=“TIME_INTERNAL”>
      <RED_VALUE RED_VALUE_TYPE_NAME=“RED_STATIC_VALUE”
    RED_VALUE_VALUE=“{ 0 0 }” />
      <RED_VALUE RED_VALUE_TYPE_NAME=“RED_STATIC_VALUE”
    RED_VALUE_VALUE=“{ 900 0 }” />
      </RED_LIST>
      </RED_ACTION>
      </RED_EVENT>
      </RED_STEP>
  • In the above event specification, one event is described within a “RED STEP” tag, although multiple events are typically describe in configuration specifications, which may be in the form of configuration files. A set of criteria describing tags provide background and contextual information followed by multiple “RED ACTION” tag sets, within each of which may be nested “RED LIST” tags, within each of which may be listed “RED VALUE” tags describing individual requirements defining conditions for satisfying the event. All events so defined must be satisfied in the preferred embodiment by the message stream corresponding to a test of interest for the results of the test to be validated. Further, parameters to be passed in the event of failure or success may also be included along with comments and other commands or instructions. As will be apparent, this is not the only scheme for specifying events in a fingerprint. Alternative tag sets may be devised with no loss of generality or the interpretation of particular tags modified without departing from the sprit of the disclosed preferred embodiment.
  • Accordingly, at runtime a more complete description of required events, a fingerprint, for each test is generated using the assay database in combination with the configuration specification. This approach not only conserves required memory resources, but also provides flexibility because updates or changes to the assay database or the configuration specification are automatically reflected in the generated fingerprint.
  • As shown in FIG. 1, Scheduler 100 communicates with various subsystems including Scripting 105, which in turn, has a two-way communications with the Subsystem Manager 110. Subsystem Manager 110 issues a command to Subsystem 115 and receives a response confirming execution of the command as is shown. The Subsystem Manager 110 typically sends the message traffic comprising commands and responses as well as asynchronous messages to an Emulator File 160 and to RED Buffer 120, which is a mechanism for transferring messages and information to Redundant Error Detector 130. A preferred RED Buffer 120 is implemented as a queue.
  • Scheduler 100 also sends commands to Postprocessor 155 by sending commands to a Postprocessing Buffer 150. Postprocessor 155 sends commands to a RED Buffer 120. RED Buffer 120 also receives a copy of the message traffic being recorded in Emulator File 160. Thus, Redundant Error Detector 130 receives as input the message traffic comprising commands and responses as well as asynchronous messages and commands from Postprocessor 155. Further, Redundant Error Detector 130 has access to Configuration File 145.
  • Control 180 controls Scheduler 100 through a two-way linkage. Similar linkages connect Sample Handler 170 to Control 180 and the Subsystem Manager 110. Under the direction of Control 180, required test instructions for an input set of patient samples are converted by a preprocessor functionality, using Assay Database 140, to commands for Scheduler 100. Scheduler 100 then allocates resources and issues commands to carryout the instructions. As noted, the expected execution order of events by Scheduler 100 may not always hold true.
  • FIG. 3 illustrates the role of Redundant Error Detector 130 in providing an independent test on the results to identify questionable results without increasing the complexity of Scheduler 100 or other components. At the start of a replicate, during step 300, Postprocessor 155 sends a Preview Replicate command to RED Buffer 120. In response to this command, Redundant Error Detector 130 allocates resources during step 310 and generates a fingerprint by using Configuration File 145 and Assay Database 140 to determine the parameters and nature of events to be checked for in the message traffic, also received at RED Buffer 120.
  • The fingerprint is generated, preferably dynamically, from (i) an assay database capable of providing protocol information for each assay or replicate, and (ii) hard-coded information in the form of a configuration file for the assay, replicate or device of interest. The fingerprint is a set of expected events required for successful completion of the test in question. At the same time, the number of events in a fingerprint typically is less than the exhaustive listing of events required to execute the test. Configuration File 145 defines a list of expected events for a given step in a protocol. The reduction in events reduces the effort required of RED as well as the complexity of RED.
  • A preferred fingerprint includes critical events, whose performance is necessary to generate a reliable result and events that improve the accuracy of the results. Critical events' successful completion may itself indicate successful completion of many events that are not expressly checked as part of the fingerprint.
  • In the preferred embodiment, parameters configurable per assay are read from the Assay Database 140 at run-time. In this preferred embodiment, the expected events of MicroSlide and SI replicates (run on two types of platforms) are completely defined by a single configuration file in Configuration File 145 while the expected events corresponding to a MicroWell and MicroTip replicate (run on two additional types of platforms) are defined by merging multiple configuration files together. In this preferred embodiment, each configuration file defines one protocol step and Assay Database 140 is queried to determine the protocol steps required to run each replicate. At runtime Redundant Error Detector 130 generates the list of expected events by merging Configuration Files 145 corresponding to each protocol step of the given assay type. Each command event within a replicate step contains an action that causes Redundant Error Detector 130 to check that the event occurred within an allowed time range. At runtime (as the steps are merged together) the allowed time ranges are modified based upon step durations and incubation durations.
  • Redundant Error Detector 130 preferably processes each event for all devices prior to processing the event for all pending replicates. Each event received by RED is compared to each expected event within each Device Sequence in the order in which the expected events are defined. If an event matches one set of criteria defined for that expected event then the actions defined within that expected event are executed. Once a received event matches one expected event within a Device Sequence (which is the definition of the events, criteria, actions, and parameters that correspond to one device) no other expected events within that Device Sequence will be checked or executed until the next event is received from RED Buffer 120.
  • FIG. 6 further describes one such exemplary preferred method for testing using RED. During step 605 an event is selected from a generated fingerprint. This is followed by the selection of a Device Event corresponding to a selected fingerprint event during step 610. Device Events are preferably specified in the configuration specification. They, however, are not merged with the assay protocol information as is the case with generating a fingerprint. Instead, they are used to independently evaluate device state. A Device Event verifies the state of a device at the time an event of interest takes place.
  • For instance, if a reading is obtained from an optical reader in a selected event, then a corresponding device event may be to check if the optical reader was tested against a control within a prescribed period of time prior to the test reading, and possibly an event after the test reading was taken to ensure the optical reader was fully functional. Failure in one or both of these device events, checked by comparing to the device state immediately preceding and following the test reading may mark a replicate as suspect. Thus, failure in a Device Event, checked by comparing to the device state immediately preceding and following the test reading, may identify a suspect replicate.
  • Control flows from step 610 to decision step 615 during which the selected device event is tested in view of a message in the message stream being monitored to ensure it is satisfied. If the event is not satisfied then control flows to step 620 to mark the replicate as suspect and the method concludes. Other embodiments may allow the method to continue to execute with no loss of generality. Alternatively, if the event is satisfied, control flows to step 625 during which the need to evaluate another device event is evaluated. If there is at least one additional device event at this stage, control flows to step 630 to select another device event and control returns to step 610.
  • Alternatively, if there are no further device events corresponding to the fingerprint event selected in step 605, the selected event is evaluated in step 635. If the selected event is satisfied, then, control flows to step 640 to determine if there are additional events in the fingerprint. Another event is selected, then, during step 645, if possible, and control returns to step 610. Else, control flows to step 650 to mark successful matching of the fingerprint with the message stream or event data being examined. The method then concludes. As is readily appreciated, this method is suitable for analyzing a message stream—including message streams not analyzed in real-time or those that are generated by a simulation.
  • Expected events may differ between platforms and analyzers. Preferred implementation of Redundant Error Detector 130 primarily evaluates two types of events: (i) Replicate Events; and (ii) Device Events. This is illustrated in the preferred exemplary embodiment illustrated in FIG. 7.
  • During step 700 a message in received as part of the monitored message traffic. During step 705, the message is used to update a description of Device State information, if applicable, maintained by RED. In a preferred embodiment, RED maintains state for each device of interest, which states are based on the message traffic monitored by RED. It should be noted that not all messages resulting in updating of device state may be part of the relevant fingerprint.
  • The received message is next employed to test for device events. A Device Event is selected during step 710. Device Events are preferably specified in the configuration specification. They, however, are not merged with the assay protocol information as is the case with generating a fingerprint. Instead, they are used to independently evaluate device state. A Device Event verifies the state of a device at the time an event of interest takes place. Failure in a Device Event, checked by comparing to the device state immediately preceding and following the test reading, may identify a suspect replicate.
  • Control flows from step 710 to decision step 715 during which the selected device event is tested in view of the message to ensure it is satisfied. If the event is not satisfied then control flows to step 720 to mark the replicate as suspect. Alternatively, if the event is satisfied, control flows to step 725 during which the need to evaluate another device event is evaluated. If there is at least one additional device event at this stage, control flows to step 730 to select another device event and control returns to step 715.
  • Alternatively, if there are no further device events, an event is selected from the corresponding fingerprint during step 735. The selected event is evaluated in step 740 in view of the message. If the selected event is satisfied, then, control flows to step 745 to determine if there are additional events in the fingerprint. Another event is selected, then, during step 750, if possible, and control returns to step 740. Else, control flows to step 755 to mark successful matching of the fingerprint with the message stream or event data being examined. The method then concludes. As is readily appreciated, this method is suitable for analyzing a message stream—including message streams not analyzed in real-time or those that are generated by a simulation.
  • Replicate Events
  • Some examples of Replicate Events in a preferred embodiment are:
  • MicroSlide PM Replicate is a set of critical events required to process a Potentimetric MicroSlide replicate An example MicroSlide PM Replicate preferably includes multiple events such as:
  • Event # 1: Aspirate command;
  • Event # 2: Aspirate response;
  • Event # 3: Plunge Slide Supply Cart command;
  • Event # 4: Plunge Slide Supply Cart response;
  • Event # 5: Dispense slide from cart command;
  • Event # 6: Dispense slide from cart response;
  • Event # 7: Dispense sample to slide command;
  • Event # 8: Dispense sample to slide response;
  • Event # 9: Dispense to slide command;
  • Event # 10: Dispense to slide response;
  • Event # 11: Push slide to Ring command;
  • Event # 12: Push slide to Ring response;
  • Event # 13: Electrometer Read command; and
  • Event # 14: Electrometer Read response.
  • Each event requires some conditions to be met as described by specified parameters. For instance, Event # 11, Push slide to PM Ring command, requires testing that the event starts within the specified time range from the start of the step. Other events, such as Event # 10 require just ‘success’ to be tested to pass. Additional non-exhaustive examples of Replicate Events include
  • MicroSlide CM Replicate Incubating in the Outer Ring is a set of critical events critical events required to process a Colormetric or Rate MicroSlide replicate that performs is major incubation period within the outer ring of the CM Rate Incubator.
  • MicroSlide CM Replicate Incubating in the Inner Ring is a set of critical events required to process a Colormetric or Rate MicroSlide replicate that performs is major incubation period within the inner ring of the CM Rate Incubator.
  • MicroSlide IR Replicate is a set of critical events required to process a ImmunoRate MicroSlide replicate.
  • MicroTip Reagent Addition Step is a set of critical events required to process the step where reagent is added to a cuvette within a MicroTip replicate.
  • MicroTip Sample Addition Step is a set of critical events required to process the step where neat sample is added to a cuvette within a MicroTip replicate.
  • MicroTip Diluted Sample Addition Step is a set of critical events required to process the step where diluted sample is added to a cuvette within a MicroTip replicate.
  • MicroTip Read Step is a set of critical events required to process the step where the cuvette is read using the Photometer within a MicroTip replicate.
  • MicroWell 1st Well Pretreatment Step is a set of critical events required to process the step where neat sample is added to a pretreatment well within a MicroWell replicate.
  • MicroWell Not 1st Well Pretreatment Step is a set of critical events required to process the step where diluted sample is added to a pretreatment well within a MicroWell replicate.
  • MicroWell 1st Well Dilution Step is a set of critical events required to process the step where neat sample is added to a dilution well within a MicroWell replicate.
  • MicroWell Not 1st Well Pretreatment Step is a set of critical events required to process the step where diluted sample is added to a dilution well within a MicroWell replicate.
  • MicroWell Reagent Addition Step To Middle Ring Well is a set of critical events required to process the step where reagent is added to a pretreatment well or a dilution well within a MicroWell replicate.
  • MicroWell Reagent Addition Step To Outer Ring Well is a set of critical events required to process the step where reaction well within a MicroWell replicate.
  • MicroWell 1st Well Sample Step is a set of critical events required to process the step where neat sample is added to a reaction well within a MicroWell replicate.
  • MicroWell Not 1st Well Sample Step is a set of critical events required to process the step where diluted or pretreated sample is added to a reaction well within a MicroWell replicate.
  • MicroWell Preliminary Wash Step is a set of critical events required to process the step where a reaction well receives its preliminary well wash within a MicroWell replicate.
  • MicroWell Final Wash Step is a set of critical events required to process the step where a reaction well receives its final well wash and is read using the Luminometer within a MicroWell replicate.
  • Sample Integrity Step is a set of critical events required to process Sample Integrity readings of a sample.
  • Device Events
  • Some preferred examples of device events include those for MicroSlide Electrometer. Independent of the sample slide, the electrometer reads a reference slide to ensure at least one successful read of a reference slide within a prescribed time period. Specifying multiple critical events included in a fingerprint captures this requirement. Other devices with critical events specified may include incubation rings or areas, conveyor or ring transport mechanisms, shuttles, metering devices, controls on metering, reflectometers, slide supplies, buffer rings, movement arms, STAT lanes, reagent supplies, washing stations, blade inserters, blade dumpers, photometers and the like. This list is not exhaustive.
  • With a generated fingerprint, Redundant Error Detector 130 reviews the message traffic and identifies discrepancies in the form of events required by the fingerprint that are absent from the message traffic. For each received message, RED preferably matches Replicate and Device Events in a fingerprint as illustrated in FIG. 7 to perform associated actions.
  • In step 340 of FIG. 3, in response to receiving a Check Replicate command in step 330, Redundant Error Detector 130 completes the discrepancy identification survey by reviewing messages with a time stamp between that of the Preview Replicate command and the Check Replicate command. Results corresponding to such discrepant events are flagged as questionable.
  • During step 350, Redundant Error Detector 130 sends the result of identifying discrepancies to Postprocessing Buffer 150, which is a mechanism for transferring messages and information to Postprocessor 155. If the result is determined to be questionable in step 360, then, Redundant Error Detector 130 also identifies the test result as suspect with logging of the corresponding conditions during step 370. Finally, during step 380 a Complete Replicate command is received. Questionable results are preferably not released by Postprocessor 155, and naturally by the clinical diagnostic analyzer. If needed, sequestered resources are released during step 390.
  • Redundant Error Detector can also review emulator files or data logging files to gain access to the message data and identify discrepancies in a non-real-time mode. This ability is useful in validating and testing software including validating newer versions of RED itself in an economical manner. FIG. 2 shows a preferred configuration for such testing while FIG. 4 illustrates some of the steps in the preferred embodiment.
  • During step 400 of FIG. 4, a script file, RED Script File 200 is used by Redundant Error Detector Driver 210 to generate ordered entries for RED Buffer 220. RED Script File 200 is a record of messages in an example operation of the clinical diagnostic analyzer coupled with some commands typically to assist in the testing. Redundant Error Detector 230 processes RED Buffer 220 to match a dynamically generated fingerprint using configuration files and Assay Database 240. The result of the processing by Redundant Error Detector 230 is sent to PostProc Buffer 250, which in turn leads back to Redundant Error Detector Driver 210, which evaluates the performance of the test software—and may lead to debugging the same. The test software preferably is Redundant Error Detector 230. Alternatively, the test software may be a module used in the example operation to generate data reflected in RED Script File 200. The data reflected in RED Script File 200 in alternative embodiments is from an emulator file generated as depicted in FIG. 1 or even a datalogger file being mined to identify errors in an analyzer under investigation. Redundant Error Detector Driver 210 is responsible for converting all these potentially different formats to one suitable for action by Redundant Error Detector 230.
  • In another aspect, RED is integrated into existing analyzers easily because it requires few modifications and leaves the subsystem input-output traffic largely unchanged. This is readily seen from FIGS. 1 and 2, wherein Redundant Error Detector 230 is connected without significantly affecting the inner message traffic of the clinical diagnostic analyzer. Preferably, RED can be configure dynamically, for instance, to release all results. Thus, preferably it is possible to control or configure the penalty imposed by RED when an anomaly is detected.
  • Goals for an efficient error detection system include timely detecting errors to prevent release of erroneous results. The configuration files define the checks performed by RED. Thus, for each assay model type RED preferably checks for events critical to result quality; events detrimental to result quality; Pass/Fail/Warn criteria, if any, for each event; and required sample consistency checks.
    In most instances, the events checked for by RED are also checked for by the routine command-response design of the clinical diagnostic analyzer or other system of interest. In view of the small impact of RED on analyzer performance, implementing RED does not significantly delay the release of acceptable results.
    In an effort to reduce the likelihood of RED preventing the prediction of good results helpful guidelines include limiting the number of events checked to only the required events; correctly setting the valid ranges of parameters to account for all scenarios; and providing configurability to prevent the RED module from being able to fail results even when it is suspected to be malfunctioning.
    Further, the RED module itself is tested to verify all checks are executing correctly in the foreseen scenarios. This may be performed by, for instance, the simulation strategy presented in FIG. 2.
  • Alternative Error Detection Strategies
  • RED detects and prevents potentially misreported results. It is customary to primarily rely upon testing and code reviews as the preferred approaches for achieving a desired quality level. However, RED greatly reduces the need for code reviews, which are necessarily difficult and resource intensive exercises when testing complicated modules.
    Closed Loop Checking is another strategy that incurs significant development, integration, and verification expenses. Further, closed loop checking performed within the module in question may not be reliable in all situations. Closed loop checking performed within a module is often not independent of the module being tested.
    The RED architecture can accept many different types of inputs. The two obvious choices are Subsystem I/O (composed of Subsystem Commands, Responses, and Asynchronous messages) and DataLogger events.
    In a preferred embodiment RED performs the requested checks and passes back a response indicating the status (pass, fail, or optionally—a warning level). RED recognizes that each assay model will have different protocol steps and therefore different integrity checks. The categories of checks associated with a replicate are listed. Each category correlates with a table of expected events that may be dynamically altered depending upon replicate protocol information. Thus, RED potentially simplifies coding for the analyzer and provides improved reliability to the results by providing a test independent of the complexity inherent in real-time handling of message traffic in an analyzer or another complex system. FIG. 5 illustrates the RED functionality. Typically, there is a mechanism for transferring messages and information, here RED Buffer 500, which provides commands and data to Redundant Error Detector Functionality 510. Redundant Error Detector Functionality 510 includes, in a preferred embodiment, a configuration file, fingerprint and its generation capability, capability to compare fingerprint to messages from RED Buffer 500, result reporting capability and resource allocation and release functions. The Assay Database 520 is accessible to Redundant Error Detector Functionality 510, which access is useful for generating the fingerprints.
  • One skilled in the art will appreciate this disclosure is susceptible to many variations and alternative implementations without departing from its teachings or spirit. For instance, complex systems other than clinical analyzers will benefit by redundant error testing as described herein based on evaluating each output for consistency evaluated using a fingerprint of events. The scope of the claims appended below includes many such modifications. Further, each reference discussed and cited herein is hereby incorporated herein by reference in its entirety.

Claims (19)

1. A module executable on a processor in a clinical diagnostic analyzer for detecting questionable laboratory results in the clinical diagnostic analyzer, the module comprising:
a mechanism for receiving messages comprising Subsystem Commands, Command Responses, and Asynchronous messages;
a fingerprint of expected events to be tested for a replicate, wherein the fingerprint is generated from at least a configuration information and protocol information, wherein the protocol information contains details for an assay required by the replicate;
instructions for completing a comparison between the received messages with the fingerprint; and
instructions for reporting a result of the comparison.
2. The module of claim 1 further comprising instructions for generating the fingerprint in response to receiving a Preview command for the replicate.
3. The module of claim 1 further comprising instructions for completing, in response to a Check command, a comparison of the fingerprint with received messages.
4. The module of claim 1 comprising instructions for releasing all resources, which were sequestered in response to receiving the Preview command for the replicate, in response to receiving a Complete command.
5. The module of claim 1 comprising instructions for dynamically reconfiguring the instructions for releasing a result of the comparison.
6. The module of claim 1 comprising instructions for logging conditions in response to determining the result is suspect.
7. The module of claim 1 comprising instructions for logging conditions in response to flagging the results of the assay with a warning.
8. The module of claim 1 further comprising instructions for generating events corresponding to a subsystem in response to receiving a Preview command for the replicate, wherein the generation is based on the configuration information.
9. A quality control method for testing a software module in a clinical diagnostic analyzer comprising the steps of:
generating subsystem commands, command responses, and asynchronous messages suitable for delivery to a redundant error detector;
reviewing the generated subsystem commands, command responses, and asynchronous messages using the redundant error detector for inconsistencies; and
debug the software module in response to detecting inconsistencies.
10. The method of claim 9, wherein the subsystem commands, command responses, and asynchronous messages are generated in a simulation of a performance of the clinical diagnostic analyzer using at least one script file which contains at least one command in addition to a record of messages in an example operation of the clinical diagnostic analyzer.
11. The method of claim 10, wherein the software module is used in the example operation of the clinical diagnostic analyzer to generate the record of messages.
12. The method of claim 9, wherein the redundant error detector uses a configuration file and an assay database to generate a fingerprint for at least one replicate in the at least one script file.
13. The method of claim 9, wherein the software module is a test redundant error detector.
14. The method of claim 9, wherein the software module is not a test redundant error detector.
15. The method of claim 9, wherein a system is tested by testing several software modules.
16. A method for preventing release of a questionable result by a clinical diagnostic analyzer, the method comprising facilitating the steps of:
receiving a preview replicate command for a replicate;
allocating resources for redundant error checking for the replicate;
generating a fingerprint of required events for the replicate using at least a configuration information and an protocol information, wherein the fingerprint has fewer events than the events performed in accordance with the protocol information for the replicate;
comparing messages corresponding to carrying out the replicate with the fingerprint;
sending a result of the comparison; and
logging conditions corresponding to the replicate.
17. The method of claim 16, wherein the configuration information is coded in eXtensible Markup Language.
18. The method of claim 16, wherein logging conditions includes writing to a log file in a specified format data relating to specified events.
19. The method of claim 16, wherein the result corresponding to the replicate being fail causes the clinical diagnostic analyzer to not release the result.
US12/493,689 2008-07-01 2009-06-29 Redundant Error Detection in a Clinical Diagnostic Analyzer Abandoned US20100005342A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US12/493,689 US20100005342A1 (en) 2008-07-01 2009-06-29 Redundant Error Detection in a Clinical Diagnostic Analyzer
US15/045,852 US10002678B2 (en) 2008-07-01 2016-02-17 Redundant error detection in a clinical diagnostic analyzer

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US7724508P 2008-07-01 2008-07-01
US12/493,689 US20100005342A1 (en) 2008-07-01 2009-06-29 Redundant Error Detection in a Clinical Diagnostic Analyzer

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/045,852 Continuation US10002678B2 (en) 2008-07-01 2016-02-17 Redundant error detection in a clinical diagnostic analyzer

Publications (1)

Publication Number Publication Date
US20100005342A1 true US20100005342A1 (en) 2010-01-07

Family

ID=41210538

Family Applications (2)

Application Number Title Priority Date Filing Date
US12/493,689 Abandoned US20100005342A1 (en) 2008-07-01 2009-06-29 Redundant Error Detection in a Clinical Diagnostic Analyzer
US15/045,852 Expired - Fee Related US10002678B2 (en) 2008-07-01 2016-02-17 Redundant error detection in a clinical diagnostic analyzer

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/045,852 Expired - Fee Related US10002678B2 (en) 2008-07-01 2016-02-17 Redundant error detection in a clinical diagnostic analyzer

Country Status (5)

Country Link
US (2) US20100005342A1 (en)
EP (1) EP2141599B1 (en)
JP (1) JP2010014711A (en)
CN (1) CN101676880A (en)
CA (1) CA2670894A1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9183117B2 (en) 2013-06-20 2015-11-10 Abbott Laboratories Inc. Method for developing and testing a connectivity driver for an instrument
US11016824B1 (en) * 2017-06-12 2021-05-25 Pure Storage, Inc. Event identification with out-of-order reporting in a cloud-based environment
US11210133B1 (en) 2017-06-12 2021-12-28 Pure Storage, Inc. Workload mobility between disparate execution environments
EP3933589A1 (en) 2020-06-30 2022-01-05 Roche Diagnostics GmbH An event chain reaction system
CN114371683A (en) * 2021-11-25 2022-04-19 江铃汽车股份有限公司 Diagnostic function verification method, system, storage medium and device of diagnostic apparatus
US11340939B1 (en) 2017-06-12 2022-05-24 Pure Storage, Inc. Application-aware analytics for storage systems
US11567810B1 (en) 2017-06-12 2023-01-31 Pure Storage, Inc. Cost optimized workload placement

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8589342B2 (en) 2011-09-16 2013-11-19 International Business Machines Corporation Log message optimization to ignore or identify redundant log messages
CN103019922B (en) * 2011-09-22 2016-09-28 北京新媒传信科技有限公司 A kind of can the component architecture implementation method of independent test
EP3502708B1 (en) * 2017-12-21 2023-09-27 Tecan Trading AG Monitoring a laboratory automation device via a simulation model
US10346287B1 (en) 2018-06-06 2019-07-09 The United States Of America, As Represented By The Secretary Of The Navy Detection of degenerate software forms in object oriented code

Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5136704A (en) * 1989-06-28 1992-08-04 Motorola, Inc. Redundant microprocessor control system using locks and keys
US5719559A (en) * 1995-06-23 1998-02-17 Limitorque Corporation System and method for the verification of a digital control system
US5980081A (en) * 1996-07-15 1999-11-09 Denso Corporation Control system having effective error detection capabilities
US6173440B1 (en) * 1998-05-27 2001-01-09 Mcdonnell Douglas Corporation Method and apparatus for debugging, verifying and validating computer software
US6397355B1 (en) * 1999-03-29 2002-05-28 International Business Machines Corporation System, method, and program for automatic error detection while utilizing a software state machine for carrying out the process flow of a software program
US6421793B1 (en) * 1999-07-22 2002-07-16 Siemens Information And Communication Mobile, Llc System and method for automated testing of electronic devices
US6640203B2 (en) * 1998-10-09 2003-10-28 Sun Microsystems, Inc. Process monitoring in a computer system
US6671874B1 (en) * 2000-04-03 2003-12-30 Sofia Passova Universal verification and validation system and method of computer-aided software quality assurance and testing
US6745343B1 (en) * 2000-07-13 2004-06-01 International Business Machines Corporation Apparatus and method for performing surveillance prior to boot-up of an operating system
US6754854B2 (en) * 2001-06-04 2004-06-22 Motorola, Inc. System and method for event monitoring and error detection
US6757714B1 (en) * 2000-07-28 2004-06-29 Axeda Systems Operating Company, Inc. Reporting the state of an apparatus to a remote computer
US6898733B2 (en) * 2001-10-31 2005-05-24 Hewlett-Packard Development Company, L.P. Process activity and error monitoring system and method
US20060136263A1 (en) * 2004-12-22 2006-06-22 Cerner Innovation, Inc. System and method for automatically verifying multiple laboratory test results in a computerized environment
US7127642B2 (en) * 2000-04-14 2006-10-24 Microsoft Corporation System and method for self-diagnosing system crashes
US7254601B2 (en) * 2001-12-20 2007-08-07 Questra Corporation Method and apparatus for managing intelligent assets in a distributed environment
US20070260932A1 (en) * 2006-04-11 2007-11-08 Ryan Prichard Event log management system
US7302677B2 (en) * 2003-05-08 2007-11-27 Microsoft Corporation Event driven graph explorer for model-based testing of software
US7366678B2 (en) * 2002-04-12 2008-04-29 International Business Machines Corporation Facilitating error checking of service elements
US20080312893A1 (en) * 2007-06-15 2008-12-18 Gary Denton Clinical diagnostic analyzer performance estimator
US7475405B2 (en) * 2000-09-06 2009-01-06 International Business Machines Corporation Method and system for detecting unusual events and application thereof in computer intrusion detection
US7584386B2 (en) * 2004-04-21 2009-09-01 Stmicroelectronics Sa Microprocessor comprising error detection means protected against an attack by error injection

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4831558A (en) * 1986-08-26 1989-05-16 The Slope Indicator Company Digitally based system for monitoring physical phenomena
JPH0810227B2 (en) * 1987-09-28 1996-01-31 株式会社日立製作所 Automatic analyzer
JPH0310161A (en) * 1989-06-08 1991-01-17 Hitachi Ltd System for instructing recheck
JP2526759B2 (en) * 1991-11-29 1996-08-21 株式会社島津製作所 Automatic chemical analyzer
JPH0675811A (en) * 1992-08-27 1994-03-18 Oki Electric Ind Co Ltd Terminal testing device
JPH07120471A (en) * 1993-10-25 1995-05-12 Hitachi Ltd Automatic analysis device
JPH08263134A (en) * 1995-03-27 1996-10-11 Oki Electric Ind Co Ltd Abnormality detecting method for process controller, and process controller
JP3505078B2 (en) * 1998-02-23 2004-03-08 株式会社日立製作所 Clinical testing system
EP0997819B1 (en) * 1999-08-06 2001-10-31 Agilent Technologies, Inc. (a Delaware corporation) Dynamic event recognition
EP1965326A3 (en) * 1999-11-30 2008-12-31 Sysmex Corporation Support method, quality control method, and device therefor
AU777102B2 (en) * 2000-01-13 2004-09-30 Ortho-Clinical Diagnostics, Inc. Failure detection in automated clinical analyzers
US7117239B1 (en) * 2000-07-28 2006-10-03 Axeda Corporation Reporting the state of an apparatus to a remote computer
JP4072724B2 (en) * 2003-04-16 2008-04-09 日立工機株式会社 Automatic dispensing device
JP2005283332A (en) * 2004-03-30 2005-10-13 Shimadzu Corp Validation system and validation program
JP4871618B2 (en) * 2006-03-14 2012-02-08 株式会社日立ハイテクノロジーズ Quality control system
JP2008021114A (en) * 2006-07-12 2008-01-31 Olympus Corp Control unit, control method and control program

Patent Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5136704A (en) * 1989-06-28 1992-08-04 Motorola, Inc. Redundant microprocessor control system using locks and keys
US5719559A (en) * 1995-06-23 1998-02-17 Limitorque Corporation System and method for the verification of a digital control system
US5980081A (en) * 1996-07-15 1999-11-09 Denso Corporation Control system having effective error detection capabilities
US6173440B1 (en) * 1998-05-27 2001-01-09 Mcdonnell Douglas Corporation Method and apparatus for debugging, verifying and validating computer software
US6640203B2 (en) * 1998-10-09 2003-10-28 Sun Microsystems, Inc. Process monitoring in a computer system
US6397355B1 (en) * 1999-03-29 2002-05-28 International Business Machines Corporation System, method, and program for automatic error detection while utilizing a software state machine for carrying out the process flow of a software program
US6421793B1 (en) * 1999-07-22 2002-07-16 Siemens Information And Communication Mobile, Llc System and method for automated testing of electronic devices
US6671874B1 (en) * 2000-04-03 2003-12-30 Sofia Passova Universal verification and validation system and method of computer-aided software quality assurance and testing
US7127642B2 (en) * 2000-04-14 2006-10-24 Microsoft Corporation System and method for self-diagnosing system crashes
US6745343B1 (en) * 2000-07-13 2004-06-01 International Business Machines Corporation Apparatus and method for performing surveillance prior to boot-up of an operating system
US6757714B1 (en) * 2000-07-28 2004-06-29 Axeda Systems Operating Company, Inc. Reporting the state of an apparatus to a remote computer
US7475405B2 (en) * 2000-09-06 2009-01-06 International Business Machines Corporation Method and system for detecting unusual events and application thereof in computer intrusion detection
US6754854B2 (en) * 2001-06-04 2004-06-22 Motorola, Inc. System and method for event monitoring and error detection
US6898733B2 (en) * 2001-10-31 2005-05-24 Hewlett-Packard Development Company, L.P. Process activity and error monitoring system and method
US7254601B2 (en) * 2001-12-20 2007-08-07 Questra Corporation Method and apparatus for managing intelligent assets in a distributed environment
US7366678B2 (en) * 2002-04-12 2008-04-29 International Business Machines Corporation Facilitating error checking of service elements
US7302677B2 (en) * 2003-05-08 2007-11-27 Microsoft Corporation Event driven graph explorer for model-based testing of software
US7584386B2 (en) * 2004-04-21 2009-09-01 Stmicroelectronics Sa Microprocessor comprising error detection means protected against an attack by error injection
US20060136263A1 (en) * 2004-12-22 2006-06-22 Cerner Innovation, Inc. System and method for automatically verifying multiple laboratory test results in a computerized environment
US20070260932A1 (en) * 2006-04-11 2007-11-08 Ryan Prichard Event log management system
US20080312893A1 (en) * 2007-06-15 2008-12-18 Gary Denton Clinical diagnostic analyzer performance estimator

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9183117B2 (en) 2013-06-20 2015-11-10 Abbott Laboratories Inc. Method for developing and testing a connectivity driver for an instrument
US11016824B1 (en) * 2017-06-12 2021-05-25 Pure Storage, Inc. Event identification with out-of-order reporting in a cloud-based environment
US11210133B1 (en) 2017-06-12 2021-12-28 Pure Storage, Inc. Workload mobility between disparate execution environments
US11340939B1 (en) 2017-06-12 2022-05-24 Pure Storage, Inc. Application-aware analytics for storage systems
US11567810B1 (en) 2017-06-12 2023-01-31 Pure Storage, Inc. Cost optimized workload placement
EP3933589A1 (en) 2020-06-30 2022-01-05 Roche Diagnostics GmbH An event chain reaction system
WO2022002861A1 (en) 2020-06-30 2022-01-06 F. Hoffmann-La Roche Ag An event chain reaction system
CN114371683A (en) * 2021-11-25 2022-04-19 江铃汽车股份有限公司 Diagnostic function verification method, system, storage medium and device of diagnostic apparatus

Also Published As

Publication number Publication date
CA2670894A1 (en) 2010-01-01
CN101676880A (en) 2010-03-24
EP2141599A3 (en) 2011-06-22
US20160162653A1 (en) 2016-06-09
US10002678B2 (en) 2018-06-19
JP2010014711A (en) 2010-01-21
EP2141599A2 (en) 2010-01-06
EP2141599B1 (en) 2018-11-14

Similar Documents

Publication Publication Date Title
US10002678B2 (en) Redundant error detection in a clinical diagnostic analyzer
Memon et al. Taming Google-scale continuous testing
JP2010014711A5 (en)
EP2666088B1 (en) Methods for hierarchically identifying root cause errors
US20090055804A1 (en) Method and device for automatically evaluating the quality of a software source code
US7917897B2 (en) Defect resolution methodology and target assessment process with a software system
US7503037B2 (en) System and method for identifying bugs in software source code, using information from code coverage tools and source control tools to determine bugs introduced within a time or edit interval
Fagan Design and code inspections to reduce errors in program development
US20070006037A1 (en) Automated test case result analyzer
US8078916B2 (en) Testing measurements
US8549483B1 (en) Engine for scalable software testing
CN112740184A (en) Method for deterministically reporting causes and effects in software systems
WO2010018415A1 (en) A method and system for testing complex machine control software
JP2010231782A (en) Method and system for function automation
Carrozza et al. Analysis and prediction of mandelbugs in an industrial software system
US20080127118A1 (en) Method and system for dynamic patching of software
Barbosa et al. Test flakiness across programming languages
Munson et al. Toward a quantifiable definition of software faults
Raghuvanshi Introduction to Software Testing
US11256612B2 (en) Automated testing of program code under development
Baral et al. Evaluating a test automation decision support tool
Cotroneo et al. Prediction of the testing effort for the safety certification of open-source software: A case study on a real-time operating system
Legenhausen et al. Repoguard: a framework for integration of development tools with source code repositories
Wendland et al. Extending the UML Testing Profile with a fine-grained test logging model
Kirilov et al. Improving Experiment Control in the Sonix+ Software Package

Legal Events

Date Code Title Description
AS Assignment

Owner name: ORTHO-CLINICAL DIAGNOSTICS, INC., NEW JERSEY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:DAMBRA, JOSEPH J.;HURPEAU, JEAN-CHRISTOPHE;REED, MARK D.;REEL/FRAME:022889/0195;SIGNING DATES FROM 20090601 TO 20090613

AS Assignment

Owner name: BARCLAYS BANK PLC, AS COLLATERAL AGENT, NEW YORK

Free format text: SECURITY INTEREST;ASSIGNORS:ORTHO-CLINICAL DIAGNOSTICS, INC;CRIMSON U.S. ASSETS LLC;CRIMSON INTERNATIONAL ASSETS LLC;REEL/FRAME:033276/0104

Effective date: 20140630

STCB Information on status: application discontinuation

Free format text: ABANDONED -- AFTER EXAMINER'S ANSWER OR BOARD OF APPEALS DECISION

AS Assignment

Owner name: CRIMSON INTERNATIONAL ASSETS LLC, NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BANK OF AMERICA, N.A.;REEL/FRAME:060219/0571

Effective date: 20220527

Owner name: CRIMSON U.S. ASSETS LLC, NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BANK OF AMERICA, N.A.;REEL/FRAME:060219/0571

Effective date: 20220527

Owner name: ORTHO-CLINICAL DIAGNOSTICS, INC., NEW JERSEY

Free format text: RELEASE BY SECURED PARTY;ASSIGNOR:BANK OF AMERICA, N.A.;REEL/FRAME:060219/0571

Effective date: 20220527