US20050251792A1 - System for adaptively determining executable application operation characteristics - Google Patents

System for adaptively determining executable application operation characteristics Download PDF

Info

Publication number
US20050251792A1
US20050251792A1 US10/980,498 US98049804A US2005251792A1 US 20050251792 A1 US20050251792 A1 US 20050251792A1 US 98049804 A US98049804 A US 98049804A US 2005251792 A1 US2005251792 A1 US 2005251792A1
Authority
US
United States
Prior art keywords
executable application
user initiated
initiated activity
duration
execution
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/980,498
Inventor
David Smith
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens Medical Solutions USA Inc
Original Assignee
Siemens Medical Solutions Health Services Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens Medical Solutions Health Services Corp filed Critical Siemens Medical Solutions Health Services Corp
Priority to US10/980,498 priority Critical patent/US20050251792A1/en
Assigned to SIEMENS MEDICAL SOLUTIONS HEALTH SERVICES CORPORATION reassignment SIEMENS MEDICAL SOLUTIONS HEALTH SERVICES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SMITH, DAVID WESLEY
Priority to DE102005020893A priority patent/DE102005020893A1/en
Publication of US20050251792A1 publication Critical patent/US20050251792A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3438Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment monitoring of user actions
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3409Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
    • G06F11/3419Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment by assessing time
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/30Monitoring
    • G06F11/34Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466Performance evaluation by tracing or monitoring
    • G06F11/3476Data logging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2201/00Indexing scheme relating to error detection, to error correction, and to monitoring
    • G06F2201/81Threshold

Definitions

  • This invention concerns a system for determining executable application operation characteristics such as application response time.
  • executable application operational characteristics such as response time and access time, for example, for a variety of purposes.
  • the operational characteristics are used to optimize a computing configuration, to determine communication bandwidth requirements or to demonstrate that an application meets guaranteed performance thresholds, for example.
  • There are existing systems (from BMC Software, Computer Associates, Compuware and IBM, for example) providing response time monitoring and alerting for transactional-based executable applications. These existing systems typically contain fixed links (hooks) enabling communication and acquisition of data with an underlying operating system or executable application in order to determine operational characteristics.
  • the existing systems also involve additional complexity and computational burden resulting from additional feature and function overhead. Further, the additional features and functions exceed those necessary for operational characteristic determination.
  • the additional features and functions provide the ability to troubleshoot a processing system and to examine operation of executable components, for example.
  • the additional features and functions also incur an execution burden on CPU resources.
  • existing systems employ different definitions of operational characteristics such as response time and acquire response time information in different ways. It is desirable for a system to be able to acquire executable application operational characteristics, with reduced processing and computational burden and without requirement for modification of a target application of operation system environment.
  • existing response time monitoring systems are typically integrated with an operating system or application and collect response time data but fail to aggregate, correlate and analyze the data until a user requests it.
  • a system according to invention principles addresses the identified requirements, deficiencies and associated problems.
  • An adaptive system provides minimally intrusive operational characteristic (e.g., response time) determination and monitoring of an executable application and provides predictive real time operational characteristic alert messages based on learned behavior.
  • a system for monitoring executable application operation characteristics includes an acquisition processor for acquiring messages from an executable application enabling determination of a duration of the user initiated activity.
  • a data processor determines a duration of the user initiated activity in response to the acquired messages and accumulates information comprising a plurality of duration values for a corresponding plurality of occurrences of the user initiated activity.
  • a data analyzer analyzes accumulated information to identify a duration of execution of a user initiated activity exceeding a normal range derived based on the accumulated information.
  • FIG. 1 shows a system for determining operational characteristics of an executable application, according to invention principles.
  • FIG. 2 shows a data record structure containing data associated with determination of operational characteristics of an executable application, according to invention principles.
  • FIG. 3 shows a graphical report presenting operational characteristics of an executable application determined by the system of FIG. 1 , according to invention principles.
  • FIG. 4 shows a tabular report presenting operational characteristics of an executable application determined by the system of FIG. 1 , according to invention principles.
  • FIG. 5 shows a report presenting a detailed analysis of excessive response times of an executable application determined by the system of FIG. 1 , according to invention principles.
  • FIG. 6 shows a flowchart of a process employed by the system of FIG. 1 for determining operational characteristics of an executable application, according to invention principles.
  • FIG. 7 shows a flowchart of a process employed by an executable application supporting determination of operational characteristics of the executable application, according to invention principles.
  • FIG. 1 shows an adaptive system for determining operational characteristics of an executable application.
  • the system determines executable application operational characteristics (e.g., response time) whilst requiring minimal intrusion into the application and operating system.
  • the system provides monitoring of an executable application and adaptively generates alert messages based on real time prediction of operational characteristic through learned behavior.
  • the system provides response time monitoring and alerting without requiring a large infrastructure to support it.
  • the system adaptively learns acceptable ranges of response times for individual applications as well as for individual users.
  • the system receives input data from an executable application being monitored. This advantageously eliminates the need for interface links (hooks) or overhead between the system and the monitored executable application (as typically required by existing monitoring systems).
  • the system also advantageously has the capability to monitor a subset of users, applications or customers (e.g., customers of an Application Service Provider (ASP) hosting one more applications for use by the customers).
  • ASP Application Service Provider
  • the system compares current response times with corresponding historical, previously acquired response times (such as response times acquired on other days, weeks, months or by different applications or for different customers) in a real time manner.
  • the system adaptively uses this comparison to identify acceptable ranges of executable application response times and reduces application management and configuration time.
  • An executable application as used herein comprises code or machine readable instruction for implementing predetermined functions including those of an operating system, healthcare information system or other information processing system, for example, in response user command or input.
  • An executable procedure is a segment of code (machine readable instruction), sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes and may include performing operations on received input parameters (or in response to received input parameters) and provide resulting output parameters.
  • a processor as used herein is a device and/or set of machine-readable instructions for performing tasks.
  • a processor comprises any one or combination of, hardware, firmware, and/or software.
  • a processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and/or by routing the information to an output device.
  • a processor may use or comprise the capabilities of a controller or microprocessor, for example.
  • a display processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof.
  • a user interface comprises one or more display images enabling user interaction with a processor or other device.
  • FIG. 2 shows a data record structure containing data associated with determination of operational characteristics of an executable application.
  • the FIG. 1 system employs this record structure and a user interface in monitoring application operational characteristics.
  • the record structure is self-maintaining and employs files of fixed size that do not need to be purged.
  • the database structure allows response time records to be collated and reported by various reporting intervals (by hour, by day, by day-of-week, by week, by month, by year, for example).
  • a response time analyzer in the system advantageously calculates and stores response time for the various intervals substantially immediately that the response time values are determined. This also advantageously allows quick response time reporting and facilitates determination of response time anomalies.
  • the data record structure is usable to collate operational characteristics for multiple applications and users. This facilitates operational characteristic analysis by an Application Service Provider (ASP) for the purpose of optimization and management of hosted applications.
  • ASP Application Service Provider
  • the FIG. 2 data record structure 1 may be created for one or more of, a particular user, a particular customer of an ASP or a particular application.
  • Record element 2 is a physical record key used for identifying and sorting data record structures.
  • Record element 3 comprises metadata describing components of embedded logical records and an index to the components.
  • a particular instance of response time data is stored in a particular logical record instance 4 .
  • the particular instance of response time data may be for a particular time interval (hour, day, week, month etc.) or may be summary data for a particular period of time.
  • Record element 4 a stores metadata describing a time frame and identifying type of logical record for logical record 4 . This metadata also identifies components of the response time data of logical record 4 that are being measured and provides an index to the components.
  • Record element 4 b contains response time criteria (e.g., identifying percentile threshold values as response time alert values) that are used to determine when an exception condition occurs. Further, record element 4 c contains other data to be
  • the system employs a web-based user interface that provides a user friendly interface that hides complexity from a user.
  • the user interface allows for flexible generation of reports in response to a wide range of user selectable report creation criteria.
  • response time data is collected, it is formatted into a desired report format.
  • the system enables various report formats to be quickly generated in response to user command.
  • the response time data is stored in common data segments for different report creation criteria, enabling mix and match of data segments and facilitating search for common instances of abnormally increased response times.
  • FIG. 1 users AAA and BBB access Application A and user CCC accesses Application B.
  • Applications A and B provide messages 20 for storage in event log 105 .
  • the messages identify user activities and corresponding times that individual activities start and end.
  • Operational characteristic analyzer 107 analyzes and parses application message data 30 accessed from event log 105 to acquire response time values (and other operational characteristic parameter values).
  • Operational characteristic analyzer 107 provides resultant response time values 40 to database repository 109 .
  • Database repository 109 collates and aggregates values 40 to provide various criteria for use in creating reports presenting application operational characteristics to a user.
  • a particular user or system administrator initiates a request for a report presenting operational characteristics of a particular executable application for the particular user via one or more messages 70 communicated to database repository 109 .
  • a database management application managing database 109 collates and formats required operational characteristic information to create a report in response to a query containing particular search criteria received in messages 70 .
  • the database repository 109 management applicator provides the created report
  • Operational characteristic analyzer 107 acquires predetermined characteristic thresholds in messages 50 from database repository 109 .
  • Analyzer 107 uses the acquired thresholds in determining whether operational characteristics of a particular executable application and a particular user exceed a predetermined threshold.
  • Analyzer 107 generates and communicates alert messages 60 to particular destinations (e.g., particular users) in response to determining an operational characteristic exceeds its predetermined thresholds (such as maximum and minimum thresholds).
  • Operational characteristic analyzer 107 also acquires updates to predetermined characteristic thresholds in messages 50 from database repository 109 .
  • the FIG. 1 system provides real time monitoring and adapts its operation through learned behavior.
  • the following activity occurs:
  • Operational characteristic analyzer 107 accesses and parses the messages stored in event log 105 to identify and extract information needed to determine response time (and other operational characteristic) behavior.
  • analyzer 107 determines response time values from the event log 105 messages as follows. Response Application User Start Finish time A AAA 9:10:21 9:10:22 0:00:01 A BBB 9:10:22 9:15:10 0:04:48 B CCC 9:10:23 Analyzer 107 , applies predetermined response time monitoring rules in acquiring threshold information from database repository 109 and generating alert messages. Specifically, analyzer 107 generates an alert message for Application A and user BBB based on a response time exceeding a two second threshold. The response time for user ‘CCC’ is not yet known at this stage in the exemplary operation, so no alert message is generated for this event.
  • the system continuously accumulates and aggregates response time records over time and uses the accumulated data to adaptively adjust analysis, alert generation and report generation.
  • the records are processed and stored in database repository 109 maintained by the system. Ultimately this database contains values of typical response times.
  • database repository 109 becomes populated as follows. Average Worst Appln.
  • Analyzer 107 examines and analyzes the accumulated response time records and adaptively adjusts its analysis and alert generation functions based on this analysis. As a result, even though user BBB experiences a 4:48 minute response time, for example, analyzer 107 does not initiate generation of an alert message because the system has determined this is reasonably close to a normal value (average response time value 5 minutes 4 seconds) based on prior activity. This adaptive operation advantageously reduces false alerts.
  • analyzer 107 detects an increase in response time and generates alert messages indicating error. However, analyzer 107 accumulates response time values and determines a changed (increased) average response time value indicating the increased response time values are within a reasonable range. As a result, analyzer 107 no longer determines the increased response times to be excessive since they are within guidelines acquired from database repository 109 and analyzer 107 no longer generates alert messages for these response times.
  • application A is altered to include an additional function (function A 1 ) that provides a summary result instead of a detailed result.
  • function A 1 an additional function that provides a summary result instead of a detailed result.
  • user BBB intermittently experiences a reduced response time upon employing the summary function. This would be expected to result in analyzer 107 either incorrectly generating false alert messages or failing to generate alert messages when genuine excessive response times occur.
  • application. A is advantageously adapted to generate new application messages indicating start and stop times of operation of added or changed functions such as summary function A 1 in this example. Thereby, analyzer 107 prevents missing of alerts or generation of false alert messages by examining messages from application A to identify new messages indicating added, deleted or changed functions.
  • analyzer 107 detects a change in functions provided by Application A (i.e., detects operation of added function A 1 ) by examining application messages stored in event log 105 .
  • User operation of added summary function A 1 results in an exemplary message classification as follows.
  • Last Average Application User Access Resp Time Occurrences Worst Occurrences A AAA 3/31/04 0:00:01 451 0:00:04 2 8:58:85 A BBB 3/30/04 0:05:04 256 0:09:14 4 14:51:40 A1 BBB 3/30/04 0:00:03 24 0:00:05 10 14:22:10 B CCC 3/30/04 0:00:02 1,761 0:00:03 45 15:10:10
  • the adaptation of application A to generate new application messages indicating start and stop times of operation of added or changed functions advantageously enables analyzer 107 to correctly determine response time deviations.
  • analyzer 107 is also able to examine even log 105 messages to analyze application behavior based on particular day of the week, time of day, month-end for example.
  • Operational characteristic analyzer 107 provides values 40 to database repository 109 which collates and aggregates values 40 to provide various criteria indicating application behavior that may be used in creating reports for presentation to a user.
  • the criteria indicating application behavior stored in database 109 advantageously addresses problems with existing passive executable application monitoring systems. A specific problem occurs, for example, if there is little or no activity occurring. Under this condition, an existing passive executable application monitoring system may not detect and report a response time of an application that exceeds a normal range as long as minimal activity occurs in the application. In contrast, analyzer 107 and database 109 are aware of typical hourly, daily and weekly application activities and associated response times.
  • This data is used by analyzer 107 to detect response times exceeding a normal range determined for a particular time period such as a particular hour of a day, particular day of a week, particular week of a month or year, for example. Thereby unit 107 generates alert messages upon detection of response times exceeding a normal range determined for such particular time period.
  • a database management application managing database 109 collates and formats required operational characteristic information to create multiple reports in response to a query containing particular search criteria received in messages 70 .
  • the created reports are provided to one or more requesting users.
  • the reports are usable to compare response time activity for a given user or application (with other users, applications etc.) on a day-by-day, week-by-week or month-by-month basis, for example. Since some applications tend to be used in a similar manner based on the day-of-the-week, database 109 supports generation of reports enabling application response time comparison (and other operational characteristic comparison) of a particular day to the same day a week ago, two weeks ago, etc.
  • the reports contain information supporting comparison of different applications or different users within the same application. This type of report would be useful to determine if a user is incorrectly or sub-optimally using a specific application, for example.
  • FIG. 3 shows a graphical report provided by a database 109 management application and presenting operational characteristics of an executable application determined by the system of FIG. 2 .
  • the report may be accessed and viewed either online or as hardcopy following printing.
  • the graphical report shows a number of response time occurrences associated with user access of an executable application (Appl—item 313 ).
  • the response time occurrences are grouped in ten bands ranging from under one second to over minute expressed as a percentage of the total number of occurrences graphed.
  • the report shows response times experienced by user C 001 ( 310 ) during 3 different periods ( 302 , 304 and 306 ) individually graphed for each response time band.
  • Column 319 shows the number of response time occurrences recorded in the three time periods 302 , 304 and 306 .
  • Column 321 shows the number of response time occurrences actually graphed in the three time periods 302 , 304 and 306 .
  • FIG. 4 shows a tabular report of the response time characteristics of an executable application presented in the graphical report of FIG. 3 .
  • Column 400 shows the ten response time occurrence bands ranging from under one second to over 1 minute.
  • Columns 403 , 407 and 409 shows the number of response time occurrences for each response time band for the three time periods 302 , 304 and 306 identified in FIG. 3 .
  • response time values that are determined to be excessive response times are displayed as underlined to indicate links representing URLs.
  • a detailed analysis of the excessive response times is presented. The detailed analysis provides an exact time and date of the occurrences of the excessive response times and identifies an associated user of the application responsible for the excessive response times.
  • FIG. 5 shows a report presenting a detailed analysis of excessive response times associated with user access of a particular executable application 523 determined by the system of FIG. 1 over a fixed period of time 520 .
  • the report shows start and stop times in columns 500 and 503 respectively, for particular response time occurrences exceeding twenty seconds (shown in column 507 ) for a particular user (column 509 ) and workstation (column 511 ). These excessive response times are identified together with the number of uses (or accesses) of the application that did not have unusual response times.
  • the report shows, for example, the number of response time occurrences that are under twenty seconds, for the same application (or the same corresponding application function), user and workstation, such as the 1489 occurrences indicated in row 530 , for example.
  • the report indicates that user F. Ford experiences response times (e.g. 48, 42 and 43 seconds in rows 513 , 515 and 517 respectively) that significantly exceed the response times of users Jones, Miller, Davis, Rowen and Harold (23, 24, 21, 21 and 22 seconds respectively). Consequently, the report indicates that F. Ford is using application 523 in a manner that is different than the other users irrespective of which workstation is used. Multiple unusual response times grouped in time may indicate a response time problem within an application.
  • FIG. 6 shows a flowchart of a process employed by analyzer 107 , operating in conjunction with a database 109 management application, for determining operational characteristics of an executable application.
  • unit 107 acquires messages from event log 105 derived from an executable application and enabling determination of a duration of execution of a user initiated activity by the executable application.
  • the messages identify start of execution of the user initiated activity and completion of execution of the user initiated activity.
  • the user initiated activity comprise one or more of, a particular function of an executable application, a particular executable procedure of an executable application and execution of a particular executable application.
  • unit 107 interprets an acquired message to determine whether the interpreted message is provided in response to one or more of, a function of an executable application, an executable procedure of an executable application and a user initiated activity. Analyzer 107 also determines whether the interpreted message is provided in response to a function of an executable application previously un-encountered by analyzer 107 .
  • step 705 unit 107 determines a duration of execution of the user initiated activity by the executable application in response to acquired messages.
  • Analyzer 107 stores determined duration values in database 109 in step 708 .
  • Over time database 109 accumulates information comprising multiple duration values for corresponding multiple occurrences of the user initiated activity as well as information identifying a particular user initiating the activity.
  • Analyzer 107 in step 713 analyzes the accumulated information to derive threshold values indicating a normal range of duration values.
  • Analyzer 107 in step 713 analyzes the accumulated information to identify a duration of execution of a user initiated activity performed by the executable application exceeding the normal range derived based on the accumulated information.
  • Analyzer 107 adaptively adjusts the normal range at predetermined time intervals in response to change in accumulated duration values in database 109 .
  • a change in the accumulated duration values in database 109 comprises growth of the accumulated information or an identified change of trend in the accumulated information, for example.
  • Analyzer 107 initiates generation of an alert message in response to identifying the duration of execution of the user initiated activity exceeding the normal range.
  • analyzer 107 communicates a message to the executable application to initiate generation of a new application message associated with a particular user initiated activity in response to analysis of accumulated duration values in database 109 .
  • the process of FIG. 6 terminates at step 723 .
  • FIG. 7 shows a flowchart of a process employed by a first executable application supporting determination of operational characteristics of the first executable application.
  • the first application generates messages identifying start of execution of a user initiated activity by the first executable application and completion of execution the user initiated activity by the first executable application.
  • the first application communicates the messages to a second different executable application.
  • the second executable application accumulates information comprising multiple duration values, derived from the received communicated messages, for corresponding multiple occurrences of the user initiated activity.
  • the second executable application analyzes the accumulated information to identify a duration of execution of a user initiated activity by the first executable application exceeding a normal range derived based on the accumulated information.
  • the messages identifying start of the user initiated activity and completion of the user initiated activity identify a duration of the user initiated activity.
  • the process of FIG. 7 terminates at step 825 .
  • the system employs activity messages generated by an application and advantageously does not require links (hooks) into an underlying operating system or application.
  • An application may already provide such messages for auditing or security purposes and may not need any modification.
  • the system analyzes response time data and aggregates and stores it. This allows the system to initially determine expected application behavior and allows determination of variation from the expected behavior.
  • the system learns normal application behavior over time and adaptively adjusts an alerting process in response to variations from expected application behavior.
  • the system capability of learning expected response times and adapting to variation in response times reduces required application implementation and support manpower. This capability also minimizes the need to adjust response time alert thresholds when monitoring a wide variety of different executable applications.
  • the system is applicable to computing processes that provide a message indicating a user is initiating an activity in an executable application (and providing an activity start time) as well as a message indicating the user activity is complete (and providing an activity end time).
  • the system may be used for monitoring mainframe batch processing, UNIX or Windows-based processing or proprietary system processing in any field not just healthcare, for example.
  • the system is able to provide response times where acquisition or implementation of other monitoring systems is burdensome, e.g., due to software licensing, infrastructure implementation. CPU, management or configuration requirements.
  • the system may be used by an Application Service Provider (ASP) in hosting executable applications as a service allowing remote users to generate reports and see response times of applications that are leased or rented, for example. Also for sites that purchase an application, the system may be employed as an additional application or service.
  • ASP Application Service Provider
  • FIGS. 1-7 are not exclusive. Other systems and processes may be derived in accordance with the principles of the invention to accomplish the same objectives.
  • this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the invention. Further, any of the functions provided by the system of FIG. 1 may be implemented in hardware, software or a combination of both and the described processes, including the processes of FIGS. 6 and 7 , may be incorporated in a tangible storage medium as machine executable code comprising one or more executable applications.

Abstract

An adaptive system adaptively determines operational characteristics of an executable application. A system for monitoring executable application operation characteristics includes an acquisition processor for acquiring messages from an executable application enabling determination of a duration of the user initiated activity. A data processor determines a duration of the user initiated activity in response to the acquired messages and accumulates information comprising a plurality of duration values for a corresponding plurality of occurrences of the user initiated activity. A data analyzer analyzes accumulated information to identify a duration of execution of a user initiated activity by the executable application exceeding a normal range derived based on the accumulated information.

Description

  • This is a non-provisional application of provisional applications Ser. No. 60/568,532 by D. W. Smith filed May 6, 2004.
  • FIELD OF THE INVENTION
  • This invention concerns a system for determining executable application operation characteristics such as application response time.
  • BACKGROUND INFORMATION
  • It is necessary to determine executable application operational characteristics (including transactional based application characteristics) such as response time and access time, for example, for a variety of purposes. The operational characteristics are used to optimize a computing configuration, to determine communication bandwidth requirements or to demonstrate that an application meets guaranteed performance thresholds, for example. There are existing systems (from BMC Software, Computer Associates, Compuware and IBM, for example) providing response time monitoring and alerting for transactional-based executable applications. These existing systems typically contain fixed links (hooks) enabling communication and acquisition of data with an underlying operating system or executable application in order to determine operational characteristics. However, the existing systems also involve additional complexity and computational burden resulting from additional feature and function overhead. Further, the additional features and functions exceed those necessary for operational characteristic determination. The additional features and functions provide the ability to troubleshoot a processing system and to examine operation of executable components, for example. The additional features and functions also incur an execution burden on CPU resources. Also, existing systems employ different definitions of operational characteristics such as response time and acquire response time information in different ways. It is desirable for a system to be able to acquire executable application operational characteristics, with reduced processing and computational burden and without requirement for modification of a target application of operation system environment. In addition, existing response time monitoring systems are typically integrated with an operating system or application and collect response time data but fail to aggregate, correlate and analyze the data until a user requests it. A system according to invention principles addresses the identified requirements, deficiencies and associated problems.
  • SUMMARY OF THE INVENTION
  • An adaptive system provides minimally intrusive operational characteristic (e.g., response time) determination and monitoring of an executable application and provides predictive real time operational characteristic alert messages based on learned behavior. A system for monitoring executable application operation characteristics includes an acquisition processor for acquiring messages from an executable application enabling determination of a duration of the user initiated activity. A data processor determines a duration of the user initiated activity in response to the acquired messages and accumulates information comprising a plurality of duration values for a corresponding plurality of occurrences of the user initiated activity. A data analyzer analyzes accumulated information to identify a duration of execution of a user initiated activity exceeding a normal range derived based on the accumulated information.
  • BRIEF DESCRIPTION OF THE DRAWING
  • FIG. 1 shows a system for determining operational characteristics of an executable application, according to invention principles.
  • FIG. 2 shows a data record structure containing data associated with determination of operational characteristics of an executable application, according to invention principles.
  • FIG. 3 shows a graphical report presenting operational characteristics of an executable application determined by the system of FIG. 1, according to invention principles.
  • FIG. 4 shows a tabular report presenting operational characteristics of an executable application determined by the system of FIG. 1, according to invention principles.
  • FIG. 5 shows a report presenting a detailed analysis of excessive response times of an executable application determined by the system of FIG. 1, according to invention principles.
  • FIG. 6 shows a flowchart of a process employed by the system of FIG. 1 for determining operational characteristics of an executable application, according to invention principles.
  • FIG. 7 shows a flowchart of a process employed by an executable application supporting determination of operational characteristics of the executable application, according to invention principles.
  • DETAILED DESCRIPTION OF INVENTION
  • FIG. 1 shows an adaptive system for determining operational characteristics of an executable application. The system determines executable application operational characteristics (e.g., response time) whilst requiring minimal intrusion into the application and operating system. The system provides monitoring of an executable application and adaptively generates alert messages based on real time prediction of operational characteristic through learned behavior. The system provides response time monitoring and alerting without requiring a large infrastructure to support it. In addition, the system adaptively learns acceptable ranges of response times for individual applications as well as for individual users. The system receives input data from an executable application being monitored. This advantageously eliminates the need for interface links (hooks) or overhead between the system and the monitored executable application (as typically required by existing monitoring systems). The system also advantageously has the capability to monitor a subset of users, applications or customers (e.g., customers of an Application Service Provider (ASP) hosting one more applications for use by the customers).
  • The system compares current response times with corresponding historical, previously acquired response times (such as response times acquired on other days, weeks, months or by different applications or for different customers) in a real time manner. The system adaptively uses this comparison to identify acceptable ranges of executable application response times and reduces application management and configuration time.
  • An executable application as used herein comprises code or machine readable instruction for implementing predetermined functions including those of an operating system, healthcare information system or other information processing system, for example, in response user command or input. An executable procedure is a segment of code (machine readable instruction), sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes and may include performing operations on received input parameters (or in response to received input parameters) and provide resulting output parameters. A processor as used herein is a device and/or set of machine-readable instructions for performing tasks. A processor comprises any one or combination of, hardware, firmware, and/or software. A processor acts upon information by manipulating, analyzing, modifying, converting or transmitting information for use by an executable procedure or an information device, and/or by routing the information to an output device. A processor may use or comprise the capabilities of a controller or microprocessor, for example. A display processor or generator is a known element comprising electronic circuitry or software or a combination of both for generating display images or portions thereof. A user interface comprises one or more display images enabling user interaction with a processor or other device.
  • FIG. 2 shows a data record structure containing data associated with determination of operational characteristics of an executable application. The FIG. 1 system employs this record structure and a user interface in monitoring application operational characteristics. The record structure is self-maintaining and employs files of fixed size that do not need to be purged. The database structure allows response time records to be collated and reported by various reporting intervals (by hour, by day, by day-of-week, by week, by month, by year, for example). A response time analyzer in the system advantageously calculates and stores response time for the various intervals substantially immediately that the response time values are determined. This also advantageously allows quick response time reporting and facilitates determination of response time anomalies. In addition, the data record structure is usable to collate operational characteristics for multiple applications and users. This facilitates operational characteristic analysis by an Application Service Provider (ASP) for the purpose of optimization and management of hosted applications. The system enables determination of an excessive response time for a single user or for multiple users, for example.
  • The FIG. 2 data record structure 1 may be created for one or more of, a particular user, a particular customer of an ASP or a particular application. Record element 2 is a physical record key used for identifying and sorting data record structures. Record element 3 comprises metadata describing components of embedded logical records and an index to the components. A particular instance of response time data is stored in a particular logical record instance 4. The particular instance of response time data may be for a particular time interval (hour, day, week, month etc.) or may be summary data for a particular period of time. Record element 4 a stores metadata describing a time frame and identifying type of logical record for logical record 4. This metadata also identifies components of the response time data of logical record 4 that are being measured and provides an index to the components. Record element 4 b contains response time criteria (e.g., identifying percentile threshold values as response time alert values) that are used to determine when an exception condition occurs. Further, record element 4 c contains other data to be acquired and stored in data record structure 1.
  • The system employs a web-based user interface that provides a user friendly interface that hides complexity from a user. The user interface allows for flexible generation of reports in response to a wide range of user selectable report creation criteria. Further, as response time data is collected, it is formatted into a desired report format. The system enables various report formats to be quickly generated in response to user command. In addition, the response time data is stored in common data segments for different report creation criteria, enabling mix and match of data segments and facilitating search for common instances of abnormally increased response times.
  • In the FIG. 1 system, users AAA and BBB access Application A and user CCC accesses Application B. Applications A and B provide messages 20 for storage in event log 105. The messages identify user activities and corresponding times that individual activities start and end. Operational characteristic analyzer 107 analyzes and parses application message data 30 accessed from event log 105 to acquire response time values (and other operational characteristic parameter values). Operational characteristic analyzer 107 provides resultant response time values 40 to database repository 109. Database repository 109 collates and aggregates values 40 to provide various criteria for use in creating reports presenting application operational characteristics to a user. A particular user or system administrator initiates a request for a report presenting operational characteristics of a particular executable application for the particular user via one or more messages 70 communicated to database repository 109. A database management application managing database 109 collates and formats required operational characteristic information to create a report in response to a query containing particular search criteria received in messages 70. The database repository 109 management applicator provides the created report to the particular user.
  • Operational characteristic analyzer 107 acquires predetermined characteristic thresholds in messages 50 from database repository 109. Analyzer 107 uses the acquired thresholds in determining whether operational characteristics of a particular executable application and a particular user exceed a predetermined threshold. Analyzer 107 generates and communicates alert messages 60 to particular destinations (e.g., particular users) in response to determining an operational characteristic exceeds its predetermined thresholds (such as maximum and minimum thresholds). Operational characteristic analyzer 107 also acquires updates to predetermined characteristic thresholds in messages 50 from database repository 109.
  • The FIG. 1 system provides real time monitoring and adapts its operation through learned behavior. In exemplary operation, the following activity occurs:
    • 1. User AAA accesses Application A beginning at 9:10:21. Application A generates a message identifying and recording this event in event log 105.
    • 2. User BBB accesses Application A beginning at 9:10:22. Application A generates a message identifying and recording this event in event log 105.
    • 3. At 9:10:22, user AAA receives processing results from Application A. Application A generates a message identifying and recording this event in event log 105.
    • 4. User CCC accesses application B beginning at 9:10:23. Application B generates a message identifying and recording this event in event log 105.
    • 5. At 9:15:10, user BBB receives processing results from application A. Application A generates a message identifying and recording this event in event log 105.
  • Operational characteristic analyzer 107 accesses and parses the messages stored in event log 105 to identify and extract information needed to determine response time (and other operational characteristic) behavior. In the exemplary operation, analyzer 107 determines response time values from the event log 105 messages as follows.
    Response
    Application User Start Finish time
    A AAA 9:10:21 9:10:22 0:00:01
    A BBB 9:10:22 9:15:10 0:04:48
    B CCC 9:10:23

    Analyzer 107, applies predetermined response time monitoring rules in acquiring threshold information from database repository 109 and generating alert messages. Specifically, analyzer 107 generates an alert message for Application A and user BBB based on a response time exceeding a two second threshold. The response time for user ‘CCC’ is not yet known at this stage in the exemplary operation, so no alert message is generated for this event.
  • The system continuously accumulates and aggregates response time records over time and uses the accumulated data to adaptively adjust analysis, alert generation and report generation. The records are processed and stored in database repository 109 maintained by the system. Ultimately this database contains values of typical response times. In the exemplary operation, database repository 109 becomes populated as follows.
    Average Worst
    Appln. User Last Access Response Time Occurrences Response Time Occurrences
    A AAA 3/31/04 0:00:01 451 0:00:04 2
    8:58:85
    A BBB 3/30/04 0:05:04 256 0:09:14 4
    14:51:40
    B CCC 3/30/04 0:00:02 1,761 0:00:03 45
    15:10:10

    Analyzer 107 examines and analyzes the accumulated response time records and adaptively adjusts its analysis and alert generation functions based on this analysis. As a result, even though user BBB experiences a 4:48 minute response time, for example, analyzer 107 does not initiate generation of an alert message because the system has determined this is reasonably close to a normal value (average response time value 5 minutes 4 seconds) based on prior activity. This adaptive operation advantageously reduces false alerts.
  • Continuing the operation example, over time, user AAA alters usage of application A to involve more time consuming activities, initially, analyzer 107 detects an increase in response time and generates alert messages indicating error. However, analyzer 107 accumulates response time values and determines a changed (increased) average response time value indicating the increased response time values are within a reasonable range. As a result, analyzer 107 no longer determines the increased response times to be excessive since they are within guidelines acquired from database repository 109 and analyzer 107 no longer generates alert messages for these response times.
  • In another example, application A is altered to include an additional function (function A1) that provides a summary result instead of a detailed result. As a consequence, user BBB intermittently experiences a reduced response time upon employing the summary function. This would be expected to result in analyzer 107 either incorrectly generating false alert messages or failing to generate alert messages when genuine excessive response times occur. However, application. A is advantageously adapted to generate new application messages indicating start and stop times of operation of added or changed functions such as summary function A1 in this example. Thereby, analyzer 107 prevents missing of alerts or generation of false alert messages by examining messages from application A to identify new messages indicating added, deleted or changed functions. Specifically, analyzer 107 detects a change in functions provided by Application A (i.e., detects operation of added function A1) by examining application messages stored in event log 105. User operation of added summary function A1 results in an exemplary message classification as follows.
    Last Average
    Application User Access Resp Time Occurrences Worst Occurrences
    A AAA 3/31/04 0:00:01 451 0:00:04 2
    8:58:85
    A BBB 3/30/04 0:05:04 256 0:09:14 4
    14:51:40
    A1 BBB 3/30/04 0:00:03 24 0:00:05 10
    14:22:10
    B CCC 3/30/04 0:00:02 1,761 0:00:03 45
    15:10:10
  • The adaptation of application A to generate new application messages indicating start and stop times of operation of added or changed functions advantageously enables analyzer 107 to correctly determine response time deviations. In addition, analyzer 107 is also able to examine even log 105 messages to analyze application behavior based on particular day of the week, time of day, month-end for example.
  • Operational characteristic analyzer 107 provides values 40 to database repository 109 which collates and aggregates values 40 to provide various criteria indicating application behavior that may be used in creating reports for presentation to a user. The criteria indicating application behavior stored in database 109 advantageously addresses problems with existing passive executable application monitoring systems. A specific problem occurs, for example, if there is little or no activity occurring. Under this condition, an existing passive executable application monitoring system may not detect and report a response time of an application that exceeds a normal range as long as minimal activity occurs in the application. In contrast, analyzer 107 and database 109 are aware of typical hourly, daily and weekly application activities and associated response times. This data is used by analyzer 107 to detect response times exceeding a normal range determined for a particular time period such as a particular hour of a day, particular day of a week, particular week of a month or year, for example. Thereby unit 107 generates alert messages upon detection of response times exceeding a normal range determined for such particular time period.
  • A database management application managing database 109 collates and formats required operational characteristic information to create multiple reports in response to a query containing particular search criteria received in messages 70. The created reports are provided to one or more requesting users. The reports are usable to compare response time activity for a given user or application (with other users, applications etc.) on a day-by-day, week-by-week or month-by-month basis, for example. Since some applications tend to be used in a similar manner based on the day-of-the-week, database 109 supports generation of reports enabling application response time comparison (and other operational characteristic comparison) of a particular day to the same day a week ago, two weeks ago, etc. The reports contain information supporting comparison of different applications or different users within the same application. This type of report would be useful to determine if a user is incorrectly or sub-optimally using a specific application, for example.
  • FIG. 3 shows a graphical report provided by a database 109 management application and presenting operational characteristics of an executable application determined by the system of FIG. 2. The report may be accessed and viewed either online or as hardcopy following printing. The graphical report shows a number of response time occurrences associated with user access of an executable application (Appl—item 313). The response time occurrences are grouped in ten bands ranging from under one second to over minute expressed as a percentage of the total number of occurrences graphed. The report shows response times experienced by user C001 (310) during 3 different periods (302, 304 and 306) individually graphed for each response time band. Column 319 shows the number of response time occurrences recorded in the three time periods 302, 304 and 306. Column 321 shows the number of response time occurrences actually graphed in the three time periods 302, 304 and 306.
  • FIG. 4 shows a tabular report of the response time characteristics of an executable application presented in the graphical report of FIG. 3. Column 400 shows the ten response time occurrence bands ranging from under one second to over 1 minute. Columns 403, 407 and 409 shows the number of response time occurrences for each response time band for the three time periods 302, 304 and 306 identified in FIG. 3. Within the table, response time values that are determined to be excessive response times are displayed as underlined to indicate links representing URLs. In response to user selection of a displayed link, a detailed analysis of the excessive response times is presented. The detailed analysis provides an exact time and date of the occurrences of the excessive response times and identifies an associated user of the application responsible for the excessive response times.
  • FIG. 5 shows a report presenting a detailed analysis of excessive response times associated with user access of a particular executable application 523 determined by the system of FIG. 1 over a fixed period of time 520. The report shows start and stop times in columns 500 and 503 respectively, for particular response time occurrences exceeding twenty seconds (shown in column 507) for a particular user (column 509) and workstation (column 511). These excessive response times are identified together with the number of uses (or accesses) of the application that did not have unusual response times. The report shows, for example, the number of response time occurrences that are under twenty seconds, for the same application (or the same corresponding application function), user and workstation, such as the 1489 occurrences indicated in row 530, for example. The report indicates that user F. Ford experiences response times (e.g. 48, 42 and 43 seconds in rows 513, 515 and 517 respectively) that significantly exceed the response times of users Jones, Miller, Davis, Rowen and Harold (23, 24, 21, 21 and 22 seconds respectively). Consequently, the report indicates that F. Ford is using application 523 in a manner that is different than the other users irrespective of which workstation is used. Multiple unusual response times grouped in time may indicate a response time problem within an application.
  • FIG. 6 shows a flowchart of a process employed by analyzer 107, operating in conjunction with a database 109 management application, for determining operational characteristics of an executable application. In step 702, following the start at step 700, unit 107 acquires messages from event log 105 derived from an executable application and enabling determination of a duration of execution of a user initiated activity by the executable application. The messages identify start of execution of the user initiated activity and completion of execution of the user initiated activity. The user initiated activity comprise one or more of, a particular function of an executable application, a particular executable procedure of an executable application and execution of a particular executable application. In step 704, unit 107 interprets an acquired message to determine whether the interpreted message is provided in response to one or more of, a function of an executable application, an executable procedure of an executable application and a user initiated activity. Analyzer 107 also determines whether the interpreted message is provided in response to a function of an executable application previously un-encountered by analyzer 107.
  • In step 705, unit 107 determines a duration of execution of the user initiated activity by the executable application in response to acquired messages. Analyzer 107 stores determined duration values in database 109 in step 708. Over time database 109 accumulates information comprising multiple duration values for corresponding multiple occurrences of the user initiated activity as well as information identifying a particular user initiating the activity. Analyzer 107 in step 713 analyzes the accumulated information to derive threshold values indicating a normal range of duration values. Analyzer 107 in step 713 analyzes the accumulated information to identify a duration of execution of a user initiated activity performed by the executable application exceeding the normal range derived based on the accumulated information. Analyzer 107 adaptively adjusts the normal range at predetermined time intervals in response to change in accumulated duration values in database 109. A change in the accumulated duration values in database 109 comprises growth of the accumulated information or an identified change of trend in the accumulated information, for example. Analyzer 107 initiates generation of an alert message in response to identifying the duration of execution of the user initiated activity exceeding the normal range. In step 717, analyzer 107 communicates a message to the executable application to initiate generation of a new application message associated with a particular user initiated activity in response to analysis of accumulated duration values in database 109. The process of FIG. 6 terminates at step 723.
  • FIG. 7 shows a flowchart of a process employed by a first executable application supporting determination of operational characteristics of the first executable application. In step 803 following the start at step 800, the first application generates messages identifying start of execution of a user initiated activity by the first executable application and completion of execution the user initiated activity by the first executable application. In step 807 the first application communicates the messages to a second different executable application. The second executable application accumulates information comprising multiple duration values, derived from the received communicated messages, for corresponding multiple occurrences of the user initiated activity. The second executable application analyzes the accumulated information to identify a duration of execution of a user initiated activity by the first executable application exceeding a normal range derived based on the accumulated information. The messages identifying start of the user initiated activity and completion of the user initiated activity identify a duration of the user initiated activity. The process of FIG. 7 terminates at step 825.
  • The system employs activity messages generated by an application and advantageously does not require links (hooks) into an underlying operating system or application. An application may already provide such messages for auditing or security purposes and may not need any modification. In response to occurrence of an activity, the system analyzes response time data and aggregates and stores it. This allows the system to initially determine expected application behavior and allows determination of variation from the expected behavior. The system learns normal application behavior over time and adaptively adjusts an alerting process in response to variations from expected application behavior. The system capability of learning expected response times and adapting to variation in response times, reduces required application implementation and support manpower. This capability also minimizes the need to adjust response time alert thresholds when monitoring a wide variety of different executable applications.
  • The system is applicable to computing processes that provide a message indicating a user is initiating an activity in an executable application (and providing an activity start time) as well as a message indicating the user activity is complete (and providing an activity end time). The system may be used for monitoring mainframe batch processing, UNIX or Windows-based processing or proprietary system processing in any field not just healthcare, for example. The system is able to provide response times where acquisition or implementation of other monitoring systems is burdensome, e.g., due to software licensing, infrastructure implementation. CPU, management or configuration requirements. The system may be used by an Application Service Provider (ASP) in hosting executable applications as a service allowing remote users to generate reports and see response times of applications that are leased or rented, for example. Also for sites that purchase an application, the system may be employed as an additional application or service.
  • The system and processes presented in FIGS. 1-7 are not exclusive. Other systems and processes may be derived in accordance with the principles of the invention to accomplish the same objectives. Although this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the invention. Further, any of the functions provided by the system of FIG. 1 may be implemented in hardware, software or a combination of both and the described processes, including the processes of FIGS. 6 and 7, may be incorporated in a tangible storage medium as machine executable code comprising one or more executable applications.

Claims (22)

1. A system for monitoring executable application operation characteristics, comprising:
an acquisition processor for acquiring messages from an executable application enabling determination of a duration of execution of a user initiated activity by said executable application;
a data processor for determining a duration of execution of said user initiated activity by said executable application in response to said acquired messages and for accumulating information comprising a plurality of duration values for a corresponding plurality of occurrences of said user initiated activity; and
a data analyzer for analyzing said accumulated information to identify a duration of execution of a user initiated activity by said executable application exceeding a normal range derived based on said accumulated information.
2. A system according to claim 1, wherein
said messages enabling determination of said duration of execution of said user initiated activity identify start of execution of said user initiated activity and completion of execution of said user initiated activity.
3. A system according to claim 1, wherein
said accumulated information includes information identifying a particular user initiating said activity and
said activity comprises at least one of, (a) a particular function of an executable application and (b) a particular executable procedure of an executable application.
4. A system according to claim 1, wherein
said accumulated information includes information identifying a particular user initiating said activity and
said activity comprises execution of a particular executable application.
5. A system according to claim 1, wherein
said data analyzer analyzes said accumulated information to derive threshold values indicating said normal range.
6. A system according to claim 1, wherein
said data processor stores said accumulated information in a repository and
said data analyzer analyzes accumulated information retrieved from said repository.
7. A system according to claim 1, wherein
said data analyzer initiates generation of an alert message in response to identifying said duration of execution of said user initiated activity exceeding said normal range.
8. A system according to claim 1, wherein
said data analyzer adaptively adjusts said normal range in response to change in said accumulated information.
9. A system according to claim 8, wherein
said data analyzer,
analyzes said accumulated information to derive threshold values indicating said normal range and
adaptively adjusts said threshold values in response to change in said accumulated information.
10. A system according to claim 8, wherein
said change in said accumulated information comprises at least one of, (a) growth of said accumulated information and (b) an identified change of trend in said accumulated information.
11. A system according to claim 1, wherein
said data analyzer adaptively adjusts said normal range at predetermined time intervals in response to change in said accumulated information.
12. A system according to claim 1, including
a message interpreter for interpreting an acquired message to determine whether said interpreted message is provided in response to at least one of, (a) a function of an executable application, (b) an executable procedure of an executable application and (c) a user initiated activity.
13. A system according to claim 1, including
a message interpreter for interpreting an acquired message to determine whether said interpreted message is provided in response to a function of an executable application previously un-encountered by said data analyzer
14. A system according to claim 1, wherein
said data analyzer communicates a message to said executable application to initiate generation of a new application message associated with a particular user initiated activity in response to analysis of said accumulated information.
15. A system according to claim 1, wherein
said normal range is for a predetermined time period.
16. A system according to claim 15, wherein
said predetermined time period is at least one of, (a) a particular portion of a day, (b) a particular portion of a week, (c) a particular portion of a month and (d) a particular portion of a year.
17. A system employed by an executable application to support monitoring of operation characteristics of said executable application, comprising:
a message generator for generating messages identifying start of execution of a user initiated activity by a first executable application and completion of execution said user initiated activity by said first executable application; and
a communication processor for communicating said messages to a second different executable application for,
accumulating information comprising a plurality of duration values for a corresponding plurality of occurrences of said user initiated activity, said duration values being derived from said communicated messages and
analyzing said accumulated information to identify a duration of execution of a user initiated activity by said first executable application exceeding a normal range derived based on said accumulated information.
18. A system according to claim 17, wherein
said messages identifying start of a user initiated activity and completion of said user initiated activity, comprise messages identifying a duration of said user initiated activity.
19. A tangible storage medium according to claim 17 embodying machine executable code comprising said executable application.
20. A system for monitoring executable application operation characteristics, comprising:
an acquisition processor for acquiring messages from an executable application identifying start of execution of a user initiated activity by said executable application and completion of said user initiated activity;
a data processor for determining a duration of execution of said user initiated activity by said executable application in response to said acquired messages and for accumulating information comprising a plurality of duration values for a corresponding plurality of occurrences of said user initiated activity; and
a data analyzer for analyzing said accumulated information to identify a duration of execution of a user initiated activity by said executable application exceeding a normal range derived based on said accumulated information.
21. A method for monitoring executable application operation characteristics, comprising the activities of:
acquiring messages from an executable application enabling determination of a duration of execution of a user initiated activity by said executable application;
determining a duration of execution of said user initiated activity by said executable application in response to said acquired messages;
accumulating information comprising a plurality of duration values for a corresponding plurality of occurrences of said user initiated activity; and
analyzing said accumulated information to identify a duration of execution of a user initiated activity by said executable application exceeding a normal range derived based on said accumulated information.
22. A method for monitoring executable application operation characteristics, comprising the activities of:
acquiring messages from an executable application identifying start of execution of a user initiated activity by said executable application and completion of said user initiated activity;
determining a duration of execution of said user initiated activity by said executable application in response to said acquired messages;
accumulating information comprising a plurality of duration values for a corresponding plurality of occurrences of said user initiated activity; and
analyzing said accumulated information to identify a duration of execution of a user initiated activity by said executable application exceeding a normal range derived based on said accumulated information.
US10/980,498 2004-05-06 2004-11-03 System for adaptively determining executable application operation characteristics Abandoned US20050251792A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US10/980,498 US20050251792A1 (en) 2004-05-06 2004-11-03 System for adaptively determining executable application operation characteristics
DE102005020893A DE102005020893A1 (en) 2004-05-06 2005-05-04 System for adaptively determining operation properties of an executable application

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US56853204P 2004-05-06 2004-05-06
US10/980,498 US20050251792A1 (en) 2004-05-06 2004-11-03 System for adaptively determining executable application operation characteristics

Publications (1)

Publication Number Publication Date
US20050251792A1 true US20050251792A1 (en) 2005-11-10

Family

ID=35240794

Family Applications (1)

Application Number Title Priority Date Filing Date
US10/980,498 Abandoned US20050251792A1 (en) 2004-05-06 2004-11-03 System for adaptively determining executable application operation characteristics

Country Status (2)

Country Link
US (1) US20050251792A1 (en)
DE (1) DE102005020893A1 (en)

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070002736A1 (en) * 2005-06-16 2007-01-04 Cisco Technology, Inc. System and method for improving network resource utilization
US20070083564A1 (en) * 2005-10-07 2007-04-12 Ramacher Mark C Automatic performance statistical comparison between two periods
US20070277155A1 (en) * 2006-05-23 2007-11-29 Casey William L Evaluating performance of software application
US20090138859A1 (en) * 2007-11-27 2009-05-28 Sun Microsystems, Inc. Sampling based runtime optimizer for efficient debugging of applications
US7664847B2 (en) 2003-08-14 2010-02-16 Oracle International Corporation Managing workload by service
US7853579B2 (en) 2003-08-14 2010-12-14 Oracle International Corporation Methods, systems and software for identifying and managing database work
US20130036405A1 (en) * 2011-08-07 2013-02-07 Guy Verbest Automated test failure troubleshooter
US20130081001A1 (en) * 2011-09-23 2013-03-28 Microsoft Corporation Immediate delay tracker tool
US20130162433A1 (en) * 2007-10-12 2013-06-27 Masimo Corporation Systems and methods for storing, analyzing, retrieving and displaying streaming medical data
WO2014208139A1 (en) * 2013-06-28 2014-12-31 日本電気株式会社 Fault detection device, control method, and program
US20150207709A1 (en) * 2014-01-21 2015-07-23 Oracle International Corporation Logging incident manager
US9218454B2 (en) 2009-03-04 2015-12-22 Masimo Corporation Medical monitoring system
EP2990950A1 (en) * 2014-08-25 2016-03-02 Tata Consultancy Services Limited Monitoring activities of a software application
US9323894B2 (en) 2011-08-19 2016-04-26 Masimo Corporation Health care sanitation monitoring system
CN107402864A (en) * 2017-06-07 2017-11-28 阿里巴巴集团控股有限公司 Access processing method, device and equipment, the computer-readable recording medium of duration
EP3316142A1 (en) * 2016-10-27 2018-05-02 Entit Software LLC Performance monitor based on user engagement
US10007758B2 (en) 2009-03-04 2018-06-26 Masimo Corporation Medical monitoring system
US10032002B2 (en) 2009-03-04 2018-07-24 Masimo Corporation Medical monitoring system
US11190512B2 (en) 2019-04-17 2021-11-30 Microsoft Technology Licensing, Llc Integrity attestation of attestation component
US11223547B1 (en) * 2013-01-07 2022-01-11 Workspot, Inc. Managing information technology infrastructure based on user experience
US11222137B2 (en) * 2019-05-03 2022-01-11 Microsoft Technology Licensing, Llc Storing and executing an application in a user's personal storage with user granted permission
US11381567B2 (en) 2019-04-29 2022-07-05 Microsoft Technology Licensing, Llc Execution of an application within a scope of user-granted permission
US11392467B2 (en) 2019-04-17 2022-07-19 Microsoft Technology Licensing, Llc Failover between decentralized identity stores
US11411959B2 (en) 2019-05-03 2022-08-09 Microsoft Technology Licensing, Llc Execution of application in a container within a scope of user-granted permission
US11429743B2 (en) 2019-04-29 2022-08-30 Microsoft Technology Licensing, Llc Localization of DID-related claims and data

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4780821A (en) * 1986-07-29 1988-10-25 International Business Machines Corp. Method for multiple programs management within a network having a server computer and a plurality of remote computers
US4796181A (en) * 1986-10-24 1989-01-03 Wiedemer John D Billing system for computer software
US5023907A (en) * 1988-09-30 1991-06-11 Apollo Computer, Inc. Network license server
US5694549A (en) * 1994-03-03 1997-12-02 Telescan, Inc. Multi-provider on-line communications system
US5740233A (en) * 1995-11-02 1998-04-14 Intervoice Limited Partnership System and method for statistical diagnosis of the operation of an automated telephone system
US5848396A (en) * 1996-04-26 1998-12-08 Freedom Of Information, Inc. Method and apparatus for determining behavioral profile of a computer user
US5861922A (en) * 1992-09-16 1999-01-19 Fujitsu Ltd. Image data coding and restoring method and apparatus for coding and restoring the same
US5930773A (en) * 1997-12-17 1999-07-27 Avista Advantage, Inc. Computerized resource accounting methods and systems, computerized utility management methods and systems, multi-user utility management methods and systems, and energy-consumption-based tracking methods and systems
US5949415A (en) * 1997-06-16 1999-09-07 Intel Corporation Method and apparatus for tracking program usage in a computer system
US6037868A (en) * 1997-07-04 2000-03-14 Samsung Electronics Co., Ltd. Alarm method and apparatus for a mobile communication repeating system
US6202199B1 (en) * 1997-07-31 2001-03-13 Mutek Solutions, Ltd. System and method for remotely analyzing the execution of computer programs
US6249802B1 (en) * 1997-09-19 2001-06-19 Silicon Graphics, Inc. Method, system, and computer program product for allocating physical memory in a distributed shared memory network
US6282701B1 (en) * 1997-07-31 2001-08-28 Mutek Solutions, Ltd. System and method for monitoring and analyzing the execution of computer programs

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4780821A (en) * 1986-07-29 1988-10-25 International Business Machines Corp. Method for multiple programs management within a network having a server computer and a plurality of remote computers
US4796181A (en) * 1986-10-24 1989-01-03 Wiedemer John D Billing system for computer software
US5023907A (en) * 1988-09-30 1991-06-11 Apollo Computer, Inc. Network license server
US5861922A (en) * 1992-09-16 1999-01-19 Fujitsu Ltd. Image data coding and restoring method and apparatus for coding and restoring the same
US5694549A (en) * 1994-03-03 1997-12-02 Telescan, Inc. Multi-provider on-line communications system
US5740233A (en) * 1995-11-02 1998-04-14 Intervoice Limited Partnership System and method for statistical diagnosis of the operation of an automated telephone system
US5848396A (en) * 1996-04-26 1998-12-08 Freedom Of Information, Inc. Method and apparatus for determining behavioral profile of a computer user
US5949415A (en) * 1997-06-16 1999-09-07 Intel Corporation Method and apparatus for tracking program usage in a computer system
US6037868A (en) * 1997-07-04 2000-03-14 Samsung Electronics Co., Ltd. Alarm method and apparatus for a mobile communication repeating system
US6202199B1 (en) * 1997-07-31 2001-03-13 Mutek Solutions, Ltd. System and method for remotely analyzing the execution of computer programs
US6282701B1 (en) * 1997-07-31 2001-08-28 Mutek Solutions, Ltd. System and method for monitoring and analyzing the execution of computer programs
US6249802B1 (en) * 1997-09-19 2001-06-19 Silicon Graphics, Inc. Method, system, and computer program product for allocating physical memory in a distributed shared memory network
US5930773A (en) * 1997-12-17 1999-07-27 Avista Advantage, Inc. Computerized resource accounting methods and systems, computerized utility management methods and systems, multi-user utility management methods and systems, and energy-consumption-based tracking methods and systems

Cited By (47)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7664847B2 (en) 2003-08-14 2010-02-16 Oracle International Corporation Managing workload by service
US7853579B2 (en) 2003-08-14 2010-12-14 Oracle International Corporation Methods, systems and software for identifying and managing database work
US20070002736A1 (en) * 2005-06-16 2007-01-04 Cisco Technology, Inc. System and method for improving network resource utilization
US20070083564A1 (en) * 2005-10-07 2007-04-12 Ramacher Mark C Automatic performance statistical comparison between two periods
US7526409B2 (en) * 2005-10-07 2009-04-28 Oracle International Corporation Automatic performance statistical comparison between two periods
US20070277155A1 (en) * 2006-05-23 2007-11-29 Casey William L Evaluating performance of software application
US7937690B2 (en) * 2006-05-23 2011-05-03 Hewlett-Packard Development Company, L.P. Evaluating performance of software application
US9142117B2 (en) * 2007-10-12 2015-09-22 Masimo Corporation Systems and methods for storing, analyzing, retrieving and displaying streaming medical data
US20130162433A1 (en) * 2007-10-12 2013-06-27 Masimo Corporation Systems and methods for storing, analyzing, retrieving and displaying streaming medical data
US8627302B2 (en) * 2007-11-27 2014-01-07 Oracle America, Inc. Sampling based runtime optimizer for efficient debugging of applications
US20090138859A1 (en) * 2007-11-27 2009-05-28 Sun Microsystems, Inc. Sampling based runtime optimizer for efficient debugging of applications
US11145408B2 (en) 2009-03-04 2021-10-12 Masimo Corporation Medical communication protocol translator
US10366787B2 (en) 2009-03-04 2019-07-30 Masimo Corporation Physiological alarm threshold determination
US11158421B2 (en) 2009-03-04 2021-10-26 Masimo Corporation Physiological parameter alarm delay
US11133105B2 (en) 2009-03-04 2021-09-28 Masimo Corporation Medical monitoring system
US11087875B2 (en) 2009-03-04 2021-08-10 Masimo Corporation Medical monitoring system
US9218454B2 (en) 2009-03-04 2015-12-22 Masimo Corporation Medical monitoring system
US10325681B2 (en) 2009-03-04 2019-06-18 Masimo Corporation Physiological alarm threshold determination
US10255994B2 (en) 2009-03-04 2019-04-09 Masimo Corporation Physiological parameter alarm delay
US10032002B2 (en) 2009-03-04 2018-07-24 Masimo Corporation Medical monitoring system
US10007758B2 (en) 2009-03-04 2018-06-26 Masimo Corporation Medical monitoring system
US11923080B2 (en) 2009-03-04 2024-03-05 Masimo Corporation Medical monitoring system
US20130036405A1 (en) * 2011-08-07 2013-02-07 Guy Verbest Automated test failure troubleshooter
US8935676B2 (en) * 2011-08-07 2015-01-13 Hewlett-Packard Development Company, L.P. Automated test failure troubleshooter
US9323894B2 (en) 2011-08-19 2016-04-26 Masimo Corporation Health care sanitation monitoring system
US11816973B2 (en) 2011-08-19 2023-11-14 Masimo Corporation Health care sanitation monitoring system
US11176801B2 (en) 2011-08-19 2021-11-16 Masimo Corporation Health care sanitation monitoring system
US20130081001A1 (en) * 2011-09-23 2013-03-28 Microsoft Corporation Immediate delay tracker tool
US11824750B2 (en) * 2013-01-07 2023-11-21 Workspot, Inc. Managing information technology infrastructure based on user experience
US20220131779A1 (en) * 2013-01-07 2022-04-28 Workspot, Inc. Managing Information Technology Infrastructure Based On User Experience
US11223547B1 (en) * 2013-01-07 2022-01-11 Workspot, Inc. Managing information technology infrastructure based on user experience
WO2014208139A1 (en) * 2013-06-28 2014-12-31 日本電気株式会社 Fault detection device, control method, and program
JPWO2014208139A1 (en) * 2013-06-28 2017-02-23 日本電気株式会社 Abnormality detection apparatus, control method, and program
US20160132359A1 (en) * 2013-06-28 2016-05-12 Nec Corporation Abnormality detection apparatus, control method, and program
US10255114B2 (en) * 2013-06-28 2019-04-09 Nec Corporation Abnormality detection apparatus, control method, and program
US20150207709A1 (en) * 2014-01-21 2015-07-23 Oracle International Corporation Logging incident manager
US9742624B2 (en) * 2014-01-21 2017-08-22 Oracle International Corporation Logging incident manager
EP2990950A1 (en) * 2014-08-25 2016-03-02 Tata Consultancy Services Limited Monitoring activities of a software application
US10257312B2 (en) * 2016-10-27 2019-04-09 Entit Software Llc Performance monitor based on user engagement
EP3316142A1 (en) * 2016-10-27 2018-05-02 Entit Software LLC Performance monitor based on user engagement
CN107402864A (en) * 2017-06-07 2017-11-28 阿里巴巴集团控股有限公司 Access processing method, device and equipment, the computer-readable recording medium of duration
US11392467B2 (en) 2019-04-17 2022-07-19 Microsoft Technology Licensing, Llc Failover between decentralized identity stores
US11190512B2 (en) 2019-04-17 2021-11-30 Microsoft Technology Licensing, Llc Integrity attestation of attestation component
US11381567B2 (en) 2019-04-29 2022-07-05 Microsoft Technology Licensing, Llc Execution of an application within a scope of user-granted permission
US11429743B2 (en) 2019-04-29 2022-08-30 Microsoft Technology Licensing, Llc Localization of DID-related claims and data
US11222137B2 (en) * 2019-05-03 2022-01-11 Microsoft Technology Licensing, Llc Storing and executing an application in a user's personal storage with user granted permission
US11411959B2 (en) 2019-05-03 2022-08-09 Microsoft Technology Licensing, Llc Execution of application in a container within a scope of user-granted permission

Also Published As

Publication number Publication date
DE102005020893A1 (en) 2005-12-01

Similar Documents

Publication Publication Date Title
US20050251792A1 (en) System for adaptively determining executable application operation characteristics
US8352867B2 (en) Predictive monitoring dashboard
US20200358826A1 (en) Methods and apparatus to assess compliance of a virtual computing environment
US20210349953A1 (en) Single click delta analysis
US7457872B2 (en) On-line service/application monitoring and reporting system
US7870244B2 (en) Monitoring performance of applications in a distributed environment
US7051244B2 (en) Method and apparatus for managing incident reports
US7379999B1 (en) On-line service/application monitoring and reporting system
US7788198B2 (en) Method for detecting anomalies in server behavior using operational performance and failure mode monitoring counters
US8589859B2 (en) Collection and processing of code development information
US7363543B2 (en) Method and apparatus for generating diagnostic recommendations for enhancing process performance
US7689688B2 (en) Multiple-application transaction monitoring facility for debugging and performance tuning
US10489711B1 (en) Method and apparatus for predictive behavioral analytics for IT operations
US20030115511A1 (en) Method, apparatus and program for diagnosing system risk
US20090210360A1 (en) Computing the values of configuration parameters for optimal performance of associated applications
US7886302B2 (en) System and methods for tracking processing unit usage
US20070168696A1 (en) System for inventing computer systems and alerting users of faults
US7685475B2 (en) System and method for providing performance statistics for application components
JP6689995B2 (en) Computer system monitoring apparatus and method
US20080232266A1 (en) Network monitoring apparatus, network monitoring method and recording medium
US7908361B2 (en) Computer maintenance support system and analysis server
US20070214193A1 (en) Change monitoring program for computer resource on network
US20150019913A1 (en) Random event capturing mechanism for application systems
US20040073657A1 (en) Indirect measurement of business processes
US8032432B2 (en) System and method for tracking a billing cycle

Legal Events

Date Code Title Description
AS Assignment

Owner name: SIEMENS MEDICAL SOLUTIONS HEALTH SERVICES CORPORAT

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SMITH, DAVID WESLEY;REEL/FRAME:015658/0013

Effective date: 20050203

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION