US20100269101A1 - Event logging and performance analysis system for applications - Google Patents
Event logging and performance analysis system for applications Download PDFInfo
- Publication number
- US20100269101A1 US20100269101A1 US12/825,334 US82533410A US2010269101A1 US 20100269101 A1 US20100269101 A1 US 20100269101A1 US 82533410 A US82533410 A US 82533410A US 2010269101 A1 US2010269101 A1 US 2010269101A1
- Authority
- US
- United States
- Prior art keywords
- event
- events
- application
- logging
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000007246 mechanism Effects 0.000 claims abstract description 23
- 238000000034 method Methods 0.000 claims description 28
- 238000012545 processing Methods 0.000 claims description 10
- 230000004931 aggregating effect Effects 0.000 claims description 4
- 230000004044 response Effects 0.000 claims description 2
- 238000004590 computer program Methods 0.000 claims 2
- 230000002123 temporal effect Effects 0.000 claims 1
- 230000002776 aggregation Effects 0.000 abstract description 13
- 238000004220 aggregation Methods 0.000 abstract description 13
- 238000005259 measurement Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000006399 behavior Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 230000004069 differentiation Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- RWSOTUBLDIXVET-UHFFFAOYSA-N Dihydrogen sulfide Chemical compound S RWSOTUBLDIXVET-UHFFFAOYSA-N 0.000 description 1
- 241000510009 Varanus griseus Species 0.000 description 1
- 230000002547 anomalous effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000001771 impaired effect Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000011112 process operation Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3409—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment
- G06F11/3419—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment for performance assessment by assessing time
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation ; Recording or statistical evaluation of user activity, e.g. usability assessment
- G06F11/3466—Performance evaluation by tracing or monitoring
- G06F11/3476—Data logging
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/86—Event-based monitoring
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
- G06F2201/875—Monitoring of systems including the internet
Definitions
- the present invention relates to software applications. More specifically, the present invention relates to the performance analysis and logging of events generated in applications.
- Performance tools have been developed which measure certain events from an operating system perspective by measuring the time taken to process operations such as computations and I/O (Input/Output). Performance can also be measured within an application itself. For instance, a videoconferencing application may have a mechanism for collecting and presenting metrics regarding the number of frames per second displayed on the screen or the number of frames skipped (not displayed). Such performance analysis mechanisms are typically built into the executable code of the application. Thus, typically, applications that do not have performance measurement code already built-in, must either completely forgo much measurement or rely on a third-party measurement tool.
- An event logging and analysis mechanism which creates an event object for event of an application to be logged.
- the event logging mechanism logs into the event object the start time, end time and other information regarding the event.
- the analysis of the collected event objects may include hierarchical and contextual grouping as well as aggregation of events considered to be identical.
- the mechanism operates independent of the application whose events it logs and can be turned on and off independently. Further, in some embodiments, a user may define the levels of hierarchy and contexts upon which to analyze the event objects.
- FIG. 1 is a flowchart describing basic event logging as utilized in one or more embodiments of the invention.
- FIG. 2 is a flowchart outlining hierarchical event grouping, aggregation and statistics computation according to one or more embodiments of the invention.
- FIG. 3 is an operational system diagram of event logging and display according to one or more embodiments of the invention.
- FIG. 4 is an organizational tree diagram of grouping, subgrouping and granularity according to one or more embodiments of the invention.
- FIG. 5 illustrates an example of recursion as applied to event logging and analysis according to one or more embodiments of the invention.
- FIG. 6 is a computer system diagram that utilizes and implements one or more embodiments of the invention.
- FIG. 1 is a flowchart describing basic event logging as utilized in one or more embodiments of the invention.
- event logging is characterized by its ability to be turned on or off at any instant in time during an applications' run-time.
- One advantage of the invention is that event logging requires no special linking to libraries or recompiling and need not be present as a feature in the design and compilation of the application whose events are to be logged. The performance of the application being measured is not impaired nor does its behavior change because the event logging is transparent from the application's viewpoint.
- One feature of the event logging mechanism is the ability to define the granularity at which “events” of an application are measured and displayed.
- the event logging mechanism of the invention also performs aggregation and grouping of event statistics such that a hierarchical view as well as contextual information is presentable to the user.
- the event logging mechanism (referred to as the “event logging center”) is built within a foundational layer that is common to a class of applications. Such a common foundational layer is sometimes referred to as a “framework.”
- the framework is a common platform upon which applications may be built such that they take advantage of linkable libraries of code and standard routines that can be utilized during a run-time or at application development. Examples of a framework are the Java virtual machine and WebObjects (a product of Apple Computer Corporation). In addition to developmental frameworks, foundational layers may also be as low as the operating system itself. In this way, any application capable of running within that operating system can make use of operating system calls that perform event logging.
- event logging For each event or family of events identified, first the operating system/framework must check to see if event logging is turned on (block 110 ) in relation to an operation to be performed. To avoid any penalty to the execution of the operation that may be subjected to measure, the test of whether event logging is on or off for the operation should be a Boolean test.
- One mechanism for implementing such a Boolean test is by setting/resetting a flag whose state can be checked. There are flags created for each and every event or family (“group”) of events that the application or operating system/framework identifies during run-time. If event logging is not on, then the operation is performed without any logging or measuring (block 100 ).
- an “event object” is created (block 120 ).
- the event object is a software object or memory space designated for the log information be stored and later, accessed for analysis. According to one embodiment of invention, the event object subsists in a memory space called an “event center” wholly independent of any other applications, including the application whose operations are being logged.
- the event object is created by the framework.
- the event object marks the time index (obtainable, for instance, by returning the system clock's time information) at the start of the event (operation) being logged (block 130 ). The marking of the time index may be as simple as a read of the system clock and subsequent store in the event object.
- the operation being timed is then performed as it normally would were event logging not present (block 140 ).
- the operation itself returns no information other than whatever software mechanism is used to indicate its end (e.g. return of a function's result).
- the time index at the event's end is marked in identical manner with that of event start demarcation (block 150 ).
- event objects may encapsulate information regarding what the operation actually did from a functional standpoint.
- Each operation whether at a high grouping level (such as the entirety of a sub-routine) or at a lower level (such as a specific computation of a function as one of many within a sub-routine) can be assigned its own event object which marks its start and end time index.
- a new event object is instantiated for every occurrence of an operation, if the operation is repeated.
- a user can select points of interest prior to invoking event logging to measure only certain specific events as opposed to a brute force logging and measurement of all events. This can serve to reduce the amount of memory/storage space utilized by the logging process and allows results displayed to the user at a later stage to be filtered of non-pertinent or non-critical event statistics.
- event object based logging aids in developing a hierarchy of event statistics, grouping of events and contextual differentiation. The structural and procedural nature of most code can be exposed in this manner and statistics can be used to monitor the execution efficiency program flow, rather than just measures of input/output timing.
- vents but should be understood to mean “event objects” which result from the logging of actual events that have occurred in an application. Once event logging is turned off, only “event objects” remain to represent the actual events which occurred with the application, and thus, “event objects” and “events” are viewed as one in the same.
- FIG. 2 is a flowchart outlining hierarchical event grouping, aggregation and statistics computation according to one or more embodiments of the invention.
- Hierarchical event grouping refers to a differentiation by structural or procedural level within the application's execution of events that take place.
- Aggregation refers to the collecting of identical events (events that are repeated) into one event via averaging and other such means.
- Contextual event grouping refers to the grouping/aggregating of similar events considered identical by virtue of their context.
- a user selects the granularity (noted by a variable “grain” in FIG. 2 ) referring to the lowest structural level of an application's code or event space that is to be aggregated (block 210 ). Further, the user can select/define contexts whose events are to be considered as identical for purpose of gathering the statistics.
- an event log is completed for a given run-time instance (i.e. when events between the start and finish of an event logging session have been logged)
- all of the events are grouped or arranged according to their functional level within the code of the application or according to user selected grouping (block 220 ).
- the grouping of events is enabled by the operational information provided by the application during event logging.
- the grouping of events in a hierarchical sense can take advantage of the “call stack”, which is a operating system ordering of executable instructions resulting from the execution of application code.
- the call stack advantageously, is by nature already organized in rough order of program flow.
- a developmental framework may also contain such clues as to the possible grouping of events.
- Such hints and call stacks allow the event logging center to determine a default grouping that may serve as a starting point or substitute for user customization.
- the grouping of events puts all events into a structural framework that starts from a coarse “granularity” to a finer and finer level of granularity and by nature, may be recursive/loop driven.
- Each group of events, at every level of granularity, will be referred to for the sake of simplicity as a “subgroup” to indicate that that it is potentially a logical subset of yet another group of events.
- a loop ensues (from block 230 ) that runs through each subgroup of event objects and recursively proceeds to a finer grain subgroup until the desired level of granularity is reached.
- the subgroup level being processed is less than the variable “grain” (granularity level) (checked at block 240 )
- a first set of processes shown in blocks 250 through 270 are performed.
- each subgroup the events therein are further divided into a finer and finer granular level of subgrouping (block 250 ).
- the statistics for that subgroup are computed (block 260 ). These statistics on the level of the subgroup are then displayed (block 290 ) (see description below).
- the display of statistics according to block 290 does not break out of the loop ensued at block 240 , but is performed in parallel with it. Recursively, for each further divided subgroup (as divided in block 250 ) flow returns to block 230 which traverses through the events of the newly created subgroups.
- each “aggregate” is a collection of events that are identical and considered to be single event.
- Statistics for each aggregate may be computed (block 285 ) by computing the duration for each individual event and combining them through averaging and by maintaining any other desired statistics such as modal or deviation information. These statistics are then displayed to the user (block 290 ).
- the display of statistics (block 290 ) may be arranged in a hierarchical manner as well, showing the user the grouping of events and structural flow of the application.
- subgroup level statistics are computed during the recursive grouping process and thus do not have to be recomputed (block 260 ).
- FIG. 3 is an operational system diagram of event logging and display according to one or more embodiments of the invention.
- FIG. 3 shows an exemplary database application 320 where queries and updates (transactions) are performed by a user upon a database 350 .
- Application 320 executes or is built on top of a operating system/framework space 300 and connects to the database 350 to perform transactions thereon.
- an event logging center 310 is provided as part of the operating system/framework 300 .
- Event logging center 310 occupies a space in memory and executes under the operating system/framework 300 but is separate and independent of application 320 .
- Other layer of abstraction The event logging center 310 , may be turned on and off by one of several means.
- the application 320 may have a provision for turning on event logging.
- event logging may be turned on and off by a local web browser which also subsists in operating system/framework 300 or a remote web browser 340 which may not be part of the operating system/framework 300 at all, but merely sends and receives data with applications within it.
- a local web browser which also subsists in operating system/framework 300 or a remote web browser 340 which may not be part of the operating system/framework 300 at all, but merely sends and receives data with applications within it.
- There may be other means of turning on and off event logging such as starting event logging from within the auspices of the event logging center 310 .
- event logging proceeds for each event by the application 320 initiating a call to create an “event object” 315 in the event logging center 310 which is a reference to the event's start time, end time, and operational information (what the event does). For each event, a process similar to that shown with respect to FIG. 1 above is performed.
- the event object 315 subsists in the event logging center 310 which handles its memory allocation and storage.
- the application 320 sends its start time to the event logging center 310 .
- the event logging center 310 then records this start time into event object 315 .
- application 320 sends its ending time index to the event logging center 310 .
- the event logging center 310 When event logging is turned off, the event logging center 310 performs hierarchical event grouping, aggregation and contextual grouping, as described above with respect to FIG. 2 . Any event objects that are incomplete (do not have ending time information) at the termination of event logging are destroyed and not counted.
- statistics are computed at each aggregate or subgroup, they are displayed to the user. These results may be displayed on a graphical user interface such as that provided by local web browser 330 or remote web browser 340 .
- the display of results may be in a drill-down or “threaded” fashion so that groupings and subgroupings can be easily visualized. As well, the display can show event details individually.
- both the local web browser 330 and remote web browser 340 have identical displays of event statistics and cannot configure their own displays based upon the raw statistics of the event logging.
- each of the local web browser 330 and remote web browser 340 may specify the grouping of events that they desire communicating this to the event logging center 310 . In this way, each user may alert the event logging center to their own preferences and see statistics in different contextual or hierarchically granular groupings on their respective displays.
- the configuration of event grouping and contexts may occur prior, during or after event logging. Control and display of results via a web browser in one embodiment is advantageous in that the event logging can be cross platform.
- a database query for a customer list of customers who have outstanding orders For Several levels of subgrouping could be defined. For instance, the operation can be contextually considered from the start of the query to the display of the data from the database.
- a list of customers involves individual fetches (one for each customer) form database 350 . Each of these fetches may be considered an “event” whose timing is recorded in event objects such as event object 315 . They are grouped under the coarser level subgroup of the total database fetch. The coarsest level under the query event subgroup consists of the subgroups: query processing, database fetch and then display of results. Each of these subgroups (such as the individual fetches within the database fetch main group) may have further subgroupings and so on.
- FIG. 4 is an organizational tree diagram of subgrouping and granularity according to one or more embodiments of the invention.
- the diagram of FIG. 4 represents an exemplary event hierarchy of an application.
- the application as a whole is considered an event that has its own start and end. For example, if the application is a web browser, then the web browser, once started may generate a number of pages. The generation of each page is also an event and the group of all page generation events are thus considered a “subgroup” of the application event.
- This subgroup of page generation events are shown as subgroup level 420 , which is finer in granularity than level 410 but coarser than its children. Note that there is no particular chronology of events represented in FIG. 4 , but rather, a hierarchical organization.
- Subgroup level 420 three page events, page 1 , page 2 and page 3 . These pages may be simultaneously generated (on simultaneous execution threads) or generated one after another, but in any case, are hierarchically related in the same manner as that shown.
- the next subgroup level 430 shows a number of component events-component A, component B, component C, component D, component E and component F-which belong variously as children of the page events belonging to subgroup level 420 .
- the event logging mechanism does not necessarily follow assumptions about events since different instance of events within the same subgroup may have a different behavior.
- Page 1 of subgroup level 420 has only member belonging to subgroup level 430 , namely, component A, while page 2 has three members in subgroup level 430 -components B, C and D-and page 3 has two members in subgroup level 430 -components E and F.
- Subgroup level 440 has children of components in level 430 which are of an even finer granularity.
- Component F of subgroup level 430 has no children below it, and thus no members belonging to subgroup level 440 .
- the vent logging center is notified of this behavior during the logged application's runtime and can adjust its analysis of the log accordingly. This allows the event logging center to dynamically arrive at the hierarchical grouping of the events logged, rather than just guessing.
- the process of subgrouping extends until the finest granularity of subgrouping is reached.
- the user of the event logging may specify how granular the event statistics and log should be. In the example shown in FIG. 4 the user desired only five levels of subgrouping down to level 450 .
- the finest granularity subgrouping level 450 is a subgroup of level 440 which is a subgroup of level 430 and so on.
- FIG. 4 some subgroup members have one child, multiple children or no children at all.
- the hierarchy is maintained regardless of whether only one child or multiple children persist for a given member of a subgroup.
- the event logging statistics can be arranged in a similar fashion to that shown in FIG. 4 but in a drill down fashion. For instance, clicking on the statistics for the page 2 event would yield statistics for component B, component C and component D and so on.
- FIG. 4 primarily depicts a hierarchical grouping, it could also be viewed of as a contextual grouping. For instance, each of the pages, page 1 , page 2 and page 3 , could be different types of pages generated: One could be an information page, the other a form page for inputting and so on.
- Each of these page events could be considered to be in different contexts and thus, forgo/supplement the aggregation of statistics that might normally occur given merely the hierarchy. This added level of differentiation may be user or application specified and can provide a different statistical view of the events of the application.
- FIG. 5 illustrates an example of recursion as applied to event logging and analysis according to one or more embodiments of the invention.
- FIG. 5 shows an exemplary mapping of events in a database query application.
- a root event or subgroup 510 at the coarsest level is one that for example is the initialize or start-up of the application itself.
- Event 510 is a “root” event since it has no “parent” event (i.e. prior to event 510 , other application events could not commence).
- the graph of FIG. 5 shows events occurring sequentially down the width of the page, and “child” or nested events to the right of their respective parent. After root event 510 ends, event generate page 520 commences.
- Event generate page 520 cannot end until its subgroup of events also end. However, this is a necessary but not sufficient condition, since the end of the subgroups under event 520 does not necessarily mark the end of event 520 .
- initialize page event 530 must be started (indicated by the arrow from event 520 to event 530 ) and end (indicated by the curved arrow proceeding back from event 530 to event 520 ).
- event generate component 540 occurs chronologically following the end of event 530 .
- Generate component 540 is not complete until it's nested or child event, the event “database fetch to initialize component” 550 is complete.
- Events 530 and 540 are at the same level of granularity.
- Event 550 is at a finest level of granularity since it has no children of its own. Generate component 540 is complete once event 550 returns its result, and this will trigger completion of the event subgroup of event 520 composed of event initialize page 530 and event generate component 540 .
- event flow proceeds chronologically to “get request” 560 .
- get request 560 is complete another event “generate reply for request” 570 is performed.
- Event logging for the events of FIG. 5 could be displayed and would be logged hierarchically in this manner:
- start 510 (A) perform 510 end 510 (B) start 520 (C) start 530 (D) perform 530 end 530 (E) start 540 (F) start 550 (G) perform 550 end 550 (H) end 540 (I) end 520 (J) start 560 (K) perform 560 end 560 (L) start 570 (M) perform 570 end 570 (N)
- the numbers refer to the event numbers therein and the letters in parenthesis are time indexes for each start and end.
- the duration of event such as event 510 would be computed as “B-A” for example.
- the duration of event 520 could be computed as “J-C” rather than computing it as the sum of durations of all nested or recursed subgroups within it.
- Event 530 has a duration “E-D”
- event 540 has “F-I”
- event 550 has “H-G”.
- event 560 has a duration “L-K” while event 570 has a duration of “N-M”.
- durations of events may be performed after event logging has been turned off. This allows the operating system/framework to have little, if any, additional load which may skew the results of the logging by introducing non-application timing delays (i.e. the time taken to compute durations, etc.). Likewise, statistics upon the durations such as averages, graphs, etc. may also be prepared after event logging has been turned off.
- the events of FIG. 5 are shown as uniquely occurring and not repeating. However, any one or more of the events depicted may be repeated or may recur many times. For instance, many pages of database results may be generated which would imply some repetition of event 520 and its children. In this case, the aggregation of the present invention would take the durations for each occurrence of event 520 and consider them a single event (i.e. an aggregate). Assume that event 520 is repeated with a start time X and end time Y. Then, in the aggregate, event generate page has a total time taken of “J-C+Y-X” and has been performed twice. Knowing the count and total duration, an average can be computed and presented to the user. Since the log contains all of the raw data, a user may peer into the details of the two instances of event 520 to see if anything is anomalous.
- event initialize application 510 refers the start-up of web browser application.
- the application has “initialized” (i.e. been allocated memory to occupy, had its instructions loaded therein, etc.)
- the next event is generate page 520 .
- Event generate page 520 must start with its own initialization.
- event initialize page 530 may equate to allocating memory and screen area to the window, but as shown in FIG. 5 , must also include content that can fill the page.
- event generate component 540 is the next sequentially occurring event.
- Event 540 could be analogized to generating the drop down lists, toolbars, tables, menus etc. that often occupy a web page (i.e. its user interface elements).
- a necessary and sufficient condition for the completion of generate component 540 is event database fetch 550 which would correspond to filling the component with the desired data (e.g. list of categories for the toolbar).
- event generate page 520 has reached its end.
- event get request 560 commences. If the user is presented with input choices from the generated page, then get request 560 would correspond to the application getting a request (such as a search query) from the user based on what the generated page (event 520 ) has made available. After the request has been obtained (i.e. event 560 has ended) a reply to that request may be generated (event 570 ).
- Event logging takes account of the hierarchy of events as shown in FIG. 5 , but also allows contextual grouping and statistics gathering. In this descriptive view of FIG.
- the non-contextual hierarchical count of all eight (book and music) events 570 could be maintained in addition to the contextual count of five events and three events as being separate.
- FIG. 6 is a computer system diagram that utilizes and implements one or more embodiments of the invention.
- Computer system 610 Illustrated is a computer system 610 , which may be any general or special purpose computing or data processing machine such as a PC (personal computer).
- Computer system 610 is coupled to a network 600 which may connect to one or more networks, such as a Local Area Network or the Internet, enabling computer system 610 to transact information with other computer systems or information devices.
- networks such as a Local Area Network or the Internet
- One of ordinary skill in the art may program computer system 610 to allow event logging that is capable of hierarchical and contextual event grouping as well as aggregation in a user customizable manner. As described above, event logging may be provided for in the operating system or framework that is common to applications executing within computer system 610 .
- Computer system 610 has a processor 612 and a memory 611 , such as RAM, which is used to store/load instructions, addresses and result data as desired.
- a memory 611 such as RAM, which is used to store/load instructions, addresses and result data as desired.
- the instructions pertaining to implementing the logging are loaded into memory 611 , executed by processor 612 and then given a memory space within 611 to execute.
- the implementation of the event logging center and associated functionality may derive from a low level source code such as assembler, a machine language or a higher level language or derivative thereof that has been embedded with the operating system/framework of the computer system 610 .
- the instructions corresponding to the event logging center as well as the log of event objects and its associated information may be stored to any combination of a disk 618 , such as a hard drive, or to memory 611 .
- Computer system 610 has a system bus 613 which facilitates information transfer to/from the processor 612 and memory 611 and a bridge 614 which couples to an I/O bus 615 .
- I/O bus 615 connects various I/O devices such as a network interface card (NIC) 616 , disk 618 and to the system memory 611 and processor 612 .
- NIC network interface card
- the NIC 616 allows the event logging results to be displayed to users connected to network 600 . For instance, referring back to FIG. 3 , the remote web browser 340 may access the event logging center and display the results therefrom via network 600 .
- event logging is completed and statistics thereafter computed, they can be displayed to a user through a web browser which may be displayed onto a display device (e.g. monitor) such as display 620 .
- a display device e.g. monitor
- the user can input their grouping preferences to the event logging center through the web browser running on the computer system 610 and displayed via display 620 .
- the NIC 616 is used to receive that remote user's grouping preferences and send event center results via network 600 .
Abstract
An event logging and analysis mechanism which creates an event object for event of an application to be logged. The event logging mechanism logs into the event object the start time, end time and other information regarding the event. The analysis of the collected event objects may include hierarchical and contextual grouping as well as aggregation of events considered to be identical. The mechanism operates independent of the application whose events it logs and can be turned on and off independently. A user may define the levels of hierarchy and contexts upon which to analyze the event objects.
Description
- 1. Field of the Invention
- The present invention relates to software applications. More specifically, the present invention relates to the performance analysis and logging of events generated in applications.
- 2. Description of Related Art
- One operation in fine-tuning and troubleshooting of information systems, such as a computer application, is to measure the “performance” (time taken) to deliver a requested result from that system. Performance tools have been developed which measure certain events from an operating system perspective by measuring the time taken to process operations such as computations and I/O (Input/Output). Performance can also be measured within an application itself. For instance, a videoconferencing application may have a mechanism for collecting and presenting metrics regarding the number of frames per second displayed on the screen or the number of frames skipped (not displayed). Such performance analysis mechanisms are typically built into the executable code of the application. Thus, typically, applications that do not have performance measurement code already built-in, must either completely forgo much measurement or rely on a third-party measurement tool. Due to the peculiarities between the way in which different software applications execute, generic measurement tools lack usefulness. If not specifically prepared for the application much of the information is geared toward sampling at given intervals what the application is doing. This does not necessarily give information that the user may be looking for or is interested in. Typically, the user must wade through piles of useless or uninteresting data regarding performance and may thus be unable to pick out those metrics of greatest importance.
- All performance tools must preferably be operating as close, in a system or network sense, to the measured entity to eliminate the added measurement noise of latency which may skew results. The latency in delivering data over a network between clients and server completely renders useless the measurement of the operations themselves. In a client-server environment, there is often no separable measurement of client from server and server from client in regard to operations. For instance, in database applications where a server is being accessed and queried or updated by a client, timing of server operations needs to be isolated from the latency of network transfer. Likewise, timing of client operations may also need to be isolated from the very same latency.
- What is often measured is only the time to gather the data from the server, and often, this is performed on the server side. Often, when preferred on the client side, this puts additional processing load on the client which can in turn degrade the accuracy of the measurement. When such measurements deal only with data collection time (request and response) rather than also the display of results, they are burdensome to the client and from the standpoint of the user, irrelevant.
- There is a need for a performance measurement tool that can work interoperably with third-party applications, which is application and platform independent, and can be quickly and easily initiated when necessary and shut down when not. There is also a need for a performance analysis system that allows measurement of selective operations of an application.
- An event logging and analysis mechanism is provided which creates an event object for event of an application to be logged. The event logging mechanism logs into the event object the start time, end time and other information regarding the event. The analysis of the collected event objects may include hierarchical and contextual grouping as well as aggregation of events considered to be identical. The mechanism operates independent of the application whose events it logs and can be turned on and off independently. Further, in some embodiments, a user may define the levels of hierarchy and contexts upon which to analyze the event objects.
-
FIG. 1 is a flowchart describing basic event logging as utilized in one or more embodiments of the invention. -
FIG. 2 is a flowchart outlining hierarchical event grouping, aggregation and statistics computation according to one or more embodiments of the invention. -
FIG. 3 is an operational system diagram of event logging and display according to one or more embodiments of the invention. -
FIG. 4 is an organizational tree diagram of grouping, subgrouping and granularity according to one or more embodiments of the invention. -
FIG. 5 illustrates an example of recursion as applied to event logging and analysis according to one or more embodiments of the invention. -
FIG. 6 is a computer system diagram that utilizes and implements one or more embodiments of the invention. -
FIG. 1 is a flowchart describing basic event logging as utilized in one or more embodiments of the invention. - In accordance with the invention, event logging is characterized by its ability to be turned on or off at any instant in time during an applications' run-time. One advantage of the invention is that event logging requires no special linking to libraries or recompiling and need not be present as a feature in the design and compilation of the application whose events are to be logged. The performance of the application being measured is not impaired nor does its behavior change because the event logging is transparent from the application's viewpoint. One feature of the event logging mechanism is the ability to define the granularity at which “events” of an application are measured and displayed. The event logging mechanism of the invention also performs aggregation and grouping of event statistics such that a hierarchical view as well as contextual information is presentable to the user.
- The event logging mechanism (referred to as the “event logging center”) is built within a foundational layer that is common to a class of applications. Such a common foundational layer is sometimes referred to as a “framework.” The framework is a common platform upon which applications may be built such that they take advantage of linkable libraries of code and standard routines that can be utilized during a run-time or at application development. Examples of a framework are the Java virtual machine and WebObjects (a product of Apple Computer Corporation). In addition to developmental frameworks, foundational layers may also be as low as the operating system itself. In this way, any application capable of running within that operating system can make use of operating system calls that perform event logging. Thus, the following description will refer variously to “operating system/framework” to indicate the lowest level of abstraction, common to all applications, that embeds the event logging center and its functionality. Further, the words “operation” and “event” will be used interchangeably to mean a piece of code that executes to perform some function during the flow of an application's run-time.
- To achieve event logging, for each event or family of events identified, first the operating system/framework must check to see if event logging is turned on (block 110) in relation to an operation to be performed. To avoid any penalty to the execution of the operation that may be subjected to measure, the test of whether event logging is on or off for the operation should be a Boolean test. One mechanism for implementing such a Boolean test is by setting/resetting a flag whose state can be checked. There are flags created for each and every event or family (“group”) of events that the application or operating system/framework identifies during run-time. If event logging is not on, then the operation is performed without any logging or measuring (block 100).
- If event logging is turned on, then an “event object” is created (block 120). The event object is a software object or memory space designated for the log information be stored and later, accessed for analysis. According to one embodiment of invention, the event object subsists in a memory space called an “event center” wholly independent of any other applications, including the application whose operations are being logged. The event object is created by the framework. The event object marks the time index (obtainable, for instance, by returning the system clock's time information) at the start of the event (operation) being logged (block 130). The marking of the time index may be as simple as a read of the system clock and subsequent store in the event object. The operation being timed is then performed as it normally would were event logging not present (block 140). The operation itself returns no information other than whatever software mechanism is used to indicate its end (e.g. return of a function's result). When the operation is completed, the time index at the event's end is marked in identical manner with that of event start demarcation (block 150). In addition to start and end times, event objects may encapsulate information regarding what the operation actually did from a functional standpoint.
- While the application is running and while event logging is still on, no specific analysis of the data present in the event object is performed. The process of marking start and end time indices according to
FIG. 1 can be repeated for any objects at any level or context as are desired. Each operation (event) whether at a high grouping level (such as the entirety of a sub-routine) or at a lower level (such as a specific computation of a function as one of many within a sub-routine) can be assigned its own event object which marks its start and end time index. A new event object is instantiated for every occurrence of an operation, if the operation is repeated. Further, a user can select points of interest prior to invoking event logging to measure only certain specific events as opposed to a brute force logging and measurement of all events. This can serve to reduce the amount of memory/storage space utilized by the logging process and allows results displayed to the user at a later stage to be filtered of non-pertinent or non-critical event statistics. Further, as demonstrated below, event object based logging aids in developing a hierarchy of event statistics, grouping of events and contextual differentiation. The structural and procedural nature of most code can be exposed in this manner and statistics can be used to monitor the execution efficiency program flow, rather than just measures of input/output timing. - The following description refers to “events” but should be understood to mean “event objects” which result from the logging of actual events that have occurred in an application. Once event logging is turned off, only “event objects” remain to represent the actual events which occurred with the application, and thus, “event objects” and “events” are viewed as one in the same.
-
FIG. 2 is a flowchart outlining hierarchical event grouping, aggregation and statistics computation according to one or more embodiments of the invention. - The invention in various embodiments presents two inter-related features, namely, hierarchical event grouping, aggregation and contextual event grouping. “Hierarchical event grouping” refers to a differentiation by structural or procedural level within the application's execution of events that take place. “Aggregation” refers to the collecting of identical events (events that are repeated) into one event via averaging and other such means. “Contextual event grouping” refers to the grouping/aggregating of similar events considered identical by virtue of their context. Each of these features may be pre-configured into the event logging or customized by the user at the time event logging is invoked or as in
FIG. 2 may be configured or reconfigured when event statistics are viewed which may be after event logging has been turned off. - In
FIG. 2 , a user selects the granularity (noted by a variable “grain” inFIG. 2 ) referring to the lowest structural level of an application's code or event space that is to be aggregated (block 210). Further, the user can select/define contexts whose events are to be considered as identical for purpose of gathering the statistics. After an event log is completed for a given run-time instance (i.e. when events between the start and finish of an event logging session have been logged), then all of the events are grouped or arranged according to their functional level within the code of the application or according to user selected grouping (block 220). The grouping of events is enabled by the operational information provided by the application during event logging. The grouping of events in a hierarchical sense can take advantage of the “call stack”, which is a operating system ordering of executable instructions resulting from the execution of application code. The call stack, advantageously, is by nature already organized in rough order of program flow. A developmental framework may also contain such clues as to the possible grouping of events. Such hints and call stacks allow the event logging center to determine a default grouping that may serve as a starting point or substitute for user customization. - The grouping of events, once completed, puts all events into a structural framework that starts from a coarse “granularity” to a finer and finer level of granularity and by nature, may be recursive/loop driven. Each group of events, at every level of granularity, will be referred to for the sake of simplicity as a “subgroup” to indicate that that it is potentially a logical subset of yet another group of events. Accordingly, once events are grouped from the event center a loop ensues (from block 230) that runs through each subgroup of event objects and recursively proceeds to a finer grain subgroup until the desired level of granularity is reached. As long as the subgroup level being processed is less than the variable “grain” (granularity level) (checked at block 240), a first set of processes shown in
blocks 250 through 270 are performed. - Within each subgroup, the events therein are further divided into a finer and finer granular level of subgrouping (block 250). Within the currently processed subgroup (from which further subgroups were extracted in block 250), the statistics for that subgroup are computed (block 260). These statistics on the level of the subgroup are then displayed (block 290) (see description below). The display of statistics according to block 290 does not break out of the loop ensued at
block 240, but is performed in parallel with it. Recursively, for each further divided subgroup (as divided in block 250) flow returns to block 230 which traverses through the events of the newly created subgroups. This process is recursive and blocks 250 through 270 are repeated and recursively so, until the (sub)group level is less than the granularity (represented by “grain”) (checked at block 240). This indicates that the desired lowest level of grouping has been achieved and that the hierarchy of event grouping is completed. The event log would at this point have been traversed completely and event objects therein hierarchically organized (for example, seeFIG. 4 and description below). - As such, all of the events that belong to a particular subgroup are aggregated and this aggregation is performed for each subgroup that has been defined by the recursion into the event log. Each “aggregate” is a collection of events that are identical and considered to be single event. Statistics for each aggregate may be computed (block 285) by computing the duration for each individual event and combining them through averaging and by maintaining any other desired statistics such as modal or deviation information. These statistics are then displayed to the user (block 290). The display of statistics (block 290) may be arranged in a hierarchical manner as well, showing the user the grouping of events and structural flow of the application. Advantageously, subgroup level statistics are computed during the recursive grouping process and thus do not have to be recomputed (block 260).
-
FIG. 3 is an operational system diagram of event logging and display according to one or more embodiments of the invention. -
FIG. 3 shows anexemplary database application 320 where queries and updates (transactions) are performed by a user upon adatabase 350.Application 320 executes or is built on top of a operating system/framework space 300 and connects to thedatabase 350 to perform transactions thereon. According to the invention, anevent logging center 310 is provided as part of the operating system/framework 300.Event logging center 310 occupies a space in memory and executes under the operating system/framework 300 but is separate and independent ofapplication 320. Other layer of abstraction. Theevent logging center 310, may be turned on and off by one of several means. Theapplication 320 may have a provision for turning on event logging. If not, event logging may be turned on and off by a local web browser which also subsists in operating system/framework 300 or aremote web browser 340 which may not be part of the operating system/framework 300 at all, but merely sends and receives data with applications within it. There may be other means of turning on and off event logging such as starting event logging from within the auspices of theevent logging center 310. - Once invoked, event logging proceeds for each event by the
application 320 initiating a call to create an “event object” 315 in theevent logging center 310 which is a reference to the event's start time, end time, and operational information (what the event does). For each event, a process similar to that shown with respect toFIG. 1 above is performed. Theevent object 315 subsists in theevent logging center 310 which handles its memory allocation and storage. Once theevent object 315 is created for a given event, theapplication 320 sends its start time to theevent logging center 310. Theevent logging center 310 then records this start time intoevent object 315. When the event is complete,application 320 sends its ending time index to theevent logging center 310. This process repeats over and over again for each event that is to be logged and continues until event logging is turned off. For each event subgroup, a Boolean flag is assigned and checked, which is independent of other flags for other events.Application 320 may inform theevent logging center 310 of special events or subgroups in addition to default clues provided by call stacks and other mechanisms that are within operating system/framework 300. - When event logging is turned off, the
event logging center 310 performs hierarchical event grouping, aggregation and contextual grouping, as described above with respect toFIG. 2 . Any event objects that are incomplete (do not have ending time information) at the termination of event logging are destroyed and not counted. When statistics are computed at each aggregate or subgroup, they are displayed to the user. These results may be displayed on a graphical user interface such as that provided bylocal web browser 330 orremote web browser 340. The display of results may be in a drill-down or “threaded” fashion so that groupings and subgroupings can be easily visualized. As well, the display can show event details individually. - In one embodiment of the invention, both the
local web browser 330 andremote web browser 340 have identical displays of event statistics and cannot configure their own displays based upon the raw statistics of the event logging. In yet another embodiment, each of thelocal web browser 330 andremote web browser 340 may specify the grouping of events that they desire communicating this to theevent logging center 310. In this way, each user may alert the event logging center to their own preferences and see statistics in different contextual or hierarchically granular groupings on their respective displays. The configuration of event grouping and contexts may occur prior, during or after event logging. Control and display of results via a web browser in one embodiment is advantageous in that the event logging can be cross platform. - As an example, consider a database query for a customer list of customers who have outstanding orders. Several levels of subgrouping could be defined. For instance, the operation can be contextually considered from the start of the query to the display of the data from the database. Within the display, a list of customers involves individual fetches (one for each customer)
form database 350. Each of these fetches may be considered an “event” whose timing is recorded in event objects such asevent object 315. They are grouped under the coarser level subgroup of the total database fetch. The coarsest level under the query event subgroup consists of the subgroups: query processing, database fetch and then display of results. Each of these subgroups (such as the individual fetches within the database fetch main group) may have further subgroupings and so on. -
FIG. 4 is an organizational tree diagram of subgrouping and granularity according to one or more embodiments of the invention. - The diagram of
FIG. 4 represents an exemplary event hierarchy of an application. At thecoarsest level 410 of granularity, is the application itself. The application as a whole is considered an event that has its own start and end. For example, if the application is a web browser, then the web browser, once started may generate a number of pages. The generation of each page is also an event and the group of all page generation events are thus considered a “subgroup” of the application event. This subgroup of page generation events are shown assubgroup level 420, which is finer in granularity thanlevel 410 but coarser than its children. Note that there is no particular chronology of events represented inFIG. 4 , but rather, a hierarchical organization. -
Subgroup level 420 three page events,page 1,page 2 andpage 3. These pages may be simultaneously generated (on simultaneous execution threads) or generated one after another, but in any case, are hierarchically related in the same manner as that shown. Thenext subgroup level 430 shows a number of component events-component A, component B, component C, component D, component E and component F-which belong variously as children of the page events belonging tosubgroup level 420. The event logging mechanism does not necessarily follow assumptions about events since different instance of events within the same subgroup may have a different behavior.Page 1 ofsubgroup level 420 has only member belonging tosubgroup level 430, namely, component A, whilepage 2 has three members in subgroup level 430-components B, C and D-andpage 3 has two members in subgroup level 430-components E andF. Subgroup level 440 has children of components inlevel 430 which are of an even finer granularity. Component F ofsubgroup level 430 has no children below it, and thus no members belonging tosubgroup level 440. The vent logging center is notified of this behavior during the logged application's runtime and can adjust its analysis of the log accordingly. This allows the event logging center to dynamically arrive at the hierarchical grouping of the events logged, rather than just guessing. The process of subgrouping extends until the finest granularity of subgrouping is reached. As described above, the user of the event logging may specify how granular the event statistics and log should be. In the example shown inFIG. 4 the user desired only five levels of subgrouping down tolevel 450. The finestgranularity subgrouping level 450 is a subgroup oflevel 440 which is a subgroup oflevel 430 and so on. - As depicted in
FIG. 4 , some subgroup members have one child, multiple children or no children at all. The hierarchy is maintained regardless of whether only one child or multiple children persist for a given member of a subgroup. The event logging statistics can be arranged in a similar fashion to that shown inFIG. 4 but in a drill down fashion. For instance, clicking on the statistics for thepage 2 event would yield statistics for component B, component C and component D and so on. WhileFIG. 4 primarily depicts a hierarchical grouping, it could also be viewed of as a contextual grouping. For instance, each of the pages,page 1,page 2 andpage 3, could be different types of pages generated: One could be an information page, the other a form page for inputting and so on. Each of these page events could be considered to be in different contexts and thus, forgo/supplement the aggregation of statistics that might normally occur given merely the hierarchy. This added level of differentiation may be user or application specified and can provide a different statistical view of the events of the application. -
FIG. 5 illustrates an example of recursion as applied to event logging and analysis according to one or more embodiments of the invention. -
FIG. 5 shows an exemplary mapping of events in a database query application. A root event orsubgroup 510 at the coarsest level is one that for example is the initialize or start-up of the application itself.Event 510 is a “root” event since it has no “parent” event (i.e. prior toevent 510, other application events could not commence). The graph ofFIG. 5 shows events occurring sequentially down the width of the page, and “child” or nested events to the right of their respective parent. Afterroot event 510 ends, event generatepage 520 commences. - Event generate
page 520 cannot end until its subgroup of events also end. However, this is a necessary but not sufficient condition, since the end of the subgroups underevent 520 does not necessarily mark the end ofevent 520. For instance, for event generatepage 520 to end, first, initializepage event 530 must be started (indicated by the arrow fromevent 520 to event 530) and end (indicated by the curved arrow proceeding back fromevent 530 to event 520). As indicated byFIG. 5 event generatecomponent 540 occurs chronologically following the end ofevent 530. Generatecomponent 540 is not complete until it's nested or child event, the event “database fetch to initialize component” 550 is complete.Events Event 550 is at a finest level of granularity since it has no children of its own. Generatecomponent 540 is complete onceevent 550 returns its result, and this will trigger completion of the event subgroup ofevent 520 composed ofevent initialize page 530 and event generatecomponent 540. - After generate
page 520 is complete (and thus, all subgroup and nestedevents request 560 is complete another event “generate reply for request” 570 is performed. - Event logging for the events of
FIG. 5 could be displayed and would be logged hierarchically in this manner: -
start 510 (A) perform 510 end 510 (B) start 520 (C) start 530 (D) perform 530 end 530 (E) start 540 (F) start 550 (G) perform 550 end 550 (H) end 540 (I) end 520 (J) start 560 (K) perform 560 end 560 (L) start 570 (M) perform 570 end 570 (N) - In the above exemplary event log for the events of
FIG. 5 the numbers refer to the event numbers therein and the letters in parenthesis are time indexes for each start and end. The duration of event such asevent 510 would be computed as “B-A” for example. The duration ofevent 520 could be computed as “J-C” rather than computing it as the sum of durations of all nested or recursed subgroups within it.Event 530 has a duration “E-D”,event 540 has “F-I” andevent 550 has “H-G”. Likewise,event 560 has a duration “L-K” whileevent 570 has a duration of “N-M”. In this regard, it should be noted that the computation of durations of events may be performed after event logging has been turned off. This allows the operating system/framework to have little, if any, additional load which may skew the results of the logging by introducing non-application timing delays (i.e. the time taken to compute durations, etc.). Likewise, statistics upon the durations such as averages, graphs, etc. may also be prepared after event logging has been turned off. - The events of
FIG. 5 are shown as uniquely occurring and not repeating. However, any one or more of the events depicted may be repeated or may recur many times. For instance, many pages of database results may be generated which would imply some repetition ofevent 520 and its children. In this case, the aggregation of the present invention would take the durations for each occurrence ofevent 520 and consider them a single event (i.e. an aggregate). Assume thatevent 520 is repeated with a start time X and end time Y. Then, in the aggregate, event generate page has a total time taken of “J-C+Y-X” and has been performed twice. Knowing the count and total duration, an average can be computed and presented to the user. Since the log contains all of the raw data, a user may peer into the details of the two instances ofevent 520 to see if anything is anomalous. - The events of
FIG. 5 may be better understood when translated from its abstract event description to a real-world scenario. Assume, for example, that event initializeapplication 510 refers the start-up of web browser application. After the application has “initialized” (i.e. been allocated memory to occupy, had its instructions loaded therein, etc.), the next event is generatepage 520. Event generatepage 520 must start with its own initialization. Thus, event initializepage 530 may equate to allocating memory and screen area to the window, but as shown inFIG. 5 , must also include content that can fill the page. After the page is initialized (event 530), then the next sequentially occurring event is event generatecomponent 540. -
Event 540 could be analogized to generating the drop down lists, toolbars, tables, menus etc. that often occupy a web page (i.e. its user interface elements). A necessary and sufficient condition for the completion of generatecomponent 540 is event database fetch 550 which would correspond to filling the component with the desired data (e.g. list of categories for the toolbar). - Once the web page is generated (i.e. filled with data and user interface elements, but also, displayed to the user), then event generate
page 520 has reached its end. After the end ofevent 520, event getrequest 560 commences. If the user is presented with input choices from the generated page, then getrequest 560 would correspond to the application getting a request (such as a search query) from the user based on what the generated page (event 520) has made available. After the request has been obtained (i.e.event 560 has ended) a reply to that request may be generated (event 570). Event logging takes account of the hierarchy of events as shown inFIG. 5 , but also allows contextual grouping and statistics gathering. In this descriptive view ofFIG. 5 , assume that the events getrequest 560 and generate reply forrequest 570 were repeated many times for different types of data items. For instance, if one request dealt with the database look-up of a movie, while the other a music album, then these events could be separated by their context. Hence, rather than aggregating the two instances ofevent 570 into a single event, they could separated by their context of music album versus book. Aggregation could still occur but within context. For instance, if there were five replies for books and three replies for music albums, then the five book-related reply events could be aggregated into one while the three music-related reply events could be aggregated into another. From a hierarchical grouping standpoint, all eight (three music and five book) reply events are identical. But differentiating by context, they could be considered separately, as the invention's event logging provides for. - In this way, it may be possible to measure the speed/efficiency at which look-ups of books are performed in contrast with the look-up of music albums. This may have utility, for instance, if the size of the table and records associated with books versus music were vastly different, thus yielding different reply event timings. According to one embodiment of the invention, the non-contextual hierarchical count of all eight (book and music)
events 570 could be maintained in addition to the contextual count of five events and three events as being separate. -
FIG. 6 is a computer system diagram that utilizes and implements one or more embodiments of the invention. - Illustrated is a
computer system 610, which may be any general or special purpose computing or data processing machine such as a PC (personal computer).Computer system 610 is coupled to anetwork 600 which may connect to one or more networks, such as a Local Area Network or the Internet, enablingcomputer system 610 to transact information with other computer systems or information devices. - One of ordinary skill in the art may program
computer system 610 to allow event logging that is capable of hierarchical and contextual event grouping as well as aggregation in a user customizable manner. As described above, event logging may be provided for in the operating system or framework that is common to applications executing withincomputer system 610. -
Computer system 610 has aprocessor 612 and amemory 611, such as RAM, which is used to store/load instructions, addresses and result data as desired. Typically, when event logging is turned on, the instructions pertaining to implementing the logging (such as event object creation) are loaded intomemory 611, executed byprocessor 612 and then given a memory space within 611 to execute. The implementation of the event logging center and associated functionality may derive from a low level source code such as assembler, a machine language or a higher level language or derivative thereof that has been embedded with the operating system/framework of thecomputer system 610. The instructions corresponding to the event logging center as well as the log of event objects and its associated information (e.g. start and end time indices), may be stored to any combination of adisk 618, such as a hard drive, or tomemory 611. -
Computer system 610 has a system bus 613 which facilitates information transfer to/from theprocessor 612 andmemory 611 and abridge 614 which couples to an I/O bus 615. I/O bus 615 connects various I/O devices such as a network interface card (NIC) 616,disk 618 and to thesystem memory 611 andprocessor 612. TheNIC 616 allows the event logging results to be displayed to users connected tonetwork 600. For instance, referring back toFIG. 3 , theremote web browser 340 may access the event logging center and display the results therefrom vianetwork 600. - Once event logging is completed and statistics thereafter computed, they can be displayed to a user through a web browser which may be displayed onto a display device (e.g. monitor) such as
display 620. Through other input devices, such as a keyboard and/or mouse (not shown), the user can input their grouping preferences to the event logging center through the web browser running on thecomputer system 610 and displayed viadisplay 620. In the case of a remote web browser, theNIC 616 is used to receive that remote user's grouping preferences and send event center results vianetwork 600.
Claims (21)
1-22. (canceled)
23. A method of logging events for a plurality of applications that operate on an operating system of a first computer, the method comprising:
receiving a request from a particular application of said applications to create an event object associated with an event of the application that has not been logged;
in response to said request, creating the event object in a first memory space that is separate from a second memory space in which said application operates;
from the application, receiving temporal data and other data regarding the event;
storing the received data regarding the event in the event object; and
processing the stored data regarding the event in order to produce processed event data for display on a web browser of a second computer that is connected to the first computer through the Internet,
wherein said receiving the request, said creating, said receiving the data, said storing, and said processing are performed by an event logging mechanism that is part of said operating system and operating independently from said application on the first computer on which said application operates.
24. A method according to claim 23 further comprising:
checking whether event logging has been turned on for the event.
25. A method according to claim 24 , wherein said creating, receiving the data, and storing are performed for each event having event logging turned on, wherein a plurality of event objects are created for a plurality of events.
26. A method according to claim 25 , wherein said processing comprises analyzing said event object after event logging is turned off.
27. A method according to claim 26 , wherein processing includes:
allowing user definition of the hierarchical levels of granularity of said events whose event objects are to be analyzed; and
allowing user definition of contexts for differentiating repeated occurrences of events deemed identical by nature of their hierarchical position.
28. A method according to claim 27 , wherein processing further includes:
grouping events into their hierarchical subgroups; and
grouping events by their context, if any are defined.
29. A method according to claim 28 , wherein processing further comprises:
traversing through the hierarchy of subgroups until the subgroup of finest granularity is traversed;
subdividing said events into further subgroups;
computing statistics for each subgroup while traversing; and
displaying said statistics.
30. A method according to claim 29 , wherein if said subgroup of finest granularity has been traversed, then:
aggregating events deemed identical by virtue of their hierarchical position into an aggregate;
computing statistics for each aggregate; and
displaying said statistics for each said aggregate.
31. A method according to claim 29 , wherein said processing includes:
aggregating events deemed identical by virtue of their context into an aggregate;
computing statistics for each aggregate; and
displaying said statistics for each said aggregate.
32. An event logging method comprising:
for each of a plurality of events that need to be logged within a plurality of applications operating on an operating system of a computer:
recording event data for each of a plurality of application events in a first memory space that is uniquely allocated for the event logging method, said first memory space separate from a second memory space allocated for the plurality of applications;
for each application in said plurality of application, grouping a plurality of said event data into a plurality of hierarchical sets of events, wherein each hierarchical set comprises a parent event and at least one child event; and
for a plurality of said hierarchical sets of events, defining a nested hierarchical display of event data that displays the parent events at one aligned level in the hierarchy and displays the child events at another aligned level in the hierarchy under the corresponding parent events, wherein said recording, grouping, and defining are performed by an event logging mechanism that is part of the operating system of said computer and that runs independently from said applications on said computer.
33. A method according to claim 32 , wherein the operating system comprises a framework, said framework comprising the event logging mechanism.
34. A method according to claim 32 further comprising checking, for each event identified by a particular application within the plurality of applications, whether event logging has been enabled.
35. A method according to claim 34 , wherein said nested hierarchical display is displayed in a web browser.
36. A method according to claim 32 , wherein the memory space occupied by the event data is within a memory space that has been allocated solely to the event logging mechanism.
37. A method according to claim 32 , wherein the events that are logged by the event logging mechanism have not been previously logged by any other application.
38. A method according to claim 32 , wherein information placed in the event data is first logged by the event logging mechanism.
39. A method according to claim 32 , wherein each event comprises an enable/disable state, wherein a disable state of a particular event precludes any system from logging said particular event.
40. A method according to claim 35 , wherein said computer is a first computer, wherein said web browser executes on a second computer that is connected to said first computer through the Internet.
41. An article comprising a computer readable medium storing a computer program for execution by at least one processor, the computer program comprising a set of instructions which when executed causes:
for each event in a plurality of events relating to an application (i) that generates web pages and responds to selections received through said generated web pages and (ii) that operates on an operating system of a computer,
receiving event data related to at least one of a request for a web page, generation of a web page, and a request received through a selection of an item of the generated web page; and
recording said event data information regarding the event,
wherein said receiving and recording are performed by an event logging mechanism that is part of the operating system of said computer and that runs independently from said application on said computer on which said application executes.
42. An article according to claim 41 , wherein the comprises a framework, said framework comprising the event logging mechanism.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/825,334 US20100269101A1 (en) | 2000-07-18 | 2010-06-28 | Event logging and performance analysis system for applications |
US13/801,902 US20130198573A1 (en) | 2000-07-18 | 2013-03-13 | Event logging and performance analysis system for applications |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US09/618,367 US7774790B1 (en) | 2000-07-18 | 2000-07-18 | Event logging and performance analysis system for applications |
US12/825,334 US20100269101A1 (en) | 2000-07-18 | 2010-06-28 | Event logging and performance analysis system for applications |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/618,367 Continuation US7774790B1 (en) | 2000-07-18 | 2000-07-18 | Event logging and performance analysis system for applications |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/801,902 Continuation US20130198573A1 (en) | 2000-07-18 | 2013-03-13 | Event logging and performance analysis system for applications |
Publications (1)
Publication Number | Publication Date |
---|---|
US20100269101A1 true US20100269101A1 (en) | 2010-10-21 |
Family
ID=42536707
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/618,367 Expired - Fee Related US7774790B1 (en) | 2000-07-18 | 2000-07-18 | Event logging and performance analysis system for applications |
US12/825,334 Abandoned US20100269101A1 (en) | 2000-07-18 | 2010-06-28 | Event logging and performance analysis system for applications |
US13/801,902 Abandoned US20130198573A1 (en) | 2000-07-18 | 2013-03-13 | Event logging and performance analysis system for applications |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/618,367 Expired - Fee Related US7774790B1 (en) | 2000-07-18 | 2000-07-18 | Event logging and performance analysis system for applications |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/801,902 Abandoned US20130198573A1 (en) | 2000-07-18 | 2013-03-13 | Event logging and performance analysis system for applications |
Country Status (1)
Country | Link |
---|---|
US (3) | US7774790B1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090164482A1 (en) * | 2007-12-20 | 2009-06-25 | Partha Saha | Methods and systems for optimizing projection of events |
US20090265451A1 (en) * | 2008-04-17 | 2009-10-22 | E.S.I. Software Ltd. | Assisted application definition |
US20140013310A1 (en) * | 2007-08-24 | 2014-01-09 | Riverbed Technology, Inc. | Selective Monitoring of Software Applications |
US20140068567A1 (en) * | 2012-09-05 | 2014-03-06 | Microsoft Corporation | Determining relevant events in source code analysis |
US20140089903A1 (en) * | 2007-11-27 | 2014-03-27 | Oracle America, Inc. | Sampling Based Runtime Optimizer for Efficient Debugging of Applications |
CN114327769A (en) * | 2021-12-29 | 2022-04-12 | 科东(广州)软件科技有限公司 | Operating system event recording method and device and computer system |
Families Citing this family (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8595214B1 (en) * | 2004-03-31 | 2013-11-26 | Google Inc. | Systems and methods for article location and retrieval |
US8099716B2 (en) * | 2005-11-28 | 2012-01-17 | Ubiquity Software Corporation Limited | Service structured application development architecture |
US8050255B2 (en) * | 2006-05-10 | 2011-11-01 | Microsoft Corporation | Routing a VoIP call with contextual information |
US8135125B2 (en) * | 2006-05-10 | 2012-03-13 | Microsoft Corporation | Multi-party information analysis in a VoIP system |
US7983247B2 (en) * | 2006-05-31 | 2011-07-19 | Microsoft Corporation | Metadata collection |
US8630191B2 (en) * | 2006-11-01 | 2014-01-14 | Microsoft Corporation | Device selection for broadcast messages |
US8271891B1 (en) * | 2007-02-02 | 2012-09-18 | Sandia Corporation | Computing environment logbook |
US8140919B2 (en) * | 2007-10-15 | 2012-03-20 | International Business Machines Corporation | Display of data used for system performance analysis |
US8141044B2 (en) * | 2007-11-14 | 2012-03-20 | International Business Machines Corporation | Method and system for identifying sources of operating system jitter |
US9298815B2 (en) * | 2008-02-22 | 2016-03-29 | Accenture Global Services Limited | System for providing an interface for collaborative innovation |
US8813096B2 (en) * | 2011-10-11 | 2014-08-19 | International Business Machines Corporation | Predicting the impact of change on events detected in application logic |
US20150127466A1 (en) | 2012-05-14 | 2015-05-07 | Hengshu Zhu | Method and apparatus for determining context-aware similarity |
US8904412B2 (en) * | 2012-09-26 | 2014-12-02 | International Business Machines Corporation | Dynamic performance data collection in large computer servers |
NL2009756C2 (en) * | 2012-11-05 | 2014-05-08 | Realworld Holding B V | Method and arrangement for collecting timing data related to a computer application. |
CN103902438A (en) * | 2012-12-24 | 2014-07-02 | 珠海市君天电子科技有限公司 | Method and system for generating readable report of APK program behavior |
US9363311B1 (en) | 2014-12-05 | 2016-06-07 | Tealium Inc. | Delivery of instructions in host applications |
US10185613B2 (en) * | 2016-04-29 | 2019-01-22 | Vmware, Inc. | Error determination from logs |
US10838841B2 (en) * | 2018-07-25 | 2020-11-17 | Vmware, Inc. | Grouping log pieces of different context within a log file in virtualization system |
US20210005206A1 (en) | 2019-07-05 | 2021-01-07 | Talkdesk, Inc. | System and method for speech-enabled automated agent assistance within a cloud-based contact center |
US11328205B2 (en) | 2019-08-23 | 2022-05-10 | Talkdesk, Inc. | Generating featureless service provider matches |
US20210117882A1 (en) | 2019-10-16 | 2021-04-22 | Talkdesk, Inc | Systems and methods for workforce management system deployment |
US20210136220A1 (en) | 2019-10-31 | 2021-05-06 | Talkdesk, Inc. | Monitoring and listening tools across omni-channel inputs in a graphically interactive voice response system |
US11599526B2 (en) * | 2019-10-31 | 2023-03-07 | Salesforce.Com, Inc. | Selectively publishing an event responsive to an operation on a database in a transaction which is rolled back or committed |
US11736615B2 (en) | 2020-01-16 | 2023-08-22 | Talkdesk, Inc. | Method, apparatus, and computer-readable medium for managing concurrent communications in a networked call center |
US11677875B2 (en) | 2021-07-02 | 2023-06-13 | Talkdesk Inc. | Method and apparatus for automated quality management of communication records |
US11856140B2 (en) | 2022-03-07 | 2023-12-26 | Talkdesk, Inc. | Predictive communications system |
US11736616B1 (en) | 2022-05-27 | 2023-08-22 | Talkdesk, Inc. | Method and apparatus for automatically taking action based on the content of call center communications |
US11943391B1 (en) | 2022-12-13 | 2024-03-26 | Talkdesk, Inc. | Method and apparatus for routing communications within a contact center |
Citations (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5423000A (en) * | 1988-09-26 | 1995-06-06 | Hitachi, Ltd. | Operation control technique for computer system by driving a command procedure with reference to events recorded on a logging storage computer operation control by automatic decision making based on events on a logging storage |
US5446878A (en) * | 1990-02-26 | 1995-08-29 | Digital Equipment Corporation | Method for selectively enabling subset of embedded event-making instructions and selecting types and items of event-based data to be collected per enabled instruction |
US5675510A (en) * | 1995-06-07 | 1997-10-07 | Pc Meter L.P. | Computer use meter and analyzer |
US5740354A (en) * | 1995-11-27 | 1998-04-14 | Microsoft Corporation | Method and system for associating related errors in a computer system |
US5819094A (en) * | 1996-03-26 | 1998-10-06 | Fujitsu Ltd. | Apparatus for log data collection and analysis |
US5857190A (en) * | 1996-06-27 | 1999-01-05 | Microsoft Corporation | Event logging system and method for logging events in a network system |
US5867659A (en) * | 1996-06-28 | 1999-02-02 | Intel Corporation | Method and apparatus for monitoring events in a system |
US5872909A (en) * | 1995-01-24 | 1999-02-16 | Wind River Systems, Inc. | Logic analyzer for software |
US5872976A (en) * | 1997-04-01 | 1999-02-16 | Landmark Systems Corporation | Client-based system for monitoring the performance of application programs |
US5893077A (en) * | 1995-08-23 | 1999-04-06 | Microsoft Corporation | Method and apparatus for generating and collecting a billing event object within an on-line network |
US6144967A (en) * | 1996-01-25 | 2000-11-07 | International Business Machines Corporation | Object oriented processing log analysis tool framework mechanism |
US6148338A (en) * | 1998-04-03 | 2000-11-14 | Hewlett-Packard Company | System for logging and enabling ordered retrieval of management events |
US6363435B1 (en) * | 1998-02-03 | 2002-03-26 | Microsoft Corporation | Event sourcing and filtering for transient objects in a hierarchical object model |
US20020083217A1 (en) * | 1997-07-25 | 2002-06-27 | Ward Alan D. | System and method asynchronously accessing a graphics system for graphics application evaluation and control |
US20020138663A1 (en) * | 1998-07-29 | 2002-09-26 | Northrop Grumman Corporation | System and method for ensuring and managing situation awareness |
US6466970B1 (en) * | 1999-01-27 | 2002-10-15 | International Business Machines Corporation | System and method for collecting and analyzing information about content requested in a network (World Wide Web) environment |
US6470388B1 (en) * | 1999-06-10 | 2002-10-22 | Cisco Technology, Inc. | Coordinated extendable system for logging information from distributed applications |
US6611498B1 (en) * | 1997-09-26 | 2003-08-26 | Worldcom, Inc. | Integrated customer web station for web based call management |
US20040054770A1 (en) * | 1994-04-05 | 2004-03-18 | Shlomo Touboul | Method and apparatus for monitoring and controlling programs in a network |
US6742143B2 (en) * | 1999-06-03 | 2004-05-25 | Microsoft Corporation | Method and apparatus for analyzing performance of data processing system |
US6754704B1 (en) * | 2000-06-21 | 2004-06-22 | International Business Machines Corporation | Methods, systems, and computer program product for remote monitoring of a data processing system events |
US6836894B1 (en) * | 1999-07-27 | 2004-12-28 | International Business Machines Corporation | Systems and methods for exploratory analysis of data for event management |
US6971110B1 (en) * | 2000-02-19 | 2005-11-29 | Hewlett-Packard Development Company, L.P. | System and method to pace event sharing collaboration across multiple distributed applications |
US6973577B1 (en) * | 2000-05-26 | 2005-12-06 | Mcafee, Inc. | System and method for dynamically detecting computer viruses through associative behavioral analysis of runtime state |
US20060085798A1 (en) * | 2000-05-05 | 2006-04-20 | Bristol Technology Inc. | Method and apparatus for correlation of events in a distributed multi-system computing environment |
US7155409B1 (en) * | 1999-03-05 | 2006-12-26 | Trade Finance Service Corporation | Trade financing method, instruments and systems |
US7243271B2 (en) * | 1999-09-09 | 2007-07-10 | Microsoft Corporation | Wrapped object for observing object events |
US7792947B1 (en) * | 1999-04-26 | 2010-09-07 | Mainstream Scientific, Llc | Apparatus and method for dynamically coordinating the delivery of computer readable media |
Family Cites Families (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE69720857T2 (en) * | 1996-05-31 | 2004-02-05 | Hewlett-Packard Co. (N.D.Ges.D.Staates Delaware), Palo Alto | Systems and methods for operating a network management station |
US5796633A (en) * | 1996-07-12 | 1998-08-18 | Electronic Data Systems Corporation | Method and system for performance monitoring in computer networks |
US5682328A (en) * | 1996-09-11 | 1997-10-28 | Bbn Corporation | Centralized computer event data logging system |
US5892898A (en) * | 1996-10-04 | 1999-04-06 | Honeywell, Inc. | Error management system for supporting the identification and logging of error messages |
US6138121A (en) * | 1998-05-29 | 2000-10-24 | Hewlett-Packard Company | Network management event storage and manipulation using relational database technology in a data warehouse |
US6347330B1 (en) * | 1998-09-04 | 2002-02-12 | International Business Machines Corporation | Dynamic selective distribution of events to server receivers |
AU3889200A (en) * | 1999-03-15 | 2000-10-04 | Smartsan Systems, Inc. | System and method of event management and early fault detection |
US7249356B1 (en) * | 1999-04-29 | 2007-07-24 | Fisher-Rosemount Systems, Inc. | Methods and structure for batch processing event history processing and viewing |
US6539542B1 (en) * | 1999-10-20 | 2003-03-25 | Verizon Corporate Services Group Inc. | System and method for automatically optimizing heterogenous multiprocessor software performance |
US6772107B1 (en) * | 1999-11-08 | 2004-08-03 | J.D. Edwards World Source Company | System and method for simulating activity on a computer network |
US7278103B1 (en) * | 2000-06-28 | 2007-10-02 | Microsoft Corporation | User interface to display and manage an entity and associated resources |
-
2000
- 2000-07-18 US US09/618,367 patent/US7774790B1/en not_active Expired - Fee Related
-
2010
- 2010-06-28 US US12/825,334 patent/US20100269101A1/en not_active Abandoned
-
2013
- 2013-03-13 US US13/801,902 patent/US20130198573A1/en not_active Abandoned
Patent Citations (28)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5423000A (en) * | 1988-09-26 | 1995-06-06 | Hitachi, Ltd. | Operation control technique for computer system by driving a command procedure with reference to events recorded on a logging storage computer operation control by automatic decision making based on events on a logging storage |
US5446878A (en) * | 1990-02-26 | 1995-08-29 | Digital Equipment Corporation | Method for selectively enabling subset of embedded event-making instructions and selecting types and items of event-based data to be collected per enabled instruction |
US20040054770A1 (en) * | 1994-04-05 | 2004-03-18 | Shlomo Touboul | Method and apparatus for monitoring and controlling programs in a network |
US5872909A (en) * | 1995-01-24 | 1999-02-16 | Wind River Systems, Inc. | Logic analyzer for software |
US5675510A (en) * | 1995-06-07 | 1997-10-07 | Pc Meter L.P. | Computer use meter and analyzer |
US5893077A (en) * | 1995-08-23 | 1999-04-06 | Microsoft Corporation | Method and apparatus for generating and collecting a billing event object within an on-line network |
US5740354A (en) * | 1995-11-27 | 1998-04-14 | Microsoft Corporation | Method and system for associating related errors in a computer system |
US6144967A (en) * | 1996-01-25 | 2000-11-07 | International Business Machines Corporation | Object oriented processing log analysis tool framework mechanism |
US5819094A (en) * | 1996-03-26 | 1998-10-06 | Fujitsu Ltd. | Apparatus for log data collection and analysis |
US5857190A (en) * | 1996-06-27 | 1999-01-05 | Microsoft Corporation | Event logging system and method for logging events in a network system |
US5867659A (en) * | 1996-06-28 | 1999-02-02 | Intel Corporation | Method and apparatus for monitoring events in a system |
US5872976A (en) * | 1997-04-01 | 1999-02-16 | Landmark Systems Corporation | Client-based system for monitoring the performance of application programs |
US20020083217A1 (en) * | 1997-07-25 | 2002-06-27 | Ward Alan D. | System and method asynchronously accessing a graphics system for graphics application evaluation and control |
US6611498B1 (en) * | 1997-09-26 | 2003-08-26 | Worldcom, Inc. | Integrated customer web station for web based call management |
US6363435B1 (en) * | 1998-02-03 | 2002-03-26 | Microsoft Corporation | Event sourcing and filtering for transient objects in a hierarchical object model |
US6148338A (en) * | 1998-04-03 | 2000-11-14 | Hewlett-Packard Company | System for logging and enabling ordered retrieval of management events |
US20020138663A1 (en) * | 1998-07-29 | 2002-09-26 | Northrop Grumman Corporation | System and method for ensuring and managing situation awareness |
US6466970B1 (en) * | 1999-01-27 | 2002-10-15 | International Business Machines Corporation | System and method for collecting and analyzing information about content requested in a network (World Wide Web) environment |
US7155409B1 (en) * | 1999-03-05 | 2006-12-26 | Trade Finance Service Corporation | Trade financing method, instruments and systems |
US7792947B1 (en) * | 1999-04-26 | 2010-09-07 | Mainstream Scientific, Llc | Apparatus and method for dynamically coordinating the delivery of computer readable media |
US6742143B2 (en) * | 1999-06-03 | 2004-05-25 | Microsoft Corporation | Method and apparatus for analyzing performance of data processing system |
US6470388B1 (en) * | 1999-06-10 | 2002-10-22 | Cisco Technology, Inc. | Coordinated extendable system for logging information from distributed applications |
US6836894B1 (en) * | 1999-07-27 | 2004-12-28 | International Business Machines Corporation | Systems and methods for exploratory analysis of data for event management |
US7243271B2 (en) * | 1999-09-09 | 2007-07-10 | Microsoft Corporation | Wrapped object for observing object events |
US6971110B1 (en) * | 2000-02-19 | 2005-11-29 | Hewlett-Packard Development Company, L.P. | System and method to pace event sharing collaboration across multiple distributed applications |
US20060085798A1 (en) * | 2000-05-05 | 2006-04-20 | Bristol Technology Inc. | Method and apparatus for correlation of events in a distributed multi-system computing environment |
US6973577B1 (en) * | 2000-05-26 | 2005-12-06 | Mcafee, Inc. | System and method for dynamically detecting computer viruses through associative behavioral analysis of runtime state |
US6754704B1 (en) * | 2000-06-21 | 2004-06-22 | International Business Machines Corporation | Methods, systems, and computer program product for remote monitoring of a data processing system events |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140013310A1 (en) * | 2007-08-24 | 2014-01-09 | Riverbed Technology, Inc. | Selective Monitoring of Software Applications |
US9189364B2 (en) * | 2007-08-24 | 2015-11-17 | Riverbed Technology, Inc. | Selective monitoring of software applications |
US20140089903A1 (en) * | 2007-11-27 | 2014-03-27 | Oracle America, Inc. | Sampling Based Runtime Optimizer for Efficient Debugging of Applications |
US9146831B2 (en) * | 2007-11-27 | 2015-09-29 | Oracle America, Inc. | Sampling based runtime optimizer for efficient debugging of applications |
US20090164482A1 (en) * | 2007-12-20 | 2009-06-25 | Partha Saha | Methods and systems for optimizing projection of events |
US20090265451A1 (en) * | 2008-04-17 | 2009-10-22 | E.S.I. Software Ltd. | Assisted application definition |
US20140068567A1 (en) * | 2012-09-05 | 2014-03-06 | Microsoft Corporation | Determining relevant events in source code analysis |
US9612937B2 (en) * | 2012-09-05 | 2017-04-04 | Microsoft Technology Licensing, Llc | Determining relevant events in source code analysis |
CN114327769A (en) * | 2021-12-29 | 2022-04-12 | 科东(广州)软件科技有限公司 | Operating system event recording method and device and computer system |
Also Published As
Publication number | Publication date |
---|---|
US7774790B1 (en) | 2010-08-10 |
US20130198573A1 (en) | 2013-08-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7774790B1 (en) | Event logging and performance analysis system for applications | |
US11934408B1 (en) | Interactive development environment for visualization of query result information | |
US11531662B2 (en) | System and method for use of a dynamic flow in a multidimensional database environment | |
US10417063B2 (en) | Artificial creation of dominant sequences that are representative of logged events | |
US6564170B2 (en) | Customizable user interfaces | |
KR101678131B1 (en) | Generating dependency maps from dependency data | |
US7853585B2 (en) | Monitoring performance of a data processing system | |
US7398512B2 (en) | Method, system, and software for mapping and displaying process objects at different levels of abstraction | |
US10241901B2 (en) | Web application performance testing | |
CN104680395B (en) | For method and system audience segment estimation | |
US20020070953A1 (en) | Systems and methods for visualizing and analyzing conditioned data | |
US20050172306A1 (en) | Systems, methods and computer programs for determining dependencies between logical components in a data processing system or network | |
US20120030220A1 (en) | Heterogeneous data source management | |
US20080159317A1 (en) | Data organization and evaluation using a two-topology configuration | |
JP2005062941A (en) | Method for analyzing performance information | |
WO2002017183A2 (en) | System and method for analysing a transactional monitoring system | |
AU2003298223A1 (en) | Scrolling through data in a graphical user interface | |
CN102236851A (en) | Real-time computation method and system of multi-dimensional credit system based on user empowerment | |
CN110347954B (en) | Complex Web application-oriented servitization method | |
CA2804441A1 (en) | Database performance analysis | |
US20020156825A1 (en) | Organization and visualization of performance data in selected display modes | |
US20050086664A1 (en) | Method and apparatus for transaction tracking in a web presentation architecture | |
CN109522179B (en) | Server running state monitoring method and device, processor and server | |
US20070033233A1 (en) | Log management system and method of using the same | |
CN112036931A (en) | Real-time label calculation method and device, computer equipment and storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO PAY ISSUE FEE |