WO2006026686A1

WO2006026686A1 - User interfaces for data integration systems

Info

Publication number: WO2006026686A1
Application number: PCT/US2005/031020
Authority: WO
Inventors: Nathan Bobbin; Jr. Meredith Russell Butler; Jonathan Dee Jaynes; Michael W. Yaklin; Yasmin Yousseff; Steven John Totman
Original assignee: Ascential Software Corporation
Priority date: 2004-08-31
Filing date: 2005-08-31
Publication date: 2006-03-09
Also published as: CN101084494B; CA2579803A1; CN101084494A; KR20070067082A; EP1810169A1; JP2008511935A; KR101033446B1

Abstract

Provided herein are methods and systems for managing and using metadata in connection with data integration in an enterprise computing environment. Through an integrated user interface, a user may have enterprise-wide access to data integration services and underlying data. The interface may be structured around theworkflow of data integration system design and implementation, and may facilitate reuse and redesign of tools and jobs in the data integration environment. The interface may present platform-independent access to all of the data integration resources and data sources across an enterprise.

Description

USER INTERFACES FOR DATA INTEGRATION SYSTEMS

RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application Number 60/606,372, filed August 31, 2004 and entitled "User Interfaces for Data Integration Systems".

BACKGROUND

1. Field.

This invention relates to the field of information technology, and more particularly to the field of data integration systems.

2. Description of the Related Art.

The advent of computer applications made many business processes much faster and more efficient; however, the proliferation of different computer applications that use different data structures, communication protocols, languages and platforms has led to great complexity in the information technology infrastructure of the typical business enterprise. Different business processes within the typical enterprise may use completely different computer applications, each computer application being developed and optimized for the particular business process, rather than for the enterprise as a whole. For example, a business may have a particular computer application for tracking accounts payable and a completely different one for keeping track of customer contacts. In fact, even the same business process may use more than one computer application, such as when an enterprise keeps a centralized customer contact database, but employees keep their own contact information, such as in a personal information manager.

While specialized computer applications offer the advantages of custom-tailored solutions, the proliferation leads to inefficiencies, such as repetitive entry and handling of the same data many times throughout the enterprise, or the failure of the enterprise to capitalize on data that is associated with one process when the enterprise executes another process that could benefit from that data. For example, if the accounts payable process is separated from the supply chain and ordering process, the enterprise may accept and fill orders from a customer whose credit history would have caused the enterprise to decline the order. Many other examples can be provided where an enterprise would benefit from consistent access to all of its data across varied computer applications. A number of companies have recognized and addressed the need for sharing of data across different applications in the business enterprise. Thus, enterprise application integration, or EAI, has emerged as a message- based strategy for addressing data from disparate sources. As computer applications increase in complexity and number, EAI efforts encounter many challenges, ranging from the need to handle different protocols, the need to address ever-increasing volumes of data and numbers of transactions, and an ever-increasing appetite for faster integration of data. Various approaches to EAI have been taken, including least-common-denominator approaches, atomic approaches, and bridge-type approaches. However, EAI is based upon communication between individual applications. As a significant disadvantage, the complexity of EAI solutions grows geometrically in response to linear additions of platforms and applications.

While data integration systems provided useful tools for addressing the needs of an enterprise, such systems are typically deployed as custom solutions. They have a lengthy development cycle, and may require sophisticated technical training to accommodate changes in business structure and information requirements. There remains a need for user interfaces to data integration systems that permit use, reuse, and modification of functionality through an intuitive interface adapted to the data integration design and workflow environments.

SUMMARY Provided herein are methods and systems for managing and using metadata in connection with data integration in an enterprise computing environment. Through an integrated user interface, a user may have enterprise-wide access to data integration services and underlying data. The interface may be structured around the workflow of data integration system design and implementation, and may facilitate reuse and redesign of tools and jobs in the data integration environment. The interface may present platform-independent access to all of the data integration resources and data sources across an enterprise.

In one aspect, there is disclosed herein methods and systems for providing a user interface for managing workflow in a computer environment, the user interface including a menu bar that displays a plurality of controls, each control accessing an interface corresponding to a phase in a workflow, each interface configured to (1) retrieve data from the computer environment about the corresponding phase in the workflow, (2) display the retrieved data in the interface, (3) receive user input relating to the corresponding phase in the workflow; and (4) transmit data to the computer environment based upon the user input.

The computer environment may include a data integration system. The workflow may relate to a data integration process. The workflow may include at least one of a data integration function, a data integration service, or a data integration job. The user interface may provide varying degrees of functionality for the plurality of controls according to a skill level of a user of the interface.

The phases may include one or more of investigate, design, develop, deploy and operate. In an investigate phase, one or more data sources in the computer environment may be profiled, metadata is obtained for one or more data sources, and/or one or more data sources may be audited for data quality. In a design phase, one or more data integration processes may be characterized and/or one or more transformations may be characterized for data in the data sources. In a develop phase, one or more data integration processes may be associated with data sources in the data integration system, one or more data integration processes may be associated with other data integration processes deployed in the data integration system, and/or data staging may be performed on one or more data sources. In a deploy phase, a data integration process may be made available to the data integration system and/or a data integration process is deployed as a real-time data integration service. In an operate phase, at least one data integration function, service, or job may be used in the data integration system.

The user interface may provide varying degrees of functionality for the plurality of controls according to a skill level of a user of the interface.

In another aspect, disclosed herein are methods for displaying related objects in a user interface including displaying a plurality of objects having relationships; and providing visual cues for relationships between the plurality of object including shape, size, color, and opacity of objects, and distance between objects. Also disclosed herein are systems for displaying related objects in a user interface including a user interface displaying a plurality of objects having relationships, the user interface providing visual cues for relationships between the plurality of object including shape, size, color, and opacity of objects, and distance between objects.

The objects may describe a business context. The objects may include metadata for one or more data sources. The objects may include data integration objects. The systems and methods may further include providing one or more tools for visualizing, navigating, and editing relationships among the plurality of objects. A hierarchy of the plurality of objects may be displayed as a projection onto a three dimensional surface. The user interface may animate a transition to a new current object by rotating point of view of the three-dimensional surface. The hierarchy may be displayed as a projection onto a hyperbolic surface. The current object may be displayed in greater detail than other objects. The current object may be displayed larger than other objects. The objects may include one or more of people, business objects, and data sources.

In another aspect, a method for providing information for an object within a user interface may include displaying a container in a user interface representing an object; and providing an active area within the container, the active area responding to an event by displaying one or more visual controls related to the object. A system for providing information may include a user interface displaying a container that represents an object; and an active area within the container, the active area responding to an event by displaying one or more visual controls related to the object.

The one or more visual controls may relate to functions performed on the object, attributes of the object, and or relationships of the object to other objects. The one or more visual controls may respond to an event by displaying additional visual controls related to the object, whereby a hierarchy of controls is progressively displayed for the object. The event may be a mouse over event. Manipulations of visual controls may cause persistent changes that may be reviewed on subsequent responses of the active area to the event. The object may relate to a data integration process.

In another aspect, a method for navigating to a page within a user interface may include: displaying one of a plurality of sequential pages within a user interface; providing a control for navigating forward and backward within the plurality of sequential pages; upon activation of the control, displaying a drop down menu of more than one of the plurality of sequential pages; receiving within the control a selection of one of the sequential pages displayed in the drop down menu; and navigating to the selected one of the sequential pages. A system for navigating to a page within a user interface may include: a user interface displaying one of a plurality of sequential pages; and a control for navigating forward and backward within the plurality of sequential pages, the control responsive to activation by displaying a drop down menu of more than one of the plurality of sequential pages, wherein a user may navigate to one of the sequential pages displayed in the drop down menu by selecting the one of the sequential pages displayed in the drop down menu. The pages may be represented by tabs along an edge of a page display of the user interface

In another aspect, a method for maintaining a list of reusable operations in a user interface may include: storing a plurality of tasks as they are performed in a user interface as a history; displaying the history within a control in the user interface; receiving a selection of one of the plurality of tasks; and repeating the selected one of the plurality of tasks in the present context of the user interface. A system for maintaining a list of reusable operations in a user interface may include: a user interface displaying a history of a plurality of tasks as they are performed; one or more controls for selecting one of the plurality of tasks and repeating the selected one of the plurality of tasks in the present context of the user interface.

The tasks may be data integration tasks. Tasks may be be stored at variable levels of detail. Displaying the history may include displaying the tasks in a hierarchy of levels of detail of the tasks. The method may further include undocking the control from the user interface for use with a plurality of user interfaces. The plurality of tasks may be sorted within the history according to a frequency of use. The tasks may include one or more of functions, operations, designs, macros, text types, stages, and wizards. The method or system may further include receiving a selection of more than one of the plurality of tasks; and repeating the selected more than one of the plurality of tasks in the present context of the user interface. The method or system may further include storing the selected more than one of the plurality of tasks for reuse.

In another aspect, there is disclosed herein a method for displaying a hierarchy of objects within a drop down menu of a user interface including: displaying list of a plurality of objects at a plurality of levels of a hierarchy within a hierarchical tree; providing visual cues to the position of each object within the hierarchy, the visual cues including identical shading of items at the same level of the hierarchy and lines around a group of adjacent items at the same level of the hierarchy; and providing a control at each transition between different levels of the hierarchy for collapsing or expanding a branch of the hierarchical tree. Also disclosed herein is a system for displaying a hierarchy of objects within a drop down menu of a user interface including: a user interface displaying a list of a plurality of objects at a plurality of levels of a hierarchy within a hierarchical tree, the each one of the plurality of objects displayed with one or more visual cues to the position of each object within the hierarchy, the visual cues including identical shading of items at the same level of the hierarchy and lines around a group of adjacent items at the same level of the hierarchy; and a control at each transition between different levels of the hierarchy for collapsing or expanding a branch of the hierarchical tree. The objects may be data integration objects and/or data integration tasks.

A method for displaying the status of a process within a user interface may include: displaying a status bar in a first mode that provides animation to indicate that a process is running; in response to positioning a cursor over the status bar, switching the status bar to a second mode by increasing the size of the status bar and displaying within the status bar a name of the process and a percentage of completion of the process, and further displaying a number indicative of a number of other processes running; and in response to positioning a cursor over the number, switching to a third mode by additionally increasing the size of the status bar and displaying within the status bar a list of every process running. A system for displaying the status of a process within a user interface may include a user interface displaying a status bar, the status bar having a first mode, a second mode, and a third mode, the first mode providing animation to indicate that a process is running, the second mode accessible by positioning a cursor over the status bar, the second mode increasing the size of the status bar and displaying within the status bar a name of the process and a percentage of completion of the process, and further displaying a number indicative of a number of other processes running, and the third mode accessible by positioning the cursor over the number in the second mode, the third mode additionally increasing the size of the status bar and displaying within the status bar a list of every process running.

After a predetermined period of time, the status bar in the second mode may cycle through status information for the other processes. The method and system may further include displaying within the status bar in the third mode a start time for every process. The method may further include, in response to positioning a cursor over a process in the list of the third mode, providing a detailed view of the process including one or more of a start time for the process, a user who initiated the process, and a percentage of resources used by the process. At least one of the processes may be a data integration process.

A method for linking a data integration task to a business context may include: providing a graphical user interface for manipulating business objects and metadata; placing at least one business object into the graphical user interface; placing at least one metadata object from a data source into the graphical user interface; and linking the at least one metadata object to the at least one business object within the graphical user interface. A system for linking a data integration task to a business context may include a graphical user interface for manipulating business objects and metadata, the graphical user interface configured to link one or more metadata objects from a data source to one or more business objects within the graphical user interface.

The method and system may further include placing at least one metadata object from a data target into the graphical user interface. The method and system may further include linking the at least one metadata object from the data target to a business object within the graphical user interface. The method and system may further include deriving a data transformation for one or more links between the data source metadata and the data target metadata.

In another aspect, a method for designing a data integration process may include: modeling a business initiative as a first data structure; modeling a business context as a second data structure; adding one or more associations between the first data structure and the second data structure to provide an information model for the business initiative in the business context; adding one or more associations between the information model and one or more enterprise data sources to provide a mapping model; and applying one or more design tools to construct a data integration process from the mapping model. A system for designing a data integration process may include: a modeling tool for modeling a business initiative as a first data structure, modeling a business context as a second data structure, and adding one or more associations between the first data structure and the second data structure to provide an information model for the business initiative in the business context; a linking tool for adding one or more associations between the information model and one or more enterprise data sources to provide a mapping model; and one or more design tools to construct a data integration process from the mapping model.

The business initiative may include one or more of an initiative, an objective, a requirement, and a statement. The business context may include one or more of a subject, a process, and a rule. The enterprise data sources may include data sources external to the enterprise. The associations among the first data structure, the second data structure, and the data sources may be created using tools within a graphical user interface. AU of the steps of the method may be performed in an integrated user environment. The design tools may be provided within a user interface that includes integrated interfaces for one or more of investigation, development, design, deployment, and operation of a data integration process. A collaborative environment may be provided for creating one or more of the business initiative, the business context, the information model, and the mapping model.

In another aspect, a method for providing a configurable user interface may include: providing a plurality of data integration tasks; providing a plurality of menus corresponding to phases of a workflow; and assigning one or more of the plurality of data integration tasks to each on of the menus with a task matrix. A system for providing a configurable user interface may include a task matrix that assigns one or more of a plurality of data integration tasks to one or more of a plurality of menus, each menu corresponding to a phase of a workflow.

At least one of the data integration tasks may assigned to more than one menu. The task matrix is user- configurable.

In other aspects, a computer program product may include a computer useable medium including computer readable program code, wherein the computer readable program code when executed on one or more computers causes the one or more computers to perform any one or more of the methods above.

"International Business Machines" or "IBM" as used herein shall refer to International Business Machines Corporation of Armonk, New York.

As used herein, "data source" or "data target" are intended to have the broadest possible meaning consistent with these terms, and shall include a database, a plurality of databases, a repository information manager, a queue, a message service, a repository, a data facility, a data storage facility, a data provider, a website, a server, a computer, a computer storage facility, a CD, a DVD, a mobile storage facility, a central storage facility, a hard disk, a multiple coordinating data storage facilities, RAM, ROM, flash memory, a memory card, a temporary memory facility, a permanent memory facility, magnetic tape, a locally connected computing facility, a remotely connected computing facility, a wireless facility, a wired facility, a mobile facility, a central facility, a web browser, a client, a laptop, a personal digital assistant ("PDA"), a telephone, a cellular phone, a mobile phone, an information platform, an analysis facility, a processing facility, a business enterprise system or other facility where data is handled or other facility provided to store data or other information, as well as any files or file types for maintaining structured or unstructured data used in any of the above systems, or any streaming, messaged, event driven, or otherwise sourced data, and any combinations of the foregoing, unless a specific meaning is otherwise indicated or the context of the phrase requires otherwise. A storage mechanism is any physical or logical device, resource, or facility capable of serving as a data source or a data target, or otherwise storing data in a retrievable form.

"Enterprise Java Bean (EJB)" shall include the server-side component architecture for the J2EE platform. EJBs support rapid and simplified development of distributed, transactional, secure and portable Java applications. EJBs support a container architecture that allows concurrent consumption of messages and provide support for distributed transactions, so that database updates, message processing, and connections to enterprise systems using the J2EE architecture can participate in the same transaction context.

"JMS" shall mean the Java Message Service, which is an enterprise message service for the Java-based J2EE enterprise architecture. "JCA" shall mean the J2EE Connector Architecture of the J2EE platform described more particularly below. It should be appreciated that, while EJB, JMS, and JCA are commonly used software tools in contemporary distributed transaction environments, any platform, system, or architecture providing similar functionality may be employed with the data integration systems described herein.

"Real time" as used herein, shall include periods of time that approximate the duration of a business transaction or business and shall include processes or services that occur during a business operation or business process, as opposed to occurring off-line, such as in a nightly batch processing operation. Depending on the duration of the business process, real time might include seconds, fractions of seconds, minutes, hours, or even days.

"Business process," "business logic" and "business transaction" as used herein, shall include any methods, service, operations, processes or transactions that can be performed by a business, including, without limitation, sales, marketing, fulfillment, inventory management, pricing, product design, professional services, financial services, administration, finance, underwriting, analysis, contracting, information technology services, data storage, data mining, delivery of information, routing of goods, scheduling, communications, investments, transactions, offerings, promotions, advertisements, offers, engineering, manufacturing, supply chain management, human resources management, data processing, data integration, work flow administration, software production, hardware production, development of new products, research, development, strategy functions, quality control and assurance, packaging, logistics, customer relationship management, handling rebates and returns, customer support, product maintenance, telemarketing, corporate communications, investor relations, and many others.

"Service oriented architecture (SOA)", as used herein, shall include services that form part of the infrastructure of a business enterprise. In the SOA, services can become building blocks for application development and deployment, allowing rapid application development and avoiding redundant code. Each service may embody a set of business logic or business rules that can be bound to the surrounding environment, such as the source of the data inputs for the service or the targets for the data outputs of the service. Various instances of SO A are provided in the following description. "Metadata," as used herein, shall include data that brings context to the data being processed, data about the data, information pertaining to the context of related information, information pertaining to the origin of data, information pertaining to the location of data, information pertaining to the meaning of data, information pertaining to the age of data, information pertaining to the heading of data, information pertaining to the units of data, information pertaining to the field of data, and/or information pertaining to any other information relating to the context of the data.

"WSDL" or "Web Services Description Language" as used herein, includes an XML format for describing network services (often web services) as a set of endpoints operating on messages containing either document- oriented or procedure-oriented information. The operations and messages are described abstractly, and then bound to a concrete network protocol and message format to define an endpoint. Related concrete endpoints are combined into abstract endpoints (services). WSDL is extensible to allow description of endpoints and their messages regardless of what message formats or network protocols are used to communicate.

BRIEF DESCRIPTION OF THE FIGURES Fig. 1 is a schematic diagram of a business enterprise with a plurality of business processes, each of which may include a plurality of different computer applications and data sources.

Fig. 2 is a schematic diagram showing data integration across a plurality of business processes of a business enterprise.

Fig. 3 is a schematic diagram showing an architecture for providing data integration for a plurality of data sources for a business enterprise.

Fig. 4A shows a layout for a user interface.

Fig. 4B shows a user interface adapted for use in a data integration system.

Fig. 4C depicts a task matrix for the design of a user interface

Fig. 5 shows an investigate phase in a workflow user interface. Fig. 6 shows a design phase in a workflow user interface.

Fig. 7 shows a develop phase in a workflow user interface.

Fig. 8 shows a deploy phase in a workflow user interface.

Fig. 9 shows an operate phase in a workflow user interface.

Fig. 10 shows a navigation tool for use with a user interface. Fig. 11 shows a tool that provides progressive disclosure of detail in a user interface.

Fig. 12 shows a page selection tool for use in a user interface.

Fig. 13 shows a tool for recalling a history of previous operations within a user interface.

Fig. 14 shows a menu tree for use with a user interface.

Fig. 15 shows a status bar for use with a user interface. Fig. 16 shows a map for a subject in a business layer of a model.

Fig. 17 depicts rules and definitions for a subject in the business layer.

Fig. 18 depicts a business process in the business layer

Fig. 19 depicts an association of elements between the business layer and the data layer.

Fig. 20 is a map of linked business and data elements. Fig. 21 depicts the addition of a data object to a workspace of a user interface.

Fig. 22 depicts an association of a data object to business objects in a workspace of a user interface. Fig. 23 shows a process for designing a data integration process from a business context. Fig. 24 shows decomposition of an initiative into objectives. Fig. 25 shows decomposition of an objective into statements. Fig. 26 shows the development of an information model Fig. 27 shows the addition of business requirements to information model objects.

Fig. 28 shows linking of an information model to an initiative. Fig. 29 shows linking of an information model to data structures.

DETAILED DESCRIPTION Throughout the following discussion, like element numerals are intended to refer to like elements, unless specifically indicated otherwise.

The invention(s) described herein can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment containing both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

Furthermore, the invention(s) can take the form of a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk - read only memory (CD-ROM), compact disk - read/write (CD-R/W) and DVD.

A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.

Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers.

Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.

Fig. 1 represents a platform 100 for facilitating integration of various data of a business enterprise. The platform includes a plurality of business processes, each of which may include a plurality of different computer applications and data sources. The platform may include several data sources 102, which may be data sources such as those described above. These data sources may include a wide variety of data types from a wide variety of physical locations. For example, the data source may include systems from providers such as such as Sybase, Microsoft, Informix, Oracle, Inlomover, EMC, Trillium, First Logic, Siebel, PeopleSoft, IBM, Apache, or Netscape. The data sources 102 may include systems using database products or standards such as IMS, DB2, ADABAS, VSAM, MD Series, UDB, XML, complex flat files, or FTP files. The data sources 102 may include files created or used by applications such as Microsoft Outlook, Microsoft Word, Microsoft Excel, Microsoft Access, as well as files in standard formats such as ASCII, CSV, GIF, TIF, PNG, PDF, and so forth. The data sources 102 may come from various locations or they may be centrally located. The data supplied from the data sources 102 may come in various forms and have different formats that may or may not be compatible with one another.

Data targets are discussed later in this description. In general, these data targets may be any of the data sources 102 noted above. This difference in nomenclature typically denotes whether a data system provides data or receives data in a data integration process. However, it should be appreciated that this distinction is not intended to convey any difference in capability between data sources and data targets (unless specifically stated otherwise), since in a conventional data integration system, data sources may receive data and data targets may provide data.

The platfonn illustrated in Fig.l also includes a data integration system 104. The data integration system may, for example, facilitate the collection of data from the data sources 102 as the result of a query or retrieval command the data integration system 104 receives. The data integration system 104 may send commands to one or more of the data sources 102 such that the data source(s) provides data to the data integration system 104. Since the data received may be in multiple formats including varying metadata, the data integration system may reconfigure the received data such that it can be later combined for integrated processing. The functions that may be performed by the data integration system 104 are described in more detail below. The platform 100 also includes several retrieval systems 108. The retrieval systems 108 may include databases or processing platforms used to further manipulate the data communicated from the data integration system 104. For example, the data integration system 104 may cleanse, combine, transform or otherwise manipulate the data it receives from the data sources 102 such that a retrieval system 108 can use the processed data ϊo produce reports 110 useful to the business. The reports 110 may be used to report data associations, answer complex queries, answer simple queries, or form other reports useful to the business or user, and may include raw data, tables, charts, graphs, and any other representations of data from the retrieval systems 108.

The platform 100 may also include a database or data base management system 112. The database 112 may be used to store information temporally, temporarily, or for permanent or long-term storage. For example, the data integration system 104 may collect data from one or more data sources 102 and transform the data into forms that are compatible with one another or compatible to be combined with one another. Once the data is transformed, the data integration system 104 may store the data in the database 112 in a decomposed form, combined form or other form for later retrieval.

Fig. 2 is a schematic diagram showing data integration across a plurality of entities and business processes of a business enterprise. In the illustrated embodiment, the data integration system 104 facilitates the information flowing between user interface systems 202 and data sources 102. The data integration system 104 may receive queries from the interface systems 202, where the queries necessitate the extraction and possibly transformation of data residing in one or more of the data sources 102. The interface systems 202 may include any device or program for communicating with the data integration system 104, such as a web browser operating on a laptop or desktop computer, a cell phone, a personal digital assistant ("PDA"), a networked platform and devices attached thereto, or any other device or system that might interface with the data integration system 104.

10 For example, a user may be operating a PDA and make a request for information to the data integration system 104 over a WiFi or Wireless Access Protocol/Wireless Markup Language ("WAP/WML") interface. The data integration system 104 may receive the request and generate any required queries to access information from a website or other data source 102 such as an FTP file site. The data from the data sources 102 may be extracted and transformed into a format compatible with the requesting interface system 202 (a PDA in this example) and then communicated to the interface system 202 for user viewing and manipulation. In another embodiment, the data may have previously been extracted from the data sources and stored in a separate database 112, which may be a data warehouse or other data facility used by the data integration system 104. The data may have been stored in the database 112 in a transformed condition or in its original state. For example, the data may be stored in a transformed condition such that the data from a number of data sources 102 can be combined in another transformation process. For example, a query from the PDA may be transmitted to the data integration system 104 and the data integration system 104 may extract the information from the database 112. Following the extraction, the data integration system 104 may transform the data into a combined format compatible with the PDA before transmission to the PDA. Fig. 3 is a schematic diagram showing an architecture for providing data integration for a plurality of data sources 102 for a business enterprise. An embodiment of a data integration system 104 may include a discover data stage 302 to perform, possibly among other processes, extraction of data from a data source and analysis of column values and table structures for source data. A discover data stage 302 may also generate recommendations about table structure, relationships, and keys for a data target. More sophisticated profiling and auditing functions may include date range validation, accuracy of computations, accuracy of if-then evaluations, and so forth. The discover data stage 302 may normalize data, such as by eliminating redundant dependencies and other anomalies in the source data. The discover data stage 302 may provide additional functions, such as drill down to exceptions within a data source 102 for further analysis, or enabling direct profiling of mainframe data. A non-limiting example of a commercial embodiment of a discover data stage 302 may be found in IBM's WebSphere ProfϊleStage product. The data integration system 104 may also include a data preparation stage 304 where the data is prepared, standardized, matched, or otherwise manipulated to produce quality data to be later transformed. The data preparation stage 304 may perform generic data quality functions, such as reconciling inconsistencies or checking for correct matches (including one-to-one matches, one-to-many matches, and deduplication) within data. The data preparation stage 304 may also provide specific data enhancement functions. For example, the data preparation stage 304 may ensure that addresses conform to multinational postal references for improved international communication. The data preparation stage 304 may conform location data to multinational geocoding standards for spatial information management. The data preparation stage may modify or add to addresses to ensure that address information qualifies for U.S. Postal Service mail rate discounts under Government Certified U.S. Address Correction. Similar analysis and data revision may be provided for Canadian and Australian postal systems, which provide discount rates for properly addressed mail. A non-limiting example of a commercial embodiment of a data preparation stage 304 may be found in IBM's WebSphere QualityStage product.

The data integration system may also include a data transformation stage 308 to transform, enrich and deliver transformed data. The data transformation stage 308 may perform transitional services such as reorganization and reformatting of data, and perform calculations based on business rules and algorithms of the system user. The data transformation stage 308 may also organize target data into subsets known as datamarts or cubes for more highly tuned processing of data in certain analytical contexts. The data transformation stage 308

11 may employ bridges, translators, or other interfaces (as discussed generally below) to span various software and hardware architectures of various data sources and data targets used by the data integration system 104. The data transformation stage 308 may include a graphical user interface, a command line interface, or some combination of these, to design data integration jobs across the platform 100. A non-limiting example of a commercial embodiment of a data transformation stage 308 may be found in IBM's WebSphere DataStage product.

The stages 302, 304, 308 of the data integration system 104 may be executed using a parallel execution system 310 or in a serial or combination manner to optimize the performance of the system 104.

The data integration system 104 may also include a metadata management system 312 for managing metadata associated with data sources 102. In general, the metadata management system 312 may provide for interchange, integration, management, and analysis of metadata across all of the tools in a data integration environment. For example, a metadata management system 312 may provide common, universally accessible views of data in disparate sources, such as IBM's WebSphere ODBC MetaBroker, CA ER win, IBM WebSphere ProfileStage, IBM WebSphere DataStage, IBM WebSphere QualityStage, IBM DB2 Cube Views, and Cognos Impromptu. The metadata management system 312 may also provide analysis tools for data lineage and impact analysis for changes to data structures. The metadata management system 312 may further be used to prepare a business data glossary of data definitions, algorithms, and business contexts for data within the data integration system 104, which glossary may be published for use throughout an enterprise. A non-limiting example of a commercial embodiment of a metadata management system 312 may be found in IBM's WebSphere MetaStage product. Fig. 4A shows a layout for a user interface. The user interface may be presented as, for example an interface 5200, which may have a layout including a header 5202, a sidebar 5204, a footer 5206 and a main section 5208, all of which may be displayed within, for example, any of the interface systems 202 described above, or any other suitable client device or terminal. The interface system 202 may use, for example, any conventional web browser technology, such as Microsoft Internet Explorer or Netscape, with Java or other client-side applets or the like providing enhanced functionality within the interface, or the entire interface, or portions thereof, may be controlled by other client-server or peer-to-peer communication technologies and/or rendered locally using applications or other software running on a client device. The interface may also, or instead, be deployed as a rich client, and may be strongly separated from underlying functionality using, for example, model-view-controller design concepts. The various features of the interfaces described below may include any known media, including animation, sound, music, graphics, and text. Furthermore, while HTML pages are a useful starting point for discussing features of a user interface, it will be appreciated that the user interface may not employ HTML at all, and/or may be generated and interpreted using many known programming languages or mark-up languages, and any associated plug-ins or add-ons, including HTML, SHTML, XML, DHTML, JavaScript, Java, Perl, PHP3, CGI, C, and C++, all of which may be used with the interfaces discussed herein. As used in the following discussion, the term

"interface" refers to any user interface, however rendered, whether or not it employs HTML, and whether or not it is provided over a network.

The header 5202 may include, for example, a title of the page, and one or more links to other pages or resources displayed as hyperlinks, labeled tabs, or the like. The sidebar 5204 may include a menu of choices for a user of the interface 5200. The footer 5206 may include information concerning the page such as a "help" feature, status information for processes, and so on. The main section 5208 may display a user workspace for content such

12 as models, design information, and so on, as described more generally in the following sections. The main section 5208 may also include, for example, tools for adding, deleting, or modifying information, such as data objects or jobs within the section 5208, managing and displaying objects, and so forth. In addition to the various controls described below, the page 5202 may employ known user interface controls either within the interface 5200 or in supplemental dialogue boxes, such as drop-down menus, check boxes, text boxes, sliders, radio buttons, and the like.

The interface 5200 may track user activities and user information. Using, for example, cookies or other client-side data stores, the system may maintain a degree of persistence between the system and a particular user or a particular client device. Content displayed within the interface 5200 may be determined in whole, or in part, by a user's previous activities (aside from persistent data, such as models that have been stored). For example, where an authorization scheme is employed, previous logged-in sessions may be used to determine a starting point within the interface 5200 of the application, such as a specific point within the design workflow described below. Or, where a web browser model is employed, a cookie may be employed to recall a previous state of the browser. A number of registration/authorization schemes and technologies are known in the art, and may usefully be employed with the systems described herein to manage access to data and other system resources through the interface 5200.

Different interfaces may be provided for different users of the system. For example, a designer may have full access to tools and functions provided by the interface. By contrast, a business analyst may view data using read-only access to all or some of the data stored in the databases, or may have a simplified tool set for a limited range of design activities. A manager may have write access to some or all of the data, in addition to read access, so that data can be created, updated, or viewed. Each level of access and use of data may be realized by providing a different interface with a different perspective on a common underlying data set, such as metadata and metadata models used within the metadata management systems described herein.

It will be appreciated that the description above is generic, and may be varied according to where a user is within a user environment related to the interface, as well as according to any available information about the user interface system 202 (such as display size, media capabilities, etc.), available information about the user of the user interface 202 (such as skill level, job title), or authorization to view or edit certain information. More generally, any layout, or combination of layouts suitable for inputting, editing, deleting, and/or otherwise managing objects and relationships within a data integration system may be usefully employed with the interface system 202.

Fig. 4B shows a user interface 5200 adapted for use in a data integration system. As depicted, the user interface 5200 may include workflow tools 5212, menu items 5214, tool buttons 5218, a search function 5220, one or more window scroll bars 5222, and a workspace 5224.

The workflow tools 5212 may be designed around a workflow, such as an investigate, design, develop, deploy, operate ("IDDDO") workflow useful in working with data integration systems such as the data integration systems 104 described above, or implementing other enterprise initiatives. The tools may be arranged in a pillar menu where a group of icons represent each phase of the IDDDO workflow, and moving a cursor over one of the icons presents a drop-down tree or "pillar" of useful tools, objects, and other resources useful at that stage in the workflow. Aspects of the pillar menu will be described in greater detail in the following figures. In Fig. 4B, a "home" icon of the pillar menu is selected, with access to system-wide functions such as navigating among open workspaces and global copy, paste, delete, rename, and cut functions. Other user interface controls may be provided within conventional drop-down menus 5214 or buttons

5218. The drop-down menus 5214, for example, may provide access to functions that are grouped according to file,

13 edit, view, tools, and help. The buttons 5218 may provide immediate access to commonly used tools, which may overlap with tools provided in the drop-down menus 5214, the workflow tools 5212, or elsewhere. A search function 5220 may be provided for searching the contents of one or more workspaces, data sources, or other areas within a system. Scroll bars 5222 may be provided to scroll through the workspace 5224, The workspace 5224 may display any relevant content, such as content that is generic to a user environment, specific to a user, specific to a phase of the workflow, specific to a project within the workflow environment, or any combination of these. The workspace 5224 may be populated with data or content from various sources within the system such as message queues, data sources, e-mail accounts, task monitors, resource monitors, and so forth. The workspace 5224 may display status information for processes such as data integration services, functions, or jobs, running on the system. The workspace 5224 may also provide controls, such as any of the controls described above, for taking actions relating to any of the foregoing, which controls may be global, or specific to a source for the item being displayed, or specific to a phase within the workflow. In one aspect, the workspace 5224 may enable collaboration by providing messaging services between users, along with the ability to provide links to various initiatives or sub-parts of initiatives within the system. The system may advantageously provide for persistence of data between various phases of the workflow, so that a user may transition forward and backward within the workflow, and between tasks in a particular phase, without loosing information or data relating to a project. At the same time, changes within one phase may have repercussions within other phases, and the results may be readily observed by navigation among the phases, such as through use of the icons within the pillar menu. Several views of a user interface for use with an IDDDO workflow will now be described in greater detail.

In general, within each phase of the workflow, a user may introduce data or content, or retrieve data from the computer environment (such as the data integration system 104 or enterprise computing system) about the corresponding phase in the workflow. The data may be displayed within the workspace 5224 of the user interface 5200, or as appropriate, within windows or frames in the workspace 5224, or in dialogue boxes or drop-down menus associated with the workspace 5224 or the user interface 5200. Through the workspace 5224 and the associated tools and controls 5212-5222, a user may provide input to the current workflow phase, which input may be transmitted back to resources within the computing environment.

Thus, as a significant advantage, a user interface as described herein provides a work environment where changes are persistent and consistent across a number of phases in a workflow, while also permitting a user to navigate seamlessly among the phases of a staged workflow from a global interface. Such an interface may be advantageously employed in complex hardware and software environments, particularly heterogeneous and/or networked environments, such as might be found in the data integration systems 104 and enterprise computing systems described above.

The example below will illustrate a design of a user interface to for an IDDDO data integration design process. It should be appreciated that, while a specific, detailed interface is described, the system may be more generally characterized as a realization of data integration as a navigation methodology. That is, aspects of an arbitrary design process may be explicitly expressed within the interface to facilitate designs according to the process. From this perspective, a workflow such as an industry accepted design methodology may serve as the architecture for the user interface. More generally, any process or result dictated by external constraints such as legal statutes, industry standards, protocols, or data structures may suggest an architecture for a user interface that

14 logically and seamlessly leads a designer to a compliant result within the interface. Similarly, internal constraints such as a business initiative within a corporation may suggest an architecture for the user interface.

Further enhancements may be provided in this context. For example, by modeling the interface around a known workflow, the interface may generate one or more recommendations for a next step, which may be displayed, for example as an item or list within a region at the bottom of the workspace 5224. This technique may be further refined by adapting to a history of user activities (either by the current user, by all users, or some combination of these) from the current location within the workflow to provide more refined recommendations at each point in the workflow. The recommendations may be dynamically updated as the user navigates through the workflow environment. Similarly, context-sensitive help may be provided for assistance specific to a users location within the workflow. More generally, the interface may be context-aware, and may provide any number of forms of assistance or feedback to the user according to the user's location within the workflow.

Fig. 4C depicts a task matrix for the design of a user interface. The user interface may advantageously be strongly separated from underlying functionality in order to maintain design flexibility. In such an architecture, the tasks dictated by a workflow may be organized within a task matrix 5250 that is interpreted to provide an interface to the user. The task matrix 5250 may include one or more contexts 5252, each including a number of tasks 5254 and a number of menus 5256 (the "pillars" described in the interface below).

The one or more contexts 5252 may relate to, for example, a number of different presentations of the tasks within the menus. Through the use of a number of separately defined contexts, the interface may be designed with a variety of optional presentations, such as skill -level sensitive presentations or security-level sensitive presentations. More generally, by providing a number of optional contexts, an additional dimension of design flexibility is realized within the task matrix 5250. This may be used in a number of ways. As an example, a health care provider and an insurer may both be responsible under a regulatory scheme such as HIPAA for maintaining internal records and transacting with others in a specific manner. In this case, the provider and the insurer may have a number of tasks in common, such as encrypting personal identification information or using a common format for a request for payment that is sent by the provider to the insurer. Data integration processes for the insurer and the provider may share a substantial number of tasks in common, while also requiring significant differences. Two contexts 5252 may be used for the provider and the insurer to define two different workflows based upon the same set of common tasks under a more general HIPAA-compliant interface definition.

Each task 5254 may be any number of user-defined tasks or common tasks, functions, services and the like provided by the system, or combinations of such tasks, functions, and/or services combined into a single useful task relating to a workflow. The tasks 5254 may also include, or be associated with, dialogs, wizards, help windows, and the like useful within the interface. The tasks 5254 identified in the task matrix 5250 may be predefined with respect to, e.g., their location within the interface, association with certain control regions of the interface, the controls, inputs, and outputs associated with the task, and so on. Thus it is simply necessary to indicate the presence of the task in the task matrix 5250 for the task to be accessible through the user interface. Optionally, the intersection of each task 5254 and menu 5256 may include one or more specifications relating to the occurrence of the task 5254. Thus, the task matrix 5250 may specify visual elements, controls, location, inputs, outputs, skill- level parameters, and so on.

Each menu item 5256 may correspond to a phase of a workflow. Using the task matrix 5250, the relevant tasks for a workflow may be realized at appropriate locations within the user environment.

15 To further assist in rapid deployment of workflow-based interface designs, the user interface itself may be defined as a collection of visual styles, controls, control panels, workspaces, tools, wizards, dialogs, and so on. At the same time, the tasks may be defined, for example, as services within a service-oriented architecture. Thus, through the use of the task matrix 5250 an arbitrary combination of menu items 5256, each with one or more supporting tasks 5254, may be conveniently arranged to express any data integration or other workflow as a navigation methodology.

The interface may advantageously operate against a common data set, so that changes are persisted among various phases (i.e., menus) of the workflow, and between tasks within a phase. Each task 5254 and each menu 5256 may represent a different functional perspective on the same data set. Thus, the architecture may provide a significant improvement over existing application-based environments in which a workflow passes data from application to application for different phases of the workflow.

While a specific construct has been identified for organizing tasks within a workflow, it should be appreciated that any number of data structures or other storage mechanisms may be used to store, retrieve, and modify definitions of user interfaces, and it should further be appreciated that the architecture of the user interface itself may be defined at various levels of specificity, according to the desired trade-off between flexibility and ease of design.

Fig. 5 shows an investigate phase in a workflow user interface. By selecting an investigate icon 5302 within the user interface 5300, which may be any of the user interfaces 5200 described above, a user may access a number of investigation-related functions and objects, such as through a pillar menu 5304, in connection with implementing an enterprise initiative. For example, the pillar menu 5304 may include selectable items for systems, users, specification, components, data model, information, information model, reports, flows, and any of the global tasks noted above, and all items may be contextual, that is adapted to the current pillar or workflow phase.

In the investigate phase, available systems may include any systems within the computer environment, including repositories such as message queues, directories, files, and databases. Systems may include servers such as DBMS servers, Transaction Coordinator servers, web servers, application servers, machine configuration servers, message queue servers, execution configuration servers, or security model servers. Systems may also include networks, such as local networks, remote networks, and cluster configurations within networks. When a system is selected, information relating to the selected system may be displayed within the workspace or in a separate window, where a user may investigate characteristics, configurations, and resources of the selected system. In certain configurations, the user with adequate authority on the system may reconfigure or otherwise alter systems through the user interface 5300.

Users may include system users having user identities within the computing environment, such as authorized employees, as well as external users such as vendors or other providers, and customers, consumers, or other external users. When a user is selected, information relating to the selected user may be displayed within the workspace or in a separate window. For each user or user type, an operator of the user interface 5300 may view (and possibly edit) role data that defines a user's roles and authority within the system, as well as identification information such as name, address, e-mail, phone, fax, and the like.

Specification may include any information related to specifying a project within the workflow management environment, such as processes and rules describing the project. Processes may include external applications or internal business processes. Rules may include business rules, configurations, or metrics.

16 Components may be software objects and tools available to the system that may be used in various phases of the workflow, such as scripts (e.g., shell scripts and filters), tools (e.g., metabrokers, metadata importers, and modeling tools), containers (e.g., custom components, shared containers, and existing jobs), services (e.g., web services and external routines), macros (e.g., OSH scripts, data filters, shell scripts, and SQL queries), connectors (e.g., files, EAI connectors, message queues, metabrokers, attached tables, import utilities, custom adapters, web service connectors, message switches, databases, applications, and analysis servers), functions, and routines. A user may select components to review more detailed information about the availability, function, and interface of each. Data models may be models for data sources within the system. Data models may include definitions of data type, structure, format, and any other metadata and the like for data sources, along with an identification of each specific data source available. Through the data model item, a user may browse data sources and corresponding models for data available to be used within phases of the workflow.

Information maybe any information relevant to a project, such as documentation (e.g., specifications, manuals, online help, online documentation, and so on), messages (e.g., message formats, notifications, and subscriptions), data (e.g., column analysis, cross table analysis, and table analysis), metrics (e.g., security, availability, goals, quality, and performance), and metadata (e.g., for objects or models). Information provided through this aspect of the user interface 5300 may be available to a user throughout the phases of a project. Similarly, an information definition may be provided for the resulting design.

Reports may be any reports useful for monitoring progress of the design, or operating data integration services or other processes. For example, investigation reports may include word investigation, pattern reports, character discreet, character concatenate, or delete options. Analysis reports may include general reports, table analysis, normalization, column analysis, relationship analysis, and exception reports. Metric reports may include various chart or statistical reports. Through this user interface feature, a user may select and configure various reports to be used during the workflow.

Flows may include data flows (e.g., JCL flow, batch ETL flow, and RTI flow), services (e.g., RTI flow or external flow), and process flow (e.g., BPM flow or job sequence flow). Through this aspect of the user interface 5300, a user may investigate existing data and process flows within the system.

More generally, the features noted above may be used to investigate the scope and content of a system that will be the subject of the workflow, and to characterize various sources, resources, services, tools and the like within the system. Fig. 6 shows a design phase in a workflow user interface. By selecting a design icon 5402 within the user interface 5400, which may be any of the user interfaces 5200 described above, a user may access a number of design-related functions and objects, such as through a pillar menu 5404, associated with the design of an enterprise initiative. For example, the pillar menu 5404 may include selectable items for users, reports, flows, systems, specification, components, and information, as well as the global tasks noted above. In general these items may be as described above, with some exceptions such as those described below that relate more explicitly to the actual use of various items in the design phase.

For example, under the information item, data may include table schema, data schema and domains, and target schema. Also under information, metadata may be more specifically characterized in terms of process definition, operational definitions, rule definitions, system definitions, data definitions, component definitions, model/object definitions, message definitions, table definitions, and so on. Similarly, specifications may explicitly include mapping specifications.

17 More generally, the features noted above may be used to design data integration processes, jobs, services, functions, and the like within a data integration system, or still more generally to design any other enterprise initiative within a computer environment.

Fig. 7 shows a develop phase in a workflow user interface. By selecting a develop icon 5502 within the user interface 5500, which may be any of the user interfaces 5200 described above, a user may access a number of development-related functions and objects, such as through a pillar menu 5504, associated with the development of an enterprise initiative or project. The pillar menu 5504 may include selectable items for flows, components, specifications, systems, and information, along with any of the global tasks, such as tasks related to data integration jobs, extraction, transformation, loading, performance of real-time integration services, or the like as described in this disclosure. In general, these items may be as described throughout this disclosure, with some exceptions such as those described below that relate more explicitly to the actual use of various items in the develop phase.

For example, the specification in the develop phase may include processes and rules, as described above, but may also include templates for the project, such as JCL templates, documentation templates, and job templates that are to be deployed with the project. As another example, within the develop phase, data may be characterized as information that is defined in terms of reference or lookup tables and data sets.

More generally, the features available in the develop phase may be used to develop a data integration process, job, service, function, or the like within a data integration system, or still more generally to develop any enterprise initiative within a computer environment.

Fig. 8 shows a deploy phase in a workflow user interface. By selecting a deploy icon 5602 within the user interface 5600, a user may access a number of deployment-related functions and objects, such as through a pillar menu 5304. The pillar menu 5604 may include selectable items for flows, systems, components, and information, as well as any of the global tasks noted above. In general, these items may be as described above, with some exceptions such as those described below that relate more explicitly to the actual use of Various items in the develop phase. In the deploy phase, the selectable systems item, for example, may relate to systems as required and/or defined for deployment. As such, the items may include tools (e.g., development tools, installation tools, administration tools, services and configuration tools), repositories (e.g., source control, databases, files, message queues, and directories), servers (e.g., security model, execution configuration, and machine configuration), and environment (e.g., disk resources, roles, CPU resources, projects, groups, and parameters). It will be noted that the "systems" as defined for deployment specifically describes an environment and platform onto which processes might be deployed.

More generally, the features available in the deploy phase may be used to deploy a data integration process, job, service, function, or the like within a data integration system, or still more generally to deploy any other enterprise initiative within a computer environment. Fig. 9 shows an operate phase in a workflow user interface. By selecting an operate icon 5702 within the user interface 5700, a user may access a number of operation-related functions, such as through a pillar menu 5704. The pillar menu 5704 may include selectable items for flows, systems, specification, components, and information, as well as any of the global tasks described throughout this disclosure. In general, these items maybe as described herein, with some exceptions such as those described below that relate more explicitly to the actual use of various items in the operate phase.

18 For example, in the operate phase the specification for the process is in terms of processes and rules. A full array of reports may also be provided, consistent with actual operation of the process or project, such s statistical reports, data lineage reports, metrics, audit reports, charts, domain analysis, metadata reports, analysis charts, exception reports, transcripts, logs, impact analysis, and trend charts. More generally, the features available in the operate phase may be used to operate a data integration process, job, service, function, or the like within a data integration system, or still more generally to operate any other enterprise initiative within a computer environment.

Figs. 1OA and 1OB shows a navigation tool for use with a user interface. The user interface 5800, which may include any of the user interfaces and user interface systems 202 described above, may include a text display of a hierarchy 5802, one or more buttons 5804-5810 or other controls, an object pallet 5812, a workspace 5814, and one or more objects 5820 displayed within the workspace 5814.

The text display of hierarchy 5802 may provide an indication, in textual format, of the relative position of objects 5820 within a hierarchy, such as by showing a path from a root object to a current object, or between two user-selected objects. Each object 5820 may represent a data item, data structure, data source, function, tool, or other resource available within the data integration system 104. It should also be appreciated that the objects 5820 may represent other entities, such as people, services, business units, business functions, projects, plans, or other entities within and enterprise, as well as entities outside the enterprise such as vendors, customers, suppliers, regulatory bodies, and so on. The objects 5820 may also, or instead, correspond to physical objects such as computer terminals, mainframes, routers, servers, repositories, resources, services (including web services), modules, databases, printers, and/or other computer resources. In general, objects 5820 within the user interface 5800 may represent anything that may have a relationship with other items, objects, entities, or the like.

The one or more buttons 5804-5810 or other controls may provide access to various tools within the user interface 5800. For example, a control panel button 5804 may provide access to a control panel for the interface, that may display descriptive information (e.g., hops or levels within the hierarchy between selected objects, or between the current object and a root object), or may be used to control how many degrees of relationship are displayed within the workspace (e.g., all objects related to the current object by two or fewer hops, three or fewer hops, etc.). The control panel may also provide "skins" or other user-customizable components for controlling visualization of objects within the workspace 5814, such as color, shape, and so on. The control panel may similarly provide control over a variety of other visual and substantive features of objects displayed within the workspace 5814. The center button 5808 may be used to center the display of objects 5820 within the workspace

5814 around the current object. This feature may be of particular value when navigating large, complex hierarchies. The clear button 5810 may be used to clear the workspace 5814.

The object pallet 5812 may contain generic object forms for addition to the workspace 5814, such as through conventional drag-and-drop or double-click manipulation with a cursor. The object pallet 5812 may also include predefined objects, such as objects already within the workspace 5814 that may be copied, or predefined object types, such as a "person" or "business unit". Predefined objects may contain a number of predefined relationships to other objects. For example, by dragging a "person" object type into the workspace 5814, the user interface 5800 may automatically generate a number of related objects such as a name, an address, a social security number, an employer, a supervisor, and so on, and relate these new objects to the person object in the workspace 5814. Optionally, by dragging a predefined person (e.g., John Doe) onto the workspace 5814, the user interface 5800 may automatically generate known relationships to other objects 5820 within the workspace 5814.

19 The workspace 5814 can provide a working area for adding, removing, and editing objects 5820. Within the workspace 5814, a user may, for example, navigate through a hierarchy of objects 5820 by selecting new current objects from among the objects 5820 visible within the workspace 5814. A search tool may also be provided for searching for objects. Within the workspace 5814, a current object may be edited, and details for an object may be reviewed, such as by clicking on the object. Relationships between objects may be added, deleted, or modified. More generally, the user interface 5800 may provide any tools or controls that a user might usefully employ to navigate, create, or modify hierarchical relationships among objects.

In the workspace 5814, visual cues may be provided for the relationships among the objects 5820. For example, the current object may appear in a foreground, while other objects may be projected at various distances in the background, for example according to their relative distance to the current object in the hierarchy. By adjusting the size, shape, and color, a visual representation of a representation of a three-dimensional distance to the current object may be created within the workspace 5814. Other visual cues to relationship, such as opacity and two- or three-dimensional distance may be further employed to reveal associations and relative distance within a hierarchy between objects. In this manner, the hierarchy among objects 5820 may be displayed as a series of projections of a three dimensional surface, such as a spherical surface or a hyperbolic surface where the current object is displayed larger and in greater detail, while progressively more remote objects are displayed as progressively smaller with less detail. Other hyper-dimensional representations may be employed to visually emphasize distance among objects.

Thus, for example, Fig. 1OB depicts a view of another current object 5822 within the same hierarchy depicted in Fig. 1OA. It will be noted that the current object 5822 has more numerous close relationships to other objects both below and above the current object 5822 within the hierarchy. The workspace 5814 may be animated so that transitions between different current objects are illustrated by moving objects 5820 within the workspace 5814, both in terms of X-Y position and in terms of a simulated Z position as described above. The animation may, for example, rotate a user point-of-view about the three-dimensional or hyper-dimensional object, such as to bring the current object 5822 into the center of the workspace 5814. In embodiments, as an object is brought to the center of the workspace 5814, other attributes of the object may be modified to enhance the view of the object; for example, the object may be increased in size, provided in a darker or more prominent font, placed on bold, provided with a more intense, saturated and/or vivid color, provided with greater opacity, provided with a pattern, or enhanced with other visual cues, such as legends or markings, to provide the user with a sense that the object is the current focus of attention. Similarly, as the hyper-dimensional projection rotates to place the particular object in the center of the workspace, other objects 5820 can be rotated out of the center of the workspace 5814. As the other objects 5820 are rotated away, they can be made smaller, more transparent, less colorful, less saturated, less intense, or the like, and markings or legends can be removed or changed to reflect the relative distance of the object from the user's focus. Thus, the user can see objects that are related to the object of current focus, but the object of focus is provided with visual cues to assist the user in focusing on that object. If the user wishes to turn focus to another object, the user can click on the other object (or move the cursor over it, or the like), so that the other object moves "forward" (enlarging) and toward the center, and the previous object recedes away and shrinks, but remains in view, preserving the opportunity to shift focus back and forth between objects without losing the connection between the objects in the overall hierarchy of objects that is represented within the multi-dimensional space of the workspace 5814. In embodiments, multiple objects, such as those at the same level of a hierarchy may be presented in enhanced format, rather than just a single object.

20 Other functions may be provided that are not specifically illustrated in Fig. 1OA. For example the user interface 5800 may provide drop-down menus or other controls to save, open, delete, and otherwise manage hierarchies as files. The user interface 5800 may provide access to help functions, tools, formatting and display functions, printing functions, and any other tools or functions useful for exploring and editing hierarchies, or investigating resources that may be associated with hierarchies. Other tools may be provided within the user interface 5800 for a user to associate objects within the current hierarchy with existing data items, data structures, metadata, and the like available within the system.

As an example of another enhancement to the user interface 5800, the user interface 5800 may include a skill-level sensitive option. For example, a sophisticated user may view all available functions within any phase of the process, while a beginning user may view only a subset of available functions or a subset of available phases, with additional functions optionally available upon user request.

As yet another example, the user interface 5800 may be adapted for specific tasks, such as an ETL process for data conforming to regulations (e.g., the Health Insurance Portability and Accountability Act ("HlPAA") or Graham-Leach-Bliley ("GLB")) or industry standards (e.g., Electronic Data Interchange ("EDl") or Society for Worldwide Interbank Financial Telecommunication ("SWIFT"). While EDI and SWIFT designate specific protocols for data exchange, HIPAA more generally provides minimum standards for storing and transferring data among covered entities. The environment may be complex and require communication among multiple actors. For example, in a HIPAA environment, a number of communications take place between health care providers and insurers, including eligibility inquiries, descriptions of services provided, claims for payment, and payment. In these environments, the user interface 5800 may provide access to pre-built metadata models, connectors, objects, hierarchies and the like for rapid prototyping and deployment of jobs and processes in a data integration environment that conform to the external standards.

It will be appreciated that the user interface 5800 described above advantageously provides a set of tools and controls specifically adapted to managing objects and relationships, so that a user may more easily create, investigate, and modify objects and relationships within large, complex hierarchies such as might be found in a data integration system 104 or an enterprise computing system, or more generally across and enterprise.

This modeling approach has broad utility. While a more detailed description of the application of these tools to modeling of business issues will be provided below, a simple use scenario for the interface 5800 in a business context should quickly illustrate its capabilities. In this example, a business statement is provided from a source within the company. The business statement may be an objective, a philosophy, a current problem or shortcoming, or any other business matter than can be clearly articulated. The business statement may be placed as an object 5820 within the workspace 5814. The object 5820 may have properties such as an owner, a creation date, completion or due date, a textual description of the business statement, and so on. The workspace 5814 may be live, so that it is immediately accessible to all or some group of users within the company. The original creator, or some other user, may add business requirements as new objects 5820 in the workspace 5814, and associate the business requirements with the business statement. Each business requirement may articulate a specific requirement for addressing the business statement. Collaboration may be enhanced through use of messaging so that, for example, if one of the business requirements is assigned to a new owner, the new owner may receive an e- mail or other message alerting her to the new business requirement. She may review the entire context of the business requirement within the workspace 5814. The new owner may respond to notification of the requirement by revising the textual statement, and creating new child objects for the business requirement that set out one or

21 more business processes for meeting the business requirements. Each business process may be further refined with child objects to provide subjects such as definitions, rules, and data mappings relevant to the process. Finally, relevant objects may be added, such as contacts, vendors, consultants, databases, and so on, to accumulate all of the relevant resources within the company and associate them with appropriate objects within the workspace 5814. It will be noted that the collaborative process for defining and refining a business statement occurred above with a clear hierarchy: business statement -> requirements -> processes -> subjects (rules, definitions, mappings). A number of positive results may be immediately noted. The visual modeling provides organizational structure to the business issue. Further, the use of a shared workspace 5814 and messaging permits seamless collaboration among users of a system. As an additional benefit, the textual portions of the resulting structure may be automatically compiled into full documentation of the business initiative. A hierarchy may be strictly enforced by the tools of the user interface 5800, or loosely enforced, or not enforced at all, to permit more creative flexibility in defining business matters.

Fig. 11 shows a tool that provides progressive disclosure of detail in a user interface. The user interface may be, for example, any of the user interfaces 5200 described above. In general a visual container 5900 may be provided within the user interface, and may include a variety of levels of detail, such as a top level 5903, a middle level 5904, and a low level 5910, each selectively exposed under user control, and each providing progressively greater detail concerning objects within the container.

For example, the container may include a top level 5903 view, in which only a title is provided, along with a control 5902 or other active area that permits a user to expand the container 5900 to view greater detail. In a middle level 5904, accessed by operating the control 5902 in the top level 5903, a user may review information relating to the top level, such as a list of objects, properties of those objects, and tools that relate to those objects. For each object, the middle level 5904 may provide a button or other control 5908 or active area to access one or more additional layers of detail, such as the bottom level 5910, or to perform a task on the selected information. By selecting the control 5908, a user may initiate a task that expands the container to display a bottom level 5910, which may include, for example, a list of all tasks associated with an object, or any other form of greater detail concerning the objects. Within the list, additional controls 5914 or active areas may be provided to selectively display atomic properties for an object, or sub-object, task, or subtask thereof. For example, the bottom level 5910 may include drop-down lists 5914, data fields for text entry, or any other controls for selecting associations, defining or redefining properties or attributes, or otherwise changing an object or characteristics thereof.

Thus control over objects and display of related data may be progressively revealed in increasing levels of detail, all under control of a user. An interface 5900 arranged in this manner may advantageously permit full access to all of the data and controls relating to an object within the current interface, while permitting a user to conserve space on a computer desktop when the controls are not needed. This approach may also advantageously reduce stacking of windows or dialog boxes that may use up workspace or otherwise confuse or clutter a user interface 5200.

Fig. 12 shows a page selection tool for use in a user interface. The user interface 6000 may be, for example, any of the user interfaces 5200 described above. Within the user interface 6000, a number of worksheets or pages in a workspace may be identified by tabs 6002, which may be similar to, for example, the tabs used to navigate among worksheets in Microsoft Excel. The user interface 6000 may also provide forward and back

22 controls 6004 for navigating sequentially forward and backward through a number of sequentially arranged worksheets or pages, each represented by a tab. A pages indicator 6008 may indicate the total number of worksheets that are open, and may also, or instead, provide the current page or worksheet number in a format such as "x of y". When a user holds a cursor over the pages indicator 6008, or provides another input, such as hovering the cursor over the controls 6004, the user interface 6000 may provide a drop down menu of all, or some subset, of the worksheets so that a user may select a worksheet for navigation to. In this manner, a user may conveniently navigate among a large number of worksheets that are open in a workspace displayed within the user interface 6000. Fig. 13 shows a tool for recalling a history of previous operations or tasks within a user interface. The user interface 6100 may be, for example, any of the user interfaces 5200 described above. Within the user interface 6100 a history palette 6102 may be provided, which includes a record of previous tasks. The tasks may be at any level within the interface 6100. For example, the task may be to navigate to one of the phases of the workflow process, or, to simply delete or save a current object. The task may specify both navigation and function, for example, by going to the Batch ETL Flow function of the deploy phase and publishing a batch ETL flow. The user may return to this task from any location within the user interface 6100 at any time. Any function within the user interface 6100 may be stored in the history palette, either associated with the current object, or with a particular phase of the workflow, or with a particular item within a particular phase of the workflow. Thus, tasks may be recalled at variable levels of granularity within a user environment. Also, a useful series of operations may be conveniently ported between projects or other enterprise initiatives for improved efficiency.

The user may select from the operations or tasks simply by activating them with a cursor or other user input. The history palette may be undocked so that it floats above a workspace of the user interface 6100. Items within the history may be deleted, and the history palette 6102 may sort task according to frequency of use, current location, or other user preferences. Fig. 14 shows a menu tree for use with a user interface. The user interface may be, for example, any of the user interfaces 5200 described above. In particular, the menu tree 6200 within the user interface, as depicted, is the design pillar menu described above, although it should be appreciated that any menu of items accessible in a user interface may be used with the control described herein.

The menu tree 6200 may include a number of objects or items, each having associated with it a control for selectively viewing additional layers of hierarchically arranged items. For example, the control may be in a closed position 6202, identified here by a horizontally directed arrow, or in an open position 6204, identified here by a vertically directed arrow. By activating the control, such as with a cursor or hot key, a user may toggle the control between the open position 6204 and the closed position 6202. In the open position 6204, another layer of hierarchy may be displayed immediately below the object or item, and in the closed position 6202, no further layers of detail may be displayed. An object or item with no additional layers of hierarchy beneath it may be represented by a different symbol, such as a bullet point 6206. The controls 6202, 6204 may be persistent, so that a user may control the degree of detail displayed to a desired level, such as with specific objects of interest visible, and the menu will retain those settings upon subsequent uses of the drop-down menu.

In addition to the control itself, other visual cues may be provided for the hierarchy of objects or items within the menu. For example, horizontal lines 6208 may explicitly separate transitions between depths of hierarchy. As another example, different colors or degrees of shading may be used for different depths. Thus, for

23 example, areas between horizontal lines 6208 may include progressively brighter shades of a color to indicate progressively higher (or lower) levels of hierarchy. Similarly, different fonts can be used to indicate the progressive levels of the hierarchy.

Arranging access to objects and items in this manner may provide significant advantages in a complex user environment. For example, a user may configure each drop-down menu to provide ready access to commonly used functions, while concealing other objects or items to maintain useable workspace within the user interface.

Fig. 15 shows a status bar for use with a user interface. The user interface may be, for example, any of the user interfaces 5200 described throughout this disclosure.

In a first view 6302 of the status bar within the user interface, the status bar may display a simple progress indicator to indicate that a process is running, such as an animation of a status indicator moving horizontally back and forth within the status bar. The status bar may also display alerts, messages, warnings, progress, and status through appropriate text and icons. The process may be a job, background task, or any other process that might be running in the computer environment associated with the user interface. The status bar may be located at the bottom of a window displaying the user interface. A second view 6304 of the status bar within the user interface may be activated, such as by hovering a cursor over the status bar. In the second view 6304, the status bar may expand and display more detailed status information. For example, in addition to the animated status indicator described above, second view 6304 may include a textual description of an active process, a progress bar and/or text indicated a percentage or degree of completion, and a textual description of the number of processes running. The second view 6304 of the status bar may also include a control for scrolling through additional processes. The status bar, although expanded, may be located at the bottom of the window displaying the user interface.

A third view 6308 of the status bar within the user interface may be activated, such as by hovering the cursor over the second view 6304, or by clicking within the status bar of the first view 6302 or the second view 6304. In the third view 6308, the status bar may expand further and display, for example, a list of all processes along with detailed information such as the start time, the user who started the process, resources used by the process, and any other appropriate information. The third view 6308 of the status bar may be located along a bottom portion of the window displaying the user interface.

Thus, there is disclosed herein a status bar that progressively displays increasing levels of detail under user control. The status bar may advantageously conserve space within the user interface, while always being available for a user.

In embodiments the methods and systems described herein may be used to provide a tool that assists a user in bringing a specific business context to a data integration initiative of an enterprise. Thus, the workflow-based user interfaces described herein may include objects that are relevant to a business context and that are conventionally maintained in separate computer systems and managed through disparate word processing tools, email exchanges and the like. For example, a business analyst may develop a process that requires one or more reports. The analyst conventionally specifies the content of the report to a programmer, who creates the report. Rather than use ad hoc reports, the workflow methods and systems described herein allow an analyst to link objects related to data integration tasks to other processes and hierarchies of an enterprise, such as hierarchies related to plans, processes, decisions, and projects of the analyst. For example, the analyst can link a particular report, such as a sales report, to the data objects required to populate the report (such as data from various sales databases for different regions of the enterprise) as well as to a hierarchy from a business process that relates to sales, such as a

24 process for determining what commissions should be paid in a given calendar quarter or what commissions should be proposed for the next calendar quarter, or the like. By connecting the data integration task to the business workflow, the analyst can more readily see other opportunities to improve either the business workflow, the data integration workflow or both. For example, a business analyst may have mistakenly believed a particular rule was necessary to enforce to ensure that a process was performed correctly and may have embodied that rule in a proposed workflow. If the data integration designer can see the relationships among various business workflows, linked in the user interface as workflows as described herein, the data integration designer can build a data integration workflow that avoids the mistaken constraint, rather than build a workflow that is mistaken because of the lack of understanding of the business context. One recurring problem with using data in a business context is the lack of coordination between the structure of data and the structure of a business problem. In a data model, a tightly coupled hierarchy is maintained among data, and specific sources are identified. In a business context, relationships are typically defined by loosely coupled associations of information, which may or may not form a definite structure or hierarchy. The following Figures 16-20 set out guidelines by which business issues may be more clearly defined and related to data structures.

Fig. 16 shows a map for a subject in a business model. The subject map 6400 illustrates a mapping of a subject, in this case an address, to other subjects within the business layer. The address may have a definition and some set of business rules. The address may also participate in several other hierarchies, such as a customer hierarchy, a vendor hierarchy, and an employee hierarchy. An address may have two defined forks, such as a billing address and a shipping address, although the address itself may be neither of these. Each fork includes its own definitions and rules. The address subject itself may have a number of related subjects, such as a location, an area, and a country. These subjects may in turn have a definition and rules, and have participating subjects, such as an area having a city, state, and/or postal code. The business layer may also have rules, as depicted below, which further complication the identification of information in the business layer. Fig. 17 depicts rules and definitions for a subject in the business layer. In the example of Fig. 17, a billing address 6502 has a definition 6504 and a plurality of business rules 6506. The definition 6504 may, for example, include a textual description of the billing address 6502, and may indicate its distinction from, and relationship to, its higher level subject, an address. The business rules 6506 may, for example, indicate conditions based on the country and other criteria for an address to be valid. These rules may be exclusive or inclusive of one another. The rules may also be inherited by other subjects, particularly subjects dependent on the subject having the rule. For example, the rules of the business address may have a child subject for zip code, which may inherit a business rule of the business address. By contrast, a related subject such as the shipping address may or may not inherit a particular rule.

Fig. 18 depicts a business process in the business layer. In this example, an address verification process 6600 involves the steps of receiving an address 6602, matching the address to a reference 6604, and certifying the address 6606. Each of these steps may have a specific definition of one or more actions to be taken, and each step may be expanded into smaller substeps in a hierarchical or other fashion to fully characterize the process 6600. The steps may include conditional rules, decision points, or other flow control mechanisms. Another example of a business process is householding, the process by which a number of individual are identified as belonging to a single household. This may be accomplished by comparing addresses for a number of people and linking people having an identical address.

25 There are other aspects of the business layer and business modeling not depicted explicitly in Figs. 16-18. One example is a business metric. The business metric, or measure, typically characterizes a degree to which a business rule or process achieves a desired result, but may more generally provide a quantitative measure of any aspect of the business, such as conformance to business rules, results of business processes, and so on. Another example is roles. Roles describe the rules by which individuals interact with other components of the business layer, and may relate to an individual's rights or responsibilities with respect to various components, such as creation, review, approval, editing, and so on. The particular aspects of a role may be defined according to the type of individual and their relationship to the business issue. For example roles may include a business process owner, business analyst, subject matter expert, manager, and so on. Fig. 19 depicts an association of elements between the business layer and the data layer. The universe of information may be represented as a business layer 6702 and a data layer 6704 interrelated by tags 6706. The business layer 6702 may include the business subjects, business processes, business rules, business metrics, business roles, and any other information and corresponding relationships gathered by modeling a business matter as described above. The data layer 6704 is much more concretely structured. Data is always contained within a data source, the source has a defined format, whether relational, hierarchic, flat/sequential, or complex. Each format has low-level components that can be termed fields or columns. Some information can be redefined into larger multi-field components (e.g. Address Line 1, Address Line 2, and Address Line 3 form an Address). Fields and columns may be combined into larger structures that may be single transactions or files or a database table. These larger structures may be contained within a repository such as a database or a file directory. All data requires specific access methods and this information is associated with the data source. Relationships between components of the business layer 6702 and components of the data layer 6704 may be specified using tags 6706. Tags 6706 represent links between any element in the data layer 6704 and any element in the business layer 6702 at any hierarchical level. Tags may be directional, and may include one-to-one, one-to-many, and many-to-one relationships between the layers 6702, 6704. Fig. 20 is a map of linked business and data elements. An information map 6800 shows the association of components of the business layer to components of the data layer using tags. In the example of Fig. 20, a central subject 6802, such as an address, may have relationships to other subjects 6804, such as a billing address and a shipping address that inherit some or all of the elements of the address, as well as sub-elements 6804 that more specifically describe and define various aspects of the central subject 6804, such as a location, and area, and a country. These, of course, may similarly have sub-elements more specifically refining them. Each subject 6802, 6804 may be mapped to a specific database component 6806 through a tag. This may include, for example, an identification of database, a table within a database, a column within a table, a reference database, or any other information specifying a relationship to a database entity or component.

This conceptual framework may be used to associate a business context to a data environment. More specifically, the user interface tools described above, such as with reference to Figs. 4A-10B, may be employed to model a business initiative and map the business initiative to data sources in a user environment that seamlessly transitions between the business layer and the data layer. The following example shows how a data object may be associated with subjects in a business layer model.

Fig. 21 depicts the addition of a data object to a workspace of a user interface. The user interface 6900, which may include any of the user interfaces 5800 or related components described above, may particularly include a menu 6902 of data objects and a workspace 6904, which may be the workspace 5814 described above. The menu

26 6902 of data objects may include any data items, data fields, tables, metadata, data sources and the like known to the interface 6900 from any data sources, such as the data sources 102 described above, within an enterprise or other business or computing entity, and may provide tools for searching and navigating among the data objects. Through the user interface 6900, a user may place a specific data object 6906 in the workspace 6902. Fig. 22 depicts an association of a data object to business objects in a workspace of a user interface. The user interface 7000, which may be any of the user interfaces 6900 described above, may include a workspace 7804 containing a data object 7806 that was placed in the workspace using tools of the user interface 7000. A user of the interface 7000 may also retrieve a model 7808 of a business initiative. Using tools provided by the interface 7000, a user may then associate the data object 7806 with the model 7808 of the business initiative using any relationships that the user chooses to define. As an example, the business initiative may be a marketing company's wish to identify household groups from lists of individual customers. The business initiative may define a business process through which groups of customers with identical addresses are characterized as a family, but only if they have an identical last name. A user may place specific address objects from the company's databases into the workspace 7804 for association with statements, processes or other components of the model 7808 for the business initiative. Once the model 7808 has been more fully associated with a company's database(s), data integration processes may be quickly prototyped and deployed from the resulting information. A more detailed example of developing data integration processes from a business initiative is provided below.

Fig. 23 shows a process for designing a data integration process from a business context. It will be appreciated that the processes described below may be realized in hardware, software, or some combination of these. The processes may be realized in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors or other programmable devices, along with internal and/or external memory for storing program instructions, program data, and program output or other intermediate or final results. It will further be appreciated that these processes may be realized as computer executable code created using a structured programming language such as C, an object oriented programming language such as C#, C++ or Java, or any other high-level or low-level programming language that may be compiled or interpreted to run on any uniform or heterogeneous group of hardware and software platforms, including a computer or computers, networks of computers, and combinations thereof. The processes may also employ a wide variety of tools, platforms and architectures to achieve a scaleable enterprise metadata management system. While specific examples of software tools and platforms have been provided above, other platforms and technologies exist and may be usefully employed with the processes described below. Finally, it should be appreciated that, while a specific example of a business initiative is provided below for purposes of explanation, this example in no way limits the scope of the systems and methods described herein.

The process 7100 may begin by setting up an initiative, as shown in step 7102. As an example, a business analyst may set up an initiative or project in the system by defining a project title, such as "Actuarial UR Project". The analyst may add description, definition, manager, business sponsor, and any other descriptive information for the project. Descriptive data may be captured, for example, through keyboard input, or may be imported from any existing documents or other materials available in electronic or other form. Once the initiative has been set up, the process 7100 may proceed to step 7104.

As shown in step 7104, the initiative maybe broken down into objectives. Continuing with the actuarial project example provided above, an analyst may decompose the business initiative into specific business objectives

27 using, for example, the hierarchy of business subjects described with reference to the business layer above. The resulting decomposition may yield the following business objectives: a. Capture 7 years of hospital data b. Capture 3 years of physician data c. Capture 2 years of pharmacy data

For each objective, the business analysts may define a name (title), description (summary), definition (detailed), business owner and any relevant notes. The format of definitions may be controlled by user-defined formats such as templates, which may be employed in the description process to establish a useful and consistent hierarchy for the modeling process. Once the business initiative has been decomposed into objectives, the process 7100 may proceed to step 7016.

As shown in step 7106, objectives may be further decomposed into statements. The business analyst may further de-compose each objective into a set of business statements that are the general requirements needed to satisfy that objective. Continuing with the actuarial example provided above, a breakdown of the first objective may result in the following more specific statement into requirements: a. Capture 7 years of hospital data

1. Identify all hospital data sources

2. Transform hospital data to a common format

3. Validate standardized hospital data.

This business initiative model may be stored as a data structure for subsequent steps of the process 7100, as well as any other use, reuse and examination by users of the system. Once the objectives have been decomposed into specific requirements for meeting the objectives, the process 7100 may proceed to step 7108.

As shown in step 7108, the model of initiative, objectives, and requirements, may be published as a functional requirements document for the initiative. A reporting feature may be provided by the process 7100, through which the functional requirements document may be automatically generated from data collected directly from the model. This may be a collaborative process through which numerous users contribute to and edit both the structure of the model and the underlying descriptions. A review process and an approval process may be conducted through which numerous participants are expressly requested to comment on and approve the document. The process 7100 may automate approvals by collecting approvals from appropriate personnel and ensuring that all needed approvals are obtained for a final version of the requirements document. The resulting requirements document may present the requirements as, for example, a hierarchical tree, an outline, a full textual description, or any combination of these. The functional requirements document may be published, for example, in paper or electronic form. It will be appreciated that publication in electronic form, in particular, may permit rapid dissemination through a business entity, and may provide the functional requirements document in an interactive form for convenient navigation among, and investigation of, the initiative, the objectives, and the requirements, at any desired level of detail. Once the functional requirements have been published, the process 7100 may proceed to step 7110.

As shown in step 7110, an information model may be developed. A business analyst may apply knowledge of an enterprise to develop an information model, a data structure that is an abstract representation of the real world in which the enterprise operates. The model may be built by defining subjects as building blocks that can be related to one another through graphical relationships that primarily consist of "is part of or "is type of. The subjects themselves can be of various types in a range from a very high abstract conceptual level (e.g., "hospital") to

28 a very atomic level (e.g., "type of service") equivalent to individual data elements. For each subject, a user may define a subject name, description (summary) and definition (detailed), business owner and any other relevant or potentially relevant information. The format of definitions may be controlled, for example, by user-defined formats such as templates to facilitate consistency, and in particular, collaboration in the development effort. It will be readily appreciated that user interface tools described generally above may be conveniently employed in this modeling step. Once the information model has been developed, the process 7100 may proceed to step 7112.

As shown in step 7112, requirements may be added to the information model to provide a data structure that describes a business context. A user may attach specific requirements to any subject in the information model to more specifically detail aspects of each. A modeling tool, such as the user interface 5800 described above in reference to Figs. 4A-10, may provide specific object types to facilitate this modeling step. For example, the following types may be defined for the hospital actuarial initiative: Business Requirement (general)

(Type of Service valid code values are "01" - "17" ) Business Rule (Place of Service must be appropriate for Type of Service)

Business Process

(Notice of Admission must precede Inpatient claim) Business Process Step

(Step 1 - Approve Notice of Admission) Business Metric

(Type of Service code must contain 100% valid data)

For each object type, a user such as a business analyst may capture an object name, description (summary), and definition (detailed), business owner and any other relevant or potentially relevant information. The format of definitions may be controlled by user-defined formats such as templates to provide consistency, and more particularly to facilitate collaboration among a number of contributors. Once the information model has been completed, the process 7100 may proceed to step 7114 below.

As shown in step 7114, the information model may be published. A reporting feature may be provided by the process 7100, through which the information model may be automatically documented in textual or other form directly from the information model. A review process and an approval process may be conducted through which numerous participants are expressly requested to comment on and approve the document. The process 7100 may automate approvals by collecting approvals from appropriate personnel and ensuring that all needed approvals are obtained for a final version of the information model. It should be appreciated that, while publication may occur at a specific point in a development process, the information model, or subsets of the model and/or related documentation and annotations, may more generally be published at any time. The model may be presented in a number of forms, such as a hierarchical tree, an outline, a full textual description, or any combination of these. The information model may be published, for example, in paper or electronic form. It will be appreciated that publication in electronic form, in particular, may permit rapid dissemination through a business entity, and may provide the model in an interactive form for convenient navigation among, and investigation of, the subjects, rules, processes, and so on at any desired level of detail. Once the information model has been published, the process 7100 may proceed to step 7116.

29 , As shown in step 7116, the information model may be linked to the business initiative, or requirements and other subcomponents thereof. At this point in the development cycle, the information model should contain a representation of all of the relevant real-world data and any other relevant requirements for subjects in the information model. The applicable data subjects may then be linked to the specific business statements in the initiative to provide detailed specifications for each business requirement of the initiative. Once the applicable areas of the information model have been linked to the initiative, the combined information model forms a formal description of relevant aspects of the business layer discussed above, and may be provided at any level of granularity desired by the architects or appropriate to a particular business initiative. This combined information model may be submitted for review so that any requirement modifications or other appropriate additions, deletions or changes to the initiative, or more generally, the combined information model, can be made. Once the combined information model is complete, the process 7100 may proceed to step 7118.

As shown in step 1118, the combined information model may be published as a functional specifications document. The combined information model of the functional specifications document may include the information model described above as well as the business initiative model that is integrated therewith. A reporting feature may be provided by the process 7100, through which the functional specifications document may be automatically generated in textual or other form directly from the combined information model. A review process and an approval process may be conducted through which numerous participants are expressly requested to comment on and approve the document. The process 7100 may automate approvals by collecting approvals from appropriate personnel and ensuring that all needed approvals are obtained for a final version of the functional specifications document. It should be appreciated that, while publication may occur at a specific point in a development process, the information contained in the functional specifications document, or subsets thereof, along with any related documentation and annotations, may more generally be published at any time. The functional specifications document may be presented in a number of forms, such as a hierarchical tree, an outline, a full textual description, or any combination of these. The functional specifications document may be published, for example, in paper or electronic form. It will be appreciated that publication in electronic form, in particular, may permit rapid dissemination through a business entity, and may provide the document and associated models in an interactive form for convenient navigation among, and investigation of, the subjects, rules, processes, and so on at any desired level of detail. Once the information model has been published, the process 7100 may proceed to step 7120.

As shown in step 7120, the combined information model of the functional specifications document may be linked to real data structures of an enterprise. It will be appreciated that the information model and portions thereof are data structures themselves. However in the following description, the term "data structure" will refer to data structures within an enterprise other than the model(s), such as data structures that might be used in implementing a business initiative or portions thereof as one or more data integration processes. This phrasing is intended to more clearly distinguish between data structures created during the modeling process to describe a business context or initiative, and data structures already existing within the design process 7100 to which the model might apply. The use of these terms in the following description is made without any loss of generality to the associated terms ("model", "information", "data", "data structure", etc.) throughout this description.

Given an approved functional specifications document, a user such as a business analyst can begin the task of linking subjects in the (combined) information model to actual data structures (e.g., tables and columns), which may, for example, be data structures from any of the data sources 102 described above. A user may collaborate with other users, for example where a business analyst collaborates with a data analyst, to identify correspondence

30 between aspects of the information model and aspects of the data structures of the enterprise, or where appropriate, data structures outside the enterprise. The links to data structures may include an identification of a data source, and any data or metadata related thereto, and a defined target data structure that may be used for any data integration process derived from the process 7100. It should be appreciated that the linking of data structures to models, which corresponds generally to the tagging described above with reference to Fig. 19, may occur at a number of different levels of abstraction and detail according to the relative complexity of the data structures and/or the information model. Thus, for example, a subject may correspond to a database, a table of a database, a column within a database, or a data instance within one or more columns, or any combination of any or all of these. In general, the more accurate and detailed the mapping, the easier subsequent development of data integration processes becomes. Once the mapping is complete, the process may proceed to step 7122 below.

As shown in step 7122, the mapping of the (combined) information model to other data structures may be published. Using the links between the information model and the data structures, a mapping model may be produced. This may include documentation of the relationship between the various aspects of the information model and the surrounding environment of enterprise data sources. This may also include more functional documentation such as a mapping of source data to target data. The mapping document may also be, or include, a logical mapping that, for each data source and target, maps a source column to a target column (n to n) to fully characterize a source-target mapping associated with the modeling process. A reporting feature may be provided by the process 7100, through which the mapping model (either the database mapping model, a more general information-to-data mapping model, or any other subset or variation of these) may be automatically generated in textual, graphical, or other form. A review process and an approval process may be conducted through which numerous participants are expressly requested to comment on and approve the document. The process 7100 may automate approvals by collecting approvals from appropriate personnel and ensuring that all needed approvals are obtained for a final version of the mapping model. It should be appreciated that, while publication may occur at a specific point in a development process, the information contained in the mapping model, or subsets thereof, along with any related documentation and annotations, may more generally be published at any time. The mapping model may be presented in a number of forms, such as a hierarchical tree, an outline, a full textual description, or any combination of these. The mapping model may be published, for example, in paper or electronic form. It will be appreciated that publication in electronic form, in particular, may permit rapid dissemination through a business entity, and may provide the document and associated models in an interactive form for convenient navigation among, and investigation of, the subjects, rules, processes, data mappings, and so on at any desired level of detail. Once the mapping model has been published, the process 7100 may proceed to step 7124.

As shown in step 7124, data integration tools may be applied to the mapping model or, stated differently, the model may be applied to data integration tools. The specifications and mappings developed during the process may be transferred to a variety of data integration tools as design processes such as, for example, the processes outlined with respect to Figs. 4A-9. This may include, for example, comprehensive analysis and cleansing of source data, the development of technical specifications for data integration and transformation, and any other tasks relating to the investigation, development, design, deployment, and operation of data integration processes. Similarly, any metadata generated by or identified during the process 7100 may be reused in other business initiatives, particularly initiatives involving similar subject areas and requirements.

31 More generally, modeling techniques and processes 7100 described above may flow directly into the workflow management tools described in Figs. 4A-9 to yield a complete, end-to-end solution for generating data integration jobs, processes, functions, and the like from business initiatives. At the same time, it should also be appreciated that many of the tools and techniques described herein may have separate utility as stand-alone products or processes. As an example, the workflow-based user interfaces described herein may be provided with a subset of the pillars, or a subset of tasks within one or more of the pillars, described above in Figs. 4A-9. Thus, for example, a user interface may provide a business context workflow design workspace for use by non-technical personnel that might include investigate and design pillars, while omitting pillars related to development, deployment or operation. All such combinations, decompositions, and variations to the processes, user interfaces, and systems described above form a part of the inventive disclosure provided herein.

While a specific example of a process 7100 is provided, it will be appreciated that the order of the steps described in the process 7100 may be changed, or other steps may be added or removed from the process, without changing the general objectives or advantages thereof. For example, the decomposition of objectives into specific statements may be revised after the information model is linked to a specific business initiative, or an additional iteration of adding business requirements may be required if the linking of the information model to actual data structures reveals potential refinements to the business initiative, or conversely if the linking of the information model to actual data structures reveals issues that cannot be addressed with currently available data. AU such variations and modifications are intended to fall within the scope of the systems described herein.

Having described a process 7100 for modeling business initiatives, certain data structures and interface features associated with the process 7100 are now described in greater detail.

Fig. 24 shows decomposition of an initiative into objectives, as described for example, with reference to step 7104 of the process 7100 above. A decomposition 7200 may include a business initiative 7202 with characteristics such as a title, description, definition, manager, business sponsor, and any other descriptive information for the initiative. The initiative 7202 may be broken down into any number of objectives 7204 that more specifically characterize the initiative. The structure of objectives 7204 may be fixed by an architect of the initiative 7202, or may permit changes by other users involved in the modeling process. It should be appreciated that, while a simple hierarchy is depicted in Fig. 24, any degree of detail and complexity may be used to decompose a business initiative 7202 into related objectives 7204.

Fig. 25 shows decomposition of an objective into statements, as described for example, with reference to step 7106 of the process 7100 above. A decomposition 7300 may include a business initiative 7302, such as the business initiative 7202 described above, at least one objective 7304, such as the objectives 7204 described above, and any number of statements 7306 more specifically setting out the requirements for meeting each objective 7304. It should be appreciated that, while a simple hierarchy is depicted in Fig. 25, any degree of detail and complexity may be used to decompose a business initiative 7302 into related objectives 7304 and statements 7306. Fig. 26 shows the development of an information model, as described for example with reference to step

7110 of the process 7100 above. The subjects of the information model may correspond, for example, to subjects of the business layer described above, and may be associated in a hierarchical or other structure. The relationship of business subjects may be modeled using, for example, any of the user interfaces 5800 or components thereof, described with reference to Figs. 4A-10B. Fig. 27 shows the addition of business requirements to information model objects, as described for example with reference to step 7112 of the process 7100 above. The subjects 7502 of the information model 7500

32 may be arranged in a hierarchy as generally described above. Other elements of the business layer, such as business requirements 7504, business processes 7506 or substeps 7508 thereof, business metrics 7510, and business rules 7512, maybe created and associated with other elements of the model. The addition of business requirements and other components may be performed using, for example, any of the user interfaces 5800 or components thereof, described with reference to Figs. 4A- 1 OB.

Fig. 28 shows linking of an information model to an initiative. In general the workspace 7600 depicted in Fig. 28 may include any number of elements from the information model (e.g., subjects, requirements, rules, processes) and the business initiative model (e.g., initiative, objectives, statements), along with any other relevant information contained therein. The components of these models, or subsets thereof, may be presented within the workspace 7600 and associated using tools of a graphical user interface, such as any of the user interfaces 5800 described above.

Fig. 29 shows linking of an information model to data structures. As noted above, this linking or tagging, referred to here as a mapping 7700, may be used to specify how aspects of the information model correspond to structures such as tables or columns within a source database 7702, which may be any of the data sources 102 described above. The mapping 7700 may further specify a target database, which may also be any of the data sources 102 described above, and more specifically indicate how structures of the source database 7702 relate to structures of the target database 7704 to permit convenient realization of data integration processes.

While the invention has been described in connection with certain preferred embodiments, it should be understood that other embodiments would be recognized by one of ordinary skill in the art. AU such variations and modifications are intended to fall within the scope of this disclosure.

33

Claims

CLAIMSWhat is claimed is:

1. A user interface for managing workflow in a computer environment, the user interface comprising a menu bar that displays a plurality of controls, each control accessing an interface corresponding to a phase in a workflow, each such interface configured to: retrieve data from the computer environment about the corresponding phase in the workflow; display the retrieved data in the interface; receive user input relating to the corresponding phase in the workflow; and transmit data to the computer environment based upon the user input.

2. The user interface of claim 1 wherein the computer environment includes a data integration system.

3. The user interface of claim 1 wherein the workflow relates to a data integration process.

4. The user interface of claim 1 wherein the workflow includes at least one of a data integration function, a data integration service, or a data integration job.

5. The user interface of claim 1 wherein the phases include one or more of investigate, design, develop, deploy and operate.

6. The user interface of claim 1 wherein the phases include an investigate phase in which one or more data sources in the computer environment are profiled.

7. The user interface of claim 1 wherein the phases include an investigate phase in which metadata is obtained for one or more data sources.

8. The user interface of claim 1 wherein the phases include an investigate phase in which one or more data sources are audited for data quality.

9. The user interface of claim 1 wherein the phases include a design phase in which one or more data integration processes are characterized.

10. The user interface of claim 1 wherein the phases include a design phase in which one or more transformations are characterized for data in the data sources.

11. The user interface of claim 1 wherein the phases include a develop phase in which one or more data integration processes are associated with data sources in the data integration system.

12. The user interface of claim 1 wherein the phases include a develop phase in which one or more data integration processes are associated with other data integration processes deployed in the data integration system.

13. The user interface of claim 1 wherein the phases include a develop phase in which data staging may be performed on one or more data sources.

14. The user interface of claim 1 wherein the phases include a deploy phase in which a data integration process is made available to the data integration system.

15. The user interface of claim 1 wherein the phases include a deploy phase in which a data integration process is deployed as a real-time data integration service.

16. The user interface of claim 1 wherein the phases include an operate phase in which at least one data integration function, service, or job is used in the data integration system.

17. The user interface of claim 1 wherein the user interface provides varying degrees of functionality for the plurality of controls according to a skill level of a user of the interface.

34

18. A computer program product comprising a computer useable medium including computer readable program code, wherein the computer readable program code when executed on one or more computers causes the one or more computers to: provide a user interface, the user interface including a menu bar that displays a plurality of controls, each control accessing an interface corresponding to a phase in a workflow; receive a selection of one of the controls; in response to the selection, navigating to an interface corresponding to one of the phases in the workflow; retrieve data from the computer environment about the corresponding phase in the workflow; display the retrieved data in the interface; receive user input relating to the corresponding phase in the workflow; and transmit data to the computer environment based upon the user input.

19. A method for linking a data integration task to a business context, comprising: providing a graphical user interface for manipulating business objects and metadata; placing at least one business object into the graphical user interface; placing at least one metadata object from a data source into the graphical user interface; and

Unking the at least one metadata object to the at least one business object within the graphical user interface.

20. The method of claim 19 further comprising placing at least one metadata object from a data target into the graphical user interface.

21. The method of claim 20 further comprising linking the at least one metadata object from the data target to a business object within the graphical user interface.

22. The method of claim 21 further comprising deriving a data transformation for one or more links between the data source metadata and the data target metadata.

23. A computer program product comprising a computer useable medium including computer readable program code, wherein the computer readable program code when executed on one or more computers causes the one or more computers to: provide a graphical user interface for manipulating business objects and metadata; place at least one business object into the graphical user interface; place at least one metadata object from a data source into the graphical user interface; and link the at least one metadata object to the at least one business object within the graphical user interface.

24. The computer program product of claim 23 wherein the computer readable program code when executed on one or more computers further causes the one or more computers to place at least one metadata object from a data target into the graphical user interface.

25. The computer program product of claim 24 wherein the computer readable program code when executed on one or more computers further causes the one or more computers to link the at least one metadata object from the data target to a business object within the graphical user interface to provide one or more links.

26. The method of claim 25 wherein the computer readable program code when executed on one or more computers further causes the one or more computers to derive a data transformation for the one or more links.

27. A method for designing a data integration process comprising: modeling a business initiative as a first data structure;

35 modeling a business context as a second data structure; adding one or more associations between the first data structure and the second data structure to provide an information model for the business initiative in the business context; adding one or more associations between the information model and one or more enterprise data sources to provide a mapping model; and applying one or more design tools to construct a data integration process from the mapping model.

28. The method of claim 27 wherein the business initiative includes one or more of an initiative, an objective, a requirement, and a statement.

29. The method of claim 27 wherein the business context includes one or more of a subject, a process, and a rule.

30. The method of claim 27 wherein the enterprise data sources include data sources external to the enterprise.

31. The method of claim 30 wherein the associations among the first data structure, the second data structure, and the data sources are created using tools within a graphical user interface.

32. The method of claim 30 wherein a collaborative environment is provided for creating one or more of the business initiative, the business context, the information model, and the mapping model.

33. A computer program product comprising a computer useable medium including computer readable program code, wherein the computer readable program code when executed on one or more computers causes the one or more computers to: model a business initiative as a first data structure; model a business context as a second data structure; add one or more associations between the first data structure and the second data structure to provide an information model for the business initiative in the business context; add one or more associations between the information model and one or more enterprise data sources to provide a mapping model; and apply one or more design tools to construct a data integration process from the mapping model.

34. The method of claim 33 wherein the business initiative includes one or more of an initiative, an objective, a requirement, and a statement.

35. The method of claim 33 wherein the business context includes one or more of a subject, a process, and a rule.

36. The method of claim 33 wherein the enterprise data sources include data sources external to the enterprise.

37. The method of claim 36 wherein the associations among the first data structure, the second data structure, and the data sources are created using tools within a graphical user interface.

38. The method of claim 36 wherein a collaborative environment is provided for creating one or more of the business initiative, the business context, the information model, and the mapping model.

39. A method for providing a configurable user interface comprising: providing a plurality of data integration tasks; providing a plurality of menus corresponding to phases of a workflow; and assigning one or more of the plurality of data integration tasks to each on of the menus with a task matrix.

40. The method of claim 39 wherein at least one of the data integration tasks is assigned to more than one menu.

41. The method of claim 39 wherein the task matrix is user-configurable.

36