US20050251556A1 - Continuous feedback-controlled deployment of message transforms in a distributed messaging system - Google Patents

Continuous feedback-controlled deployment of message transforms in a distributed messaging system Download PDF

Info

Publication number
US20050251556A1
US20050251556A1 US10/841,297 US84129704A US2005251556A1 US 20050251556 A1 US20050251556 A1 US 20050251556A1 US 84129704 A US84129704 A US 84129704A US 2005251556 A1 US2005251556 A1 US 2005251556A1
Authority
US
United States
Prior art keywords
broker
message
transform
messages
network
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/841,297
Inventor
Roman Ginis
Robert Strom
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US10/841,297 priority Critical patent/US20050251556A1/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GINIS, ROMAN, STROM, ROBERT EVAN
Publication of US20050251556A1 publication Critical patent/US20050251556A1/en
Priority to US12/115,798 priority patent/US7962646B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/30Definitions, standards or architectural aspects of layered protocol stacks
    • H04L69/32Architecture of open systems interconnection [OSI] 7-layer type protocol stacks, e.g. the interfaces between the data link level and the physical level
    • H04L69/322Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions
    • H04L69/329Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/60Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources
    • H04L67/63Routing a service request depending on the request content or context

Definitions

  • the present invention relates to data processing systems and, in particular, to messaging systems in a distributed processing environment. Still more particularly, the present invention provides a distributed messaging system supporting stateful subscriptions.
  • a publish-subscribe messaging middleware is a system in which there are two types of clients.
  • Publishers generate messages, also referred to as events, containing a topic and some data content.
  • Subscribers request a criterion, also called a subscription, specifying what kind of information, based on published messages, the system is to deliver in the future.
  • Publishers and subscribers are anonymous, meaning that publishers do not necessarily know how many subscribers there are or where they are and, similarly, subscribers do not necessarily know where publishers are.
  • mediators may be introduced into the flow of traditional messaging middleware. This is a useful concept; however, in their current manifestations, mediators are complex to program, require external database services in order to store and access state, and groups of mediators cannot be automatically combined.
  • a stateful publish-subscribe system extends the functionality of the content-based approach to include more general state-valued expressions.
  • Stateful subscriptions may refer to one or more message histories and may include more complex expressions. Therefore, subscribers may receive different information than that provided in the published messages.
  • a plurality of broker machines is provided to deliver messages sent by publishing clients toward subscribing clients based upon the contents of the messages and stateful transformations requested by the subscribing clients. These broker machines form an overlay network.
  • Subscription specifications are either given as a directed transformation graph (with nodes being the transforms required to derive the subscription from published messages and edges representing the ordering of the transforms) or in a language that can be analyzed by a compiler and converted into a collection of message transforms and views.
  • the messaging system builds a structure containing all message transforms and views needed for all intermediate and subscribed views of all subscriptions.
  • the messaging system uses this structure to allocate message transforms and views to broker machines in the overlay network.
  • a deployment service component deploys tasks to optimize system performance.
  • a monitoring services component detects a possible need to reconfigure.
  • a performance optimization service component computes new assignment of transforms.
  • a continuous deployment service implements a redeployment protocol that installs changes to transform placement while the existing publish-subscribe system continues to operate.
  • FIG. 1 depicts a pictorial representation of a network of data processing systems in which the present invention may be implemented
  • FIG. 2 is a block diagram of a data processing system that may be implemented as a server in accordance with a preferred embodiment of the present invention
  • FIG. 3 is a block diagram of a data processing that may serve as a client of a service in accordance with a preferred embodiment of the present invention
  • FIG. 4 illustrates a broker network for a publish-subscribe system in accordance with a preferred embodiment of the present invention
  • FIG. 5 illustrates how a stateful publish-subscribe service of the present invention appears to clients
  • FIG. 6 illustrates an example of a operator that transforms input view objects to an output view object in accordance with a preferred embodiment of the present invention
  • FIG. 7 illustrates an example dataflow hypergraph distributed over multiple brokers in accordance with a preferred embodiment of the present invention
  • FIG. 8 depicts a process for deploying transform objects and view objects when a dataflow specification is a declarative specification in accordance with a preferred embodiment of the present invention
  • FIG. 9 illustrates a redeployment mechanism in a distributed messaging system in accordance with a preferred embodiment of the present invention.
  • FIG. 10 is a flowchart illustrating the operation of a moving a message transform from a source broker to a target broker in a distributed messaging system in accordance with a preferred embodiment of the present invention.
  • the present invention provides a method, apparatus and computer program product for continuous feedback-controlled deployment of message transforms in a distributed messaging system.
  • the data processing device may be a stand-alone computing device or may be a distributed data processing system in which multiple computing devices are utilized to perform various aspects of the present invention. Therefore, the following FIGS. 1-3 are provided as exemplary diagrams of data processing environments in which the present invention may be implemented. It should be appreciated that FIGS. 1-3 are only exemplary and are not intended to assert or imply any limitation with regard to the environments in which the present invention may be implemented. Many modifications to the depicted environments may be made without departing from the spirit and scope of the present invention.
  • FIG. 1 depicts a pictorial representation of a network of data processing systems in which the present invention may be implemented.
  • Network data processing system 100 is a network of computers in which the present invention may be implemented.
  • Network data processing system 100 contains a network 102 , which is the medium used to provide communications links between various devices and computers connected together within network data processing system 100 .
  • Network 102 may include connections, such as wire, wireless communication links, or fiber optic cables.
  • servers 112 , 114 , 116 are connected to network 102 along with storage unit 106 .
  • clients 122 , 124 , and 126 are connected to network 102 .
  • These clients 122 , 124 , and 126 may be, for example, personal computers or network computers.
  • servers 112 , 114 , 116 provide data, such as boot files, operating system images, and applications to clients 122 , 124 , 126 .
  • Clients 122 , 124 , and 126 are clients to servers 112 , 114 , 116 .
  • Network data processing system 100 may include additional servers, clients, and other devices not shown.
  • network data processing system 100 provides a distributed messaging system that supports stateful subscriptions.
  • a subset of clients 122 , 124 , 126 may be publishing clients, while others of clients 122 , 124 , 126 may be subscribing clients, for example.
  • Published events may also be generated by one or more of servers 112 , 114 , 116 . Any computer may be both a publisher and a subscriber.
  • a stateful publish-subscribe system is a distributed messaging system in which at least one subscription is stateful.
  • Other subscriptions may be content-based or, in other words, stateless.
  • a stateful publish-subscribe system must compute information that requires multiple messages of one or more streams. For example, a stateful subscription may request, “Give me the highest quote within each one-minute period.”
  • a stateful subscription may entail delivering information other than simply a copy of the published messages. For example, a stateful subscription may request, “Tell me how many stocks fell during each one-minute period.”
  • the stateful publish-subscribe system is implemented within an overlay network, which is a collection of service machines, referred to as brokers, that accept messages from publisher clients, deliver subscribed information to subscriber clients, and route information between publishers and subscribers.
  • An overlay network which is a collection of service machines, referred to as brokers, that accept messages from publisher clients, deliver subscribed information to subscriber clients, and route information between publishers and subscribers.
  • servers 112 , 114 , 116 may be broker machines.
  • Both content-based and stateful publish-subscribe systems support a message delivery model based on two roles: (1) publishers produce information in the form of streams of structured messages; and, (2) subscribers specify in advance what kinds of information in which they are interested. As messages are later published, relevant information is delivered in a timely fashion to subscribers.
  • Content-based subscriptions are restricted to Boolean filter predicates that can only refer to fields in individual messages. For example, a content-based subscription may request, “Deliver message if traded volume>1000 shares.”
  • stateful subscriptions are more general state-valued expressions and may refer to one or more messages, either by referring to multiple messages of a single message stream or by referring to multiple message streams or both.
  • a content-based publish-subscribe system because subscriptions can only specify filtering, all published messages are either passed through to subscribers or filtered out. Therefore, messages received by subscribers are identically structured copies of messages published by publishers.
  • subscriptions may include more complex expressions and, therefore, subscribers may receive information that is not identical to the published messages with different formatting.
  • a published message may have only integer prices, while subscriptions to average prices may have non-integer averages.
  • Published message streams are associated with topics. Each topic is associated with a base relation.
  • a base relation is a table of tuples, each tuple corresponding to an event in the particular message stream.
  • Subscriptions are expressed as view expressions in a relational algebraic language, although other representations may be used, such as eXtensible Markup Language (XML), for example.
  • the language defines a cascade of views of base relations and derived views computed from either base relations or other views.
  • the set of subscriptions is compiled into a collection of objects that are deployed and integrated into messaging brokers.
  • publishers and subscribers connect to these brokers.
  • Published events are delivered to objects associated with base relations. The events are then pushed downstream to other objects that compute how each derived view changes based on the change to the base relation.
  • Those derived views associated with subscriptions then deliver events to the subscriber informing the subscriber of each change in state.
  • network data processing system 100 is the Internet with network 102 representing a worldwide collection of networks and gateways that use the Transmission Control Protocol/Internet Protocol (TCP/IP) suite of protocols to communicate with one another.
  • TCP/IP Transmission Control Protocol/Internet Protocol
  • At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational and other computer systems that route data and messages.
  • network data processing system 100 also may be implemented as a number of different types of networks, such as for example, an intranet, a local area network (LAN), or a wide area network (WAN).
  • FIG. 1 is intended as an example, and not as an architectural limitation for the present invention.
  • Data processing system 200 may be a symmetric multiprocessor (SMP) system including a plurality of processors 202 and 204 connected to system bus 206 . Alternatively, a single processor system may be employed. Also connected to system bus 206 is memory controller/cache 208 , which provides an interface to local memory 209 . I/O bus bridge 210 is connected to system bus 206 and provides an interface to I/O bus 212 . Memory controller/cache 208 and I/O bus bridge 210 may be integrated as depicted.
  • SMP symmetric multiprocessor
  • Peripheral component interconnect (PCI) bus bridge 214 connected to I/O bus 212 provides an interface to PCI local bus 216 .
  • PCI Peripheral component interconnect
  • a number of modems may be connected to PCI local bus 216 .
  • Typical PCI bus implementations will support four PCI expansion slots or add-in connectors.
  • Communications links to clients 108 - 112 in FIG. 1 may be provided through modem 218 and network adapter 220 connected to PCI local bus 216 through add-in connectors.
  • Additional PCI bus bridges 222 and 224 provide interfaces for additional PCI local buses 226 and 228 , from which additional modems or network adapters may be supported. In this manner, data processing system 200 allows connections to multiple network computers.
  • a memory-mapped graphics adapter 230 and hard disk 232 may also be connected to I/O bus 212 as depicted, either directly or indirectly.
  • FIG. 2 may vary.
  • other peripheral devices such as optical disk drives and the like, also may be used in addition to or in place of the hardware depicted.
  • the depicted example is not meant to imply architectural limitations with respect to the present invention.
  • the data processing system depicted in FIG. 2 may be, for example, an IBM eServerTM pSeries® system, a product of International Business Machines Corporation in Armonk, N.Y., running the Advanced Interactive Executive (AIX) operating system or LINUX operating system.
  • AIX Advanced Interactive Executive
  • Data processing system 300 is an example of a computer, such as client 108 in FIG. 1 , in which code or instructions implementing the processes of the present invention may be located.
  • data processing system 300 employs a hub architecture including a north bridge and memory controller hub (MCH) 308 and a south bridge and input/output (I/O) controller hub (ICH) 310 .
  • MCH north bridge and memory controller hub
  • I/O input/output controller hub
  • Processor 302 , main memory 304 , and graphics processor 318 are connected to MCH 308 .
  • Graphics processor 318 may be connected to the MCH through an accelerated graphics port (AGP), for example.
  • AGP accelerated graphics port
  • local area network (LAN) adapter 312 audio adapter 316 , keyboard and mouse adapter 320 , modem 322 , read only memory (ROM) 324 , hard disk drive (HDD) 326 , CD-ROM driver 330 , universal serial bus (USB) ports and other communications ports 332 , and PCI/PCIe devices 334 may be connected to ICH 310 .
  • PCI/PCIe devices may include, for example, Ethernet adapters, add-in cards, PC cards for notebook computers, etc. PCI uses a cardbus controller, while PCIe does not.
  • ROM 324 may be, for example, a flash binary input/output system (BIOS).
  • BIOS binary input/output system
  • Hard disk drive 326 and CD-ROM drive 330 may use, for example, an integrated drive electronics (IDE) or serial advanced technology attachment (SATA) interface.
  • a super I/O (SIO) device 336 may be connected to ICH 310 .
  • IDE integrated drive electronics
  • SATA serial
  • An operating system runs on processor 302 and is used to coordinate and provide control of various components within data processing system 300 in FIG. 3 .
  • the operating system may be a commercially available operating system such as Windows XPTM, which is available from Microsoft Corporation.
  • An object oriented programming system such as Java may run in conjunction with the operating system and provides calls to the operating system from Java programs or applications executing on data processing system 300 .
  • “JAVA” is a trademark of Sun Microsystems, Inc.
  • Instructions for the operating system, the object-oriented programming system, and applications or programs are located on storage devices, such as hard disk drive 326 , and may be loaded into main memory 304 for execution by processor 302 .
  • the processes of the present invention are performed by processor 302 using computer implemented instructions, which may be located in a memory such as, for example, main memory 304 , memory 324 , or in one or more peripheral devices 326 and 330 .
  • FIG. 3 may vary depending on the implementation.
  • Other internal hardware or peripheral devices such as flash memory, equivalent non-volatile memory, or optical disk drives and the like, may be used in addition to or in place of the hardware depicted in FIG. 3 .
  • the processes of the present invention may be applied to a multiprocessor data processing system.
  • data processing system 300 may be a personal digital assistant (PDA), which is configured with flash memory to provide non-volatile memory for storing operating system files and/or user-generated data.
  • PDA personal digital assistant
  • FIG. 3 and above-described examples are not meant to imply architectural limitations.
  • data processing system 300 also may be a tablet computer, laptop computer, or telephone device in addition to taking the form of a PDA.
  • a plurality of broker machines are responsible for delivery of message sent by publishing clients towards subscribing clients based upon the content of the messages and the stateful transformations requested by the subscribing clients. These broker machines form an overlay network.
  • Some broker machines may be specialized for hosting publishing clients, referred to as publisher hosting brokers (PHB), and others for hosting subscribing clients, referred to as subscriber hosting brokers (SHB).
  • PHB publisher hosting brokers
  • SHB subscriber hosting brokers
  • IBs intermediate brokers
  • FIG. 4 illustrates a broker network for a publish-subscribe system in accordance with a preferred embodiment of the present invention.
  • a publishing client such as one of publishers 402 a - 402 d , establishes a connection to a PHB, such as PHB 404 a or PHB 404 b , over a corresponding one of client connections 406 a - 406 d .
  • the client connection may be, for example, any reliable synchronous or asynchronous first-in/first-out (FIFO) connection, such as a Transmission Control Protocol/Internet Protocol (TCP/IP) socket connection.
  • FIFO first-in/first-out
  • a subscribing client such as one of subscribers 412 a - 412 d , establishes a connection to a SHB, such as SHB 410 a or SHB 410 b , over a corresponding one of client connections 414 a - 414 d , which may be similar to client connections 406 a - 406 d .
  • the PHBs and SHBs are connected, via intermediate brokers 408 a - 408 b , through broker-to-broker links.
  • the publish-subscribe system of the present invention may include a fault-tolerant protocol that tolerates link failures and message re-orderings, in which case it is not necessary for the broker-to-broker connections to use reliable FIFO protocols, such as TCP/IP, but may advantageously use faster, less reliable protocols.
  • Each broker machine may be a stand-alone computer, a process within a computer, or, to minimize delay due to failures, a cluster of redundant processes within multiple computers.
  • the links may be simple socket connections, or connection bundles that use multiple alternative paths for high availability and load balancing.
  • FIG. 5 illustrates how a stateful publish-subscribe service of the present invention appears to clients.
  • Clients are unaware of the physical broker network or its topology.
  • a client application may connect to any broker in the role of publisher and/or subscriber.
  • Publishing clients are aware only of particular named published message streams, such as message streams 502 , 504 . Multiple clients may publish to the same message stream.
  • derived views based on functions of either published message streams or of other derived views.
  • message streams may be represented as relations.
  • Derived views are represented as relations derived from published message streams or from other derived views by means of relational algebraic expressions in a language, such as Date and Darwen's tutorial-D, Structured Query Language (SQL), or XQUERY.
  • derived view 510 is defined as a function of stream relations 502 and 504 by means of a JOIN expression with relations 502 and 504 as inputs and relation 510 as an output.
  • relation 512 indicated as a subscriber view, is derived from relation 510 by client-specified relational expressions.
  • subscriber view 512 may be a request to group the stock trades of relation 510 by issue and hour and compute the running total volume and max and min price for each issue-hour pair.
  • Each subscribing client subscribes to one or more particular derived views. As published events enter the system from publishing clients, they are saved in their respective streams. The system is then responsible for updating each derived view according to the previously specified relational expressions and then continuously delivering messages to each subscribing client representing the changes to the state of the respective subscribed view.
  • the term “message transform” denotes a software component, such as a JavaTM programming language object instance, which takes as inputs one or more sequences of messages and produces as output one or more sequences of messages.
  • a message transform may keep some local state and is repeatedly triggered upon receipt of a new message (or a batch of new messages) on one of its inputs. The message transform then performs some computation the effect of which is to possibly modify its state and to possibly generate one or more new messages on one or more of its output sequences.
  • the input messages of a transform may represent published events, such as stock trades, and the output messages of the transform might represent changes to a table of total stock volume per day grouped by stock issue. The function of the transform is to compute the change to the total in the appropriate row based upon the current state whenever a new published event arrives.
  • a sequence processing graph consists of a multiplicity of such message transforms connected together into a flow graph such that the output sequence produced by one message transform is fed into the instance of another transform.
  • subscribers are interested in the results of one or more functions computed over one or more published sequences (subscriptions).
  • the system may compute and propagate many message sequences concurrently.
  • subscription specifications are analyzed by a compiler and converted into a collection of message transform objects and view objects.
  • Each operator that derives a view from one or more inputs corresponds to a one or more transform objects.
  • Each view corresponds to a view object.
  • View objects hold the state of a view.
  • Transform objects express the logic for incrementally updating an output view constituting the result of an operator in response to individual changes to input views constituting the arguments to that operator. Without a compiler, the transform objects can be specified directly. The invention does not depend on the method of transform specification.
  • FIG. 6 illustrates an example of an operator that transforms input view objects to an output view object in accordance with a preferred embodiment of the present invention.
  • views 610 and 620 are view objects that are inputs to some operator, such as, for example, a JOIN operator.
  • Transform 650 is a transform object for that operator, which produces a derived view shown as view object 670 .
  • Transform 650 receives the messages representing changes to its inputs 610 , 620 , computes how the result of the operator changes given the announced changes it has received, and then delivers the computed results to its output view object 670 in the form of change messages. Output view object 670 then propagates in its turn such change messages, either to further transforms, if view object 670 is an intermediate view, or to subscribers, if view object 670 is a subscriber view.
  • FIG. 6 illustrates the objects and message pathways for a single transform implementing a single computational operation.
  • the mechanism of the present invention builds a structure containing all of the transform objects and view objects needed for all intermediate and subscribed views of all subscriptions.
  • This structure is called a dataflow hypergraph.
  • the dataflow hypergraph has nodes corresponding to each view object and hyperedges, which may possibly have more than one input feeding an output, representing each transform object associated with an operation in the subscription specification.
  • the view objects and transform objects are then allocated to actual brokers in the overlay network, either manually by an administrator or automatically via a continuous deployment service as described herein.
  • the published streams and the subscribed views may be constrained to be located on brokers where the publishers and subscribers actually connect.
  • the placement of the intermediate transform objects and view objects is not constrained. That is intermediate transform objects and view objects may be placed wherever suitable, taking into consideration the capacities of the broker machines and the links, as well as the desired performance. After such allocation of objects to brokers, the result is a distributed transform graph.
  • FIG. 7 illustrates an example dataflow hypergraph distributed over multiple brokers in accordance with a preferred embodiment of the present invention.
  • the physical network consists of brokers 710 , 720 , 730 , and 740 .
  • the publishing clients are publishing to three separate published message streams: “buys” 722 on broker 720 , “sells” 734 on broker 730 , and “matches” 712 on broker 710 .
  • the subscribing client 750 subscribes to derived view 748 on broker 740 .
  • Broker 710 also includes transforms 714 and 716 , which feed change messages to brokers 720 and 730 , respectively.
  • Broker 720 includes view objects 724 , 726 and transform objects 725 , 727 .
  • view object 726 represents an intermediate derived view or relation, which is based on transform 725 , published stream 722 , and view 724 .
  • Broker 730 includes views 732 and 736 , in addition to published stream 734 , and also includes transforms 735 , 737 .
  • Broker 740 includes views 742 , 744 , 748 , and transform 746 .
  • View 748 is a subscriber view for subscriber 750 .
  • the transform graph consists of multiple transform and view objects distributed over all brokers.
  • the paths between objects will sometimes lie within a broker, as is the case between transform object 725 and intermediate view object 726 .
  • the path must cross over an inter-broker link.
  • within-broker communications between objects may use cheaper communications means, such as parameter passing between objects, whereas inter-broker communications requires generating physical messages or packets that will cross the link.
  • the protocols of all view objects and transform objects will be able to recover from lost, out-of-order, or duplicate messages, and, therefore, will work correctly regardless of which paths between objects cross broker boundaries and which do not.
  • a history of states is stored in a data storage device.
  • messages from the “matches” published stream 712 are stored in storage 782
  • messages from the “buys” published stream 722 are stored in storage 784
  • messages from the “sells” published stream 734 are stored in storage 786 .
  • Storage 782 , 784 , 786 may be a portion of system memory or may be a persistent storage, such as a hard drive, or a combination thereof.
  • Messages may be stored as records in a table, spreadsheet, or database, for example. While the example shown in FIG. 7 stores published messages in storage, other changes to derived views may also be stored in history storage. For example, changes to views 742 , 744 may be stored in history storage for use by transform 746 .
  • the transforms involved in the computation must be placed on the nodes of the broker network intelligently in order to maximize some measure of performance.
  • the placement of transforms is performed continuously in response to system state changes, resource availability, and client demands.
  • the system monitors itself using a set of agents that submit performance feedback information to the performance optimization service that uses the data to issue new transform deployment instructions to the broker network. Regardless of the placement algorithm, such a closed-loop system may sometimes decide to move some transforms from resource to resource to improve performance.
  • FIG. 8 depicts a process for deploying transform objects and view objects when a dataflow specification is a declarative specification in accordance with a preferred embodiment of the present invention.
  • a declarative specification is a program, such as middleware program source 802 .
  • Program source 802 may be written in a language similar to the SQL syntax, for example.
  • Program source 802 may be compiled using compiler 804 into a dataflow hypergraph.
  • Compiler 804 includes particular algorithms for compiling relational algebraic operators into tailored object code that implements transform objects specialized to incrementally evaluate operators, such as join, select, aggregate, etc., for relations of known data types and key signatures.
  • the object code may be, for example, JavaTM programming language code.
  • Compiler 804 represents values of derived states as monotonic knowledge.
  • a monotonic state can change value only in one direction. Even state values that oscillate back and forth are represented as the difference between two monotonically increasing values. Hence, the system can always detect which of two values of a single data field is older. As another example, the system distinguishes internally between a “missing value,” which is missing because its value is not yet known, and one that is missing because it is known to be absent. As yet another example, the system distinguishes between a field having a value that is currently ten, but which may get larger later, from a field whose value is currently ten, but is final and will never change.
  • Compiler 804 generates an object for each relation and an object for each transform.
  • Each algebraic operation with n operands is associated with one transform object with n input interfaces and one relation object with an output interface.
  • the compiler may then generates a script that is executed at deployment time to instantiate all transform and relation objects, connecting an instance of the output interface from each relation to an instance of the input to each transform that uses the relation as an input.
  • Base relations are fed from an “input transform” object, which delivers published messages to the relation.
  • the objects form the knowledge flow graph, or hypergraph described earlier.
  • Each transform in the hypergraph such as join, project, group-by, select, etc., is associated with a template used by the compiler to produce an incremental transform.
  • a transform has one or more relations feeding inputs to it and a single relation receiving its output.
  • the incremental transform is an object that, given a message indicating that one component of its input has changed, computes the appropriate changes to its output and invokes a method in the output relation indicating what has changed.
  • the hypergraph may be specified explicitly or, alternatively, the hypergraph may be optimized using transform placement service 806 and deployed using deployment service 808 .
  • the hypergraph is optimized and deployed to broker network 824 by passing broker instructions 812 to broker network 824 and receiving performance information 814 from broker network 824 .
  • An optimization step consolidates multiple subscriptions to exploit common computations and performs other simplifications to reduce the total computational load on the system.
  • a placement step allocates the transform objects of the knowledge flow graph to particular brokers for the purpose of load balancing, reduction of bandwidth, increasing update throughput and overall reduction of delays between publishers and subscribers.
  • Broker network 824 receives the knowledge flow graph at deployment time. At execution time, publishers 822 publish message streams to broker network 824 . The brokers receive events and propagate messages representing view changes towards the subscribers. Subscribers 828 may then receive messages from subscriber views in broker network 824 .
  • FIG. 9 illustrates a redeployment mechanism in a distributed messaging system in accordance with a preferred embodiment of the present invention.
  • the continuous feedback-controlled deployment system of the present invention includes deployment system 910 that has three components. Monitoring services component 912 detects the possible need to reconfigure. Performance optimization services component 914 computes a new assignment of transforms. Continuous deployment services component 916 implements a redeployment protocol that installs the changes to transform placement while the existing publish-subscribe system continues to operate.
  • An efficient stream transform redeployment algorithm addresses the problem of disruption that happens when a transform needs to be moved from one hosting resource to another. Since transforms continuously exchange message sequence data, there is a high probability of having messages in the transform-to-transform links during the time of the move. Furthermore, new messages are likely to appear at the moving transform inputs. If the transform is absent, the effects of these messages would not reach the subscribers and cause information gaps and delays.
  • the continuous transform redeployment service of the present invention limits the disruption in the message sequences as perceived by its subscribers while one or more of the subscription transforms are being redeployed.
  • the redeployment algorithm does this by ensuring that each message sequence continues to reach its subscribers without interruption.
  • the redeployment system may temporarily deal with message sequences through redundant paths during periods of switchover.
  • the continuous deployment service of the present invention in the computing middleware works in two phases.
  • the first phase is initial deployment of the service itself.
  • the second phase is transform deployment instructed by performance optimization service 916 .
  • a program (agent) is installed on each computer (broker) that needs to participate in the computing middleware.
  • agent 922 is installed on broker 920
  • agent 932 is installed on broker 930
  • agent 942 is installed on broker 940 .
  • Each agent turns the computer into a network broker and has several capabilities: a) the capability to start/stop a particular transform in the broker program; b) the capability to receive a transform assignment from continuous deployment service 916 ; c) the capability to copy a transform state to an agent on another machine; d) the capability to receive/send instructions to establish/break connections between its broker program and other broker programs; e) the capability to monitor the broker program and machine performance and send the statistics to monitoring service 912 ; f) the capability to register its network broker as a resource with deployment system 910 so that the broker can be used for computation and message routing; g) the capability to perform diagnostics of the network broker it manages to ensure that the broker works properly; and, h) the capability to request a diagnostic report from another agent about its broker.
  • Each agent enters the second phase of the continuous deployment service as soon as its registration with deployment system 910 is complete. In this phase, the agent is always ready to execute the steps of the efficient stream transform redeployment algorithm discussed above, when instructed by continuous deployment service 916 .
  • the efficient stream transform redeployment algorithm is as follows:
  • the agent associated with network broker A sends instructions to the agent associated with network broker B about the connections network broker B would need to establish as inputs. Once the instructions are received by the agent associated with broker B, the agent instructs the broker to initialize the connections and ensure that the messages start flowing into B using these connections.
  • the agent associated with network broker A sends the copy of the state of transform T to the agent associated with network broker B.
  • This state is a tuple that contains all of the information necessary for T to continue being evaluated properly at broker B as if it were at broker A, as well as the sequence marker indicating which last message in the sequence was processed to compute this state. The latter is necessary in order for the broker B to know what messages it can use to compute a next correct version of the state.
  • the agent associated with broker B loads the received state of T into the broker and ensures that T functions correctly with new state.
  • the agent associated with network broker A sends instructions to the agent associated with network broker B about the connections network broker B would need to establish as outputs. Once the instructions are received by the agent associated with broker B, the agent instructs the broker to initialize the connections and ensure that the messages start flowing out of B using these connections.
  • the agent associated with network broker A requests a diagnostic report from the agent associated with broker B. Once the agent associated with broker A receives a satisfactory diagnostic report, the agent associated with network broker A stops the transform T in its broker program.
  • FIG. 10 is a flowchart illustrating the operation of a moving a message transform from a source broker to a target broker in a distributed messaging system in accordance with a preferred embodiment of the present invention.
  • the process begins and instantiates a copy of the transform on the target broker (block 1002 ). Then, the process duplicates all of the inputs of the source broker to the target broker (block 1004 ). The process then copies the state of the transform from the source broker to the target broker (block 1006 ). Next, the process duplicates all of the outputs of the source broker from the target broker to the same children (block 1008 ). The process terminates the transform on the source broker when the target broker passes a self-diagnostic and works properly (block 1010 ). Thereafter, the process ends.
  • the stateful publish-subscribe system of the present invention is able to be deployed on a wide-area, distributed overlay network of broker machines, communicating by message passing, and being subjected to the possibility of duplicate and out-of-order messages between links.
  • the distributed messaging system of the present invention allows subscriptions expressed as relational algebraic expressions on published message histories. Relational algebra may include, in particular, operators such as select, project, join, extend, group-by, sum, count, and average, for example.
  • the relational algebraic expressions may be mapped to form various query languages, such as SQL and XQUERY.
  • the messaging system of the present invention may allow service specifications that are deterministic and “eventually consistent,” meaning that multiple identical subscriptions eventually receive the same result, but weaker, and hence cheaper to implement, than fully consistent database systems.

Abstract

The present invention solves the disadvantages of the prior art and provides a distributed messaging system supporting stateful subscriptions. A stateful publish-subscribe system extends the functionality of the content-based approach to include more general state-valued expressions. A plurality of broker machines is provided to deliver messages sent by publishing clients toward subscribing clients based upon the contents of the messages and stateful transformations requested by the subscribing clients. Subscription specifications are analyzed by a compiler and converted into a collection of message transforms and views. A deployment service component deploys tasks to optimize system performance. A monitoring services component detects a possible need to reconfigure. A performance optimization service component computes new assignment of transforms. A continuous deployment service implements a redeployment protocol that installs changes to transform placement while the existing publish-subscribe system continues to operate.

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • The present application is related to U.S. patent application Ser. No. ______, attorney docket no. YOR920040154US1, entitled “DISTRIBUTED MESSAGING SYSTEM SUPPORTING STATEFUL SUBSCRIPTIONS,” filed on an even date herewith, assigned to the same assignee, and incorporated herein by reference.
  • BACKGROUND OF THE INVENTION
  • 1. Technical Field
  • The present invention relates to data processing systems and, in particular, to messaging systems in a distributed processing environment. Still more particularly, the present invention provides a distributed messaging system supporting stateful subscriptions.
  • 2. Description of Related Art
  • A publish-subscribe messaging middleware is a system in which there are two types of clients. Publishers generate messages, also referred to as events, containing a topic and some data content. Subscribers request a criterion, also called a subscription, specifying what kind of information, based on published messages, the system is to deliver in the future. Publishers and subscribers are anonymous, meaning that publishers do not necessarily know how many subscribers there are or where they are and, similarly, subscribers do not necessarily know where publishers are.
  • A topic-based, or content-based, publish-subscribe system is one in which the delivered messages are a possibly filtered subset of the published messages and the subscription criterion is a property that can be tested on each message independent of any other message. For example, a filter may determine whether “topic=stock-ticker” or “volume>1000.” Content-based or topic-based publish-subscribe systems are referred to herein as “stateless.”
  • There are pre-existing and emerging alternative technologies to solve the deficiencies of content-based publish-subscribe systems. Message mediators may be introduced into the flow of traditional messaging middleware. This is a useful concept; however, in their current manifestations, mediators are complex to program, require external database services in order to store and access state, and groups of mediators cannot be automatically combined.
  • Traditional database systems may also be used. Each published message can be fed to a database and give rise to a cascade of transactions updating the message history. Subscriptions can be expressed as views of these histories. Technologies are being developed to allow views to be updated incrementally. Such an approach is easier to program; however, it can be costly and slow if each new message results in a transaction involving a large number of subscribers.
  • An emerging technology still being researched is continuous queries on data streams. These approaches preserve the simpler programming model of the database system approach above and attempt to reduce the cost of traditional databases by a combination of approaches, including batching message updates and restricting the available operations to ones allowing the use of bounded-sized, in-memory sliding windows. However, this approach is restricting and limited.
  • SUMMARY OF THE INVENTION
  • The present invention solves the disadvantages of the prior art and provides a distributed messaging system supporting stateful subscriptions. A stateful publish-subscribe system extends the functionality of the content-based approach to include more general state-valued expressions. Stateful subscriptions may refer to one or more message histories and may include more complex expressions. Therefore, subscribers may receive different information than that provided in the published messages. A plurality of broker machines is provided to deliver messages sent by publishing clients toward subscribing clients based upon the contents of the messages and stateful transformations requested by the subscribing clients. These broker machines form an overlay network. Subscription specifications are either given as a directed transformation graph (with nodes being the transforms required to derive the subscription from published messages and edges representing the ordering of the transforms) or in a language that can be analyzed by a compiler and converted into a collection of message transforms and views. The messaging system builds a structure containing all message transforms and views needed for all intermediate and subscribed views of all subscriptions.
  • The messaging system uses this structure to allocate message transforms and views to broker machines in the overlay network. A deployment service component deploys tasks to optimize system performance. A monitoring services component detects a possible need to reconfigure. A performance optimization service component computes new assignment of transforms. A continuous deployment service implements a redeployment protocol that installs changes to transform placement while the existing publish-subscribe system continues to operate.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:
  • FIG. 1 depicts a pictorial representation of a network of data processing systems in which the present invention may be implemented;
  • FIG. 2 is a block diagram of a data processing system that may be implemented as a server in accordance with a preferred embodiment of the present invention;
  • FIG. 3 is a block diagram of a data processing that may serve as a client of a service in accordance with a preferred embodiment of the present invention;
  • FIG. 4 illustrates a broker network for a publish-subscribe system in accordance with a preferred embodiment of the present invention;
  • FIG. 5 illustrates how a stateful publish-subscribe service of the present invention appears to clients;
  • FIG. 6 illustrates an example of a operator that transforms input view objects to an output view object in accordance with a preferred embodiment of the present invention;
  • FIG. 7 illustrates an example dataflow hypergraph distributed over multiple brokers in accordance with a preferred embodiment of the present invention;
  • FIG. 8 depicts a process for deploying transform objects and view objects when a dataflow specification is a declarative specification in accordance with a preferred embodiment of the present invention;
  • FIG. 9 illustrates a redeployment mechanism in a distributed messaging system in accordance with a preferred embodiment of the present invention; and
  • FIG. 10 is a flowchart illustrating the operation of a moving a message transform from a source broker to a target broker in a distributed messaging system in accordance with a preferred embodiment of the present invention.
  • DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
  • The present invention provides a method, apparatus and computer program product for continuous feedback-controlled deployment of message transforms in a distributed messaging system. The data processing device may be a stand-alone computing device or may be a distributed data processing system in which multiple computing devices are utilized to perform various aspects of the present invention. Therefore, the following FIGS. 1-3 are provided as exemplary diagrams of data processing environments in which the present invention may be implemented. It should be appreciated that FIGS. 1-3 are only exemplary and are not intended to assert or imply any limitation with regard to the environments in which the present invention may be implemented. Many modifications to the depicted environments may be made without departing from the spirit and scope of the present invention.
  • With reference now to the figures, FIG. 1 depicts a pictorial representation of a network of data processing systems in which the present invention may be implemented. Network data processing system 100 is a network of computers in which the present invention may be implemented. Network data processing system 100 contains a network 102, which is the medium used to provide communications links between various devices and computers connected together within network data processing system 100. Network 102 may include connections, such as wire, wireless communication links, or fiber optic cables.
  • In the depicted example, servers 112, 114, 116 are connected to network 102 along with storage unit 106. In addition, clients 122, 124, and 126 are connected to network 102. These clients 122, 124, and 126 may be, for example, personal computers or network computers. In the depicted example, servers 112, 114, 116 provide data, such as boot files, operating system images, and applications to clients 122, 124, 126. Clients 122, 124, and 126 are clients to servers 112, 114, 116. Network data processing system 100 may include additional servers, clients, and other devices not shown.
  • In accordance with a preferred embodiment of the present invention, network data processing system 100 provides a distributed messaging system that supports stateful subscriptions. A subset of clients 122, 124, 126 may be publishing clients, while others of clients 122, 124, 126 may be subscribing clients, for example. Published events may also be generated by one or more of servers 112, 114, 116. Any computer may be both a publisher and a subscriber.
  • A stateful publish-subscribe system is a distributed messaging system in which at least one subscription is stateful. Other subscriptions may be content-based or, in other words, stateless. In other words, a stateful publish-subscribe system must compute information that requires multiple messages of one or more streams. For example, a stateful subscription may request, “Give me the highest quote within each one-minute period.” A stateful subscription may entail delivering information other than simply a copy of the published messages. For example, a stateful subscription may request, “Tell me how many stocks fell during each one-minute period.”
  • The stateful publish-subscribe system is implemented within an overlay network, which is a collection of service machines, referred to as brokers, that accept messages from publisher clients, deliver subscribed information to subscriber clients, and route information between publishers and subscribers. One or more of servers 112, 114, 116, for example, may be broker machines.
  • Both content-based and stateful publish-subscribe systems support a message delivery model based on two roles: (1) publishers produce information in the form of streams of structured messages; and, (2) subscribers specify in advance what kinds of information in which they are interested. As messages are later published, relevant information is delivered in a timely fashion to subscribers.
  • Content-based subscriptions are restricted to Boolean filter predicates that can only refer to fields in individual messages. For example, a content-based subscription may request, “Deliver message if traded volume>1000 shares.” On the other hand, stateful subscriptions are more general state-valued expressions and may refer to one or more messages, either by referring to multiple messages of a single message stream or by referring to multiple message streams or both. In a content-based publish-subscribe system, because subscriptions can only specify filtering, all published messages are either passed through to subscribers or filtered out. Therefore, messages received by subscribers are identically structured copies of messages published by publishers. In contrast, in a stateful publish-subscribe system, subscriptions may include more complex expressions and, therefore, subscribers may receive information that is not identical to the published messages with different formatting. For example, a published message may have only integer prices, while subscriptions to average prices may have non-integer averages.
  • Published message streams are associated with topics. Each topic is associated with a base relation. A base relation is a table of tuples, each tuple corresponding to an event in the particular message stream. Subscriptions are expressed as view expressions in a relational algebraic language, although other representations may be used, such as eXtensible Markup Language (XML), for example. The language defines a cascade of views of base relations and derived views computed from either base relations or other views. At compile-time, the set of subscriptions is compiled into a collection of objects that are deployed and integrated into messaging brokers. At run-time, publishers and subscribers connect to these brokers. Published events are delivered to objects associated with base relations. The events are then pushed downstream to other objects that compute how each derived view changes based on the change to the base relation. Those derived views associated with subscriptions then deliver events to the subscriber informing the subscriber of each change in state.
  • In the depicted example, network data processing system 100 is the Internet with network 102 representing a worldwide collection of networks and gateways that use the Transmission Control Protocol/Internet Protocol (TCP/IP) suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational and other computer systems that route data and messages. Of course, network data processing system 100 also may be implemented as a number of different types of networks, such as for example, an intranet, a local area network (LAN), or a wide area network (WAN). FIG. 1 is intended as an example, and not as an architectural limitation for the present invention.
  • Referring to FIG. 2, a block diagram of a data processing system that may be implemented as a server, such as server 104 in FIG. 1, is depicted in accordance with a preferred embodiment of the present invention. Data processing system 200 may be a symmetric multiprocessor (SMP) system including a plurality of processors 202 and 204 connected to system bus 206. Alternatively, a single processor system may be employed. Also connected to system bus 206 is memory controller/cache 208, which provides an interface to local memory 209. I/O bus bridge 210 is connected to system bus 206 and provides an interface to I/O bus 212. Memory controller/cache 208 and I/O bus bridge 210 may be integrated as depicted.
  • Peripheral component interconnect (PCI) bus bridge 214 connected to I/O bus 212 provides an interface to PCI local bus 216. A number of modems may be connected to PCI local bus 216. Typical PCI bus implementations will support four PCI expansion slots or add-in connectors. Communications links to clients 108-112 in FIG. 1 may be provided through modem 218 and network adapter 220 connected to PCI local bus 216 through add-in connectors.
  • Additional PCI bus bridges 222 and 224 provide interfaces for additional PCI local buses 226 and 228, from which additional modems or network adapters may be supported. In this manner, data processing system 200 allows connections to multiple network computers. A memory-mapped graphics adapter 230 and hard disk 232 may also be connected to I/O bus 212 as depicted, either directly or indirectly.
  • Those of ordinary skill in the art will appreciate that the hardware depicted in FIG. 2 may vary. For example, other peripheral devices, such as optical disk drives and the like, also may be used in addition to or in place of the hardware depicted. The depicted example is not meant to imply architectural limitations with respect to the present invention. The data processing system depicted in FIG. 2 may be, for example, an IBM eServer™ pSeries® system, a product of International Business Machines Corporation in Armonk, N.Y., running the Advanced Interactive Executive (AIX) operating system or LINUX operating system.
  • With reference now to FIG. 3, a block diagram of a data processing that may serve as a client of a service in accordance with a preferred embodiment of the present invention. Data processing system 300 is an example of a computer, such as client 108 in FIG. 1, in which code or instructions implementing the processes of the present invention may be located. In the depicted example, data processing system 300 employs a hub architecture including a north bridge and memory controller hub (MCH) 308 and a south bridge and input/output (I/O) controller hub (ICH) 310. Processor 302, main memory 304, and graphics processor 318 are connected to MCH 308. Graphics processor 318 may be connected to the MCH through an accelerated graphics port (AGP), for example.
  • In the depicted example, local area network (LAN) adapter 312, audio adapter 316, keyboard and mouse adapter 320, modem 322, read only memory (ROM) 324, hard disk drive (HDD) 326, CD-ROM driver 330, universal serial bus (USB) ports and other communications ports 332, and PCI/PCIe devices 334 may be connected to ICH 310. PCI/PCIe devices may include, for example, Ethernet adapters, add-in cards, PC cards for notebook computers, etc. PCI uses a cardbus controller, while PCIe does not. ROM 324 may be, for example, a flash binary input/output system (BIOS). Hard disk drive 326 and CD-ROM drive 330 may use, for example, an integrated drive electronics (IDE) or serial advanced technology attachment (SATA) interface. A super I/O (SIO) device 336 may be connected to ICH 310.
  • An operating system runs on processor 302 and is used to coordinate and provide control of various components within data processing system 300 in FIG. 3. The operating system may be a commercially available operating system such as Windows XP™, which is available from Microsoft Corporation. An object oriented programming system such as Java may run in conjunction with the operating system and provides calls to the operating system from Java programs or applications executing on data processing system 300. “JAVA” is a trademark of Sun Microsystems, Inc. Instructions for the operating system, the object-oriented programming system, and applications or programs are located on storage devices, such as hard disk drive 326, and may be loaded into main memory 304 for execution by processor 302. The processes of the present invention are performed by processor 302 using computer implemented instructions, which may be located in a memory such as, for example, main memory 304, memory 324, or in one or more peripheral devices 326 and 330.
  • Those of ordinary skill in the art will appreciate that the hardware in FIG. 3 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disk drives and the like, may be used in addition to or in place of the hardware depicted in FIG. 3. Also, the processes of the present invention may be applied to a multiprocessor data processing system.
  • For example, data processing system 300 may be a personal digital assistant (PDA), which is configured with flash memory to provide non-volatile memory for storing operating system files and/or user-generated data. The depicted example in FIG. 3 and above-described examples are not meant to imply architectural limitations. For example, data processing system 300 also may be a tablet computer, laptop computer, or telephone device in addition to taking the form of a PDA.
  • In accordance with a preferred embodiment of the present invention, a plurality of broker machines are responsible for delivery of message sent by publishing clients towards subscribing clients based upon the content of the messages and the stateful transformations requested by the subscribing clients. These broker machines form an overlay network. Some broker machines may be specialized for hosting publishing clients, referred to as publisher hosting brokers (PHB), and others for hosting subscribing clients, referred to as subscriber hosting brokers (SHB). Between the PHBs and the SHBs, there may be any number of intermediate nodes that include routing and filtering. The brokers at the intermediate nodes are referred to as intermediate brokers or IBs. For expository purposes, this separation is assumed; however, in actual deployment, some or all of the broker machines may combine the functions of PHB, SHB, and/or IB.
  • FIG. 4 illustrates a broker network for a publish-subscribe system in accordance with a preferred embodiment of the present invention. A publishing client, such as one of publishers 402 a-402 d, establishes a connection to a PHB, such as PHB 404 a or PHB 404 b, over a corresponding one of client connections 406 a-406 d. The client connection may be, for example, any reliable synchronous or asynchronous first-in/first-out (FIFO) connection, such as a Transmission Control Protocol/Internet Protocol (TCP/IP) socket connection. Independently, a subscribing client, such as one of subscribers 412 a-412 d, establishes a connection to a SHB, such as SHB 410 a or SHB 410 b, over a corresponding one of client connections 414 a-414 d, which may be similar to client connections 406 a-406 d. The PHBs and SHBs are connected, via intermediate brokers 408 a-408 b, through broker-to-broker links.
  • The publish-subscribe system of the present invention may include a fault-tolerant protocol that tolerates link failures and message re-orderings, in which case it is not necessary for the broker-to-broker connections to use reliable FIFO protocols, such as TCP/IP, but may advantageously use faster, less reliable protocols. Each broker machine may be a stand-alone computer, a process within a computer, or, to minimize delay due to failures, a cluster of redundant processes within multiple computers. Similarly, the links may be simple socket connections, or connection bundles that use multiple alternative paths for high availability and load balancing.
  • FIG. 5 illustrates how a stateful publish-subscribe service of the present invention appears to clients. Clients are unaware of the physical broker network or its topology. A client application may connect to any broker in the role of publisher and/or subscriber. Publishing clients are aware only of particular named published message streams, such as message streams 502, 504. Multiple clients may publish to the same message stream.
  • Administrators and clients may defined derived views based on functions of either published message streams or of other derived views. In the depicted example, message streams may be represented as relations. Derived views are represented as relations derived from published message streams or from other derived views by means of relational algebraic expressions in a language, such as Date and Darwen's Tutorial-D, Structured Query Language (SQL), or XQUERY. For example, derived view 510 is defined as a function of stream relations 502 and 504 by means of a JOIN expression with relations 502 and 504 as inputs and relation 510 as an output. Similarly, relation 512, indicated as a subscriber view, is derived from relation 510 by client-specified relational expressions. For example, subscriber view 512 may be a request to group the stock trades of relation 510 by issue and hour and compute the running total volume and max and min price for each issue-hour pair.
  • Each subscribing client subscribes to one or more particular derived views. As published events enter the system from publishing clients, they are saved in their respective streams. The system is then responsible for updating each derived view according to the previously specified relational expressions and then continuously delivering messages to each subscribing client representing the changes to the state of the respective subscribed view.
  • As used herein, the term “message transform” denotes a software component, such as a Java™ programming language object instance, which takes as inputs one or more sequences of messages and produces as output one or more sequences of messages. In typical use, a message transform may keep some local state and is repeatedly triggered upon receipt of a new message (or a batch of new messages) on one of its inputs. The message transform then performs some computation the effect of which is to possibly modify its state and to possibly generate one or more new messages on one or more of its output sequences. By way of example, the input messages of a transform may represent published events, such as stock trades, and the output messages of the transform might represent changes to a table of total stock volume per day grouped by stock issue. The function of the transform is to compute the change to the total in the appropriate row based upon the current state whenever a new published event arrives.
  • A sequence processing graph consists of a multiplicity of such message transforms connected together into a flow graph such that the output sequence produced by one message transform is fed into the instance of another transform. On a distributed network, it is possible for some of the transforms in the sequence processing graph to be deployed on different machines. In the system of the present invention, subscribers are interested in the results of one or more functions computed over one or more published sequences (subscriptions). Thus, the system may compute and propagate many message sequences concurrently.
  • In a preferred embodiment of the present invention, subscription specifications are analyzed by a compiler and converted into a collection of message transform objects and view objects. Each operator that derives a view from one or more inputs corresponds to a one or more transform objects. Each view corresponds to a view object. View objects hold the state of a view. Transform objects express the logic for incrementally updating an output view constituting the result of an operator in response to individual changes to input views constituting the arguments to that operator. Without a compiler, the transform objects can be specified directly. The invention does not depend on the method of transform specification.
  • FIG. 6 illustrates an example of an operator that transforms input view objects to an output view object in accordance with a preferred embodiment of the present invention. In the depicted example, views 610 and 620 are view objects that are inputs to some operator, such as, for example, a JOIN operator. Transform 650 is a transform object for that operator, which produces a derived view shown as view object 670. When one of the input objects changes, either because it itself is a published input stream or because it is a derived view that has changed as a result of changes to its inputs, messages reflecting the changes are sent to transform object 650. Transform 650 receives the messages representing changes to its inputs 610, 620, computes how the result of the operator changes given the announced changes it has received, and then delivers the computed results to its output view object 670 in the form of change messages. Output view object 670 then propagates in its turn such change messages, either to further transforms, if view object 670 is an intermediate view, or to subscribers, if view object 670 is a subscriber view.
  • FIG. 6 illustrates the objects and message pathways for a single transform implementing a single computational operation. When subscriptions are entered, the mechanism of the present invention builds a structure containing all of the transform objects and view objects needed for all intermediate and subscribed views of all subscriptions. This structure is called a dataflow hypergraph. The dataflow hypergraph has nodes corresponding to each view object and hyperedges, which may possibly have more than one input feeding an output, representing each transform object associated with an operation in the subscription specification.
  • The view objects and transform objects are then allocated to actual brokers in the overlay network, either manually by an administrator or automatically via a continuous deployment service as described herein. The published streams and the subscribed views may be constrained to be located on brokers where the publishers and subscribers actually connect. The placement of the intermediate transform objects and view objects is not constrained. That is intermediate transform objects and view objects may be placed wherever suitable, taking into consideration the capacities of the broker machines and the links, as well as the desired performance. After such allocation of objects to brokers, the result is a distributed transform graph.
  • FIG. 7 illustrates an example dataflow hypergraph distributed over multiple brokers in accordance with a preferred embodiment of the present invention. In the depicted example, the physical network consists of brokers 710, 720, 730, and 740. There are three publishing clients 702, 704, and 706, and one subscribing client 750. The publishing clients are publishing to three separate published message streams: “buys” 722 on broker 720, “sells” 734 on broker 730, and “matches” 712 on broker 710. The subscribing client 750 subscribes to derived view 748 on broker 740.
  • Broker 710 also includes transforms 714 and 716, which feed change messages to brokers 720 and 730, respectively. Broker 720 includes view objects 724, 726 and transform objects 725, 727. As an example, view object 726 represents an intermediate derived view or relation, which is based on transform 725, published stream 722, and view 724. Broker 730 includes views 732 and 736, in addition to published stream 734, and also includes transforms 735, 737. Broker 740 includes views 742, 744, 748, and transform 746. View 748 is a subscriber view for subscriber 750.
  • As shown in FIG. 7, the transform graph consists of multiple transform and view objects distributed over all brokers. The paths between objects will sometimes lie within a broker, as is the case between transform object 725 and intermediate view object 726. In other cases, such as the path between transform 727 and intermediate view object 742 (shown with a dotted line), the path must cross over an inter-broker link. It is clear to those versed in the art that the within-broker communications between objects may use cheaper communications means, such as parameter passing between objects, whereas inter-broker communications requires generating physical messages or packets that will cross the link. In a preferred embodiment, the protocols of all view objects and transform objects will be able to recover from lost, out-of-order, or duplicate messages, and, therefore, will work correctly regardless of which paths between objects cross broker boundaries and which do not.
  • In order to support stateful subscriptions, a history of states is stored in a data storage device. For example, messages from the “matches” published stream 712 are stored in storage 782, messages from the “buys” published stream 722 are stored in storage 784, and messages from the “sells” published stream 734 are stored in storage 786. Storage 782, 784, 786 may be a portion of system memory or may be a persistent storage, such as a hard drive, or a combination thereof. Messages may be stored as records in a table, spreadsheet, or database, for example. While the example shown in FIG. 7 stores published messages in storage, other changes to derived views may also be stored in history storage. For example, changes to views 742, 744 may be stored in history storage for use by transform 746.
  • To compute and deliver message sequences to subscribers efficiently, the transforms involved in the computation must be placed on the nodes of the broker network intelligently in order to maximize some measure of performance. In accordance with a preferred embodiment of the present invention, the placement of transforms is performed continuously in response to system state changes, resource availability, and client demands. The system monitors itself using a set of agents that submit performance feedback information to the performance optimization service that uses the data to issue new transform deployment instructions to the broker network. Regardless of the placement algorithm, such a closed-loop system may sometimes decide to move some transforms from resource to resource to improve performance.
  • Designs with various types of dynamic process migration and task assignment for mainframe, distributed, and parallel systems are known in the art. These designs often deal with tasks that exchange data rarely, such as at an end of a computational block. In a stream-processing system the tasks performing the transforms exchange data very often. A transform object continuously receives message sequence data from its parent transform objects and sends the derived data to its children. If a transform needs to be redeployed to another network node, it should be done in a way that causes minimal disruption to the subscriptions that use it. The problem of redeploying a transform at runtime is analogous to changing a segment of a water pipe without interrupting the water flow and spilling as little water as possible.
  • FIG. 8 depicts a process for deploying transform objects and view objects when a dataflow specification is a declarative specification in accordance with a preferred embodiment of the present invention. A declarative specification is a program, such as middleware program source 802. Program source 802 may be written in a language similar to the SQL syntax, for example. Program source 802 may be compiled using compiler 804 into a dataflow hypergraph. Compiler 804 includes particular algorithms for compiling relational algebraic operators into tailored object code that implements transform objects specialized to incrementally evaluate operators, such as join, select, aggregate, etc., for relations of known data types and key signatures. The object code may be, for example, Java™ programming language code. Compiler 804 represents values of derived states as monotonic knowledge. A monotonic state can change value only in one direction. Even state values that oscillate back and forth are represented as the difference between two monotonically increasing values. Hence, the system can always detect which of two values of a single data field is older. As another example, the system distinguishes internally between a “missing value,” which is missing because its value is not yet known, and one that is missing because it is known to be absent. As yet another example, the system distinguishes between a field having a value that is currently ten, but which may get larger later, from a field whose value is currently ten, but is final and will never change.
  • Compiler 804 generates an object for each relation and an object for each transform. Each algebraic operation with n operands is associated with one transform object with n input interfaces and one relation object with an output interface. The compiler may then generates a script that is executed at deployment time to instantiate all transform and relation objects, connecting an instance of the output interface from each relation to an instance of the input to each transform that uses the relation as an input. Base relations are fed from an “input transform” object, which delivers published messages to the relation. The objects form the knowledge flow graph, or hypergraph described earlier. Each transform in the hypergraph, such as join, project, group-by, select, etc., is associated with a template used by the compiler to produce an incremental transform. As discussed above, a transform has one or more relations feeding inputs to it and a single relation receiving its output. The incremental transform is an object that, given a message indicating that one component of its input has changed, computes the appropriate changes to its output and invokes a method in the output relation indicating what has changed.
  • The hypergraph may be specified explicitly or, alternatively, the hypergraph may be optimized using transform placement service 806 and deployed using deployment service 808. The hypergraph is optimized and deployed to broker network 824 by passing broker instructions 812 to broker network 824 and receiving performance information 814 from broker network 824. An optimization step consolidates multiple subscriptions to exploit common computations and performs other simplifications to reduce the total computational load on the system. A placement step allocates the transform objects of the knowledge flow graph to particular brokers for the purpose of load balancing, reduction of bandwidth, increasing update throughput and overall reduction of delays between publishers and subscribers.
  • Broker network 824 receives the knowledge flow graph at deployment time. At execution time, publishers 822 publish message streams to broker network 824. The brokers receive events and propagate messages representing view changes towards the subscribers. Subscribers 828 may then receive messages from subscriber views in broker network 824.
  • FIG. 9 illustrates a redeployment mechanism in a distributed messaging system in accordance with a preferred embodiment of the present invention. The continuous feedback-controlled deployment system of the present invention includes deployment system 910 that has three components. Monitoring services component 912 detects the possible need to reconfigure. Performance optimization services component 914 computes a new assignment of transforms. Continuous deployment services component 916 implements a redeployment protocol that installs the changes to transform placement while the existing publish-subscribe system continues to operate.
  • An efficient stream transform redeployment algorithm addresses the problem of disruption that happens when a transform needs to be moved from one hosting resource to another. Since transforms continuously exchange message sequence data, there is a high probability of having messages in the transform-to-transform links during the time of the move. Furthermore, new messages are likely to appear at the moving transform inputs. If the transform is absent, the effects of these messages would not reach the subscribers and cause information gaps and delays.
  • The continuous transform redeployment service of the present invention limits the disruption in the message sequences as perceived by its subscribers while one or more of the subscription transforms are being redeployed. The redeployment algorithm does this by ensuring that each message sequence continues to reach its subscribers without interruption. The redeployment system may temporarily deal with message sequences through redundant paths during periods of switchover.
  • The continuous deployment service of the present invention in the computing middleware works in two phases. The first phase is initial deployment of the service itself. The second phase is transform deployment instructed by performance optimization service 916. In the first phase, a program (agent) is installed on each computer (broker) that needs to participate in the computing middleware. In the example depicted in FIG. 9, agent 922 is installed on broker 920, agent 932 is installed on broker 930, and agent 942 is installed on broker 940.
  • Each agent turns the computer into a network broker and has several capabilities: a) the capability to start/stop a particular transform in the broker program; b) the capability to receive a transform assignment from continuous deployment service 916; c) the capability to copy a transform state to an agent on another machine; d) the capability to receive/send instructions to establish/break connections between its broker program and other broker programs; e) the capability to monitor the broker program and machine performance and send the statistics to monitoring service 912; f) the capability to register its network broker as a resource with deployment system 910 so that the broker can be used for computation and message routing; g) the capability to perform diagnostics of the network broker it manages to ensure that the broker works properly; and, h) the capability to request a diagnostic report from another agent about its broker.
  • Each agent enters the second phase of the continuous deployment service as soon as its registration with deployment system 910 is complete. In this phase, the agent is always ready to execute the steps of the efficient stream transform redeployment algorithm discussed above, when instructed by continuous deployment service 916. The efficient stream transform redeployment algorithm is as follows:
  • Assuming task T needs to be moved from network broker A to network broker B,
      • 1. Instantiate a copy of T on system B;
      • 2. Duplicate all inputs into A to additionally go to B;
      • 3. Copy T's state from A to B;
      • 4. Duplicate all outputs from A to also go from B to the same children;
      • 5. Terminate T on A when B passes self-diagnostics and works properly.
        To perform task 1 of the algorithm, the agent associated with network broker B receives the program for transform T either from its local storage or from deployment system 910 or a delegate thereof. The agent then instructs the broker to load the program and prepare to execute the transform.
  • To perform task 2 of the algorithm, the agent associated with network broker A sends instructions to the agent associated with network broker B about the connections network broker B would need to establish as inputs. Once the instructions are received by the agent associated with broker B, the agent instructs the broker to initialize the connections and ensure that the messages start flowing into B using these connections.
  • To perform task 3 of the algorithm, the agent associated with network broker A sends the copy of the state of transform T to the agent associated with network broker B. This state is a tuple that contains all of the information necessary for T to continue being evaluated properly at broker B as if it were at broker A, as well as the sequence marker indicating which last message in the sequence was processed to compute this state. The latter is necessary in order for the broker B to know what messages it can use to compute a next correct version of the state. To complete the task the agent associated with broker B loads the received state of T into the broker and ensures that T functions correctly with new state.
  • To perform task 4 of the algorithm, the agent associated with network broker A sends instructions to the agent associated with network broker B about the connections network broker B would need to establish as outputs. Once the instructions are received by the agent associated with broker B, the agent instructs the broker to initialize the connections and ensure that the messages start flowing out of B using these connections.
  • To perform task 5 of the algorithm, the agent associated with network broker A requests a diagnostic report from the agent associated with broker B. Once the agent associated with broker A receives a satisfactory diagnostic report, the agent associated with network broker A stops the transform T in its broker program.
  • FIG. 10 is a flowchart illustrating the operation of a moving a message transform from a source broker to a target broker in a distributed messaging system in accordance with a preferred embodiment of the present invention. The process begins and instantiates a copy of the transform on the target broker (block 1002). Then, the process duplicates all of the inputs of the source broker to the target broker (block 1004). The process then copies the state of the transform from the source broker to the target broker (block 1006). Next, the process duplicates all of the outputs of the source broker from the target broker to the same children (block 1008). The process terminates the transform on the source broker when the target broker passes a self-diagnostic and works properly (block 1010). Thereafter, the process ends.
  • The stateful publish-subscribe system of the present invention is able to be deployed on a wide-area, distributed overlay network of broker machines, communicating by message passing, and being subjected to the possibility of duplicate and out-of-order messages between links. The distributed messaging system of the present invention allows subscriptions expressed as relational algebraic expressions on published message histories. Relational algebra may include, in particular, operators such as select, project, join, extend, group-by, sum, count, and average, for example. The relational algebraic expressions may be mapped to form various query languages, such as SQL and XQUERY. Furthermore, the messaging system of the present invention may allow service specifications that are deterministic and “eventually consistent,” meaning that multiple identical subscriptions eventually receive the same result, but weaker, and hence cheaper to implement, than fully consistent database systems.
  • It is important to note that while the present invention has been described in the context of a fully functioning data processing system, those of ordinary skill in the art will appreciate that the processes of the present invention are capable of being distributed in the form of a computer readable medium of instructions and a variety of forms and that the present invention applies equally regardless of the particular type of signal bearing media actually used to carry out the distribution. Examples of computer readable media include recordable-type media, such as a floppy disk, a hard disk drive, a RAM, CD-ROMs, DVD-ROMs, and transmission-type media, such as digital and analog communications links, wired or wireless communications links using transmission forms, such as, for example, radio frequency and light wave transmissions. The computer readable media may take the form of coded formats that are decoded for actual use in a particular data processing system.
  • The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention, the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.

Claims (23)

1. A method for moving a message transform from a source broker to a target broker in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, wherein a plurality of message transforms are deployed in the network of broker machines, and wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages, the method comprising:
starting on the target broker a copy of a given message transform from the source broker;
duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and
duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system.
2. The method of claim 1, further comprising:
copying a state of the given message transform from the source broker to the copy of the given message transform on the target broker;
3. The method of claim 1, further comprising:
terminating the given message transform on the source broker.
4. The method of claim 3, wherein terminating the given transform on the source broker is performed responsive to the copy of the given message transform passing a self-diagnostic.
5. The method of claim 1, further comprising:
receiving a plurality of input messages associated with input streams;
applying the copy of the given transform at the target broker to the plurality of input messages to form an output message, wherein the output message depends on at least two input messages from the plurality of input messages; and
delivering the output message in an output stream.
6. A method for continuous deployment of message transforms in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, the method comprising:
deploying a plurality of message transforms in the network of broker machines, wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages;
receiving performance information from the network of broker machines; and
redeploying at least a subset of the plurality of message transforms in the network of broker machines based on the performance information.
7. The method of claim 6, wherein redeploying at least a subset of the plurality of message transforms includes sending one or more broker instructions to at least one broker machine in the network of broker machines.
8. The method of claim 6, wherein redeploying at least a subset of the plurality of message transforms includes moving a given message transform from a source broker to a target broker in the distributed messaging system.
9. The method of claim 8, wherein moving a given message transform includes:
starting on the target broker a copy of the given message transform from the source broker;
duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and
duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system.
10. An apparatus for implementing a stateful subscription service, the apparatus comprising:
an overlay network including a plurality of broker machines;
a plurality of publishing clients that publish messages to published message streams;
a performance optimization service that determines an assignment of message transforms to broker machines in the overlay network; and
a continuous deployment service that deploys the assignment of message transforms to the plurality of broker machines.
11. The apparatus of claim 10, further comprising:
a plurality of execution engines running on the plurality of broker machines, wherein each execution engine within the plurality of execution engine receives input messages, processes the input messages using the transform objects, and routes output messages toward subscribers.
12. The apparatus of claim 11, wherein at least one message transform in the assignment of message transforms has at least one message delivered to its output stream that depends on more than one input message from an input stream of the message transform.
13. An apparatus for moving a message transform from a source broker to a target broker in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, wherein a plurality of message transforms are deployed in the network of broker machines, and wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages, the apparatus comprising:
means for starting on the target broker a copy of a given message transform from the source broker;
means for duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and
means for duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system.
14. The apparatus of claim 13, further comprising:
means for copying a state of the given message transform from the source broker to the copy of the given message transform on the target broker;
15. The apparatus of claim 13, further comprising:
means for terminating the given message transform on the source broker.
16. The apparatus of claim 15, wherein the means for terminating the given transform on the source broker terminates the given transform responsive to the copy of the given message transform passing a self-diagnostic.
17. The apparatus of claim 13, further comprising:
means for receiving a plurality of input messages associated with input streams;
means for applying the copy of the given transform at the target broker to the plurality of input messages to form an output message, wherein the output message depends on at least two input messages from the plurality of input messages; and
means for delivering the output message in an output stream.
18. An apparatus for continuous deployment of message transforms in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, the apparatus comprising:
means for deploying a plurality of message transforms in the network of broker machines, wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages;
means for receiving performance information from the network of broker machines; and
means for redeploying at least a subset of the plurality of message transforms in the network of broker machines based on the performance information.
19. The apparatus of claim 18, wherein the means for redeploying at least a subset of the plurality of message transforms includes means for sending one or more broker instructions to at least one broker machine in the network of broker machines.
20. The apparatus of claim 18, wherein the means for redeploying at least a subset of the plurality of message transforms includes means for moving a given message transform from a source broker to a target broker in the distributed messaging system.
21. The apparatus of claim 20, wherein the means for moving a given message transform includes:
means for starting on the target broker a copy of the given message transform from the source broker;
means for duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and
means for duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system.
22. A computer program product, in a computer readable medium, for moving a message transform from a source broker to a target broker in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, wherein a plurality of message transforms are deployed in the network of broker machines, and wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages, the computer program product comprising:
instructions for starting on the target broker a copy of a given message transform from the source broker;
instructions for duplicating all inputs into the given message transform on the source broker to additionally go to the copy of the given message transform on the target broker; and
instructions for duplicating all outputs of the given message transform on the source broker to also go from the copy of the given message transform on the target broker to appropriate children nodes within the distributed messaging system.
23. A computer program product, in a computer readable medium, for continuous deployment of message transforms in a distributed messaging system, wherein the distributed messaging system includes a network of broker machines, the computer program product comprising:
instructions for deploying a plurality of message transforms in the network of broker machines, wherein the plurality of message transforms take as input one or more sequences of messages and produce as output one or more sequences of messages;
instructions for receiving performance information from the network of broker machines; and
instructions for redeploying at least a subset of the plurality of message transforms in the network of broker machines based on the performance information.
US10/841,297 2004-05-07 2004-05-07 Continuous feedback-controlled deployment of message transforms in a distributed messaging system Abandoned US20050251556A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US10/841,297 US20050251556A1 (en) 2004-05-07 2004-05-07 Continuous feedback-controlled deployment of message transforms in a distributed messaging system
US12/115,798 US7962646B2 (en) 2004-05-07 2008-05-06 Continuous feedback-controlled deployment of message transforms in a distributed messaging system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/841,297 US20050251556A1 (en) 2004-05-07 2004-05-07 Continuous feedback-controlled deployment of message transforms in a distributed messaging system

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US12/115,798 Continuation US7962646B2 (en) 2004-05-07 2008-05-06 Continuous feedback-controlled deployment of message transforms in a distributed messaging system

Publications (1)

Publication Number Publication Date
US20050251556A1 true US20050251556A1 (en) 2005-11-10

Family

ID=35240638

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/841,297 Abandoned US20050251556A1 (en) 2004-05-07 2004-05-07 Continuous feedback-controlled deployment of message transforms in a distributed messaging system
US12/115,798 Expired - Fee Related US7962646B2 (en) 2004-05-07 2008-05-06 Continuous feedback-controlled deployment of message transforms in a distributed messaging system

Family Applications After (1)

Application Number Title Priority Date Filing Date
US12/115,798 Expired - Fee Related US7962646B2 (en) 2004-05-07 2008-05-06 Continuous feedback-controlled deployment of message transforms in a distributed messaging system

Country Status (1)

Country Link
US (2) US20050251556A1 (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060149840A1 (en) * 2005-01-06 2006-07-06 Tervela, Inc. End-to-end publish/subscribe middleware architecture
US20060146999A1 (en) * 2005-01-06 2006-07-06 Tervela, Inc. Caching engine in a messaging system
US20060271384A1 (en) * 2005-05-31 2006-11-30 Microsoft Corporation Reference data aggregate service population
US20060271382A1 (en) * 2005-05-27 2006-11-30 Microsoft Corporation Entity projection
US20070174233A1 (en) * 2006-01-10 2007-07-26 International Business Machines Corporation Method for predicting performance of distributed stream processing systems
WO2007096298A1 (en) * 2006-02-21 2007-08-30 International Business Machines Corporation Method and system for mediating published message streams for selective distribution
US20070297327A1 (en) * 2006-06-27 2007-12-27 International Business Machines Corporation Method for applying stochastic control optimization for messaging systems
US7380005B1 (en) 2007-11-20 2008-05-27 International Business Machines Corporation Systems, methods and computer program products for improving placement performance of message transforms by exploiting aggressive replication
US20080209440A1 (en) * 2004-05-07 2008-08-28 Roman Ginis Distributed messaging system supporting stateful subscriptions
US20080244025A1 (en) * 2004-05-07 2008-10-02 Roman Ginis Continuous feedback-controlled deployment of message transforms in a distributed messaging system
US20080306888A1 (en) * 2007-06-07 2008-12-11 Chitra Dorai Stochastic control optimization for sender-based flow control in a distributed stateful messaging system
US20090287805A1 (en) * 2008-05-13 2009-11-19 International Business Machines Corporation System & method for non-http session based publish/subscribe support using pre-emptive subscriptions
US20120016979A1 (en) * 2010-07-15 2012-01-19 International Business Machines Corporation Propagating changes in topic subscription status of processes in an overlay network
US20130091232A1 (en) * 1999-03-11 2013-04-11 Easyweb Innovations, Llc. Message publishing with prohibited or restricted content removal
US8589536B2 (en) 2010-08-02 2013-11-19 International Business Machines Corporation Network monitoring system
EP3026869A4 (en) * 2013-07-26 2016-07-27 Zte Corp Method, device and system for carrying out telecommunication capability group sending
US10341921B2 (en) * 2012-06-13 2019-07-02 All Purpose Networks, Inc. Active hot standby redundancy for broadband wireless network
US10827019B2 (en) 2018-01-08 2020-11-03 All Purpose Networks, Inc. Publish-subscribe broker network overlay system
US10884883B2 (en) 2012-06-13 2021-01-05 All Purpose Networks, Inc. Methods and systems of an all purpose broadband network with publish-subscribe broker network
US11026090B2 (en) 2018-01-08 2021-06-01 All Purpose Networks, Inc. Internet of things system with efficient and secure communications network

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7886180B2 (en) * 2004-05-14 2011-02-08 International Business Machines Corporation Recovery in a distributed stateful publish-subscribe system
JP2007328692A (en) * 2006-06-09 2007-12-20 Canon Inc Algebra operation method and device therefor, and program
US8429238B2 (en) * 2009-08-25 2013-04-23 International Business Machines Corporation Method for providing feedback to a publisher
US9537747B2 (en) 2010-06-11 2017-01-03 International Business Machines Corporation Publish/subscribe overlay network control system
US8793322B2 (en) * 2011-02-20 2014-07-29 International Business Machines Corporation Failure-controlled message publication and feedback in a publish/subscribe messaging environment
US8843580B2 (en) 2011-02-20 2014-09-23 International Business Machines Corporation Criteria-based message publication control and feedback in a publish/subscribe messaging environment
WO2014046672A1 (en) 2012-09-21 2014-03-27 Hewlett-Packard Development Company, L.P. Monitor usable with continuous deployment

Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5870605A (en) * 1996-01-18 1999-02-09 Sun Microsystems, Inc. Middleware for enterprise information distribution
US5974417A (en) * 1996-01-18 1999-10-26 Sun Microsystems, Inc. Database network connectivity product
US5987455A (en) * 1997-06-30 1999-11-16 International Business Machines Corporation Intelligent compilation of procedural functions for query processing systems
US20010049743A1 (en) * 2000-05-31 2001-12-06 International Business Machines Corporation Message transformation selection tool and method
US20020069244A1 (en) * 1999-11-24 2002-06-06 John Blair Message delivery system billing method and apparatus
US6502213B1 (en) * 1999-08-31 2002-12-31 Accenture Llp System, method, and article of manufacture for a polymorphic exception handler in environment services patterns
US6510429B1 (en) * 1998-04-29 2003-01-21 International Business Machines Corporation Message broker apparatus, method and computer program product
US6643682B1 (en) * 1999-09-22 2003-11-04 International Business Machines Corporation Publish/subscribe data processing with subscription points for customized message processing
US6681220B1 (en) * 1999-05-28 2004-01-20 International Business Machines Corporation Reduction and optimization of information processing systems
US6748555B1 (en) * 1999-09-09 2004-06-08 Microsoft Corporation Object-based software management
US6748583B2 (en) * 2000-12-27 2004-06-08 International Business Machines Corporation Monitoring execution of an hierarchical visual program such as for debugging a message flow
US20050268146A1 (en) * 2004-05-14 2005-12-01 International Business Machines Corporation Recovery in a distributed stateful publish-subscribe system
US6983463B1 (en) * 1998-10-02 2006-01-03 Microsoft Corporation Network independent profiling of applications for automatic partitioning and distribution in a distributed computing environment
US7010538B1 (en) * 2003-03-15 2006-03-07 Damian Black Method for distributed RDSMS
US7162524B2 (en) * 2002-06-21 2007-01-09 International Business Machines Corporation Gapless delivery and durable subscriptions in a content-based publish/subscribe system
US7177859B2 (en) * 2002-06-26 2007-02-13 Microsoft Corporation Programming model for subscription services
US7349980B1 (en) * 2003-01-24 2008-03-25 Blue Titan Software, Inc. Network publish/subscribe system incorporating Web services network routing architecture
US7360202B1 (en) * 2002-06-26 2008-04-15 Microsoft Corporation User interface system and methods for providing notification(s)
US7406537B2 (en) * 2002-11-26 2008-07-29 Progress Software Corporation Dynamic subscription and message routing on a topic between publishing nodes and subscribing nodes
US20080239951A1 (en) * 2006-06-27 2008-10-02 Robert Evan Strom Method for applying stochastic control optimization for messaging systems
US20080301053A1 (en) * 2007-05-29 2008-12-04 Verizon Services Organization Inc. Service broker

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2215567B (en) * 1988-03-05 1992-11-18 Plessey Co Plc Improvements in or relating to equalisers
EP0753979A1 (en) * 1995-07-13 1997-01-15 International Business Machines Corporation Routing method and system for a high speed packet switching network
SE507118C2 (en) * 1996-08-26 1998-03-30 Ericsson Telefon Ab L M Procedure for optimizing a mainly optical ATM network
AU724356B2 (en) * 1996-10-08 2000-09-21 Tiernan Communications, Inc. Apparatus and method for multi-service transport multiplexing
US6859438B2 (en) 1998-02-03 2005-02-22 Extreme Networks, Inc. Policy based quality of service
US6298455B1 (en) * 1998-12-29 2001-10-02 International Business Machines Corporation Publish and subscribe data processing with failover using cascaded sequence numbers
US7392279B1 (en) * 1999-03-26 2008-06-24 Cisco Technology, Inc. Network traffic shaping using time-based queues
US7565403B2 (en) * 2000-03-16 2009-07-21 Microsoft Corporation Use of a bulk-email filter within a system for classifying messages for urgency or importance
US20030067874A1 (en) * 2001-10-10 2003-04-10 See Michael B. Central policy based traffic management
US6859138B2 (en) * 2001-10-31 2005-02-22 General Motors Corporation Key detection switch for vehicle entry and security systems
US20060200333A1 (en) * 2003-04-10 2006-09-07 Mukesh Dalal Optimizing active decision making using simulated decision making
US7308711B2 (en) * 2003-06-06 2007-12-11 Microsoft Corporation Method and framework for integrating a plurality of network policies
JPWO2005034446A1 (en) * 2003-10-03 2006-12-14 富士通株式会社 Policy rule application network system
US20050251556A1 (en) * 2004-05-07 2005-11-10 International Business Machines Corporation Continuous feedback-controlled deployment of message transforms in a distributed messaging system
JP2006121667A (en) * 2004-09-27 2006-05-11 Matsushita Electric Ind Co Ltd Packet reception control device and method
CA2594020C (en) * 2004-12-22 2014-12-09 Wake Forest University Method, systems, and computer program products for implementing function-parallel network firewall
US20070002750A1 (en) * 2005-07-01 2007-01-04 Nec Laboratories America, Inc. Generic Real Time Scheduler for Wireless Packet Data Systems
US20070116822A1 (en) * 2005-11-23 2007-05-24 The Coca-Cola Company High-potency sweetener composition with saponin and compositions sweetened therewith
GB2450257B (en) * 2006-03-29 2012-01-04 Intel Corp Optimization of network protocol options by reinforcement learning and propagation

Patent Citations (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5974417A (en) * 1996-01-18 1999-10-26 Sun Microsystems, Inc. Database network connectivity product
US6021443A (en) * 1996-01-18 2000-02-01 Sun Microsystems, Inc. Systems, software, and methods for routing events among publishers and subscribers on a computer network
US5870605A (en) * 1996-01-18 1999-02-09 Sun Microsystems, Inc. Middleware for enterprise information distribution
US5987455A (en) * 1997-06-30 1999-11-16 International Business Machines Corporation Intelligent compilation of procedural functions for query processing systems
US6510429B1 (en) * 1998-04-29 2003-01-21 International Business Machines Corporation Message broker apparatus, method and computer program product
US6983463B1 (en) * 1998-10-02 2006-01-03 Microsoft Corporation Network independent profiling of applications for automatic partitioning and distribution in a distributed computing environment
US6681220B1 (en) * 1999-05-28 2004-01-20 International Business Machines Corporation Reduction and optimization of information processing systems
US20040107290A1 (en) * 1999-05-28 2004-06-03 International Business Machines Corporation Reduction and optimization of information processing systems
US6996625B2 (en) * 1999-05-28 2006-02-07 International Business Machines Corporation Reduction and optiminization of operational query expressions applied to information spaces between nodes in a publish/subscribe system
US6502213B1 (en) * 1999-08-31 2002-12-31 Accenture Llp System, method, and article of manufacture for a polymorphic exception handler in environment services patterns
US6748555B1 (en) * 1999-09-09 2004-06-08 Microsoft Corporation Object-based software management
US6643682B1 (en) * 1999-09-22 2003-11-04 International Business Machines Corporation Publish/subscribe data processing with subscription points for customized message processing
US20020069244A1 (en) * 1999-11-24 2002-06-06 John Blair Message delivery system billing method and apparatus
US20010049743A1 (en) * 2000-05-31 2001-12-06 International Business Machines Corporation Message transformation selection tool and method
US6748583B2 (en) * 2000-12-27 2004-06-08 International Business Machines Corporation Monitoring execution of an hierarchical visual program such as for debugging a message flow
US7162524B2 (en) * 2002-06-21 2007-01-09 International Business Machines Corporation Gapless delivery and durable subscriptions in a content-based publish/subscribe system
US7177859B2 (en) * 2002-06-26 2007-02-13 Microsoft Corporation Programming model for subscription services
US7360202B1 (en) * 2002-06-26 2008-04-15 Microsoft Corporation User interface system and methods for providing notification(s)
US7406537B2 (en) * 2002-11-26 2008-07-29 Progress Software Corporation Dynamic subscription and message routing on a topic between publishing nodes and subscribing nodes
US7349980B1 (en) * 2003-01-24 2008-03-25 Blue Titan Software, Inc. Network publish/subscribe system incorporating Web services network routing architecture
US7010538B1 (en) * 2003-03-15 2006-03-07 Damian Black Method for distributed RDSMS
US20050268146A1 (en) * 2004-05-14 2005-12-01 International Business Machines Corporation Recovery in a distributed stateful publish-subscribe system
US20080239951A1 (en) * 2006-06-27 2008-10-02 Robert Evan Strom Method for applying stochastic control optimization for messaging systems
US20080301053A1 (en) * 2007-05-29 2008-12-04 Verizon Services Organization Inc. Service broker

Cited By (45)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130091232A1 (en) * 1999-03-11 2013-04-11 Easyweb Innovations, Llc. Message publishing with prohibited or restricted content removal
US20080244025A1 (en) * 2004-05-07 2008-10-02 Roman Ginis Continuous feedback-controlled deployment of message transforms in a distributed messaging system
US20080209440A1 (en) * 2004-05-07 2008-08-28 Roman Ginis Distributed messaging system supporting stateful subscriptions
US7962646B2 (en) 2004-05-07 2011-06-14 International Business Machines Corporation Continuous feedback-controlled deployment of message transforms in a distributed messaging system
US8533742B2 (en) 2004-05-07 2013-09-10 International Business Machines Corporation Distributed messaging system supporting stateful subscriptions
US20060168070A1 (en) * 2005-01-06 2006-07-27 Tervela, Inc. Hardware-based messaging appliance
US8321578B2 (en) 2005-01-06 2012-11-27 Tervela, Inc. Systems and methods for network virtualization
US9253243B2 (en) 2005-01-06 2016-02-02 Tervela, Inc. Systems and methods for network virtualization
US7970918B2 (en) 2005-01-06 2011-06-28 Tervela, Inc. End-to-end publish/subscribe middleware architecture
US20060168331A1 (en) * 2005-01-06 2006-07-27 Terevela, Inc. Intelligent messaging application programming interface
US20060146991A1 (en) * 2005-01-06 2006-07-06 Tervela, Inc. Provisioning and management in a message publish/subscribe system
US20060146999A1 (en) * 2005-01-06 2006-07-06 Tervela, Inc. Caching engine in a messaging system
US20060149840A1 (en) * 2005-01-06 2006-07-06 Tervela, Inc. End-to-end publish/subscribe middleware architecture
US20060271382A1 (en) * 2005-05-27 2006-11-30 Microsoft Corporation Entity projection
US7720904B2 (en) * 2005-05-27 2010-05-18 Microsoft Corporation Entity projection
US20060271384A1 (en) * 2005-05-31 2006-11-30 Microsoft Corporation Reference data aggregate service population
US7818417B2 (en) * 2006-01-10 2010-10-19 International Business Machines Corporation Method for predicting performance of distributed stream processing systems
US8499069B2 (en) 2006-01-10 2013-07-30 International Business Machines Corporation Method for predicting performance of distributed stream processing systems
US20080184262A1 (en) * 2006-01-10 2008-07-31 Business Machines Corporation Method for Predicting Performance of Distributed Stream Processing Systems
US20070174233A1 (en) * 2006-01-10 2007-07-26 International Business Machines Corporation Method for predicting performance of distributed stream processing systems
WO2007096298A1 (en) * 2006-02-21 2007-08-30 International Business Machines Corporation Method and system for mediating published message streams for selective distribution
US7792038B2 (en) 2006-06-27 2010-09-07 International Business Machines Corporation Method for applying stochastic control optimization for messaging systems
US20080239951A1 (en) * 2006-06-27 2008-10-02 Robert Evan Strom Method for applying stochastic control optimization for messaging systems
US20070297327A1 (en) * 2006-06-27 2007-12-27 International Business Machines Corporation Method for applying stochastic control optimization for messaging systems
US20080306888A1 (en) * 2007-06-07 2008-12-11 Chitra Dorai Stochastic control optimization for sender-based flow control in a distributed stateful messaging system
US7761401B2 (en) * 2007-06-07 2010-07-20 International Business Machines Corporation Stochastic control optimization for sender-based flow control in a distributed stateful messaging system
US7487247B1 (en) 2007-11-20 2009-02-03 International Business Machines Corporation Systems, methods and computer program products for improving placement performance of message transforms by exploiting aggressive replication
US7380005B1 (en) 2007-11-20 2008-05-27 International Business Machines Corporation Systems, methods and computer program products for improving placement performance of message transforms by exploiting aggressive replication
US7836123B2 (en) * 2008-05-13 2010-11-16 International Business Machines Corporation System and method for non-HTTP session based publish/subscribe support using pre-emptive subscriptions
US20090287805A1 (en) * 2008-05-13 2009-11-19 International Business Machines Corporation System & method for non-http session based publish/subscribe support using pre-emptive subscriptions
US20120016979A1 (en) * 2010-07-15 2012-01-19 International Business Machines Corporation Propagating changes in topic subscription status of processes in an overlay network
US8661080B2 (en) * 2010-07-15 2014-02-25 International Business Machines Corporation Propagating changes in topic subscription status of processes in an overlay network
US8589536B2 (en) 2010-08-02 2013-11-19 International Business Machines Corporation Network monitoring system
US11490311B2 (en) 2012-06-13 2022-11-01 All Purpose Networks, Inc. Methods and systems of an all purpose broadband network with publish subscribe broker network
US10341921B2 (en) * 2012-06-13 2019-07-02 All Purpose Networks, Inc. Active hot standby redundancy for broadband wireless network
US10841851B2 (en) 2012-06-13 2020-11-17 All Purpose Networks, Inc. Methods and systems of an all purpose broadband network with publish subscribe broker network
US10884883B2 (en) 2012-06-13 2021-01-05 All Purpose Networks, Inc. Methods and systems of an all purpose broadband network with publish-subscribe broker network
US11422906B2 (en) 2012-06-13 2022-08-23 All Purpose Networks, Inc. Methods and systems of an all purpose broadband network with publish-subscribe broker network
US11647440B2 (en) 2012-06-13 2023-05-09 All Purpose Networks, Inc. Methods and systems of an all purpose broadband network with publish subscribe broker network
US11711741B2 (en) 2012-06-13 2023-07-25 All Purpose Networks, Inc. Methods and systems of an all purpose broadband network with publish subscribe broker network
US9832281B2 (en) 2013-07-26 2017-11-28 Xi'an Zhongxing New Software Co., Ltd. Method, device and system for carrying out telecommunication capability group sending
EP3026869A4 (en) * 2013-07-26 2016-07-27 Zte Corp Method, device and system for carrying out telecommunication capability group sending
US10827019B2 (en) 2018-01-08 2020-11-03 All Purpose Networks, Inc. Publish-subscribe broker network overlay system
US11026090B2 (en) 2018-01-08 2021-06-01 All Purpose Networks, Inc. Internet of things system with efficient and secure communications network
US11683390B2 (en) 2018-01-08 2023-06-20 All Purpose Networks, Inc. Publish-subscribe broker network overlay system

Also Published As

Publication number Publication date
US20080244025A1 (en) 2008-10-02
US7962646B2 (en) 2011-06-14

Similar Documents

Publication Publication Date Title
US7962646B2 (en) Continuous feedback-controlled deployment of message transforms in a distributed messaging system
US8533742B2 (en) Distributed messaging system supporting stateful subscriptions
US7886180B2 (en) Recovery in a distributed stateful publish-subscribe system
US9049196B1 (en) Method for distributed RDSMS
US7093004B2 (en) Using execution statistics to select tasks for redundant assignment in a distributed computing platform
Grover et al. Data Ingestion in AsterixDB.
US20030154284A1 (en) Distributed data propagator
CN100570607C (en) The method and system that is used for the data aggregate of multiprocessing environment
US20020133504A1 (en) Integrating heterogeneous data and tools
US20070005610A1 (en) Method and system for processing directory events
WO2009035810A1 (en) Concurrency in event processing networks for event server
US20090282097A1 (en) Method and System for Ensuring Consistency Over Time of Data Gathered By Distinct Software Applications
US20070240140A1 (en) Methods and systems for application load distribution
US9740735B2 (en) Programming language extensions in structured queries
US20070245314A1 (en) Method, system and computer program for collecting inventory information through a data mover
US10621163B2 (en) Tracking and reusing function results
Campbell Service oriented database architecture: App server-lite?
Pandey et al. Comparative Study on Realtime Data Processing System
Cisco CiscoTaskManagedObjects.idl
Cisco CiscoTaskManagedObjects.idl
Liu et al. Supporting internet applications beyond browsing: Trigger processing and change notification
US11816177B2 (en) Streaming static web page generation
Wang et al. Fast log replication in highly available data store
Wang et al. Formalization and Verification of Go-based New Simple Queue System
Cappello et al. A Development and Deployment Framework for Distributed Branch-and-Bound

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:GINIS, ROMAN;STROM, ROBERT EVAN;REEL/FRAME:014696/0884

Effective date: 20040506

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION