NETWORK-BASED TALK SHOW SYSTEM
FIELD OF THE INVENTION
The present invention relates to a system and method for creating, managing and broadcasting a real-time, interactive, multimedia presentation such as a talk show, or talk-radio show, across a packet-based computer network allowing direct, real-time interaction between the host, guests and the audience. More specifically, the present invention relates a method and system for the audience of a show to participate in the show by submitting questions to the show's guests or host, a method for sorting and managing these questions in an efficient manner, a method for selecting questions for discussion, a method for establishing a voice communications channel for conversation between the host and the caller, and a method for efficiently broadcast the resulting conversation as part of the show across a packet-based network such as the Internet.
BACKGROUND OF THE INVENTION
Current talk shows and talk radio shows are broadcast over the radio, television or the Internet with questions commonly received using voice only over a telephone system, or text only by email or text chat. For example, radio shows commonly publicize a telephone number, which people call when they have a question or comment on the topic of the show. In some cases, people submit questions by text, either using email or text chat technology which are then commented on by the host or guests on the program.
An alternate format for talk shows, especially over the Internet, is to use an all text format with both questions and answers using text chat technology.
Voice chat technology over the Internet tends to be limited to relatively small numbers of people because large groups quickly become unmanageable, both because of interruptions of the current speaker, and because it is not possible to determine what someone wishes to say without letting them speak. This is problematic when an audience for a show might be very large, with many people competing for the host's attention, or when the audience is distributed over a wide geographic area where network lag and network traffic desynchronize the show and audience members.
There are also serious issues of network lag and quality of service and quality of the audio if it is necessary to maintain a very large group of people connected in real time to the talk show, as would be necessary if everyone has access to the host.
Another problem with implementing talk radio on the Internet is that it is difficult to pre-screen large numbers of prospective callers within a real-time network system, where the large number of voice conversation would require a large group of moderators. Such interviews would need to be conducted in real-time, and it is difficult for a person to effectively monitor multiple audio conversations.
Another problem with hosting a live, interactive talk show on the Internet is that current networks delivering streaming data work best if the data can be properly buffered to overcome problems with network lag. In a real time voice communication system, this means there is a pause each time a different person speaks, such as during a two way conversation. This is a problem that grows worst as the number of potential speakers increases, and the amount of data sent increases.
SUMMARY OF THE INVENTION
It is an object of the present invention to provide a novel, multi-user real-time talk show, or talk radio, system for show broadcast from a computer across a packet- based network such as the internet, and method which obviates or mitigates at least one problem or disadvantage of the prior art.
Host software
According to a first aspect of the present invention, there is provided a computer interface used by the host of a talk radio program or presentation, comprising:
User interface including displays of the status of a show as well as the status of the connection to the digital network;
Means for the host to initiate a broadcast;
Means for the host to control the audio properties and network connections of the show to the digital network;
Text list of questions submitted by the audience to the host of a show, such that selecting an audience member name, or selecting the text of the question will
create a direct voice link to that audience member. This list consists of either all questions submitted by the Audience, or only those questions submitted by the audience and pre-screened show by the moderator(s) and made visible to the host;
Means for the host to manage, sort or prioritize the list of submitted questions; Means for the host to manage any open voice links to the audience, by opening, closing, muting or removing open voice connections;
Means for the host to remotely manage the audio properties such as the microphone level of the caller;
The show host computer interface is part of a larger system that also includes the methods by which a show is encoded and prepared for broadcast, comprising;
Means to acquire an analog electronic representation of the sound or speech and digitization means to convert the analogue representation of the speech to digital format; A codec, or codecs, to compress the digital representation of the sound or speech and prepare it for transmission across a packet-based network, such as the Internet, as well as decode encoded speech for audible playback;
Means to package the host's encoded sound or speech and transmit the resulting packets across a network, such as the Internet, to an audience member with a live voice connection to the host's computer using a direct 'peer to peer' connection; A sound mixing system, such that digital sound from multiple sources, such as the host and a live caller connection, are combined into a single digital audio stream, such that the stream consists of the data from a single source, either host or caller, passed directly to the rebroadcast server when all other sources are quiet, or as a mixed stream otherwise;
Optional means to capture a video image of the host, such as a video camera, and digitization mean to convert the video stream to digital format;
A video codec, or codecs, to compress the digital representation of the video and prepare it for transmission across a packet-based network, such as the Internet. Means to package the encoded show and transmit the resulting packets across a network such as the Internet to a rebroadcast Server for broadcast to the entire audience using a server-client architecture.
Audience software
According to another aspect of the present invention, there is provided a computer program, used by the audience of a networked talk radio program, comprising; Means to connect across a packet-based network such as the Internet to remote computers acting as broadcast servers, and request permission to join a broadcast;
Means to receive a stream of packets, comprising the show broadcast, across a packet-based network such as the Internet;
A codec, or codecs, to decode digitally encoded speech, sound to analogue form for audible playback of the sound or speech;
Optional means to decode digitally encoded video, and display the video on the computer screen, synchronized with the decoded audio;
An onscreen display of text questions and the name or alias of the submitter selected by the host for discussion, such that it is obvious which question is currently being discussed, and which questions are upcoming;
An onscreen button or hyperlink, clearly labeled or marked to identify it, that when clicked will launch the software that allows the audience member to submit a question to the host;
Optionally, the computer program that is playing the show can provide space onscreen for the display of images, videos, text or other information.
Audience Caller software
According to another aspect of the present invention, there is provided a computer interface, used by the audience of a networked talk radio program, and invoked by clicking an onscreen button or hyperlink, comprising:
A computer software program including a user interface to be used by an audience member submitting a question, that includes an on-screen interface, comprising;
An area to type text questions that, by pressing a button, submits such questions to the host of a program;
A button to withdraw text questions, or modify text questions submitted by the user;
A button, slider or dial allowing the audience member to adjust local audio setting such as the volume of playback, or the recording volume of the microphone connected to the local computer,
The computer software also provides text and audio feedback from the host, such that the audience member is aware of the status of their submitted question, and has warning when their question is next or has been selected for discussion
The computer software also includes the methods by which the audience member connects to the host's computer as well as the means to encode and decode sound or speech, comprising,
Means to acquire an analog electronic representation of the sound or speech and digitization means to convert the analogue representation of the speech to digital format, A codec, or codecs, to compress the digital representation of the sound or speech and prepare it for transmission across a packet-based network, such as the Internet, as well as to decode encoded speech to analogue form for audible playback,
Means to package the audience member's encoded sound or speech and transmit the resulting packets across a packet-based network, such as the Internet, Means to connect the audience member's computer to the host's computer using a direct 'peer to peer' connection across a packet based network such as the Internet,
Means to mute the broadcast of the show currently playing on the local computer of the caller to eliminate audio feedback and echoes that would otherwise occur when the audience member is speaking with the host of the show
Moderator software
According to another aspect of the present invention, there is provided computer software, used by a moderator or moderators of a networked talk radio program, comprising
An onscreen computer user interface including,
Displays of the status of a show as well as the status of the connection to the
computer servers hosting the show and to the packet-based digital network;
List of text questions submitted by the audience to a show;
Means to select an audience member name and text question by clicking the interface that changes the status of the question, as stored on the servers, so that the question becomes visible on the host's computer as a pre-screened question. Text questions are evenly distributed among moderators, of which there can be any number;
Means for the moderator to manage, sort or prioritize the list of submitted questions; Means for the moderator's to enter text that is displayed to all other moderators, and / or to the host, allowing live text-based communications to synchronize and organize the activities of the host and moderators of the show;
Means for the moderator's to share questions submitted by the audience to the show with other moderators; The computer software interface is part of a larger system that also includes the methods by which moderator interface is managed and integrated into the system, comprising;
Means to start the software, and connect the moderator to the proper show and question database; Means to package data and transmit the resulting packets across a packet- based network such as the Internet to a server;
A persistent connection with the servers to allow constant data exchange between the servers, the host and the moderators.
Server software
According to another aspect of the present invention, there are computers acting as servers that control the show's client software, broadcast the show and maintain databases required by the shows, comprising;
Means to connect audience members to a show selected from a list of current shows by clicking a link or button on an HTML page displayed on the user's computer screen;
Means to check the permission of the audience member to join a show, by
compaπng login information against a user database, or a host to broadcast a show ,
Means to stream a show to the audience as data packets across a packet-based network such as the Internet,
Means to allow a host to connect to a server to broadcast a show, by clicking on a hypertext link or button displayed as part of an HTML page displayed on the host's computer monitor,
Means to accept an incoming stream of packets comprising a show sent by the host to the servers,
Means to accept questions and store the questions for a show in a database structure that can be accessed as necessary by the audience to submit questions, the moderators to prioritize and sort questions, and the host to prioritize and sort questions, and request the IP address necessary to establish the peer to peer voice connection
Means to provide streams of data of the shows to the audience in real time, or near real-time, during a show's broadcast,
Means to archive the shows for later on-demand viewing
Summary
The present invention provides a method and system to limit the number of real-time, live conversations that must be maintained, while at the same time making massive participation by the audience possible and maintaining the impression of a real-time, interactive transmission The present invention also provides a method and system that simplifies the screening of large numbers of questions, and allows a host, moderator or guest to quickly direct their attention to the most appropriate questions and callers The present invention also provides a method and system that makes it possible, during large scale broadcasts with many audience members, for a host to converse directly to the single person, selected in real time from a list of questions submitted, and allows the conversation resulting to be broadcast to the audience m as near to real-time as is technically possible Because the connection to the caller is logically before the connection to the rebroadcast servers, the show stream can be normalized or enhanced without compromising the real-time nature of the live connection to the caller
BRIEF DESCRIPTION OF THE DRAWINGS
Preferred embodiments of the present invention will now be described, by way of example only, with reference to the attached Figures, wherein
Figure 1 shows a block diagram of a system in accordance with an embodiment of the present invention including one host and the audience submitting questions, and
Figure 2 shows a block diagram of a system in accordance with an embodiment of the present invention including one host, a moderator and the audience submitting questions, and Figure 3 1 shows an example of the interface for the host software, and
Figure 3 2 shows an example of the interface for the moderator software, and
Figure 3 3 shows an example of the caller or question submission software, and
Figure 4 shows a block diagram of an audience member connecting to the servers to join a show, and
Figure 5 shows a block diagram of an audience member submitting a question to a show, and
Figure 6 shows a block diagram of a show host connecting to the servers to begin a show broadcast; and Figure 7 shows a block diagram of a moderator connecting to moderate a show and sorting, prioritizing and screening questions, and
Figure 8 shows a block diagram of a host connecting to an audience member for a voice chat to be broadcast as part of the show, and
Figure 9 shows the audio mixer used to include the audio from an audience member conversing with the host, and
Figure 10 shows a complete server configuration for the system with redundancy
DETAILED DESCRIPTION OF THE INVENTION There is provided a method and system of transmitting audio and text data across the network, and displaying the text to a computer screen, and playing the audio data The system and method includes an encoding means to encode speech
and audio for transmission across the network and a decoding means to decode speech and audio received across the network The system captures audio using a microphone connected to a personal computer equipped with a sound card The audio is converted from an analog signal to a digital representation This digital representation is then converted using audio compression codecs to highly compressed digital audio A video camera may be included to transmit the video of the host, and any video image would be encoded and sent m the same way as the host's audio signal
Also included is a transmission means to send encoded speech and audio to and from client applications, and to and from the servers broadcasting the show The highly compressed digital audio is converted to packets of data suitable for transmission across a packet-based network such as the Internet The packets are sent using various network protocols such as streaming HTTP, TC/P and UDP Conversation between the host and the live connected audience member utilize UPD and the SOCKS 5 standard. All communication between the software and the control applications hosted on the system's servers utilize UPD and the SOCKS 5 standard All communication with the servers originating on HTML pages, or opening HTML pages utilizes HTTP. The shows are broadcast using the most efficient protocol available, given the precise network configuration an audience member has. The streaming servers first attempt a multicast connection, then attempt a UDP connection, and then attempt a TC/P connection If these all fail a final attempt to stream the show is made using streaming HTTP
The audience connects to show s by entering an IP address or URL using an Internet browser This downloads an HTML script that the browser uses to render a web page [Fig 4- block 1] On this web page are hyperlinks [Fig 4- block 2] that will send request to the web server to join a particular show [Fig 4- block 3] using HTTP This invokes a server script, such as an ASP or CGI script that will verif that the usei has permission to join the requested show [Fig 4-block 4], requesting a password if necessary [Fig 4- block 5] It is envisioned that authorization could also be implemented using IP addresses, cookies or certificates instead of typing in a password Once a user is authenticated the server sends another HTML page [Fig 4- block 6] that includes scripts to launch the multimedia player that will decode the packets and cause the digital data to be played as analogue sound, as well as format
the "Ask Host" button or link [Fιg.4-block 7]. The HTML page can also provide scripts to decode instructions contained as part of the audio stream [Fιg.4-block 8] to display video, open files such as images or modify the HTML page itself. Such images would be served by Content servers using HTTP [Fig 10-block 4] Also included is a means to send and receive text data between the audience, the servers, moderators and the host, as well as provide text feedback and status updates to the users of the system The text and feedback comments are converted to packets of data suitable for transmission across a packet-based network such as the Internet These packets are them sent utilizing UDP and SOCKS 5 Audience members submit questions by text, which the host or moderator can then sort, accept, reject or drop The audience member can also withdraw the question at any time, or change the question at any time until it has been accepted by the host or moderator. Questions submitted by the audience are stored on the servers by an application designed specifically to receive, manage and store these questions. When a show is created, the application on the server creates space in its database to store questions for the show [Fιg.6-block 8], as well as space to store text chat comments between the host and moderators. These comments are automatically forwarded to all moderators and hosts as a group. During a show the audience is presented as part of the HTML page used to play the show a hypertext link or button labeled "Ask Host", or some variant there of, that can be 'clicked' to submit a question [Fιg.5-block 2] This sends a request to the web server using HTTP [Fig.5 -block 3], that invokes a server script, such as an ASP or CGI, that will generate a ticket used to launch the client software [Fig.5 -block 4]. The ticket contains the Internet address of the question server, the ID number of the show and a security verification number The user can then type a text question using client software specifically designed and created to submit questions to the question server. The audience member types the question into a text box available on the interface of the software [fig5-block 5], then presses "Submit" [Fιg.5-block 6]. This sends the question [Fιg.5-block 7], plus information about which show that the question is for, authorization information that is checked against a database of authorized users, the network address of the submitter, such as an IP address and port, and the type of request this is (Submit or Withdraw question) [Fιg5-block 6, 1 1, 12, 9]. The question is then registered on the database [Fιg.5-block
10, 13] and the host's or moderator's software is sent updated information [Fig.6- block 14, Fig.7-block 13] based on the contents of the database.
When the status of an audience member's question changes, a text notification, possibly together with an attention-getting sound, is sent back from the server to the audience member, for example when a question is 'Next' or 'Live' [Fig5 block 14, Fig 7-block 15, Fig.8-bock 4, 8, 1 1]. When the host or a moderator makes changes to a question's position in the list of questions, or its status (Next, Live. Ignored, Delete), this information is sent as UPD packets to the question server, where status flags associated with the questions are changed [Fig.5-block 10. 1 1 Fig.7-block 14, Fig.8- block 3, 7, 15].
The data packet referred to in Block 8 of Fig. 5 contains; Show ID #, User Verification #, User Name, User IP address and port, [Text Question OR Delete question command].
The database referred to in Block 10 of Fig. 5 has entries which include Show ID, User Name, Text Question, User IP address and port, Status of question
[submitted, queued, next, live, ignored], time stamp, position in moderator list [which moderator, 0 for none; position in list as sorted by moderator], position in host list as sorted by host.
The Ticket referred to in Block 7 of Fig. 6 has IP address of server, port or socket to use for control stream, port or socket to use for question data, Show ID #, show authorizations
The step summarized by Block 8 of Fig. 6 involves Create channel and station for show stream, storage area for archive, entry in question server database plus space for storage, create session on Control Service and open persistent UDP connection for control and monitoring for quality of service.
The Verification step referred to in Block 9 of Fig. 6 includes whether all channels, ports and sockets on servers are accessible, status of control channel, software client self-diagnosis, address of broadcast point used by Host client for testing In the step summarized by Block 16 of Fig. 6, the Host client prepares a broadcast point on the local computer, limited to one connection and sends this information to the servers. The servers call back to this point to receive the show
stream, which is then rebroadcast to the audience by the rebroadcast servers
There is also provided a means for the host or moderator to control remotely the ability of an audience member to transmit voice and audio data from their personal computer and microphone to the host or moderator's computei Using the software client used by the audience to submit questions, a UDP connection is opened directl) between the host and the audience member Control information flows from the host to the audience member such that the host can adjust the recording level of the audience member's microphone in real time, can disconnect the audience member from the conversation, and can disable the audience member's microphone using functionality built into the audience client Also, when the connection is first established between the host and the audience member, the question submission client instructs the show player to mute the show stream to avoid feedback or echoes It is envisioned that this same connection, with the same abilities, can also be opened between the audience member and a moderator allowing the moderators to voice interview selected audience members, and send notification to the question server about the success of such interviews
When an audience member's question is accepted by the host [Fιg.8-block 1, 6], and it is the audience member's turn to be 'on air', their talk radio client application controls their ability to broadcast The client allows a direct channel is opened between the audience member, or caller, and the host [Fig 8-block 9], who can then engage m conversation with the host [Fig 8-block 12] Because this is a direct connection with a small number of participants, there will be minimal lag and netwoik quality issues The resulting conversation is then mixed into a single stream [Fig 6- block 14, Fig 8-block 13] and streamed to the broadcast servers for distribution to the audience in real-time Fig 8-block 14] To establish such a connection the host must first assign the 'Next' status to a question [Fig 8-block 1] This is done by dragging the question to the 'Next' area of the host's interface or by double-clicking the question This sends a packet to the question server using UPD and SOCK5 that changes a flag associated with the question [Fig 8-block 3] When the flag changes a notification to the submitter of the question is automatically sent to [Fιg.8-block 4] and the host is sent the necessary connection information such as the submitter's IP address and port [Fig 8-block 5] The host can then advance the question from 'Next'
to live, either by dragging it, or by pressing 'Connect' [Fig 8-block 6] This sends a packet to the question server that changes a flag associated with the question [Fig 8- block 7], and generates a notification to the submitter [Fig 8-block 8] Using the provided IP address a peer to peer, direct connection utilizing UDP and SOCKS5 is created and verified [Fig 8-block 10] The host and audience member can now talk directly [Fig 8-block 12] m real time, while the conversation is mixed [Fig 8-block 13 Fig 9] and rebroadcast to the servers [Fig 8-block 14] When either the host disconnects the callei. or the caller closes the question client [Fig 8-block 15], the status of the question in the question ser er database is changed and the question is deleted
There is also provided a server infrastructure to stream encoded speech and audio to a multitude of clients, as well as authorizing users and maintaining Internet addresses of users The host, and any conversations with audience members the host has, are captured and broadcast by the server infrastructure Usually, this will happen in near real-time, as soon as the voice and audio has been mixed and prepared by the server for broadcast, increasing the quality of the broadcast The usual delay used is the same delay used in radio broadcasts, approximately 7 second.
The server architecture consists of one or more web servers [Fig.10-block 1 ] responsible for running scripts used to generate tickets used to launch software, web pages used to navigate and select show lists, templates used to play and participate m shows and maintain SQL databases used in authorization, as well as indexing all content such as shows available, and sho s stored Shows are streamed for rebroadcast to a master control server [Fig 10-block 2] responsible for receiving shows, running the services that allow questions to be submitted, running services that allow the automation of broadcasting show s and distribution of shows to the rebroadcast servers There are one or more rebroadcast servers that receive the show streams from the master control serv er [Fig 10-block 3] These rebroadcast servers are connected to using the player embedded [Fig 4-block 6] in the web pages served by the web servers [Fig 10-block 1 ] and are responsible for replicating the streams as necessary to serve all of the audience In cases where other content is used m shows by including references to the content with the audio stream, this content is served using HTTP requests from Content servers [Fig 10-block 4] All data is stored on disk
arrays to allow easy backup, failsafe storage and fast delivery [Fig.10-block 5]. Shows are archived automatically and stored on the disk arrays.
There is also provided a method to mix the resulting conversation between the host and the caller into a single stream of packets that is included as part of the host software running on the host's computer. Speech data from the host [Fig.9 "H"] and the caller [Fig.9 "C"] is examined many times a second |Fig.9-block3,6] to determine when the host alone is speaking [Fig.9-block 8], when the caller alone is speaking [Fig.9-block 7], and when both are speaking simultaneously [Fig.9-block 9]. Where the host alone is speaking the host's audio data is compressed using an audio codec [Fig.9-block 12], converted to packets [Fig.9-block 13] and is streamed to the master server [Fig.9-block 14]. When the caller alone is speaking [Fig.9-block 7] the stream of compressed audio is converted to packets [Fig.9-block 13] and streamed to the master server [Fig.8-block 14]. When both are speaking simultaneously [Fig.9-block 9], the caller's compressed data is decompressed using a codec to decode it to digital audio [Fig.9-blocl 10], the uncompressed digital audio is mixed with the never compressed digital audio from the host [Fig.9-block 1 1] and the result is compressed using a codec [Fig.9-block 12], converted to packets [Fig.9-blockl3] and streamed to the broadcast server [Fig.9-block 14]. The host and caller audio streams are samples at least ten times per second. Because decompressing and recompressing audio data results in lowered sound quality, the sound data is only decompressed and recompressed when made absolutely necessary by overlapping speech. Because of the mathematical nature of compression algorithms, and the sacrifices made to achieve extremely small data sizes, it is not feasible to mix speech will compressed. The time slicing mixer described above eliminates most of these difficulties and maintains the best possible sound for most situations.
The advantage of the network based talk show system described here is that it allows a host, or a host and moderator, to screen large numbers of question and focus the show only on appropriate question. By providing text descriptions of what a caller wishes to discuss, plus a means to sort, organize and identify callers with their questions, the host and moderators can quickly eliminate unwanted questions by deleting them, or 'setting their flags to 'Ignore'. Questions can be sorted into prioritized lists, and hosts can connect to callers in a controlled, moderated fashion.
Callers can be pre-screened based on their text questions, speeding the process and minimizing the amount of bandwidth This system also allows the host and caller to talk without needing to maintain a live, synchronized connection with the entire audience The show stream is broadcast from a number of broadcast servers that replicate the stream uploaded by the host to the master control server The show can be broadcast with a slight delay to allow normalization, or monitoring for inappropriate content, without disrupting the interactive feel of the show. Because
network delays are minimized for the host and caller, which is heard uninterrupted by the audience, the show appears to be live and real-time The connection between the host and caller is before any server induced delay, similar to the way a radio broadcast is transmitted using a seven second delay, but telephone calls to the radio station are patched in before the delay, so an illusion of live broadcast is maintained. The advantage of the described audio mixing method is that it allows conversations to be broadcast using the same network bandwidth the host uses when there is no caller connected. Another advantage of the described audio mixing is that it preserves the integrity of the host's voice at all times and the callers voice at most times, sacrificing audio quality only when both voices overlap and quality is already compromised. Bandwidth is never doubled, because the voices are mixed into a single digital audio source before being encoded and packaged when the host and caller are speaking simultaneously, and passed directly past through the mixer when only one or the other is speaking. Doubling the bandwidth when both speak presents serious problems because it causes a spike in utilized bandwidth that can cause network problems especially during large show broadcast, while always decoding and mixing the callers voice causes degradation common when compressed audio is decoded and recompressed
The above-described embodiments of the invention are intended to be examples of the present invention and alterations and modifications may be effected thereto, by those of skill m the art, without departing from the scope of the invention