WO2001067214A2 - System and method for tracking user interaction with a graphical user interface - Google Patents

Abstract

A system and method for tracking user interactions with a graphical user interface (e.g., those used by web browser applications). User activity may be monitored in a number of ways, e.g., by using a video camera to detect user movement, or by detecting mouse or cursor movements. Information generated by monitoring user activity may be used to determine the user's response to certain displayed content (e.g., advertising). Information regarding user activity may be stored locally along with information about the content being displayed and other data (e.g., the date and time that the content was displayed). This information may then be transmitted to a remote server (e.g., in a background task performed when the application generating the user interface is idle). The remote server may be configured to accumulate this information for multiple users.

Description

TITLE: SYSTEM AND METHOD FOR TRACKING USER INTERACTION WITH A GRAPHICAL

USER INTERFACE

Background of the Invention

1. Field of the Invention

The present invention generally relates to network browsers and graphical user interfaces. More particularly, the present invention relates to systems and methods for customizing graphical user interfaces.

2. Description of the Related Art

The Internet is an important resource for reference, research, browsing, online shopping, and e-mail. The Internet provides worldwide access to materials in a variety of subject areas. As used herein, the "Internet" is a decentralized global network connecting a large number of computers through standard commmiication and data protocols. One of the most well-known aspects of the Internet is the World Wide Web. As used herein, the "World Wide Web" is a plurality of web pages interconnected by a set of hypertext links and accessible to computers on the Internet. As used herein, a "web page" may include an information-bearing document or application which is accessible on the World Wide Web. Web pages may also include graphics and data files containing images, video clips, three-dimensional graphics, and audio information. A group of web pages sharing a particular address may be collectively referred to as a "web site".

Information accessible via the Internet such as web pages are typically accessed using a browser. A browser is typically a software application that generates a graphical user interface so that a user may access the information in a user-friendly manner. A number of different browsers are currently available, e.g., Microsoft's Internet Explorer, Netscape's Navigator, and Opera from Opera Software.

Browsers have contributed to the recent growth in popularity of the Internet much in the same way that graphical user interfaces have contributed to the increased popularity of personal computers. The graphical user interface of browsers have allowed more people to access the Internet without specialized training and have spawned a deluge of new applications, including electronic banking and stock trading, interactive news, and online shopping. As web usage increases and as new users, unfamiliar with computers wish to use the Internet, a new trend toward Internet appliances has begun. Internet appliances is a term used to refer to any simplified computer or appliance that may be used to access the Internet and or perform other networked functions. Thus, Internet appliances are also sometimes referred to as network appliances. In their first generation form, most Internet appliances are inexpensive limited-purpose personal computers. They typically have displays, keyboards, touchpads, and other input devices, but they may not have all of the processing power or peripherals that a tradition personal computer has. For example, some Internet appliances may be configured without diskette drives, CD- ROMs, or DVD-ROMs. Because they are designed solely for Internet access (i.e., web browsing and email), they may have less expensive components (e.g., less memory and smaller or no hard drives).

The large numbers of people now spending time browsing the Internet have encouraged merchants, service providers, and traditional retailers (collectively "retailers") to establish a presence on the Internet. The low hardware cost of Internet appliances is now allowing some retailers to provide customers with free Internet appliances and Internet access in exchange for the right to control some aspects of the user's browsing experience. These controlled aspects may include the ability to display particular advertisements and branding information while the user is browsing and the ability to set the default web page that appears whenever the user first turns on the appliance or accesses the web (sometimes referred to as a "home page"). Next-generation Internet appliances may take this trend further by integrating browsers and Internet access functions into traditional appliances (e.g., refrigerators) that perform specific functions using the Internet (e.g., automatically ordering groceries). Thus,

Internet appliances may require customized user interfaces specifically tailored to the particular application and the particular needs of end users.

To implement these customized features, programmers have typically had to write new browser applications for each customization. While some software code may be reused, writing new code to change the appearance or functionality of the user interface is time consuming, expensive, and may require specialized expertise. For these reasons, there is a need for an improved method for customizing the user interface of browsers.

Another particularly desirable feature for retailers interested in this market model is the ability to effectively force users to view advertisements. From the retailer's point of view, since they are paying for the hardware and software (and possibly the Internet connection), the retailer wants to ensure that the user views the advertisements. This is currently a problem because end-user applications are available to filter out banner advertisements. If an end user were to download and use one of the filtering plug-ins, the retailer's investment in the browser and appliance could be undermined. Similarly, it may be particularly advantageous for the retailer to control certain aspects of the user interface in addition to or in lieu of advertising.

Yet another potentially desirable feature for retailers interested in advertising using the Internet would be the ability to more accurately track consumer's responses to particular web pages and/or advertisements. As the amount spent on Internet advertisement increases, demand for this feature may increase.

Summary of the Invention

The problems mentioned above are at least in part solved by a system and method that implements a customizable framework for a graphical user interface based on a configuration file storing information that is usable to generate a customized user interface. The configuration file may be configured to be easily updateable via a remote server without any required interaction from the end user. In some embodiments, each end user may have their own customized configuration data set that may be automatically updated and may be based on personal information, user-specific browsing history, and/or user-specific appliance usage. Please note that as used herein, the terms "configuration file" and "configuration information" are used interchangeably. Configuration file is a broad term meant to encompass all data storage mechanisms capable of storing information for use in generating the graphical user interface. For example, one or more individual data arrays, strings, variables, memory locations, or traditional files within a directory structure may all be considered to fall within the definition of "configuration file" and "configuration information" as the terms are used herein. In one embodiment, the method may be embodied as a browser software program that is stored on a computer-readable medium (e.g., diskette, CD-ROM, DVD-ROM, tape, RAM, ROM). The browser may include a plurality of instructions that are configured to access a configuration file upon start-up (e.g., power-up, application start, or user login). The configuration data set may include, inter alia, data defining a customized user interface appearance and functionality. The customized user interface may include one or more user interface objects (also referred to as controls or user interface elements), and the configuration data set may include specific data corresponding to each of the customized or customizable user interface objects. For example, an object may be a graphical icon or menu item that is linked to a particular function or web site. The browser may be configured to use the information stored in the configuration file to generate and display a customized user interface on a display device. The browser may be further configured to monitor user interaction with the user interface objects and respond to user interaction with the objects. For example, the browser may be configured to access a particular web page that is linked to a particular icon that the user has selected (i.e., clicked on).

In some embodiments the browser may be configured to prompt the user for a password. The browser may then be configured to use this password to select a particular configuration from a number of different configuration files, wherein each different configuration file may correspond to a different user or user profile. Advantageously, this may allow the browser to efficiently support customized user interfaces for multiple users. The configuration file may contain pointers or links to graphics files that contain the data needed to generate icons and other visible and audio portions of the user interface. In some embodiments, multiple configuration files may share some or all of the graphics files. The configuration file may also include size and position information that is needed by the browser (or an underlying operating system) to position and render the visual information in the graphics files as part of the user interface. The configuration file may further include linking information describing which functional modules (e.g., accesses to a particular web page) should be invoked in response to a particular user interface object being selected by said user.

In addition to position, size, and linking information for the graphics files, the configuration file may also include similar information necessary to invoke one or more browser plug-in modules (e.g., audio or digital video stream players). For example, one browser plug-in may be an electronic wallet module. The electronic wallet module may be passed information by the browser (or operating system) based on the information provided by the user when logging in (e.g., thus requiring the user to only enter their password once). In some embodiments, the configuration file may further include features such as a default home page address and default search engines that are also customizable and updateable via a remote server.

Advantageously, the use of the configuration file allows the browser to efficiently update the configuration file. For example, the browser may be configured to access a server on a periodic basis (e.g., at each power up, application start, log-in, or every hour) to update the configuration file. These updates may include updated linking information, new graphics files, new position and size information, and new plug-in modules and stand alone applications for additional functionality. The browser may be configured to wait until a predetermined idle state is reached before performing an update. In some embodiments, the user may be locked out from modifying some or all of the information in the configuration file. Depending upon the implementation and security needs, one or more of the graphics files and/or configuration files may reside on a server instead of residing locally on the Internet appliance/computer system executing the browser application. In some embodiments, the browser may be configured to convey information to a predetermined server (or servers) about a user's actions while executing the browser, wherein the server is configured to update the browser's configuration file based on the information about the user's actions. Similarly, the server or servers may be configured to update the configuration file based on geographical information stored in the configuration file (e.g., the user's address) or provided by the end user. This may advantageously enable location-specific customized user interfaces. The update may be configured to support "checkpoints", which allow the update process to be restarted from a checkpoint instead of the beginning of the update in response to the interruption of the update process (e.g., to devote more resources to user activities). Since the update process may entail downloading entire new applications, supporting checkpoints may reduce the time needed to complete an interrupted update. In some embodiments, the data for the update is downloaded while the appliance or browser is in an idle mode, but the data may not be used to modify the configuration file (i.e., configuration information) until the next time the browser or appliance is started up (e.g., after a sleep mode or power down).

An Internet appliance configured to execute a customizable browser is also contemplated. In one embodiment the appliance may include a display device (e.g., an LCD panel), a user input device (e.g., a keyboard), a network communications device (e.g., a modem), and a memory (e.g., a flash memory) configured to store the browser application and the configuration file, and one or more processors configured to execute the browser application. The processors are configured to receive data from the input device, the network communications device, and the memory, and are configured to output data to the display device and the network communications device. The memory may store an operating system (e.g., Linux, a Windows variant, or other operating system), the browser program, and the user interface configuration file. The operating system may be configured to automatically execute the browser program (e.g., upon power up), and the browser program may be configured to automatically read the configuration file upon execution. As noted above, the browser may be configured to use the information in the configuration file to define the appearance and functional attributes of a customized user interface for the browser.

In some embodiments, the Internet appliance may include additional hardware (e.g., a thumb or retinal scanner, or a microphone) which the browser may be configured to use to automatically select one of a plurality of configuration files. Advantageously, this may improve the speed of the login process and also improve security.

These and other benefits and advantages of the present invention shall become apparent from the detailed description of the invention presented below in conjunction with the figures accompanying the description.

Brief Description of the Drawings

A better understanding of the present invention may be obtained when the following detailed description of the preferred embodiment is considered in conjunction with the following drawings, in which: Figure 1 is a network diagram of one embodiment of a wide area network which is suitable for implementing various embodiments;

Figure 2 is an illustration of one embodiment of a typical computer system which is suitable for implementing various embodiments;

Figure 2A is an illustration of one embodiment of an Internet appliance; Figure 3 is an illustration of one embodiment of a sample web browser user interface;

Figure 4 is an illustration of another specific embodiment of a web browser user interface; Figure 5 is an illustration of yet another specific embodiment of a web browser user interface; Figure 6 is a block diagram illustrating one embodiment of a customizable framework for a browser user interface; Figure 7 is a block diagram illustrating one embodiment of a configuration file usable to implement a customizable browser framework;

Figure 8 is a flowchart of one embodiment of a method for generating a configuration file usable to implement a customizable browser framework;

Figure 9 is a flowchart of one embodiment of a method for using the configuration file and customizable framework in a browser; and

Figure 10 is a flowchart illustrating one method for storing data generated by browser users. While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the present invention as defined by the appended claims. Please also note that the headings used herein are for organizational purposes only and are not meant to have any effect on the interpretation of the claims or the detailed description.

Detailed Description of Several Embodiments Before describing a number of different embodiments in greater detail, some information regarding networks, personal computers, and Internet appliances is discussed.

Figure 1: wide area network

Figure 1 illustrates one embodiment of a wide area network (WAN) 102. WAN 102 is a network that spans a relatively large geograpliical area. The Internet is one example of a WAN. WAN 102 typically includes a plurality of computer systems which are interconnected through one or more networks. Although one particular configuration is shown in Figure 1, the WAN 102 may include a variety of heterogeneous computer systems and networks which are interconnected in a variety of ways and which run a variety of software applications.

One or more mainframe computer systems 120 may also be coupled to WAN 102. As shown, mainframe 120 may be coupled to a storage device or file server 124 and mainframe terminals 122a, 122b, and 122c. The mainframe terminals 122a, 122b, and 122c may access data stored in the storage device or file server 124 coupled to or included in the mainframe computer system 120. WAN 102 may also include computer systems 112b, personal d gital assistants (PDAs) 128, and Internet appliances 126 and 113 (e.g., a refrigerator configured to order groceries using the Internet) which are connected to WAN 102 individually. For example, WAN 102 may include computer systems which are geographically remote and connected to each other via the Internet.

One or more local area networks (LANs) 104 may be coupled to WAN 102. LAN 104 is a network that spans a relatively small area. Typically, a LAN 104 is confined to a single room, floor, building or group of buildings. Each node (i.e., individual computer system or device) on LAN 104 preferably has its own CPU with which it executes programs, and each node is also able to access data and devices anywhere on LAN 104. Thus LAN 104 allows many users to share devices (e.g., printers) as well as data stored on file servers. LAN 104 may be characterized by any of a variety of types of topology (i.e., the geometric arrangement of devices on the network), of protocols (i.e., the rules and encoding specifications for sending data, and whether the network uses a peer-to- peer or client/server architecture), and of transmission media (e.g., twisted-pair wire, coaxial cables, fiber optic cables, radio waves). LAN 104 typically includes a plurality of interconnected computer systems and optionally one or more other devices: for example, one or more workstations 110a, one or more personal computers 112a, one or more laptop or notebook computer systems 114, one or more server computer systems 116, and one or more network printers 118. As illustrated in Figure 1, LAN 104 may include one of each of computer systems 110a, 112a, 114, and 116, and one printer 118. LAN 104 may be coupled to other computer systems and/or other devices and/or other LANs 104 through the WAN 102. Figure 2: typical computer system

Figure 2 illustrates a typical computer system 150 which is suitable for implementing various embodiments of the system and method for creating a customizable browser as described in greater detail below. Computer system 150 typically includes components such as a CPU 152 with an associated memory medium 160. The memory medium may store program instructions for computer programs, wherein the program instructions are executable by the CPU 152 (or more specifically, by the one or more processors within CPU 152). The computer system 150 may further include a display device such as a monitor 154 (e.g., a liquid crystal display or "LCD", a cathode ray rube display or "CRT", a head mounted display, or a projection display), an alphanumeric input device such as a keyboard 156, and a directional input device such as a mouse 158. Alternatively, keyboard 156 and/or mouse 158 may be replaced or supplemented with a microphone and speech recognition hardware and/or software, a video camera, and/or a track pad. The computer system 150 may be operable to execute the computer programs to implement creation of a customizable framework for a browser to be used with the World Wide Web as described herein.

The computer system 150 preferably includes memory medium 160 on which computer programs according to various embodiments may be stored. The term "memory medium" is intended to include an installation medium, e.g., a CD-ROM, or floppy disks, a computer system memory such as DRAM, SRAM, EDO RAM, Rambus RAM, etc., or a non-volatile memory such as a magnetic media, e.g., a hard drive, or optical storage. The memory medium may include other types of memory as well, or combinations thereof. In addition, the memory medium may be located in a first computer in which the programs are executed, or may be located in a second different computer which connects to the first computer over a network. In the latter instance, the second computer provides the program instructions to the first computer for execution. Also, the computer system 150 may take various forms, including a traditional personal computer system, an Internet appliance as previously discussed, a mainframe computer system, a workstation, personal digital assistant (PDA), television system or other device. In general, the term "computer system" may be broadly defined to encompass any device having a processor which executes instructions from a memory medium. The memory medium preferably stores a software program or programs for creating a customizable framework for a browser as described herein. The software program(s) may be implemented in any of various ways, including procedure-based techniques, component-based techniques, and or object-oriented techniques, among others. For example, the software program may be implemented using ActiveX controls, C++ objects, JavaBeans, Microsoft Foundation Classes (MFC), Java, traditional programs, or other technologies or methodologies, as desired. A CPU, such as the host CPU 152, executing code and data from the memory medium includes a means for creating and executing the software program or programs according to the methods and/or block diagrams described below.

Figure 2A: sample Internet appliance Figure 2A illustrates one embodiment of an Internet appliance 180. In this embodiment, Internet appliance

180 is an iBrow™ brand Internet Appliance from Merinta Corporation. Appliance 180 may comprise display device 154 (e.g., a 10" LCD display), CPU 152, and keyboard 156. In some embodiments, keyboard 156 may be wireless (e.g., infrared) and may include an integrated pointing devices such as a track ball, mouse, or joystick. Appliance 180 may also comprise speakers 170A-B, and a video camera 172. In some embodiments, appliance 180 may further comprise indicators 174 and additional buttons (e.g., power button 176). While each configuration may vary, in one embodiment CPU 152 may comprise one or more universal serial bus connectors, a 56k v.90 internal modem, a microphone input, and amplified headphone output. CPU 152 may also be configured with an external volume control, external contrast control, an AC power adapter, dual RJ-11 phone jacks for line-in and telephone- out, and both flash memory and SDRAM (synchronous dynamic random access memory). Indicators 174 may be configured to inform users of power up status or other information.

Other optional hardware may be included with Internet appliance 180, for example, a bar code scanner (for user identification and product identification), a smart card reader (for user identification), a Java™ ring interface (for user identification), camera (for user identification and for video conferencing), a retinal scanner (for user identification), a thumb print scanner (for user identification), a microphone (for voice recognition and speech recognition), or a voice synthesizer (for text-to-speech).

Figure 3: sample web browser user interface

Figure 3 illustrates one embodiment of a web browser user interface 300. At the top of the web browser user interface 300, several toolbars are represented: a sizing toolbar 303, a title bar 304, a navigation toolbar 305, a URL toolbar 306, and a persistent portal 307. Below the persistent portal 307, a main content section 301 is shown. Scroll bars 302A-B may be found to the right and also below the main content section 301. At the bottom of the web browser user interface 300, additional toolbars are represented: a status toolbar 308 and a live feed toolbar 309. To the right of the status toolbar 308 and the live feed toolbar 309, an advertisement 310 is shown. Note, advertisement 310 may be configured to display a different image periodically (e.g., a new image every 15 seconds or every 0.10 seconds). This allows animation within the advertisement and allows different advertisements to be displayed to the end user. Logo bar 312 may be implemented to display a retailer's or manufacturer's logo. Logo bar 312 may also be animated. Depending upon the exact implementation, logo bar 312 may also be persistent, i.e., ever-present regardless of what the user is doing, which page the user is browsing, and regardless of which applications the user is executing.

Elements of the sizing toolbar 303 may include, for example, a logo, a minimize button, a maximize button, and a close button (not shown). The title bar 304 may include the name of the browser, the name of a web page currently behig viewed, or the name of a sponsor of the web browser (not shown). Elements of the navigation toolbar 305 may include a back button, a forward button, a stop button, a reload button, a home button, a print button, a font button, a window button, a help button, a favorites button, a search button, and an e-mail button (not shown). The URL toolbar 306 may include an entry box into which a user may enter a URL address, and a GO button to load the entered URL address (not shown). Elements of the persistent portal 307 may include labeled buttons, tabs, or other links, along with a logo of the sponsor of the web browser (not shown).

The main content section 301 displays the contents of web pages and may change based on the choice of button that the user selects (i.e., from the navigation toolbar 305 or from the persistent portal 307) or based on the URL address that the user enters in the URL toolbar 306. The scroll bars 302A-B allow the user to view consecutive lines of data on the display screen, both vertically and horizontally. As used herein, "scroll" means that once the screen is full, each new line appears at the edge of the screen and all other lines move over one position. For example, when you scroll down, each new line appears at the bottom of the screen and all the other lines move up one row, so that the top line disappears. Vertical scrolling refers to the ability to scroll up or down. Horizontal scrolling refers to the ability to scroll left or right. The status toolbar 308 may be configured to indicate the current status of the browser. For example, status toolbar 308 may indicated what percentage of the current page has been received across the network connection (e.g., "50% complete) or the address that is currently being accessed (e.g., connecting to "www. merinta.com...").

The live feed toolbar 309 may be configured to display data that is periodically updated. Examples of content that may be displayed in the live feed toolbar 309 includes a stock ticker, sports scores, and news headlines. This content may be automatically updated on a periodic basis (e.g., every five minutes). Please note that other types of toolbars, portals, and content areas may also be implemented using the customizable framework disclosed herein. The examples cited herein are merely for illustrative purposes and are not meant to be limiting.

The advertisement 310 may be a banner advertisement that links to a particular web page when the advertisement area is clicked on by the user. The banner advertisement may be animated to attract the viewer's interest, and the advertisement may change periodically (e.g., once every minute) or based on the user's browsing activity.

Specific examples of the various elements of sample browser user interfaces 300 are shown in Figures 4 and 5.

Figures 4-5: specific browser user interface embodiments

Figure 4 illustrates a specific embodiment of web browser user interface 300. Figure 5 illustrates another specific embodiment of web browser user interface 300. As these figures illustrate, the visual appearance of user interface 300 may change dramatically based on the different customizations made. Examples include the different organization of icons, different positions for the toolbars, different fonts, and different titles in title bars 304.

In addition to changing the visual appearance of user interface 300, functional changes may also be implemented. For example, the web pages linked to the different icons in persistent portal 307 may be customized. A bank providing an Internet appliance to its customers may wish to have links as illustrated in Figure 4 (e.g., to pages that perform electronic bill payment, loan application pages, or account information), while a retailer such as Amazon.com may wish to have links as illustrated in Figure 5 (e.g., to pages that sell music and videos, books, and travel reservations). In some embodiments, one or more of the toolbars such as persistent portal 307 may be fixed (i.e., the user cannot change, hide, or close the toolbars). In other embodiments, users may be allowed to move the toolbars around, resize them, or even hide them. In some embodiments users may also be allowed to select the icons and the web pages to which the icons link.

Figure 6: customizable browser framework embodiment

Turning now to Figure 6, a block diagram illustrating one embodiment of a framework for a customizable browser is shown. As the figure illustrates, a user interface construction application 400 is used to generate one or more configuration files 402A-402B. These configuration files store, inter alia, information on linking, position, size, color, geometry, graphic image, behavior, associated applications, and fonts. This information is useable by web browser 410 to generate a customized user interface. The configuration files 402A-402B may include linking and position information for one or more graphics files 404A-404B, one or more third party modules 406, and one or more Internet appliance software applications 408. Advantageously, web browser 410 may be configured to access one or more configuration files 402A-402B to generate a customized user interface. As shown in the Figure, web browser 410 is also configured to interact with operating system 412, which is responsible for managing and interfacing with the hardware of the computer (or Internet appliance). In some embodiments, user interface construction application 400 may be configured as a stand alone software application that is configured to operate on a computer system (e.g., a personal computer). In other embodiments, user interface construction application 400 may be bundled with web browser 410 as a restricted access feature available only to users with certain access privileges. User interface construction application 400 may be configured with its own graphical user interface that allows user interface designers the ability to "drag-and- drop" icons or graphical images representing different user interface features to generate a user interface layout for web browser 410. User interface construction application 400 may then be configured to store the customized user interface information into one or more configuration files 402A-402B. In some embodiments, configuration files 402A-402B may actually include the graphical information used to generate the icons and images that are incorporated as part of the customized user interface. However, in other embodiments (as shown in Figure 6) configuration files 402A-402B may merely include links (e.g., file name references or pointers) to the appropriate icon and/or graphics files 404A-404B. Configuration files 402A-402B may also include subroutines or objects that call third party modules 406. Examples of these third party modules include streaming video clip players, streaming audio players, three-dimensional graphics decompression software, encryption software and electronic wallet modules. Configuration files 402A-B may further include software that calls, executes or invokes additional software applications 408. Some possible examples of such additional Internet appliance software applications include email programs, chat programs, calculator programs, word processors, news readers, telephone and video phone applications, and computer games. For example, the user interface construction application 400 may link a particular icon that is configured to invoke an email composition application in response to the user clicking on the particular icon. In some embodiments, configuration files 402A-B may also include links to particular functions within operating system 412. For example, in some customized user interfaces, a particular icon in the graphics user interface for web browser 410 may be linked to a power down or sleep function implemented within operating system 412.

Advantageously, each configuration file 402A-402B may be linked to one or more particular users or user profiles. For example, web browser 410 and/or operating system 412 may be configured to prompt the user for a password or user name. Web browser 410 may be configured to use this information to select a particular configuration file corresponding to that user name or password. As previously noted, the customizable framework illustrated in Figure 6 may be used within a browser configured to be executed on a personal computer or on an Internet appliance. Please note that Figure 6 merely illustrates one possible embodiment for this customizable framework. As those skilled in that art will appreciate, additional features may be incorporated into this framework. For example, graphics files 404A-404B may be supplemented by audio files that are configured to play customized audio clips or sounds in response to particular user actions.

Figure 7: configuration file embodiment Turning now to Figure 7, one embodiment of configuration file 402A is shown. In this embodiment, configuration file 402A comprises a plurality of different types of information including corresponding password/user id information 420. This information may be used by the web browser or operating system to select which particular configuration file will be used for a particular user. In some embodiments, a particular configuration file may be designated as the default configuration file for any new users that do not have a password or user id. This information may also be stored with the corresponding password/user id information 420.

Configuration file 402A may also comprise general user interface information 422. Examples of this type of information may include the number of tool bars to be displayed, and whether or not the tool bars may be used by the user or hidden by the user. Configuration file 402A may also comprise specific information for each tool bar to be displayed. For example, configuration file 402A may include title bar information 424 that describes position and function information about the title bar (see item 304 in Figure 3). Title bar information 424 may include position information 426 (including minimum and maximum X and Y coordinates in screen pixels), color information 428, font information 430 (including typeface, font size, font attributes such as underlining, and font color), and information usable to link to a graphics file 432. The graphics file link information may be used to display an image (e.g., a logo) on the title bar. Note, the information described in the Figure is merely for explanatory purposes, and additional information may be included according to a particular embodiment. For example, links to graphics file information 432 may further include position information and gamma correction information.

As the Figure illustrates, configuration file 402A may also include information for an options tool bar (or persistent portal) 434. In addition to the position, color, font, and graphics file information (as described in connection with title bar information 424), icon-specific information may also be included. For example, information corresponding to a first icon 436 may also be included. In addition to the position, color, font, and graphics file information for the icon, a link or link information describing one or more associated web pages may also be included (see item 440). In some embodiments, as previously noted, a second set of information for a second icon 438 may include link information to the operating system (see item 442 in the figure). This information may be used to invoke an operating system function such as powering down the computer or Internet appliance. An arbitrary number of additional sets of information may also be included. Furthermore, additional information for each icon or control may also be included. For example, the icons may support mouse-over or rollover effects (e.g., changing appearance in response to the cursor moving over the icon) or animation. In another embodiment, each icon may comprise multiple images, and one of the multiple images is selected in response to a particular user logging on. For example, there may be a large and a small set of icons, wherein the small set of icons are used to maximize the size of the main content area 450, while the larger set may be more suitable for users that have difficulty seeing smaller icons.

Configuration file 402A may also include information regarding the content area (see item 450 in the Figure), information regarding advertisements to be displayed (see item 460 in the Figure), and live feed tool bar information 470. In some embodiments, the advertisement information 460 and/or the live feed tool bar information 470 may include links to one or more web servers for content and advertisements (see items 462 and 472, respectively, in the Figure).

As previously noted, configuration file 402A may further include information necessary to invoke one or more plug-ins 480. This information may vary depending upon the plug-in being invoked, e.g., position information. As described in greater detail below, some embodiments of web browser 410 may be configured to store information regarding users' activities while the browser is executing. In some embodiments, configuration file 402A may include such data, or may include a pointer to a data file generated using such data (see item 490 in the Figure). In yet other embodiments, a plurality of pointers 490 may be included to data files with different types of user information stored therein.

Advantageously, in some embodiments the configuration file may be used to indicate to the browser which elements of the user interface should remain constant regardless of the user's actions. For example, in some embodiments persistent portal 434 may be configured to remain on the display device regardless of which content the user is viewing. As noted in the Description of the Related Art section of the application, this may be a particularly useful feature for retailers seeking to generate high traffic volumes for e-commerce web sites. The example user interface of Figure 5 illustrated a series of icons in the persistent portal labeled "Clothing", "Banking", "Health", "News", "Music", "Movies", and "Food". As the names of these icons indicates, each icon may be linked to a particular e-commerce or advertising site. Advantageously, the retailer that provided the customized browser (and possibly the Internet appliance) may configure each icon to link to a particular site of the retailer's choice (e.g., the retailer's own web pages or the web pages of a third party that has agreed to pay the retailer a commission based on the volume generated by the persistent portal links). In some embodiments, persistent portal 434 may be configured such that the user is prevented from obscuring the portal with any windows. For example, the user interface may be configured with a pop-up calculator that may be dragged around the screen. However, the user interface may be configured such that the calculator may not be dragged over the region that defines the persistent portal. This may be a desirable feature for some retailers. Similarly, the user interface may be configured to prevent users from obscuring other toolbars, e.g., advertisement bar 310.

Unlike traditional browsers, the persistent portal may be displayed regardless of what function the user is performing (e.g., drafting an email message), and may be configured to change periodically as updated by a server (e.g., the server may update the browser's configuration file automatically). In some embodiments, as described in greater detail below, the browser may be configured to send usage information (e.g., information about which web pages were visited, for how long, content information, information about which parts of the screen the user's cursor is on or travels across, user presence information, scrolling behavior, etc.) back to the advertisement server or to another server (e.g., one operated by the retailer or browser developer). The server may then be configured to analyze the data to determine usage patterns and then change the configuration of the persistent portal to have links that better target the user's interests. As with other customization information stored in the configuration file, the configuration information for the persistent portal may be customized on a per-user basis. Thus, if an Internet appliance is used by an entire family, the persistent portal for the mother may include links that reflect the mother's interest. For example, the "Clothing" icon may display a woman's outfit and may be linked to a retailer of women's clothing. Similarly, when the father logs onto the appliance, the same icon in the persistent portal may change to display a "Golf icon comprising a golf ball, and the icon may be linked to a golfing retailer's web site based on the father's frequent visits to golf web pages. Similarly, for a child the persistent portal icon may be linked to an auction site for comic books. In some embodiments, the configuration file may also include geographical information about the user and/or the Internet appliance. For example, the configuration file may include the city and state and/or the street address of the user. The server may then be configured to customize the content of the persistent portal (or other non-persistent portions of the user interface) based on the geographic location of the user. Thus a "Restaurant" link may be linked to web pages for local restaurants that are near the user. In some embodiments the configuration file may be stored in the Internet appliance's memory (e.g., in

RAM, flash memory, or in a hard drive). In other embodiments the configuration file may be stored on a server which the browser accesses upon login or power-up.

Additional information may also be stored in the configuration file, depending upon the exact implementation. For example, if an Internet appliance is configured to be used in the lobby of a bank, the configuration file may be configured to limit the viewable web pages to a predetermined domain stored in the configuration file (e.g., the bank's domain). Similarly, the configuration file may contain information directing the browser to limit the viewable pages to a predetermined set of domains (e.g., ones approved by the bank) or to a predetermined set of web pages stored on a specific server. These limitations on which web pages the browser may access may be particularly useful if the sponsor of the Internet appliance (i.e., the retailer in the previous examples) desires to censor certain content (e.g., pornography or a competitor's web sites). Similarly, the configuration file may indicate that the user input field in which users normally enter URLs (i.e., web page addresses) should not be displayed. This limits the possible displayable web pages to only those pages that are directly linked to icons or menu entries in the customized user interface. This may once again be useful in the bank lobby scenario.

In other embodiments, additional information may be included as part of the configuration information. For example, information for cursor effects, overall "look-and-feel" of the interface, icon rollover effects, button press effects, and language information (e.g., is the user a native English or Spanish speaker).

Figure 8: generating the configuration file

Turning now to Figure 8, one embodiment of a method for generating a configuration file usable in a customizable browser framework is shown. Figure 8 illustrates a flowchart depicting one embodiment of user interface construction application 400 (also referred to herein as the "design program"). First, a user interface designer executes the design program or user interface construction application 400 (step 510). The design program may be configured to generate a default configuration file based upon one or more default selections (step 512). For example, the default configuration file may include a standard user interface with a title bar but without any advertising spaces or third party plug-ins. Next, the design program may be configured to wait until a particular user interface element is selected

(step 514). For example, the designer may be presented with a palette of different user interface elements from which to select. Next, the design program may be configured to prompt the designer for the name of a corresponding graphics file which stores an image to be displayed as the icon (step 516). Next, the design program may be configured to prompt the designer for the size (e.g., in pixels) of the icon or element (step 518). Next, the design program may be configured to prompt the designer to select a location for the icon or element (step 520). For example, the user may be presented with an image of the user interface being designed and the user may be allowed to "drag-and-drop" the icon to a particular position.

If the new icon element can be activated (e.g., selected) by the browser user (step 522), then the design program may be configured to prompt the designer for an action to be linked to the users selection or activation of the element (step 524). For example, the designer may select the power down function or the designer may select a particular web page address to be linked in response to the user's selection or activation of the element. If, however, the element is merely for aesthetic purposes, then step 524 may be bypassed.

If the element is tied to a plug-in (step 526), then the design program may be configured to prompt the designer to select one or more plug-ins and one or more corresponding messages to generate upon activation (step 528). For example, the browser and/or operating system may be configured to use "messaging" to invoke a certain plug-in or a selected action. However, if the element or icon is not tied to a plug-in, this step 528 may be bypassed. This process for configuring selected user interface elements may be repeated a number of times (step 532) until the designer has configured all of the user interface elements that were selected. The design program also may be configured to prompt the designer to select one or more additional plug-in modules for incorporation into the customized user interface, and it may prompt the designer to specify position information for any visible aspects of the selected plug-in modules. The design program may also be configured to prompt the designer for a default home page address (step 534) and for other attributes (step 536). For example, the design program may prompt the designer to enter the interval at which a particular displayed web page, advertisement, or frame is updated (e.g., for a stock ticker). Once all of the selected information is completed, the design program may be configured to store the information into a configuration file that is accessible by web browser 410 (step 538). This process may be repeated for each different profile (step 540). As previously noted, the design program may be configured to prompt the designer for a password or user id which may be used to correlate the completed configuration file with a particular user.

While a custom program for interactively generating the graphical user interface and configuration file are preferred, a more simple approach may be to utilize an editor program (e.g., an XML editor) to create and edit the configuration file. Other types of graphical user interface tool kits may also be used in some embodiments.

Figure 9: using the configuration file

Turning now to Figure 9, a flowchart of one method for implementing a customizable browser framework and for using a previously generated configuration file is shown. In this embodiment, in response to the user first powering up the Internet appliance (step 600), the operating system boots (step 602) and prompts the user for a log in or a password (step 604). The operating system then begins execution of the browser and passes the browser the appropriate user profile information (step 606). The browser is then configured to read the configuration file that corresponds to the log in information or password (step 608). Next, the browser accesses and uses the graphics files pointed to by the configuration file (step 610). The browser may also be configured to read executable information from plug-ins pointed to by configuration file (step 612). The browser may then be configured to use this information to display the user interface (step 614). As noted above, the browser may be configured to further read the address of a default home page and display this together with the customized user interface (step 14), as specified by the configuration file. As noted above, the layout, visual appearance, and functionality of the browsers user interface may be dictated by the configuration file. Once the user interface is displayed, the browser may operate in a normal manner by displaying web pages that are selected by the user (step 615). In response to the browser and/or operating system entering into a predefined "idle" state (step 616), and if sufficient time has passed since a previous update, or if there is an update available on a specified update server (step 618), the browser may be configured to automatically contact the update server and download information to update the browser. This information may include, but is not limited to, one or more of the following: the graphics files, the configuration files, browser plug-ins, the operating system, other applications, and the application browser itself (step 620). In some embodiments, the browser may be configured to automatically check a specific update server at each power up. Furthermore, in some embodiments, the browser may be configured to upload usage information while in the "idle" state (step 622). For example, this usage information may include web pages the user has visited, the amount of time the browser has been active, cursor movement information in response to particular web pages and/or advertisements, login information for each of the different user profiles, and other information. While the definition of "idle" state may vary on an embodiment by embodiment basis, generally an idle state may be declared if a predetermined amount of time has elapsed since the browser has received any user input (e.g., a cursor movement or key stroke) or received data from a server.

In yet other embodiments, the browser may also be configured to update cached web pages (e.g., based on the configuration file information) to speed browsing of pages that are likely to be viewed (step 624). For example, the configuration file may include information listing the five most commonly accessed web pages for the user. The browser may then be configured to access these web pages and download their content periodically during idle periods (e.g., once per hour). Advantageously, this may speed upthe browsers performance and reduce latency when the user selects a particular commonly accessed web page.

Figure 10: storing information generated by the browser user

Turning now to Figure 10, one embodiment of a method for storing data based on a user's browsing activities is shown. In one embodiment, in response to the user logging on to the appliance or browser (step 650), the browser is configured to use the login information to read a corresponding configuration file (step 652). As previously described, the browser is configured to generate a corresponding user interface (step 654). The browser may be configured to determine whether or not the accessed profile is configured to allow tracking of the user's browsing activities (step 656). In some embodiments, some user profiles and corresponding configuration files may be configured to prevent the browser from tracking the user's browsing activities (step 657). This may be particularly advantageous when multiple users are using the browser, or when a particular user desires privacy for their browsing activities. If, however, the user profile allows tracking (or the user consents to tracking via a "click-through" agreement), then the web page address of each loaded web page may be stored (step 658). Additional information may also be stored, e.g., the time from the start of load to the completion of loading of the web page (step 660). Examples of other types of information that may be stored include the time from the completion of a web page load until the user performs a predetermined action (e.g., movement of the cursor, a mouse click, or a key stroke) (step 662). Similarly, the location of the user action (e.g., mouse click) and the type of action may be stored (step 664). After a predetermined amount of time and after the browser is in its idle state (step 668), the web browser may be configured to process the data (step 670). Examples of processing the data may include compressing the data or compiling statistics based on the data (step 670). The processed data may then be transmitted to a predetermined data server (step 672). Advantageously, this configuration may allow a browser vendor to store usage information garnered from a large number of browser users. This usage information may be particularly valuable (e.g., it may be sold to advertisers).

Please note that although the flowcharts in the figures are detailed, they are meant for explanatory purposes only and that additional steps may be added or that steps may be combined or performed in a different sequence, depending upon the exact implementation. In some embodiments, the browser may be configured to signal a predetermined server in response to the user either logging in or accessing a particular web site. The server may then be configured to send an indication to a telemarketing firm that the user is at home. The telemarketing firm may thus avoid the wasted time spent calling people that are not home. The telemarketing firm may then call, email, or send an instant message to the user. For users that only have one traditional phone line for computer and voice (i.e., instead of a DSL or cable modem configuration), the browser may be configured to send a signal to the server before disconnecting, thus allowing the server to inform the telemarketing firm that the user just disconnected from the Internet. Advantageously, the user should still be at home and the phone line should be available for a call. A similar configuration could be used for providing follow-up calls to users that have browsed certain web pages. For example, if the user browsed an automobile manufacturer's web site or a listing of homes for sale, the server may be configured to signal the appropriate realty agency or automobile company so that they may call the user while the user is browsing or immediately thereafter. This may be useful for performing web satisfaction surveys while the web browsing experience is still fresh in the user's mind.

Although the system and method of the present invention have been described in connection with several embodiments, the invention is not intended to be limited to the specific forms set forth herein, but on the contrary, it is intended to cover such alternatives, modifications, and equivalents as may be reasonably included within the spirit and scope of the invention as defined by the appended claims.

Claims

What is claimed is:

1. A method for tracking user activity during web browsing, the method comprising: displaying a graphical user interface for a web browser on a display device; detecting user activity; storing a first set of information regarding the user activity; storing a second set of information regarding web content being displayed at the time of the user activity; and transmitting the stored information regarding the user activity and the content being displayed to a server.

2. The method of claim 1, wherein said user activity comprises physical movement as detected by a digital camera.

3. The method of claim 2, further comprising using said digital camera to detect user identities.

4. The method of claim 1, wherein said user activity comprises pressing a stop control before a page being loaded has completed loading.

5. The method of claim 1, wherein said user activity comprises mouse movement, scrolling, or cursor movement.

6. The method of claim 1, wherein said content is a banner advertisement, wherein the method further comprises changing the banner advertisement in response to detecting user activity consistent with disinterest in the banner advertisement.

7. The method of claim 6, further storing information regarding each of a plurality of different users' interests in different categories of banner advertisements, and transmitting the stored information to the server.

8. The method of claim 1, wherein said user activity comprises mouse clicks and key strokes.

9. The method of claim 1, further comprising detecting user identities, and storing information corresponding to said user identities.

10. The method of claim 1, wherein said user activity comprises spoken words.

11. The method of claim 1, further comprising: detecting when the browser is idle; and performing said transmitting in response to said browser being idle.

12. A method for telemarketing comprising: using a browser to detect when a user is accessing the Internet; using the browser to send a first message to a predetermined server; and forwarding information corresponding to the first message from the server to a recipient, wherein said second message is indicative that the user is likely to be reachable at a particular location corresponding to the browser location.

13. The method as recited in claim 12, further comprising waiting for said user to log off said browser before sending said first message.

14. The method as recited in claim 12, further comprising tracking the user's actions to ensure that the user is actually using the browser.

15. The method as recited in claim 12, further comprising sending an instant message to the user in response to said first message.

16. The method as recited in claim 12, further comprising sending an email message to the user in response to said first message.

17. A method for billing internet advertisers comprising: using a browser to access a server storing a plurality of banner advertisements; using the browser to display the banner advertisements; tracking user actions to ensure that one or more users are actually using the browser while the banner advertisements is being displayed; storing information about the user's actions during the time the banner advertisement is displayed; and transmitting the stored information to the server.

18. The method as recited in claim 17, wherein the user's actions include one or more of the following: key presses, cursor movement, cursor clicks, and scrolling.

19. The method as recited in claim 17, further comprising analyzing the user's actions in response to display of a particular web page.

20. The method as recited in claim 17, further comprising analyzing the user's actions in response to display of a particular advertisement.

21. The method as recited in claim 17, further comprising analyzing the number of times the user visits one or more particular web pages.

22. The method as recited in claim 17, further comprising tracking the time spent by the user in front of the display device while at one or more particular web pages.

23. The method as recited in claim 17, further comprising tracking changes made to a favorite list by the user.

24. The method as recited in claim 17, further comprising tracking keyword searches perfoixned by the user.

25. The method as recited in claim 17, further comprising using a user behavior monitoring device to estimate the number of users viewing a particular advertisement, and billing advertisers according to the number of times the particular advertisement is displayed multiplied by the number of users viewing the particular advertisement during each display.

26. The method as recited in claim 17, further comprising: prompting the user to enter a password before beginning a browsing session, wherein said password is linked to a previously completed user profile, and transmitting information corresponding to the user profile to the server with the stored information.

27. The method as recited in claim 17, further comprising estimating demographic information about the user based on the user's actions, and transmitting the estimated demographics data to the server.

28. A computer-readable medium comprising a software program, wherein said software program comprises a plurality of instructions configured to: display a graphical user interface on a display device; detect user interaction with the graphical user interface; store information about the user's interaction with the graphical user interface; and transmit the stored information to a server.

29. The medium as recited in claim 28, wherein said software program is further configured to enter an idle mode in response to said software program being idle for a predetermined amount of time, wherein said transmitting the stored information to the server is performed once said software program enters said idle mode.

30. The medium as recited in claim 28, wherein said software program is further configured to enter an idle mode in response to said software program being idle for a predetermined amount of time, wherein said transmitting the stored information to the server is performed once said software program enters said idle mode.

31. The medium as recited in claim 30, wherein said software program is configured to enter said idle mode in response to not receiving any user input for a first predetermined time period and not receiving any data from a network connection for a second predetemiined time.

32. The medium as recited in claim 30, wherein said plurality of instructions are configured to receive customized content from said server based on said transmitted information.

33. The medium as recited in claim 32, wherein said customized content comprises customized advertising banners.

34. The medium as recited in claim 32, wherein said customized content comprises a customized desktop.

35. The medium as recited in claim 32, wherein said customized content comprises customized links.

36. A computer-readable medium comprising a software program, wherein said software program comprises a plurality of instructions configured to: display a graphical user interface on a display device; display a user feedback control object, detect user interaction with said feedback control object; store information regarding the user's interaction with the feedback control object; store information regarding a web page being displayed at the time of the user's interaction with the feedback control object; and transmit the stored information the user's interaction and the web page being displayed to a server.

37. The medium of claim 36, wherein said user feedback control object comprises a floating pop-up control.

38. The medium of claim 36, wherein said user feedback control object comprises a slider attached to a frame work of controls for the browser.

3 . An internet appliance comprising: a display device; a user input device; a user behavior monitoring device; a network communications device; a memory; and a processor configured to receive data from said input device, said network device, and said memory, and configured to output data to said display device and said network communications device, wherein said memory is loaded with: an operating system, and a browser program.

40. The internet appliance as recited in claim 39, wherein said user behavior monitoring device is a motion detector configured to detect whether or not a user is positioned in front of the display device.

41. The internet appliance as recited in claim 39, wherein said user behavior monitoring device is a motion detector configured to detect a total number of users positioned in front of the display device.

42. The internet appliance as recited in claim 39, wherein said user behavior monitoring device is a thermal detector configured to detect whether or not a user is positioned in front of the display device.

43. The internet appliance as recited in claim 39, wherein said user behavior monitoring device is a thermal detector configured to detect a total number of users positioned in front of the display device.

44. The internet appliance as recited in claim 39, wherein said user behavior monitoring device is a microphone configured to detect whether or not a user is positioned in front of the display device.

45. The internet appliance as recited in claim 39, wherein said user behavior monitoring device is a camera configured to detect whether or not a user is positioned in front of the display device.

46. The internet appliance as recited in claim 39, wherein said user behavior monitoring device is a camera configured to detect a total number of users positioned in front of the display device.

47. The internet appliance as recited in claim 39, wherein said browser program is configured to perform updating tasks in response to said user behavior monitoring device indicating that the user is not positioned in front of the display device.

48. The internet appliance as recited in claim 39, wherein said browser program is configured to process data for a server in response to said user behavior monitoring device indicating that the user is not positioned in front of the display device.

49. The internet appliance as recited in claim 39, wherein said display device is a head mounted display, and wherein said user behavior monitoring device comprises one or more sensors configured to determine if the user is wearing said display device.

50. The internet appliance as recited in claim 39, wherein said display device is a head mounted display, and wherein said user behavior monitoring device comprises one or more sensors configured to determine if the user is moving.

51. A computer-readable medium comprising a web browser software program, wherein said web browser program comprises a plurality of instructions configured to: store requested web address information from one or more users; transmit the requested web addresses over a network; receive web content corresponding to the requested web address; store information corresponding the latency between the user's entry of the requested web address and the receipt of all of the web content corresponding to the requested web address; and transmit the stored information to a server.

52. The method as recited in claim 51, wherein the instructions are further configured to detect when the browser is idle, wherein the instructions are configured transmit the stored information to the server when the browser is idle.

53. The method as recited in claim 51, wherein the instructions are further configured to: detect the number of times a particular user logins in, store the number; and transmit the number to the server.

54. The method as recited in claim 51, wherein the instructions are further configured to: monitor the amount of time a particular user spends viewing a particular web page, store the amount; and transmit the amount and the web page's address to the server.