US20110252328A1

US20110252328A1 - System and method in a network controller for remotely monitoring and/or controlling devices

Info

Publication number: US20110252328A1
Application number: US13/042,223
Authority: US
Inventors: Jeyhan Karaoguz
Original assignee: Broadcom Corp
Current assignee: Avago Technologies International Sales Pte Ltd
Priority date: 2010-04-12
Filing date: 2011-03-07
Publication date: 2011-10-13
Also published as: US20110248877A1; US8812656B2; US20110252131A1

Abstract

A system and method for remotely monitoring and/or controlling devices, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS/INCORPORATION BY REFERENCE

This patent application is related to and claims priority from provisional patent application Ser. No. 61/323,223 filed Apr. 12, 2010, and titled “SYSTEMS AND METHODS FOR PROVIDING IMAGE-BASED REMOTE CONTROL,” the contents of which are hereby incorporated herein by reference in their entirety. This patent application is also related to U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23010US02; U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR AUTOMATICALLY MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23011US02; and U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD PROVIDING REMOTE USER SELECTION OF A DEVICE”, Attorney Docket No. 23012US02. The contents of each of the above-mentioned applications are hereby incorporated herein by reference in their entirety.

FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[Not Applicable]

SEQUENCE LISTING

[Not Applicable]

MICROFICHE/COPYRIGHT REFERENCE

[Not Applicable]

BACKGROUND OF THE INVENTION

Present communication networks are incapable of providing for and/or conveniently providing for user-selection of communication network devices (e.g., for remote monitoring and/or control). Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with the present invention as set forth in the remainder of the present application with reference to the drawings.

BRIEF SUMMARY OF THE INVENTION

Various aspects of the present invention provide a system and method for remotely monitoring and/or controlling devices, substantially as shown in and/or described in connection with at least one of the figures, as set forth more completely in the claims. These and other advantages, aspects and novel features of the present invention, as well as details of illustrative aspects thereof, will be more fully understood from the following description and drawings.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 shows a flow diagram of a non-limiting exemplary method for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention.

FIG. 2 shows a flow diagram of a non-limiting exemplary method for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention.

FIG. 3 shows a block diagram of a non-limiting exemplary database that may be utilized for device determination, determining device communication characteristics and/or determining device user interface characteristics, in accordance with various aspects of the present invention.

FIG. 4 shows a block diagram of a non-limiting exemplary system for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention.

FIG. 5 shows a block diagram of a non-limiting exemplary system for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention.

FIG. 6 shows a block diagram of a non-limiting exemplary system for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention.

FIG. 7 shows a block diagram of a non-limiting exemplary device for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention.

FIG. 8 shows a block diagram of a non-limiting exemplary device for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention.

DETAILED DESCRIPTION OF VARIOUS ASPECTS OF THE INVENTION

The following discussion will refer to various modules, components or circuits. Such modules, components or circuits may generally comprise hardware and/or a combination of hardware and software (e.g., including firmware). Such modules may also, for example, comprise a computer readable medium (e.g., a non-transitory medium) comprising instructions (e.g., software instructions) that, when executed by a processor, cause the processor to perform various functional aspects of the present invention. Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of particular hardware and/or software implementations of a module, component or circuit unless explicitly claimed as such. For example and without limitation, various aspects of the present invention may be implemented by one or more processors (e.g., a microprocessor, digital signal processor, baseband processor, microcontroller, etc.) executing software instructions (e.g., stored in volatile and/or non-volatile memory). Also for example, various aspects of the present invention may be implemented by an application-specific integrated circuit (“ASIC”) and/or other hardware components.
Additionally, the following discussion will refer to various system modules (e.g., communication modules, processor modules, memory or database modules, user interface modules, etc.). It should be noted that the following discussion of such various modules is segmented into such modules for the sake of illustrative clarity. However, in actual implementation, the boundaries between various modules may be blurred. For example, any or all of the functional modules discussed herein may share various hardware and/or software components. For example, any or all of the functional modules discussed herein may be implemented wholly or in-part by a shared processor executing software instructions. Additionally, various software sub-modules that may be executed by one or more processors may be shared between various software modules. Accordingly, the scope of various aspects of the present invention should not be limited by arbitrary boundaries between various hardware and/or software components, unless explicitly claimed.
The following discussion may also refer to communication networks and various aspects thereof. For the following discussion, a communication network is generally the communication infrastructure through which a communication device (e.g., a portable communication device, a personal computer, a network controller, a user-selectable device with network communication capability, a consumer electronic device with network communication capability, etc.) may communicate with other systems. For example and without limitation, a communication network may comprise a cable and/or satellite television communication network, a cellular communication network, a wireless metropolitan area network (WMAN), a wireless local area network (WLAN), a wireless personal area network (WPAN), any home or premises communication network, etc. A particular communication network may, for example, generally have a corresponding communication protocol according to which a communication device may communicate with other devices in and/or via the communication network. Unless so claimed, the scope of various aspects of the present invention should not be limited by characteristics of a particular type of communication network.
In a non-limiting exemplary scenario that comprises various aspects of the present invention, a camera may capture an image (e.g., a real-time image) of an environment (e.g., a home environment, office environment, etc.) comprising devices that may be monitored and/or controlled by a user. Such image may, for example, be a real-time image, a periodically updated image, an image acquired during system configuration, etc.). A graphical map image may also (and/or alternatively), for example, be similarly utilized. The system may then present the image to the user (e.g., on a display device, a touch input device, etc.). The user may then identify a device in the image with which the user desires to interact (e.g., in a monitoring and/or controlling capacity). For example, the user may touch the device in the image on a touch screen, identify the device with a movable cursor, indicate the image with a light pen, etc. The system may then determine the user-identified device. Such determination may, for example, comprise analyzing a database of devices in a particular area. Such database may, for example, be maintained manually and/or automatically without user interaction.
The system may, for example, establish a communication link (directly or indirectly) with the user-identified device. The system may then determine user interface options to present to the user. Such user interface options may, for example, be related to monitoring and/or controlling the user-identified device. The system may present the determined user interface options to the user. The system may then interact with the user (e.g., presenting user interface options to the user and receiving user input corresponding to such user interface options). The system may then form signals recognized by the user-identified device, where such signals correspond to the user input. Such signals may, for example, comprise data structures, command identifiers, etc., corresponding to a communication protocol understood by the user-identified device. The system may then communicate the formed signal to the user-identified device (e.g., utilizing a communication protocol understood by the user-identified device over a communication network to which the user-identified device is communicatively coupled. The system may then, depending on the nature of the user input, continue to interact with the user-identified device, for example transmitting additional commands/information to the user-identified device and/or receiving information from the user-identified device.
Turning first to FIG. 1, shows a flow diagram of a method 100 illustrating a method for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention. The method 100 may, for example, be implemented in a network controller device, gateway, router, access point, etc. Such network controller device may, for example, be implemented in a home device (e.g., a hub of a home computing network, a home-based desktop computer, a home gateway and/or router, etc.). The method 100 may also, for example, be implemented in a portable computing device. Such a portable computing device may, for example, be a laptop or notebook computer, handheld computing device, cellular phone, personal digital assistant, any personal electronic device, etc. Also note that various aspects of the exemplary method 100 (and all methods herein) may also be performed by a distributed system (e.g., a first portion may be performed in a communication network infrastructure device, for example a network controller, and a second portion may be performed in a personal electronic device or terminal).
The method 100 may begin executing at step 105. The method 100 may begin executing in response to any of a variety of causes and/or conditions, non-limiting examples of which will now be provided.
For example, the method 100 may begin executing in response to a user command to begin. For example, a user may input a command to begin monitoring and/or controlling a device (e.g., by executing a control program). Also for example, the method 100 may begin executing in response to detection of a change in the device makeup of a particular environment. Various non-limiting examples of such detection are provided in U.S. Provisional Patent Application No. 61/323,223 and U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR AUTOMATICALLY MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23011US02, each of which is hereby incorporated herein by reference in its entirety. For example, the method 100 may begin executing in response to detecting the presence of an unknown device within a particular geographic area (e.g., a premises, office, room, suite of rooms, etc.). Further for example, the method 100 may begin executing in response to a determination that a device being tracked has moved or has potentially moved within a particular environment. Additionally, for example, the method 100 may begin executing in response to failure to establish a communication link with a device in a particular area (e.g., when such device is predicted to be in such area).
Additionally for example, the exemplary method 100 may, for example, begin executing periodically (e.g., at a constant or substantially constant period, for example hourly, daily, etc.), surveying one or more particular areas. The exemplary method 100 may, for example, begin in response to a timer and/or in response to a predefined time schedule. Additionally for example, the method 100 may begin executing in response to a system power-up or reset. Further example, the method 100 may begin executing in response to an alarm (e.g., an alarm indicating a malfunctioning device, an alarm indicating a potential issue in need of user oversight, an intrusion alarm, an alarm indicating an environmental control situation, etc.).
In general, the exemplary method 100 may begin executing at step 105. Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular initiating cause or condition unless explicitly claimed.
The exemplary method 100 may, for example at step 110, comprise receiving image-based device identification information from a user. Various non-limiting examples of receiving such user input are provided in U.S. Provisional Patent Application No. 61/323,223 and U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR AUTOMATICALLY MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23011US02, each of which is hereby incorporated herein by reference in its entirety.
For example, step 110 may comprise providing an image (e.g., one or more images) of a device environment to a user. Such image may comprise characteristics of any of a variety of types of images. For example and without limitation, the image may correspond to a camera (or photographic) image of a particular area (e.g., a room, office, set of rooms, etc.). Additionally for example, step 110 may comprise providing a map image (e.g., a 2-dimensional and/or 3-dimensional map image) to a user.
Step 110 may, for example, comprise presenting the image on a display (e.g., a touch screen) of a user device (e.g., a personal electronic device, a personal computer, a personal digital assistant, a smart phone, etc.). Step 110 may comprise presenting the image on a display of any of a variety of other devices (e.g., a network controller with a screen, an access point with a screen, a gateway with a screen, etc.).
Step 110 may also, for example, comprise communicating the image (e.g., camera image and/or map image) information to a device (e.g., a user device) if such information does not originate at such device. For example, step 110 may comprise utilizing a previously established communication link or may also comprise establishing a communication link configured for the communication of video and/or graphics information. In a non-limiting exemplary scenario, step 120 may be performed in communication network infrastructure apparatus (e.g., a network controller) and comprise transmitting the image information to a personal computer terminal.
Step 110 may comprise presenting graphics features in the presented image to assist in user selection of a device in the environment and/or assist the user in understanding various devices in the environment (e.g., device status, accessibility, etc.). For example, step 110 may comprise presenting graphics features in the presented image, where such graphics features indicate selectable devices in the image (e.g., devices for which communication has occurred in the past, devices for which communication information exists in a database, devices that are presently communicatively coupled to one or more known communication networks, etc.). Such graphical features may also indicate status of the device (e.g., currently on-line, currently off-line, sleeping, monitorable device, controllable device, device for which the user is authorized to interact with or not, new undefined device, failing device, device for which communication should be possible but which is presently not possible, etc. Also for example, step 110 may comprise providing an output image or graphics feature indicating where a user is presently indicating. For example, step 110 may comprise outputting a graphical feature (e.g., a cursor, etc.) indicating where the user is presently pointing, touching, looking, etc. Also for example, step 110 may comprise outputting a graphical indication of an area being selected by a user (e.g., as such area is being defined by the user).
The image presented at step 110 may comprise a non-real-time stored image and/or a real-time image acquired as such image is needed (e.g., during present execution of the exemplary method 100, during present execution of step 110, etc.). In a stored-image scenario, step 110 may comprise retrieving the image from a database (e.g., a local database and/or a remote networked database). Such a database may, for example and without limitation, have been formed and/or maintained in accordance with procedures or systems outlined in U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23010US02 and/or U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR AUTOMATICALLY MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23011US02, each of which is hereby incorporated herein by reference in its entirety.
For example, step 110 may comprise presenting an image to the user that was obtained at some point in the past (e.g., at a regular periodic time point in the past, during system calibration defining user-selectable devices, during a most recent network maintenance and/or definition operation, etc.). In a non-limiting exemplary scenario, step 110 may comprise presenting a camera image of a family room to a user, where such image was taken prior to the present execution of the exemplary method 100. In environments that have a relatively static device configuration, such previously obtained image may be utilized for a substantial duration of time before updating. For example, such an image need only be updated when a user-selectable device is added to and/or removed from and/or moved within the pictured or mapped environment.
In a real-time scenario, step 110 may comprise retrieving the image utilizing a camera. For example, in an exemplary scenario, a user may utilize a user interface provided at step 110 to specify a particular room, in response to which step 110 may comprise utilizing a camera in the room to obtain a current image of the room. Note that such selection may be multi-tiered, for example, a user may first specify a building or system, then a room of the specified building or system, then a camera of the specified room, etc.
A previous example mentioned a user indicating a room, in response to which step 110 comprised presenting an image of such room to the user. Step 110 may similarly comprise providing a user interface by which the user may select a particular camera from which an image is desired. In an exemplary scenario, step 110 may present the user with a list of cameras from which to select (e.g., a kitchen camera, family room camera 1, family room camera 2, patio camera, pool camera, rec room camera 1, rec room camera 2, office camera 1, office camera 2, front door camera, etc.). The user may then select the desired camera. Such a list of cameras may, for example be textual, graphical, image-based (e.g., showing a thumbnail image from each camera), etc.
In an implementation including utilization of camera images, step 110 may also comprise providing for user control of a selected camera. For example, step 110 may comprise providing a user interface to the user by which the user may control movement of the camera, operation of the camera lens (e.g., panning, zooming and/or focusing), operation of camera lighting, filtering, resolution, etc. In a non-limiting exemplary scenario, step 110 may comprise providing the user with the ability to zoom in on a desired device to assist the user in selecting a device and/or defining an area and/or volume occupied by such device. Step 110 may thus provide for additional control of an image being presented to the user for use by the user in selecting a device, inputting information identifying location of the device, orientation of the device, spatial characteristics of the device, etc.
In general, step 110 may comprise providing an image (e.g., one or more images) of a device environment to a user. Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular image (e.g., a camera image, map image, etc.) and/or any particular manner of presenting such an image unless explicitly claimed.
After presenting an image to the user, step 110 may then comprise receiving an image selection input from a user. Step 110 may comprise receiving such an image selection input in any of a variety of manners, non-limiting examples of which will now be provided.
For example, step 110 may comprise receiving a touch screen input from a user. Also for example, step 110 may comprise receiving a cursor movement and selection input from a user (e.g., utilizing a mouse, touch pad, track ball, etc.). Further for example, step 130 may comprise receiving a light pen input from a user, a camera input from a user, glove-based pointing device, user finger tracking device, etc.
Step 110 may comprise receiving a user indication of a specific point in the image as input (e.g., a touch point, a cursor click point, etc.). Also for example, step 110 may comprise receiving a user input indicative of a selected area (e.g., an area enclosed by a figure drawn by the user) or volume.
For example, step 110 may comprise receiving information descriptive of a square, rectangle or other polygon drawn by the user. Similarly for example, step 110 may comprise receiving information descriptive of a circle or oval drawn by the user. For example, step 110 may comprise providing the user with a graphical toolset, by which the user may draw a polygon around the device, circle the device, paint the device, draw an ellipse around the device, define the device region with a plurality of figures, etc. Such graphical toolset may also comprise tools that automatically provide borders on a selected object (e.g., by performing pixel analysis to search for device edges).
Additionally for example, step 110 may comprise receiving a user input that toggles between and/or traverses a plurality of selectable devices in the image presented to the user. For example, step 110 may comprise receiving a user inputting a “next” command that causes traversal of the selectable devices (e.g., one at a time) until the user reaches the desired device. In such a scenario, during a traversal of devices, step 110 may comprise outputting a graphical indication (e.g., a highlighted area or polygon) indicating the presently selected device.
In general, step 110 may comprise receiving image-based device identification information from a user. Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular manner of receiving such input unless explicitly claimed.
The exemplary method 100 may, for example at step 120, comprise identifying a device that has been selected by the user. Step 120 may comprise identifying the selected device in any of a variety of manners, non-limiting examples of which will now be provided.
For example, step 120 may comprise correlating the user device selection information received at step 110 with known device selection areas in the presented image. For example, as discussed in U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23010US02 and/or U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR AUTOMATICALLY MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23011US02, each of which is hereby incorporated herein by reference in its entirety, a database may be formed that includes information defining device selection areas within an image (e.g., a pictorial or map image). In such a scenario, step 120 may comprise comparing a user-selected location (or area or volume) in the image presented at step 110, as received at step 110, with such database information to determine whether the user-selected location (or area or volume) corresponds to a defined region of the image corresponding to a particular device. For example, if a user selects location (X, Y) in the image, step 120 may comprise determining that the location (X, Y) has been previously defined to correspond to the location of Device A. Various exemplary manners of forming such a database were discussed in the above-mentioned incorporated patent applications.
Also for example, step 120 may comprise processing the user device selection information received at step 110 to determine a location in the image at which to begin performing image matching analysis. For example, as explained in the above-mentioned incorporated patent applications, pixel characteristics with a previously defined image of a device may be processed and stored in a fashion (e.g., as pixel histograms, etc.) that may be processed to identify such device appearing in an image. For example, respective pixel histograms may be formed that correspond to respective devices (e.g., respective pixel histograms for a television, a stereo, a light switch, a receiver, a thermostat, etc.). Step 120 may, in such a scenario, comprise determining the previously defined histogram that best matches the area surrounding a user-selected point in the image and/or that best matches the characteristics of an area defined by a user input received at step 110.
Further for example, step 120 may comprise performing character recognition processing. Such processing may, for example, comprise analyzing an area of the image near a point specified by the user to determine whether any recognizable characters (e.g., alphabetical characters, numerical characters, fanciful logos, etc.) are present in the area of the image. In a non-limiting example, the system implementing the method 100 may know that there is only one device in the family room that has been produced by a particular manufacturer. The presence of a logo associated with such manufacturer in the image relatively near a user's designated point in the image may be utilized by step 120 as an indication that such device has been selected by the user.
Additionally for example, step 120 may comprise analyzing location information to identify a particular device that has been selected by a user. For example, the camera's location and field of view may be processed in conjunction with a user-identified location in the image to draw a line extending out from the camera through the camera image plane. In such a scenario, step 120 may comprise determining that the first device encountered along such line (in other words, the first device, the known location of which lies along such line) extending outward from the camera lens corresponds to the user-selected device.
Still further for example, step 120 may comprise correlating the user-identified image location and field of view with a generally analogous field of received RF energy. For example, user-identified image location may be relatively closer to an identified RF hot-spot in such image field of view than other RF hot-spots. In such a scenario, step 120 may comprise determining that the user has identified the device corresponding to the closest RF hot-spot in the image field of view. Similarly, infrared processing may identify devices emitting heat energy and determining the closest heat-emitting device in the field of view relative to the user-identified location in the image.
Note that step 120 may also comprise establishing a communication link with a determined device (or candidate device) to determine additional information about such device that may be utilized in determining whether the user has identified such device. For example, such communication link(s) may be established to communicate current device location information, to communicate test messages, to determine whether the user has access to a particular device, etc.
In general, step 120 may comprise identifying a device that has been selected by the user (e.g., identifying a user-selected device in an image that has been presented to the user). Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular manner of performing such identification unless explicitly claimed.
The exemplary method 100 may, for example at step 130, comprise establishing a communication link (or a plurality of communication links) with the user-identified device (e.g. as determined at step 120). Step 130 may comprise forming such communication link(s) in any of a variety of manners, non-limiting examples of which will be presented below.
For example, as discussed previously (and as will be exemplified in FIG. 3), a database (e.g., a data structure in a memory) may comprise records of information associated with individual devices and/or groups of devices. A record of information associated with a device may, for example, comprise information describing how communication with the associated device(s) may be performed.
For example, such a record may comprise information identifying one or more protocols according to which the device(s) is able to communicate. Such a record may, for example, comprise information identifying any or all communication networks via which communication with such device(s) may be established. Such communication network information may, for example, comprise information describing theoretically available and/or actually available communication pathways (each pathway comprising one or more communication networks) that may be established between the device (or system) implementing the method 100 and the identified device. Such a record may, for example, comprise information indicating static and/or dynamic communication network addresses associated with such device(s). Such a record may, for example, comprise secure access information (e.g., authorized user identification information, password information, encryption/decryption information, etc.). Such a record may, for example, comprise information indicating whether the identified device is presently communicatively coupled to a particular communication network and/or communication pathway (e.g., on-line), and if so, the identification of such network/pathway and/or other information indicative of how such network/pathway may be accessed. Such a record may also, for example, comprise respective operational status information for one or more devices (e.g., whether such device is presently on-line, whether such device is presently powered up, presently powered up but in a sleep mode, presently in communication with another entity, presently malfunctioning, etc.). It should be noted that the above-mentioned examples are merely exemplary and not intended to constitute an exhaustive list of device communication related information that may be stored in a database.
In an exemplary scenario involving a database, step 130 may, for example, comprise utilizing the device identity (e.g., as determined at step 120) as an index (or search term) in such a database. For example, upon determining that a user has indicated a particular device, step 130 may comprise accessing communication information in a database associated with the particular device.
Step 130 may, for example, comprise accessing communication information for a device (or group thereof) from a variety of sources of such information. For example, step 130 may comprise accessing a database associated with a particular network or set of networks, a particular premises, a particular user, a particular home and/or office, etc. Step 130 may also, for example, comprise accessing device communication information from a database associated with the manufacturer or other supplier of such device.
In an exemplary scenario, upon user-selection of a particular type of television (e.g., as selected and/or identified at steps 110 and 120), step 130 may comprise (e.g., in real-time) establishing a communication link with the manufacturer of such television to identify protocols with which such television may communicate. In another exemplary scenario, step 130 may comprise establishing a communication link with the identified device in order to determine all available communication options. For example, a relatively low-bandwidth data communication link may be initially established in order to determine the manner in which a relatively high-bandwidth video link may be established (e.g., determining communication network characteristics, video communication protocol characteristics, etc.).
After determining how to establish a communication link with a user-selected device, step 130 may comprise establishing such a communication link. Step 130 may comprise establishing such a communication link in any of a variety of manners.
For example, step 130 may comprise establishing one or more communication links utilizing any of a variety of communication media (e.g., wired, wireless, tethered optical, non-tethered optical, etc.). Also for example, step 130 may comprise establishing one or more communication links via one or more of a plurality of different types of communication networks (e.g., the Internet, telecommunication network, local area network, wireless local area network, personal area network, home network, cellular telecommunication network, cable television network, cellular telecommunication network, metropolitan area network, wide area network, etc.).
For example, step 130 may comprise establishing one or more communication links between a user device (e.g., a personal computing device) and the selected device via one or more of a plurality of intervening systems, devices and/or independent communication networks. For example, step 130 may comprise establishing a communication link between a user device and the user-selected device via a home gateway, network controller and/or wireless access point. Also for example, step 130 may comprise establishing a communication link between a user device and the user-selected device via a public wireless LAN, the Internet, cable network and home-based wireless LAN.
In general, step 130 may comprise establishing a communication link (or a plurality of communication links) with the user-identified device (e.g. as determined at step 120). Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular type of communication link and/or any particular manner of establishing such a communication link unless explicitly claimed.
The exemplary method 100 may, for example at step 140, comprise providing a user interface between the user and the user-identified device. Such a user interface may, for example, provide the user with remote control capability for the user-specified device. Step 140 may comprise providing such user interface in any of a variety of manners, non-limiting examples of which will be presented below.
For example, step 140 may comprise determining the characteristics of the user interface to provide to the user. For example, though step 140 may comprise providing the user with a generic interface that is applicable to a group of devices, various aspects of step 140 may also comprise providing a device-specific user interface to the user. For example, step 140 may comprise only providing the user with a set of user interface capabilities specifically associated with the user-selected device.
Step 140 may comprise determining the characteristics of the respective user interface for the device (or a group of devices) in any of a variety of manners. For example, as discussed previously (and as will be exemplified in FIG. 3), a database (e.g., a data structure in a memory) may comprise records of information associated with individual devices and/or groups of devices. A record of information associated with a device may, for example, comprise information describing how user interaction with the associated device(s) may be performed. For example, such a record may comprise information identifying one or more user interface protocols according to which the device(s) is able to communicate. Such a record may, for example, comprise information identifying Graphical User Interface (“GUI”) information (e.g., information describing windows, buttons, text blocks, slides, dials, graphical feature screen locations, logical associations between features, etc.).
Such information may also comprise information describing actions to take in response to user interaction with the user interface features. For example, in response to a user input, the information may indicate that a particular signal (e.g., particular data, a particular message, a particular type of message, etc.) is to be communicated to the user-selected device (e.g., to a remote user-selected device from the user device and/or any intervening device). Also for example, such information may comprise information describing response signaling from the user-selected device to the user device or other intervening device. In other words, the information may comprise information describing signaling that is related to the utilization of remote user interface features with the user-selected device.
Such information may also, for example, comprise information describing the layering (or nesting) of user interface features to be provided to a user for remote interaction with the user-selected device. For example, selecting a particular user interface button may result in other screens and/or GUI features being presented to a user.
Such information may additionally, for example, comprise information describing the user interface state machine of the user-identified device. Information describing such state machine may, for example, represent a complete user interface description, a complete user interface description tailored for remote control, a portion of the user interface tailored to the permissions of a particular user, a portion of the user interface that is relevant to a particular monitoring and/or controlling activity, etc.
Such information may, for example, comprise information describing user interface features to present to a user in response to a particular signal being received from the user-selected device. As a non-limiting example, a signal received from a user-selected device may indicate that the user-selected device is in an error state. In such a scenario, the information may comprise information describing a debugging user interface to provide to the user (e.g., at the user device for remote monitoring and/or control of the user-selected device).
Step 140 may, for example, comprise (e.g., instead of interfacing with a database and/or in addition to interfacing with a database) interfacing with the user-selected device (or any other device, for example a central controller) to determine characteristics of the user interface to provide to the user. In such a scenario, step 140 may comprise receiving any of the exemplary types of user interface information discussed previously. For example, step 140 may comprise interfacing with the user-selected device to communicate (e.g., in real-time) information descriptive of graphical user interface features to present to the user. In such an exemplary scenario, the user-selected device (e.g., the remotely monitored and/or controlled device) may maintain the entire user interface state machine (e.g., in its entirety) so that the user device or some intervening device does not need to.
Step 140 may (e.g., after determining the manner in which to provide the user interface to the user) comprise communicating U/I output information to the user and/or receiving U/I input information from the user. Such user interaction may, for example, comprise utilizing any of a wide variety of user interface mechanisms, various non-limiting examples of which are presented herein. For example, step 140 may comprise outputting U/I information on a display of a user device (or other device) or other video output device, outputting U/I information from an audio output device, outputting U/I information from a printer, fax machine, tactile output device, etc. Also for example, step 140 may comprise receiving input U/I information from a touch screen, light pen, keypad, touch pad, mouse, microphone, camera, motion tracking device, etc.
Step 140 may also, for example, comprise converting between user interface information and information understood by the user-selected device. For example, step 140 may comprise translating a user command entered via a GUI into a control and/or data command that may be communicated to the user-selected device via a communication link. Also, step 140 may comprise translating a control and/or data signal received from the user-selected device into a user-perceivable output (e.g., a visual output, audio output, tactile output, physical output, etc.).
Step 140 may, for example, comprise communicating user interface information with the user-selected device via the communication link(s) established at step 130. For example, step 140 may comprise communicating with the user-selected device utilizing a relatively higher layer protocol (e.g., an application layer user interface protocol) on top of relatively lower layer transport, network, data link and physical layer protocols. In an exemplary scenario in which a communication link comprises at least two different (e.g., independent) communication networks comprising at least two different respective communication protocols, the higher layer user interface protocol may stay consistent via the different communication networks (e.g., only the lower layer network protocol aspects might change).
In general, step 140 may comprise providing a user interface between the user and the user-identified device. Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular type of user interface and/or any particular manner of operating such a user interface unless explicitly claimed.
The exemplary method 100 may, for example at step 195, comprise performing continued processing. Various non-limiting examples of such processing will be provided below.
For example, step 195 may comprise returning execution flow of the method 100 to any previous step. Also for example, step 195 may comprise performing the exemplary method 100 with a plurality of user-selected devices (e.g., in parallel and/or serially (e.g., in a time multiplexed manner)).
Additionally, step 195 may, for example, comprise ending the user interaction with a user-selected device (e.g., in response to a user command to end such interaction, in response to a command from the user-selected device to end such interaction, etc.).
Further, step 195 may, for example, comprise interfacing with the user and/or the user-selected device to modify various aspects of the user interface (e.g., customizing the user interface to conform to user preferences). For example, step 195 may comprise providing the user with a user interface adapted to interact with the user to customize the user interface (e.g., for a particular user-selected device, a group of devices, all devices, etc.).
Also for example, step 195 may comprise providing the user with a user interface mechanism by which the user may pass control of a particular user-selected device to another user (e.g., a technical support operator), security service, etc. Further for example, step 195 may comprise providing a user interface mechanism by which the user may define and/or redefine communication links between the user (e.g., a personal computing device) and the user-selected device. For example, step 195 may comprise modifying the network via which communications between the user and the user-selected device take place (e.g., moving a device from a first home network to a second home network, configuring communication with such device via any of a plurality of alternative networks, etc.). Further, step 195 may comprise establishing a communication network with the identified device and/or establishing a remote control environment between the user and the identified device.
In general, step 195 may comprise performing continued processing. Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular type of continued processing unless explicitly claimed.
The following discussion will include various non-limiting exemplary scenarios illustrating various aspects.
For example, in a first exemplary scenario, a user identifies a television presented to the user in a camera image of a room. Upon determining the user identification of the television, the system determines how to communicate with the television. The system then establishes a communication link with the television (creating an overall communication pathway between the user and the television). For example, a user may touch a television presented in an image, resulting in a user interface being presented to the remote user for controlling the television. Such user interface may, for example, comprise on/off buttons, channel change buttons, volume buttons, audio equalization settings, secure user access interface features, screen settings, input device settings, etc.
In another exemplary scenario, the process begins when a motion detector in a home detects motion and sends a message to the user's mobile computing device. The user then may specify a room (e.g., the room in which motion was detected). The user may then, for example, identify a light switch to turn on lights, and identify a camera with which to acquire images (e.g., such interaction may also include panning, zooming, filtering, etc.). The user may then identify an audio alarm to active (e.g., by touching such an audio alarm shown in an image presented to the user on a touch screen), identify a telephone with which to initiate a “911” call or identify a communication device with which to contact a security company.
In another exemplary scenario, the process begins when a temperature sensor at a home determines that the home temperature has dropped below a threshold level. The user is notified and selects a thermostat presented to the user in a camera image and/or graphical map image (e.g., by touching such thermostat on a touch screen, by moving a graphical cursor on the thermostat of the image, etc.). The system then presents a thermostat user interface for the user, who then operates thermostat to determine whether the environmental controls for the home are functioning properly. The user may then identify a phone or other communication device with which to contact a repair company if needed, resulting in the respective user interface being provided to the user for operation of the phone or other communication device. The user may then identify and activate a door camera to await arrival of the repairman, resulting in the user being provided with a user interface for the identified camera. When the repairman is detected, the user may select the door lock in an image provided to the user and activate the door lock via a user interface provided to the user for the lock. The user may also activate an intercom system in the home with which to communicate with the repairman on-site, resulting in a user interface being provided to the user corresponding to the intercom system. The user may then select a utility room camera and/or other home cameras to monitor the repairman's activity, resulting in various camera user interfaces being provided to the user. The user may then activate a fax machine or scanner to scan the repairman's bill and automatically initiate payment to the repairman. When repairs are complete, the use may identify a lighting system with a corresponding user interface and utilize the provided user interface to operate lighting to assist the repairman in his efforts to leave the premises. Note that any or all of the above-mention user interfaces may be provided to the user (e.g., in respective windows) simultaneously.
Additional examples of such continued processing may, for example, comprise any or all of the functionality discussed in co-pending applications U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23010US02; U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD FOR AUTOMATICALLY MANAGING A NETWORK OF USER-SELECTABLE DEVICES”, Attorney Docket No. 23011US02; and U.S. patent application Ser. No. ______, filed concurrently herewith, titled “SYSTEM AND METHOD PROVIDING REMOTE USER SELECTION OF A DEVICE”, Attorney Docket No. 23012US02. Each of such applications is hereby incorporated herein by reference in its entirety.
In general, step 195 may comprise performing continued processing. Accordingly, the scope of various aspects of the present invention should not be limited by characteristics of any particular type of continued processing unless explicitly claimed.
Turning next to FIG. 2, such figure shows a flow diagram of a non-limiting exemplary method 200 for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention. The exemplary method 200 may share any or all aspects with the exemplary method 100 discussed above.
The exemplary method 200 may, for example at steps 205-220, share any or all aspects with steps 105-120 of the exemplary method 100 illustrated in FIG. 1 and discussed previously.
The exemplary method 200 may, for example at step 230, comprise establishing a communication link (or a plurality of communication links) with the user-identified device (e.g., as determined at step 220). Step 230 may, for example, share any or all characteristics with step 130 discussed above.
Step 230 may, for example at sub-step 232, comprise determining how to establish a communication link with the user-identified device. Various non-limiting examples of such operation were presented previously in the discussion of step 130.
Step 230 may, for example at sub-step 234, comprise establishing one or more communication links between the user device and the user-identified device. Various non-limiting examples of such operation were presented previously in the discussion of step 130.
The exemplary method 200 may, for example at step 240, comprise providing a user interface between the user and the user-identified device. Step 240 may, for example, share any or all characteristics with step 140 discussed above.
Step 240 may, for example at sub-step 242, comprise determining a user interface for the identified device. Various non-limiting examples of such operation were presented previously in the discussion of step 140.
Step 240 may then, for example at sub-step 244, comprise providing the determined user interface to the user. Various non-limiting examples of such operation were presented previously in the discussion of step 140.
Step 240 may then, for example at sub-step 246, comprise communicating user interface information between the user device and the user-identified device. Various non-limiting examples of such operation were presented previously in the discussion of step 140.
The exemplary method 200 may, for example at step 295, share any or all aspects with step 195 of the exemplary method 100 illustrated in FIG. 1 and discussed previously.
FIG. 3 shows a block diagram of a non-limiting exemplary system 300 comprising a database 361 that may be utilized for device determination, determining device communication characteristics and/or determining device user interface characteristics, in accordance with various aspects of the present invention.
The previous discussion (e.g., the discussion of steps 120 and 220) mentioned that identification of a user-specified device may be performed via interaction with a database. Additionally, the discussions of method steps 130 and 230 mentioned determining how to establish a communication link with the user-specified device. Further, the discussions of method steps 140 and 240 mentioned determining a user interface for the user-specified device. The acquisition of information related to a selected device (e.g., device identity information, device communication information, device user interface information, additional information related to a device, etc.) may comprise interfacing with one or more local and/or remote databases. FIG. 3 provides a non-limiting illustration of such a database 361. Additionally, other method steps may comprise the formation and/or maintenance of a database where devices of an image and/or map may be related to respective device records that include any of a variety of device information (many examples of which were provided above).
The database may, for example, comprise a memory 361. Such a memory 361 may, for example, be a non-volatile memory (or other non-transitory data retention device) such as a magnetic disk hard drive, optical disk drive, CD, DVD, laser disc, Blueray disc, diskette, flash drive, EPROM, EEPROM, flash memory, etc. The memory 361 may, in the illustrated example, include information of an image 320 (or plurality of images), where areas or features of the image may be related to records, which may also be stored in the memory 361. For example, the image 320 may comprise a still photograph or other camera image.
The image 320 may comprise an image of a thermostat that is linked to a thermostat record 331. The image 320 may also comprise an image of a DVR that is logically linked to a DVR record 332, an image of a stereo that is logically linked to a stereo record 333, an image of a television that is logically linked to a television record 334, an image of a set top box (“STB”) and/or TV receiver that is logically linked to an STB/Receiver record 335, an image of a power switch that is logically linked to a power switch record 336 and an image of a camera that is logically linked to a camera record 337. As discussed previously, the devices of the image 320 (or environment) may be defined by respective areas (e.g., boxes, outlines, etc.) input by a user and/or automatically defined (e.g., utilizing pixel analysis).
Each of the device records (e.g., exemplary records 331-337) may comprise any of a variety of information associated with the respective device, many non-limiting examples of such information are presented herein. For example, such information may comprise information indicating one or more manners in which communication may be established with the respective device. As mentioned above, such information may, for example, comprise information indicating and/or describing protocols with which the respective device may communicate (e.g., relatively low-level network protocols, relatively high-level application and/or user interface protocols, etc.). Also for example, such information may comprise information describing communication networks via which communication may be established with the respective device. Additionally for example, such information may comprise information describing addressing information, secure communication information, communication capability information, processing capability, energy resources, etc., for the respective device.
Each of the device records may also, for example, comprise information describing one or more user interfaces that may be utilized to provide an interface between the user and the user-selected device. For example, as mentioned above, such information may comprise information describing one or more user interface state machines associated with operating the device. Additionally, for example, such information may describe characteristics of graphical user interface features that may be presented to the user. Further for example, such information may comprise information describing the manner in which various types of information may be communicated between the device and a device operated by the user (and/or intervening system devices).
The database system 300 (or database 361) may comprise a database interface 390. Such a database interface 390 may, for example, comprise an interface by which other processes may access and utilize the information stored in the memory 361. Such interface 390 may be implemented in a variety of manners. For example, such interface 390 may be implemented by a processor 360 or other hardware device executing instructions, which may, for example, be stored in the memory 361 or another memory (e.g., either collocated with the memory 361 or geographically distinct from the memory 361). The database interface 390 may, for example, comprise functional modules stored in a variety of locations (e.g., at a central controller, server, gateway, access point, and/or user computing device).
The database interface 390 may, for example, comprise routines and/or methods that, when executed by a processor, cause the processor to operate in a manner (e.g., interfacing with a database) to retrieve particular information. For example, the database interface 390 may comprise an interface mechanism by which device identity information may be input to result in an output of any of a variety of types of communication information for the device. Also for example, the database interface 390 may comprise an interface mechanism by which device identity information may be input to result in an output of any of a variety of user interface information for the device.
The system 300 illustrated in FIG. 3 may also comprise one or more processors, communication interface modules, or any other hardware components 360 that may interact with the memory 361 via the database interface 390. For example, one of such processors may be operate to interface with the database 361 while implementing the exemplary methods 100, 200 illustrated in FIGS. 1-2 and discussed previously.
Turning next to FIG. 4, such figure shows a block diagram of a non-limiting exemplary system 400 for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention. The exemplary system 400 (or components thereof) may operate to perform any or all of the functional characteristics discussed herein (e.g., with regard to the exemplary methods 100 and 200, with regard to the exemplary database system 300, etc.).
The exemplary system 400 may, for example, comprise a personal computer system 480, which is communicatively coupled to a variety of devices via one or more communication networks 405 (e.g., the Internet, a telecommunication network, cable television network, satellite communication network, local area network, personal area network, metropolitan area network, wide area network, campus network, home network, etc.). Such network(s) may, for example, be wired, wireless RF, tethered optical, non-tethered optical, etc.
As a non-limiting example, the personal computer system 480 (e.g., a laptop or notebook computer, desktop computer, handheld computer, cellular telephone with computing capability, smart phone, personal digital assistant, etc.) may comprise one or more processors 460 and a database 461. The personal computer system 480 may, for example, operate to perform any or all of the method steps discussed previously. For example, the database 461 may comprise any or all database characteristics discussed herein.
The personal computer system 480 may, for example, operate to provide for user monitoring, controlling and/or generally interacting with an electronic device (e.g., a device remote from the user) in a network that comprises any or all of the non-limiting exemplary devices (e.g., the thermostat 431, DVR 432, stereo 433, television 434, STB/Rcvr 435, power switch 436 and camera 437). As a non-limiting example, the processor 460 may operate to provide for user monitoring and/or controlling any or all of such devices in a same communication network and/or in separate communication networks.
For example, the personal computer system 480 may, for example, operate to interact with the user to implement any of the above-mentioned functionality. For example, the personal computer system 480 may operate to obtain image information (or map information) from a database 461 and/or camera 437 to present an image (or map) to a user thereof. The personal computer system 480 may then operate to receive image selection input from a user of such system, identify the user-selected device, and perform continued processing regarding such user-selected device. The personal computer system 480 may then, for example, operate to establish communication links and/or provide for user interface operation between the user device and the user-selected device.
FIG. 5 shows a block diagram of a non-limiting exemplary system 500 for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention. The exemplary system 500 (or components thereof) may operate to perform any or all of the functional characteristics discussed herein (e.g., with regard to the exemplary methods 100 and 200 illustrated in FIGS. 1 and 2, with regard to the exemplary database illustrated in FIG. 3, with regard to the exemplary system 400 illustrated in FIG. 4, etc.).
The exemplary system 500 may, for example, comprise a personal computer system 580, which is communicatively coupled to a gateway, network controller and/or access point 585 via one or more communication networks 505 (e.g., the Internet, a telecommunication network, cable television network, satellite communication network, local area network, personal area network, metropolitan area network, wide area network, campus network, home network, etc.). Such network(s) may, for example, be wired, wireless RF, tethered optical, non-tethered optical, etc.
The gateway, network controller and/or access point 585, in turn, is communicatively coupled to a variety of devices via one or more local communication networks 506 (e.g., local area network(s), personal area network(s), home network(s), office network(s), etc.). Such local communication network(s) may, for example, be wired, wireless RF, tethered optical, non-tethered optical, etc.
As a non-limiting example, the personal computer system 580 (e.g., a laptop or notebook computer, desktop computer, handheld computer, cellular telephone with computing capability, personal digital assistant, etc.) may comprise one or more processors 560 and a database 561. The personal computer system 580 may, for example, operate to perform any or all of the method steps discussed previously. For example, the database 561 may comprise any or all database characteristics discussed herein.
The personal computer system 580 may, for example and without limitation, operate to provide for user monitoring, controlling and/or generally interacting with an electronic device in a network that comprises any or all of the previously exemplified thermostat 531, DVR 532, stereo 533, television 534, STB/Rcvr 535, power switch 536 and camera 537. As a non-limiting example, the processor 560 may operate to provide for user monitoring and/or controlling any or all of such devices in a same communication network and/or in separate communication networks.
For example, the personal computer system 580 may, for example, operate to interact with the user to implement any of the above-mentioned functionality. The personal computer system 580 may (e.g., utilizing a communication module) operate to establish a communication link with any of the devices illustrated (e.g., including the camera 537) via the communication network(s) 505; local gateway, network controller and/or access point 585; and communication network(s) 506; and provide the capability to view and/or select a device. Such operation may, for example, comprise interfacing with the camera 537 to obtain a photographic image of a device environment and/or obtaining image or map information from the database 561. Such operation may also, for example, comprise interfacing with a database 561 to obtain communication information and/or user interface information associated with a user-identified device. Such operation may, for example, comprise operating to establish communication links and/or provide for user interface operation between a user's device and the user-identified device.
The personal computer system 580 may, for example, operate to receive an image selection input from a user of such system, identify the user-selected device, determine a manner in which to establish and/or maintain communication with the user-selected device, establish and/or maintain one or more communication links, determine user interface characteristics for interacting with the user-selected device, provide for user interaction (e.g., remote interaction) with the user-selected device and perform continued processing regarding such user-selected device.
FIG. 6 shows a block diagram of a non-limiting exemplary system 600 for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention. The exemplary system 600 (or components thereof) may operate to perform any or all of the functional characteristics discussed herein (e.g., with regard to the exemplary methods 100 and 200 illustrated in FIGS. 1 and 2, with regard to the exemplary database illustrated in FIG. 3, with regard to the exemplary systems 400 and 500 illustrated in FIGS. 4 and 5, etc.).
The exemplary system 600 may, for example, comprise a personal computer system 680, which is communicatively coupled to a gateway, network controller and/or access point 685 via one or more communication networks 605 (e.g., the Internet, a telecommunication network, cable television network, satellite communication network, local area network, personal area network, metropolitan area network, wide area network, campus network, home network, etc.). Such network(s) may, for example, be wired, wireless RF, tethered optical, non-tethered optical, etc.
The gateway, network controller and/or access point 685, in turn, is communicatively coupled to a variety of devices via one or more local communication networks 606 (e.g., local area network(s), personal area network(s), home network(s), office network(s), etc.). Such local communication network(s) may, for example, be wired, wireless RF, tethered optical, non-tethered optical, etc.
As a non-limiting example, the gateway, network controller and/or access point 685 may comprise one or more processors 660 and a database 661. The gateway, network controller and/or access point 685 may, for example, operate to perform any or all of the method steps discussed previously. For example, the database 661 may comprise any or all database characteristics discussed herein.
The gateway, network controller and/or access point 685 may, for example and without limitation, operate to provide for user monitoring, controlling and/or generally interacting with an electronic device in a network that comprises any or all of the previously exemplified thermostat 631, DVR 632, stereo 633, television 634, STB/Rcvr 635, power switch 636 and camera 637. As a non-limiting example, the processor 660 may operate to provide for user monitoring, controlling and/or generally interacting with any or all of such devices in a same communication network and/or in separate communication networks.
For example, the personal computer system 680 and/or local gateway(s), network controller(s) and/or access point(s) 685 may, for example, operate to perform any of the above-mentioned functionality. For example, the personal computer system 680 may operate receive an indication from the user that the user desires to select a device from a particular environment (e.g., a particular room, premises, office, etc.). The personal computer system 680 may then, for example, operate to establish a communication link with the local gateway(s), network controller(s) and/or access point(s) 685. In such an exemplary scenario, functional aspects discussed previously (apart from direct user interaction) may be performed by the local gateway, network controller and/or AP 685 instead of by the personal computer system 680. Alternatively, in such an exemplary scenario, the personal computer system 680 and local gateway, network controller or AP 685 may distribute performance of any of the previously discussed method steps among the various system entities.
For example, in an exemplary scenario, where the user desires a real-time image of a particular environment, the local gateway, network controller and/or access point 685 may operate to provide a communication link between the personal computer system 680 and the camera, by which the personal computer system 680 may obtain an image from the camera and present such image to the user. Such a communication link may also comprise characteristics providing for user control of the camera 637.
Also for example, in an exemplary scenario where the user desires a non-real time image, the personal computer system 680 may operate to establish a communication link with the local gateway, network controller and/or access point 685 over which such desired image may be communicated from the database 661 to the personal computer system 680.
In the particular example illustrated in FIG. 6, the processor 660 may operate to perform any or all of the device identifying, communication link establishing, user interface providing, and continued processing functionality discussed previously.
For example, the gateway, network controller and/or access point 685 may (e.g., utilizing one or more modules) operate to identify a user-selected device, interface with the database 661 to determine a manner in which to establish and/or maintain communication with the user-selected device, establish and/or maintain one or more communication links (e.g., between the personal computer system 680 and the user-identified device), determine user interface characteristics for interacting with the user-selected device, provide for user interaction with the user-selected device and perform continued processing regarding such user-selected device. The gateway, network controller and/or access point 685 may then operate to provide to the user the capability to monitor and/or control the selected device.
Note that, as opposed to the exemplary systems 400, 500 illustrated in FIGS. 4-5, the exemplary system 600 locates most of the previously discussed functionality in the gateway, network controller and/or access point 685 (e.g., as opposed to the personal computer system 680). Such an implementation may, for example, remove system complexity from the personal computer 680, which may for example have limited energy, memory and/or processing capabilities, and place such complexity in a central location, which may for example have relatively increased energy, memory and/or processing capabilities. Such an arrangement also provides for a plurality of personal computer systems to perform the disclosed operation with a minimum of additional complexity.
FIG. 7 shows a block diagram of a non-limiting exemplary device (or system) 700 for remotely monitoring and/or controlling devices, in accordance with various aspects of the present invention. The exemplary device 700 (or various components thereof) may, for example, operate to perform any or all functionality discussed previously with regard to FIGS. 1-6. Also for example, the exemplary device 700 may share any or all characteristics with the database system 300 and/or personal computer systems 480, 580 and 680 discussed previously. Additionally for example, the exemplary device 700 may share any or all characteristics with the local gateways, remote controllers and/or access points 585, 685 discussed previously. Further for example, the exemplary device 700 may share any or all characteristics with the exemplary device (e.g., terminal devices) 331-337, 431-437, 531-537 and 631-637 discussed previously.
The exemplary device 700 may, for example, comprise a first communication interface module 710. The first communication interface module 710 may, for example, operate to communicate over any of a variety of communication media and utilizing any of a variety of communication protocols. For example, though the first communication interface module 710 is illustrated coupled to a wireless RF antenna via a wireless port 712, the wireless medium is merely illustrative and non-limiting. The first communication interface module 710 may, for example, operate to communicate with one or more communication networks (e.g., cable television networks, satellite television networks, telecommunication networks, the Internet, local area networks, personal area networks, metropolitan area networks, etc.) via which data is communicated.
The exemplary device 700 comprises a second communication interface module 720. The second communication interface module 720 may, for example, operate to communicate over any of a variety of communication media and utilizing any of a variety of communication protocols. For example, the second communication interface module 720 may communicate via a wireless RF communication port 722 and antenna, or may communicate via a non-tethered optical communication port 724 (e.g., utilizing laser diodes, photodiodes, etc.). Also for example, the second communication interface module 720 may communicate via a tethered optical communication port 726 (e.g., utilizing a fiber optic cable), or may communicate via a wired communication port 728 (e.g., utilizing coaxial cable, twisted pair, HDMI cable, Ethernet cable, any of a variety of wired component and/or composite video connections, etc.). The second communication interface module 720 may, for example, operate to communicate with one or more communication networks (e.g., cable television networks, satellite television networks, telecommunication networks, the Internet, local area networks, personal area networks, metropolitan area networks, etc.) via which data is communicated. Also for example, the second communication module 720 may operate to communicate with local devices.
The exemplary device 700 may also comprise additional communication interface modules, which are not illustrated. Such additional communication interface modules may, for example, share any or all aspects with the first 710 and second 720 communication interface modules discussed above.
The exemplary device 700 may also comprise a communication module 730. The communication module 730 may, for example, operate to control and/or coordinate operation of the first communication interface module 710 and the second communication interface module 720 (and/or additional communication interface modules as needed). The communication module 730 may, for example, provide a convenient communication interface by which other components of the device 700 may utilize the first 710 and second 720 communication interface modules. Additionally, for example, in an exemplary scenario where a plurality of communication interface modules are sharing a medium and/or network, the communication module 730 may coordinate communications to reduce collisions and/or other interference between the communication interface modules 710, 720. The communication module 730 may, for example, operate to utilize one or more of the communication interface modules 710, 720 to perform any or all of the communication functionality discussed herein (e.g., with regard to any of the steps illustrated in FIGS. 1 and 2 and discussed previously).
The exemplary device 700 may additionally comprise one or more user interface modules 740. The user interface module(s) 740 may generally operate to provide user interface functionality to a user of the device 700. For example, the user interface module(s) 740 may operate to perform any or all of the exemplary user interface functionality discussed herein (e.g., with regard to any of FIGS. 1-6). For example, and without limitation, the user interface module(s) 740 may operate to provide for user control of any or all standard device commands. The user interface module(s) 740 may, for example, operate and/or respond to user commands utilizing user interface features disposed on the device 700 (e.g., buttons, touch screen, microphone, etc.) and may also utilize the communication module 730 (and/or first 710 and second 720 communication interface modules) to communicate with the device 700 and/or any device that is communicatively coupled thereto (e.g., to control any monitoring and/or controlling functionality discussed herein). The user interface module 740 may also operate to interface with and/or control operation of any of a variety of sensors that may be utilized to ascertain an on-screen pointing location. As discussed elsewhere herein, the user may specify location and/or other characteristics of a device.
The exemplary device 700 may comprise a display 750. As discussed elsewhere herein, the user may utilize a display to view and/or interact with maps and/or images of network environments (e.g., for device location input and/or definition, for device selection, etc.). Such interaction may, for example, utilize a display 750 of the device 700, which may also, for example, be utilized for user input (e.g., as a touch screen, utilizing cursor control, etc.).
The exemplary device 700 may comprise one or more processors 760. The processor(s) 760 may, for example, comprise a general purpose processor, digital signal processor, application-specific processor, microcontroller, microprocessor, etc. The processor(s) 760 may, for example, share any or all characteristics with processors discussed elsewhere herein (e.g., processors 360, 460, 560 and 660). For example, the processor 760 may operate in accordance with software (or firmware) instructions. As mentioned previously, any or all functionality discussed herein may be performed by a processor executing instructions. For example, though various modules are illustrated as separate blocks or modules in FIG. 7 for illustrative clarity, such illustrative modules, or a portion thereof, may be implemented by the processor(s) 760. For example, the processor(s) 760 may operate to perform any or all of the steps discussed previously with regard to FIGS. 1 and 2.
The exemplary device 700 may comprise one or more memories 761. As discussed above, various aspects may be performed by one or more processors executing instructions. Such instructions may, for example, be stored in the one or more memories 761. Such memory 761 may, for example, comprise characteristics of any of a variety of types of memory. For example and without limitation, such memory 761 may comprise one or more memory chips (e.g., ROM, RAM, EPROM, EEPROM, flash memory, one-time-programmable OTP memory, etc.), hard drive memory, CD memory, DVD memory, etc. Also for example, the memory 761 may share any or all characteristics with any of the databases discusses herein (e.g., databases 361, 461, 561 and 661).
The exemplary device 700 may also comprise one or more image-based (or map-based) device selection modules 772. Such module(s) 772 may, for example and without limitation, operate to perform any or all of the functionality discussed herein with regard to soliciting, receiving and/or processing image-based device identification information from a user (e.g., at steps 110 and 210).
The exemplary device 700 may also comprise one or more device identification modules 774. Such module(s) 774 may, for example and without limitation, operate to perform any or all of the functionality discussed herein with regard to determining a device that has been identified by a user (e.g., at steps 120 and 220).
The exemplary device 700 may also comprise one or more communication link manager modules 776. Such module(s) 776 may, for example and without limitation, operate to perform any or all of the functionality discussed herein with regard to establishing and/or maintaining communication links, for example between the user device and the user-selected device (e.g., at steps 130 and 230).
The exemplary device 700 may also comprise one or more remote control user interface modules 778. Such module(s) 778 may, for example and without limitation, operate to perform any or all of the functionality discussed herein with regard to providing a remote control user interface, for example between the user device and the user-selected device (e.g., at steps 140 and 240).
The exemplary device 700 may also comprise one or more database interface modules 790. Such module(s) 790 may, for example and without limitation, operate to perform any or all of the database interface functionality discussed herein.
FIG. 8 shows a block diagram of a non-limiting exemplary device 800 for automatically maintaining a network of user-selectable devices and/or a database characterization thereof, in accordance with various aspects of the present invention. FIG. 7 provided a diagram illustrating an exemplary device (or system) 700 in accordance with various aspects of the present invention. FIG. 8 provides another diagram illustrating an exemplary device (or system) 800 in accordance with various aspects of the present invention.
The exemplary device 800 may share any or all aspects with any of the devices (e.g., portable computer devices, access points, gateways, network controllers, terminal devices, etc.) discussed herein (e.g., with regard to FIGS. 1-7). For example, the exemplary device 800 (or various modules thereof) may operate to perform any or all functionality discussed herein. As with the exemplary device 700, the components of the exemplary device 800 may be co-located a single housing.
For example, the device 800 comprises a processor 860. Such a processor 860 may, for example, share any or all characteristics with the processor 760 discussed with regard to FIG. 7. Also for example, the device 800 comprises a memory 861. Such memory 861 may, for example, share any or all characteristics with the memory 761 discussed with regard to FIG. 7.
Also for example, the exemplary device (or system) 800 may comprise any of a variety of user interface module(s) 840. Such user interface module(s) 840 may, for example, share any or all characteristics with the user interface module(s) 740 discussed previously with regard to FIG. 7. For example and without limitation, the user interface module(s) 840 may comprise: a display device, a camera (for still or moving picture acquisition), a speaker, an earphone (e.g., wired or wireless), a microphone, a video screen (e.g., a touch screen display), a vibrating mechanism, a keypad, a remote control interface, and/or any of a variety of other user interface devices (e.g., a mouse, a trackball, a touch pad, touch screen, light pen, game controlling device, etc.).
The exemplary device 800 may also, for example, comprise any of a variety of communication modules (805, 806, and 830). Such communication module(s) may, for example, share any or all characteristics with the communication interface module(s) 710, 720 and the communication module 730 discussed previously with regard to FIG. 7. For example and without limitation, the communication interface module(s) 830 may comprise: a Bluetooth interface module; an IEEE 802.11, 802.15, 802.16 and/or 802.20 module; any of a variety of cellular telecommunication interface modules (e.g., GSM/GPRS/EDGE, CDMA/CDMA2000/1x-EV-DO, WCDMA/HSDPA/HSUPA, TDMA/PDC, WiMAX, etc.); any of a variety of position-related communication interface modules (e.g., GPS, A-GPS, etc.); any of a variety of wired/tethered communication interface modules (e.g., USB, Fire Wire, RS-232, HDMI, component and/or composite video, Ethernet, wire line and/or cable modem, etc.); any of a variety of communication interface modules related to communicating with external memory devices; etc. The exemplary device 800 is also illustrated as comprising various wired 806 and/or wireless 805 front-end modules that may, for example, be included in the communication interface modules and/or utilized thereby.
The exemplary device (or system) 800 may also comprise any of a variety of signal processing module(s) 865. Such signal processing module(s) 865 may, for example, be utilized to assist in processing various types of information discussed previously (e.g., with regard to sensor processing, position or location determination, orientation determination, video processing, image processing, audio processing, general user interface information data processing, etc.). For example and without limitation, the signal processing module(s) 890 may comprise: video/graphics processing modules (e.g. MPEG-2, MPEG-4, H.263, H.264, JPEG, TIFF, 3-D, 2-D, MDDI, etc.); audio processing modules (e.g., MP3, AAC, MIDI, QCELP, AMR, CMX, etc.); and/or tactile processing modules (e.g., Keypad I/O, touch screen processing, motor control, etc.).
In summary, various aspects of the present invention provide a system and method (e.g., an image-based system and method) for remotely monitoring and/or controlling devices. While the invention has been described with reference to certain aspects and embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from its scope. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed, but that the invention will include all embodiments falling within the scope of the appended claims.

Claims

1. A method for remotely controlling and/or monitoring a device, the method comprising, automatically without direct user interaction:

receiving an image-based device selection input from a user, the image-based device selection input based at least in part on an image of a device environment;

identifying a user-selected device based, at least in part, on the received image-based device selection input;

establishing a communication link with the identified user-selected device; and

providing a user interface via which a user may interact with the identified user-selected device.

2. The method of claim 1, wherein the image of a device environment is a camera image.

3. The method of claim 1, wherein the image of a device environment is a map image.

4. The method of claim 1, wherein the image of a device environment comprises graphical features to indicate user-selectable devices in the image.

5. The method of claim 1, wherein said receiving an image-based device selection input from a user comprises receiving a user indicating a location of a device in the image.

6. The method of claim 1, wherein said establishing a communication link comprises accessing database information for the identified user-selected device, where the accessed database information comprises information indicating a manner in which to establish the communication link.

7. The method of claim 6, wherein the accessed database information comprises communication network address information for the identified user-selected device.

8. The method of claim 6, wherein the accessed database information comprises communication protocol information for the identified user-selected device.

9. The method of claim 6, wherein accessing database information comprises accessing information from a database separate from a system implementing the method.

10. The method of claim 1, wherein said establishing a communication link comprises establishing an initial communication link with the identified user-selected device and determining, via the initial communication link, how to establish additional communication links with the identified user-selected device.

11. The method of claim 1, wherein said providing a user interface comprises providing a generic user interface via which a user may interact with a plurality of different types of devices.

12. The method of claim 1, wherein said providing a user interface comprises providing a user interface comprising features specifically associated with the identified user-selected device.

13. The method of claim 1, wherein said providing a user interface comprises accessing database information for the identified user-selected device, where the accessed database information comprises user interface information.

14. The method of claim 13, wherein the accessed database information describes respective signals to communicate to the identified user-selected device corresponding to respective user inputs.

15. The method of claim 1, wherein said providing a user interface comprises communicating, with the identified user-selected device, information descriptive of graphical user interface features to present to a user.

16. The method of claim 1, comprising receiving a user input via the provided user interface, and converting the received user input to information understood by the identified user-selected device.

17. A system for remotely controlling and/or monitoring a device, the system comprising:

at least one module operable to, at least:

receive an image-based device selection input from a user, the image-based device selection input based at least in part on an image of a device environment;

identify a user-selected device based, at least in part, on the received image-based device selection input;

establish a communication link with the identified user-selected device; and

provide a user interface via which a user may interact with the identified user-selected device.

18. The system of claim 17, where the image of a device environment is a camera image.

19. The system of claim 17, where the image of a device environment is a map image.

20. The system of claim 17, where the image of a device environment comprises graphical features to indicate user-selectable devices in the image.

21. The system of claim 17, wherein said at least one module is operable to receive an image-based device selection input from a user by, at least in part, operating to receive a user indicating a location of a device in the image.

22. The system of claim 17, wherein said at least one module is operable to establish a communication link with the identified user-selected device by, at least in part, operating to access database information for the identified user-selected device, where the accessed database information comprises information indicating a manner in which to establish the communication link.

23. The system of claim 22, where the accessed database information comprises communication network address information for the identified user-selected device.

24. The system of claim 22, where the accessed database information comprises communication protocol information for the identified user-selected device.

25. The system of claim 22, wherein said at least one module operates to access database information for the identified user-selected device by, at least in part, operating to access information from a database separate from the system.

26. The system of claim 17, wherein said at least one module is operable to establish a communication link with the identified user-selected device by, at least in part, operating to establish an initial communication link with the identified user-selected device and determine, via the initial communication link, how to establish additional communication links with the identified user-selected device.

27. The system of claim 17, wherein said at least one module is operable to provide a user interface via which a user may interact with the identified user-selected device by, at least in part, operating to provide a generic user interface via which a user may interact with a plurality of different types of devices.

28. The system of claim 17, wherein said at least one module is operable to provide a user interface via which a user may interact with the identified user-selected device by, at least in part, operating to provide a user interface comprising features specifically associated with the identified user-selected device.

29. The system of claim 17, wherein said at least one module is operable to provide a user interface via which a user may interact with the identified user-selected device by, at least in part, operating to access database information for the identified user-selected device, where the accessed database information comprises user interface information.

30. The system of claim 29, where the accessed database information describes respective signals to communicate to the identified user-selected device corresponding to respective user inputs.

31. The system of claim 17, wherein said at least one module is operable to provide a user interface via which a user may interact with the identified user-selected device by, at least in part, operating to communicate, with the identified user-selected device, information descriptive of graphical user interface features to present to a user.

32. The system of claim 17, wherein the at least one module is operable to receive a user input via the provided user interface, and convert the received user input to information understood by the identified user-selected device.