WO2000036484A2

WO2000036484A2 - Improvements relating to control of computer interfaces

Info

Publication number: WO2000036484A2
Application number: PCT/NZ1999/000217
Authority: WO
Inventors: Nelson Donald Lipinski; Wayne Michael Brand
Original assignee: Aoraki Corporation Limited
Priority date: 1998-12-11
Filing date: 1999-12-13
Publication date: 2000-06-22
Also published as: AU1899900A; NZ333328A; WO2000036484A3

Abstract

A system for implementing controls (12, 13) on a software interface. Each control is formed by an image layer (30) which is visible on the interface, and one or more masks which are linked to control instructions. The masks are generally bitmaps in which colour information determines the instructions. A user positions an interface device such as a mouse (35) pointer over appropriate parts of the image to request an action. The position of the pointer is related to each mask and an instruction is determined according to the colour of the mask at that position.

Description

IMPROVEMENTS RELATING TO CONTROL OF COMPUTER INTERFACES

FIELD OF THE INVENTION

This invention relates to user control of graphical computer interfaces and particularly but not solely to control systems involving images which are displayed for a user on the screen of a computer system. The invention also relates to retrieval and presentation of information on such an interface.

BACKGROUND TO THE INVENTION

Graphical interfaces are included with most computer operating systems such as the WindoWSTm and Macintosh' systems. Controls are generally provided on each interface by way of active screen areas which contain more or less complex images. commonly referred to as buttons. Users generally actuate the controls using a screen pointer and a manual pointer controller such as a mouse. Positioning the pointer on a control image and clicking the mouse causes an event such as

of additional information on the interface.

Controls are typically defined as a square or rectangular area on the screen having an origin point, a width and a depth. Each area typically represents a single control and a single event which may be generated by the user. A meaningful image is usually associated with each control, such as the shape of a button, and is often displayed with a realistic three dimensional appearance. Pixel data representing each image is stored as a graphical file which usually includes a bitmap. Several versions of each image are usually stored and displayed alternately as a particular control is selected and operated by the user, to create visual effects such as a button being highlighted, pushed down, or raised from the screen for example.

Various routines run within the graphical interface software to monitor the position of the pointer on the screen, which may take the form of an arrow for example. Each movement of the pointer generates an interrupt and returns the position of a reference point on the pointer, such as the screen coordinates of the tip of the arrow. If the reference point is positioned in a control area then the control image associated with that area may be varied in some way. Each actuation of the pointer controller generates an interrupt causing assessment of the current position of the pointer and determination whether an event must take place. If the pointer is positioned on a control area then the image associated with that area is further varied and the event associated with that area is generated.

Applications which may be run by the user under the operating system often require complex controls. Numerous alternative control events must often be presented side by side for selection by the user within a relatively small screen area. The possibilities for doing so are complicated or limited by the traditional use of a single square or rectangular area to represent a single control. The possibilities are also limited by the practice of displaying a relatively sophisticated image in relation' to each control, as this prevents pixel sensitive control information being derived from the position of the pointer on the image.

Another limitation on traditional interface control systems arises where multiple events are to be triggered by operation of a single control. Similarly, different combinations of related events may be triggered by operation of closely spaced controls. These effects require special sensitivity in the arrangement of controls on the interface. A particular application has arisen in relation to display of various combinations of database information for computer systems operated by utility organisations such as electricity supply companies.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide for interface control systems which are able to conveniently incorporate multiple and perhaps irregularly shaped controls within a given interface area, or at least to provide an alternative to existing systems. In general terms the invention generally involves use of one or more control masks in association with a control image, and use of pixel based control information derived from the masks to determine the events which are generated on activation of the particular control.

Accordingly in one aspect the invention may broadly be said to consist in a control system for a graphical user interface, including: a control image having one or more image portions which are displayed on the interface, a control mask having one or more mask portions which correspond at least in part to respective image portions, a first process which determines control information from the mask when an interface pointer is positioned on the control image by an operator, and a second process which carries out one or more first predetermined actions on the interface according to the control information.

In another aspect the invention may be said to consist in a method of operating a computer control on a graphical user interface, including: displaying a control image on the interface, determining position information for an interface pointer which is actuated by an operator, detecting when the pointer is positioned on the control image, determining control information from a control mask which corresponds at least in part to the image, and varying the control image according to the control information.

In a third aspect the invention may be said to consist in a method of presenting information using a control system on a graphical user interface, including: determining position information for a location on the interface, determining control information from a plurality of control masks which include the position on the interface, selecting information items from a plurality' of sources corresponding to respective control masks in the control system, and displaying the information items on the interface or carrying out some other action.

The invention may also broadly be said to consist in any alternative combination of parts or features here referred to or shown in the accompanying drawings. Known equivalents of these parts or features not expressly set out are nevertheless deemed to be included.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the invention will be described with respect to the drawings of which:

Figure 1 shows a control image having two portions displayed on a graphical user interface, by way of example.

Figure 2 shows a control mask which corresponds to the displayed image in accord with the invention. Figures 3a and 3b show an interface pointer positioned over alternative portions of the control image which are correspondingly highlighted.

Figures 4a and 4b show alternative portions of the image displayed in up and down forms when selected by actuation of a pointer controller. Figure 5 outlines a control process which causes highlighting of the image portions on movement of the pointer.

Figure 6 outlines a control process which causes image portions to be displayed in the down form on actuation of the pointer controller.

Figure 7 outlines a process which causes an event on the interface according to control information in the mask.

Figure 8 outlines a process which enables events to take place according to control information in a plurality of masks which correspond to the control image.

Figures 9a to 9f show a simplified control image and a plurality of mask images relating to the control image, for a geographical application by way of example.

Figure 10 is a display which might be generated by actuation of a portion of the control image in Figure 9a at a particular geographical location.

Figure 1 1 is a personal computer system on which a graphical user interface may be implemented, also by way of example, and Figure 12 schematically shows a computer system including an operating system, graphical interface and database arrangement.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to these drawings it will be appreciated that the invention may be implemented in various forms and as part of a wide range of graphical interface applications. The following description is given by way of example only. Operation of those parts of a graphical interface which are standard in nature will be well understood by a skilled person and need not be described in detail.

Figure 1 shows a reasonably complex control image 10 which could be associated with a control system in a graphical interface, according to the invention. The image is a multicoloured double spiral formed by two interleaved arms 11 and 12 on a plain background, and is designed to have a three dimensional aspect which appears raised from the interface. Graphical information which determines the image is stored as a square bitmap indicated in dashed outline. The information includes data which determines the colour of each pixel within the image when displayed in an area on a computer screen as part of a control. Due to the complex visual nature of the image each portion is represented by a range of colours as required to create the various visual effects. Each arm of the image is a separate image portion which represents a separate control area having a spiral shape and is separated from the other arm by a non-control area 13. Separate controls which can be operated to generate distinct events can be thereby provided for the user. A two part control of this kind would be difficult to implement using conventional control systems.

Figure 2 is a simplified mask image 20 which may be associated with the control image 10 of Figure 1. The mask includes separate mask portions 21 and 22 which correspond in shape and size to arms 11 and 12 of the image 10. A plain portion 23 corresponds to portion 13 which separates the arms. The mask is not displayed to a user and each of the mask portions is represented by a single colour. Graphical information which determines the mask image is stored as a bitmap indicated in dashed outline. The single colour which is allocated to each portion of the mask forms control information for the control system which incorporates the image 10. The colour may be considered a code which is used by a control process to generate the event which is associated with the control. It should be noted that the shapes, sizes, colours and locations of the mask portions of image 20 preferably correspond closely with the control portions of image 10 but need not necessarily do so.

Figures 3a and 3b are control images 30 and 31 which might be displayed in response to the position of an interface pointer 35 in the form of an arrow. In Figure

3 a the pointer has been moved by the user to a position on image portion 12 and the control system has responded by changing the colour of that portion to provide visual confirmation. In Figure 3b the pointer has been positioned on portion 1 1 and the control system has responded by changing the colour of that portion. It is the tip 36 of the arrow which is used by the control system as a reference point when determining the position of the pointer at any instant. An action of this kind provides a highlight for the user and confirms that actuation of the pointer controller will generate the event which is associated with the particular control. Positioning the pointer on portion 13 produces no change in the image to indicate a non-active area. Various highlight actions are possible, including a conversion of the pointer by colour or shape, as will be familiar to a skilled reader. The pointer could be converted to a cross for example.

Figures 4a and 4b are control images 40 and 41 which might be displayed by the control system in response to actuation of the pointer controller in Figures 3a and 3b. In Figure 4a the controller has been actuated with the pointer 35 positioned on image portion 12, typically by depressing a mouse button, and that portion of the image 40 is now displayed with an aspect which appears inwards on the screen. In Figure 4b the pointer controller has been actuated with the pointer positioned on portion 1 1 and that portion has been similarly depressed. Depression and release of a mouse button are sometimes termed mouse down and mouse up actions respectively. It is generally the mouse up action which generates an event associated with the particular control although a detail of this kind can vary among different interfaces.

Figure 5 is a process flowchart which outlines how a control system may vary a control image to highlight placement of the pointer 35 on an image portion, such as described in relation to Figures 3a and 3b. Movement of the pointer controller is detected in step 50 and the system then determines whether the pointer is positioned on a control image or other control area of the interface in step 51. If the pointer is positioned on a control image the system retrieves control information from the associated mask in step 52. Control information is preferably pixel colour data from the corresponding position in a bitmap which forms the mask. If in step 53 the control information indicates that the pointer is positioned on an active image portion then in step 54 the system varies the control image, or perhaps the appearance of the pointer.

Figure 6 is a process flowchart which indicates how a control system may vary a control image to indicate a mouse down event as described in relation to Figures 4a and 4b. Actuation of the pointer controller is detected in step 60 and the system then determines whether the pointer is positioned on a control image or other control area of the interface in step 61. If the pointer is positioned on a control image the system retrieves control information from the associated mask in step 62. If in step 63 the control information indicates that the pointer is positioned on an active image portion then in step 64 the system varies the control image to confirm that the mouse down event has taken place. This indicates to the user that on further actuation of the controller, typically by releasing a mouse button as described in relation to Figure 7. a control event will be generated.

Figure 7 is a process flowchart which indicates how a control system may retrieve control information from a mask on detection of a mouse up event and trigger a control event as described in relation to Figure 10. The mouse event is detected in step 70 and the system determines whether the pointer is still positioned on a control image in step 7 1. If the pointer is still on a control image then control information is retrieved from the corresponding mask in step 72 and compared with previous control information in step 73. The pointer is normally required to be positioned on the same control for both the mouse down and mouse up events. If the pointer has moved elsewhere on the interface then any potential action arising from the mouse down event is preferably cancelled with an appropriate visual confirmation for the user. If the control information and therefore the pointer position is consistent the control system varies the control image in step 74. generally by raising a depressed portion, and then highlighting the portion. The image might be restored to its original appearance, for example. In step 75 the event associated with the control is generated, such as a display of information for the user.

Figure 8 is a flowchart which outlines how the process of Figure 7 may be extended to trigger multiple control events as described in relation to Figure 10. More than one mask may be associated with a particular control on the interface. Each mask provides control information for a respective event according to the position of the pointer in the control image and the colour of the pixel at the corresponding position in the mask. Following step 75 in Figure 7. step 80 in Figure 8 determines whether any further masks are associated with the control image. If a further mask exists then control information is retrieved from the bitmap in the mask in step 8 1. The associated event is then generated in step 82. The process terminates once each associated mask has been considered.

Figure 9a is a control image 90 which might be used in association with a geographical control system for an interface relating to New Zealand. The interface might be provided as part of a website containing New Zealand related information, for example, a kiosk at a public library, or a private computer system operated by a utility organisation. The image has been considerably simplified for purposes of clarity and would show detailed topographical or municipal features in practice, for example. Pointer 35 is shown positioned by a user near a northern part of the South Island. The user may be seeking or providing data relating to an individual address or a physical site located in that part of the country. The data could be related to subscription services provided to that address by the utility organisation, for example. In the example given below, the data is geophysical and demographic in nature. A control image of this kind might cover most or all of an entire computer screen in area.

Figures 9b to 9f are a series of mask images 91 to 95 associated with the control image 90. As bitmaps they may be considered as being superimposed on a common screen area. Each mask represents the distribution of a particular characteristic or statistic of New Zealand, once again by way of example. Each mask has a number of portions which in turn represent regions where the feature or statistic takes a common value, or a value within a predetermined range, with the values and perhaps other information being stored in a database. The mask in Figure 9E represents a distribution of soil types, while that in Figure 9F represents population statistics, and the others are also self evident. Regions having a common soil type or population density in a given range are represented in the mask by a common colour. The arrangement of coloured portions in these masks will generally not be indicated in the control image of Figure 9a. although images of this kind may well be accessible elsewhere.

In Figure 9a the control image 90 has two active control portions 97 and 98 which represent the North and South Islands separated by a non-active portion. A user positioning a pointer or actuating the pointer controller on either portion will observe the effect of processes such as those described in relation to Figures 5 to 8. The image portion corresponding to the North Island may be highlighted, depressed and restored to its original appearance by movement and actuation of the pointer controller, for example. Each mask image has a number of distinct coloured portions which correspond to only part of the portions of image 90. However, in this example each mask portion is wholly within the area of an image portion, although this need not generally be the case. Alternatively, each control image portion may be wholly within the area of a respective mask portion. Figure 10 is a display table 100 which might be generated on the interface by a user when actuating the pointer controller at a position as indicated in Figure 9a. The table contains individual elements of data retrieved from a database according to control information from each of the masks in Figures 9b to 9f. Each retrieval of a data element such as a level of rainfall is achieved as an event generated according to the processes of Figures 7 and 8. It will be appreciated that a very wide range of effects can be conveniently created on an interface or within a computer system using a control system of the present kind. Other actions may also take place in the computer system instead of or in addition to the display of information. Information could be added to the database for example, or a fresh screen display could be produced.

Figures 11 and 12 indicate in highly schematic forms, hardware and software aspects of a computer system which might implement one or more interface control systems according to the invention. The hardware and software which is shown in these systems will be self evident to a skilled reader. It will also be appreciated that these systems are a few of many possible examples, only some of which have been mentioned above.

Claims

CLAIMS:

1. A control system for a graphical user interface, including: a control image having one or more image portions which are displayed on the interface, a control mask having one or more mask portions which correspond at least in part to respective image portions, a first process which determines control information from the mask when an interface pointer is positioned on the control image by an operator, and a second process which carries out one or more first predetermined actions on the interface according to the control information.

2. A system according to claim 1 further including: a plurality of control masks each having one or more portions which correspond at least in pan to image portions, wherein the first process determines control information from each of the masks when the interface pointer is positioned on the control image.

3. A system according to claim 2 further including: a third process which carries out one or more second predetermined actions according to the control information when a pointer controller is actuated by the operator.

4. A system according to claim 3 further including: a plurality of information sources which correspond to the plurality of control masks. wherein the second predetermined actions include selection and display of information from the sources according to the control information.

5. A system according to claim 1 wherein: the first predetermined actions include display of one or more alternative control images on the interface.

6. A system according to claim 1 wherein: the control mask includes a bitmap and the control information includes pixel colour data.

7. A system according to claim 1 wherein: the control mask is a graphical image file derived from the control image.

8. A method of operating a computer control on a graphical user interface, including: displaying a control image on the interface. determining position information for an interface pointer which is actuated by an operator. detecting when the pointer is positioned on the control image, determining control information from a control mask which corresponds in part to the image, and varying the control image according to the control information.

10. A method according to claim 8 further including: detecting actuation of a pointer controller by the operator, and presenting information on the interface for the operator according to the control information.

10. A method according to claim 8 further including: determining control information from a plurality of control masks which correspond to the control image.

1 1. A method according to claim 10 further including: selecting and presenting information on the interface from a plurality of sources corresponding to the plurality of control masks.

12. A method according to claim 8 further including: varying the control image by selecting an alternative image from a plurality of images.

13. A method according to claim 8 wherein: the position of the pointer on the interface is provided as pixel coordinate information.

14. A method according to claim 8 wherein: the control mask is a bitmap and the control information includes pixel colour data.

15. A method according to claim 1 1 wherein: the control image represents a geographical region and the plurality of sources contain information relating to operation of utilities within the region.

16. A method of processing information using a control system on a graphical user interface, including: determining position information for a location on the interface. determining control information from a plurality of control masks which include the position on the interface. selecting information items from a plurality of sources corresponding to respective control masks in the control system, and carrying out a computer action related to the information items.

17. A method according to claim 16 further including: displaying a control image on the interface representing a geographical region, receiving geographical coordinate information relating to an activity at a site within the region, and converting the coordinate information into interface position information relating to the site as located on the control image.

18. A method according to claim 16 further comprising: displaying the information items on the interface.

19. A control system substantially as herein described with reference to the accompanying drawings.

20. A method of operating a control system substantially as herein described with reference to the accompanying drawings.

21. A method of presenting information substantially as herein described with reference to the accompanying drawings.