WO2005109189A1

WO2005109189A1 - Method and system for graphical user interface layout generation, computer program product therefor

Info

Publication number: WO2005109189A1
Application number: PCT/IB2004/001454
Authority: WO
Inventors: Luca Buriano; Walter Laurent Goix
Original assignee: Telecom Italia S.P.A.
Priority date: 2004-05-07
Filing date: 2004-05-07
Publication date: 2005-11-17

Abstract

The invention deals with a method and a system for dynamically generating the layout for a Graphic User Interface or GUI of an application aimed to be shown in a destination display context, for example through a communications network. Characteristic of the invention is the arrangement of a collection (12) of graphical elements for graphic interface, including a representation of said elements independently from the destination display context, and the following optimisation of the GUI layout configuration depending on the final display context, in particular through selection of said elements of the interface depending on said display context and the application of a sequence of transformations (400) starting from an initial layout configuration, in sight of optimising the fitness degree of such configuration. The invention also deals with a set of modules aimed to create a layout for a Graphic User Interface or GUI, in addition to a communications network and a program or group of programs that can be executed by the system, arranged for actuating the above method.

Description

Method and System for Graphical User Interface layout generation, computer program product therefor

TECHNICAL FIELD

The present invention deals with Graphical User Interfaces (GUI) , and in particular with a method and a system for dynamically generating a layout for a graphical interface.

Graphical User Interfaces (GUI) are more and more wide-spread in software applications for displaying information on a display device , in order to allow users to interact more easily with application programs being used. However, the development and adaptation of a Graphical User Interface can still be a difficult task in network applications due to the extreme heterogeneity of terminal and display devices on which GUIs themselves must be used.

In many cases, GUI designers and software applications (application programs) programmers do not know the size, geometric arrangement and graphical capabilities of final display devices or units that the user will use for interacting with the software application.

This inconvenience is further worsened by the current trends in software applications for end users. Even on a single display device, a user can in fact customise the display geometry (display area) for a specific software application, for example dynamically resizing the software application window on the screen of a personal computer. BACKGROUND ART

An arrangement for creating a layout for Graphical User Interfaces is offered by the Java programming language and in particular by Java Foundation Classes (JFC) technology and Swing components, as described in manual "The JFC Swing Tutorial: A Guide to Constructing GUIs" by Kathy Walrath and Mary Campione, ISBN 0-201-43321-4, Addison- esley, June 1999.

The "Swing" graphical interfaces programming library includes different types of layout (for example: FlowLayout, GridLayout, GridBagLayout) , each one of which allowing realising specialised layouts to be used by GUIs that can be adapted to different display contexts.

However, the assignment of a group of graphical elements to a given type of "Swing" layout must be previously decided by the software application (application) programmer. Moreover, each type of "Swing" layout is limited in managing only a specific set of GUI adaptation features: for example, the GridBagLayout type can flexibly resize a complex grid of graphical elements, but, for example, is not able to dynamically modify their row/column position in the grid itself, or their mutual alignment.

Therefore, the known approach still leaves under the application programmer's responsibility to solve the majority of problems pertaining to the graphical elements layout adaptation on the GUI of THE final display devices.

In the Graduation Thesis made in the Helsinki Technology University, entitled "Automated Pagination of the Generalized Newspaper using Simulated Annealing", Krista Lagus, November 1995, a method is known for optimising the creation of a page layout for a newspaper, a task that has some common features with the problem of generating a layout for GUIs.

The Graduation thesis deals with the problem of how to fill- in a display area of an electronic newspaper with different fields containing newspaper articles, taken from a set of available articles. The thesis author proposes a solution based on the optimisation technique known as "Simulated Annealing", searching for possible configurations of different fields positions in the display area and guiding the search process by means of a potential function for evaluating the validity of a chosen configuration.

This optimisation method allows the automation of many aspects of layout generation and frees the software application programmer from his responsibility of defining all details for the final adaptation of GUI layout. However, the proposed optimisation method is rather heavy from the computational (and the processing time) point of view, requires an iterative acquisition of graphical elements and does not lend itself to be applied to the generic problem of generating a layout for GUIs since being addressed to a single display context.

Moreover, in the known application, a single type of graphic element is present (the rectangle that contains an article) . A system and a method are also known from US-A-6 556 217 for adapting the contents of pages to be displayed depending on the features of the destination terminals.

The known system and method allow determining the available space on terminal and adapting the sizes of contents to be displayed if the space is insufficient. Synthetically, the applicant deems that the prior art does not solve the general problem of generating a layout for GUI in which there are numerous types of simple graphical elements and/or "widgets" (graphical display and user interaction elements) , a high number of graphical elements per page (also for simple layouts) and a complex set of constraints between graphical elements themselves.

DISCLOSURE OF THE INVENTION

Object of the invention therefore is proposing a dynamically generating method to be used by a software application of a GUI layout operating in different display contexts. The method for generating GUI layouts, when adapted to a specific display context, does not require the definition of GUI details by a programmer, but makes it possible to automatically manage the assignment and arrangement of graphical elements for every type of element and independently from the number of graphical elements being present .

For such purposes, objects of the invention are a method and a system for dynamically generating a Graphical User Interface layout having the features recalled in the independent claims.

Further objects of the invention are a set of modules adapted to define a GUI layout, in addition to a communication network and a computer program product or a group of computer program products as claimed.

Synthetically, the invention proposes a methodology for generating a GUI layout, in which a layout engine is adapted to dynamically perform the adaptation of the graphical elements presentation for a specific context of a display device (in particular, but not only, to the geometry of such specific device) .

The layout engine determines a better final GUI layout, through the search for an optimum layout transforming sequence .

The transformations belong to the following two categories: - Moves : these are actions that modify the spatial configuration of a group of graphical elements. They include simple transformations (such as, for example, a graphical element resizing) , and complex transformations (such as, for example, the execution of an existing layout generation method, for example the FlowLayout Swing method of the Java language) ; - Rules : these are moves that can be applied only if a predetermined set of conditions is satisfied by a group of graphical elements . The rules embed the knowledge of the expert designer in graphical interfaces about the usability of graphical elements.

The search for the optimum layout (namely the optimum sequence of Moves and Rules) is guided by the optimisation of a potential function for evaluating every examined layout solution, that takes into account the layout functionality and usability.

The method of the invention therefore joins together the advantages of an approach based on expert knowledge in realising a GUI (namely, a layout adaptation by applying predetermined rules) with those of an approach based on the optimisation of a potential function. In fact, as pointed out above, the only optimisation of a potential function would be heavy from the computational point of view in solving a generic problem of generating GUI layouts . The use of Rules allows guiding the optimisation and speeding it, orienting the search process towards more promising directions. On the other hand, the only use of Rules would generate some difficulties, among which the one of determining the order of application of a great number of rules to the layout, and the related one of solving the conflicts that arise from the existence of conflicting rules.

The optimisation process based on moves and rules and on the potential function solves the above-mentioned prior art problems, by allowing to obtain an optimum GUI layout in a quick and efficient way.

BRIEF DESCRIPTION OF DRAWINGS

Further characteristics and advantages of the invention will clearly appear from the detailed description that follows, provided as a non-limiting example, with reference to the enclosed drawings, in which: figure 1 shows a functional architecture of the system for dynamically generating GUI layouts according to the invention; figure 2 is a flow diagram of the method for dynamically generating GUI layouts according to the invention; and figures 3a-3d are application examples of a method and a system object of the invention to a data inputting (form) GUI.

BEST MODE OF CARRYING OUT THE INVENTION Figure 1 shows the functional architecture of a system for dynamically generating Graphic User Interface layouts according to the invention.

In the following description, such a GUI is aimed to be used on a final display device (not shown) that a user uses for interacting with a software application, both in case of an application provided on a local or geographic communication network, for example residing on a network server, or in case of an application residing on the same processing device provided with the display device .

A layout generation module (or layout engine) for Graphical User Interface is designated with 10. Module 10 comprises a processing and computing program or group of processing and computing programs that realize the method according to the invention; such programs or modules can be executed on a processing system such as, for example, a server of a services centre of a network applications provider (for example an Internet Service Provider) . An alternative embodiment envisions that such programs can be executed on a user processing device in case of applications residing on the device itself, such as for example browser type applications.

In general, the software application has associated therewith, or comprises, the module 10 for dynamically generating GUI layouts and a set of databases (12, 14, 16, 18) as described below.

In an embodiment it is envisioned that the application is aimed to provide the module 10, depending on data to be displayed, with a set of information that are dynamically built or are found from databases.

In a preferred embodiment, the application provides the module 10 (Figure 1) , every time, with such a pointer to allow the engine 10 to find from the databases all information to be used for generating the GUI layout. Information databases to which the module 10 for example refers, comprise, for example: a first input database 12 comprising a collection of graphical elements to be represented; a second input database 14 comprising a collection of information, such as for example constraints, rules, moves and potentials representing the relationships between graphical elements in a single displaying context, as it will be better described below; a third input database 16 comprising a collection of information, such as for example constraints, rules, moves and potentials representing the specific relationships between the graphical elements related to sets of display contexts, as it will be better described below; and a fourth database 18 of information on the display device, comprising a collection of information related to the display area and the widgets available on the display device for which the GUI layout must be generated.

The logically distinct input databases 12-18 represent collections of statically or dynamically generated data, coming from one or more sources, that can be defined both externally and internally to the layout generation module 10, some information being able to be automatically inferred or automatically generated by the above module even if not explicitly defined. In case such databases are externally defined, such databases are, for example, files that are part of the application itself or databases associated with the application or residing on the processing system or device.

For example, the first three input databases (12, 14, 16) can be defined through a single file, written in a XML type of language, while the database 18, related to the display device, can be extracted from a database or from protocol information transmitted by the user device to the server through a standard protocol such as the HTTP protocol . As known, in fact, the HTTP connection protocol can comprise in its header information about characteristics, for example geometric ones, of the display device or the application run by the user terminal .

The layout generation module 10 is adapted to provide as output, after suitable processing, as will be described below in detail, an optimised geometric layout, specifically generated for the display device, to a graphic presentation module or GUI module 20.

In detail, the databases comprise, for example, the following pieces of information.

1. Display area

The display area is the graphic interface area on which all elements of the interface (defined below) lay, and can be acquired from the information data base 18 on the display device .

A display area is defined by: - a geometric shape (for example the rectangular shape commonly used for graphic interface display areas) ; - a set of geometric parameters (for example, display area size, namely width and height if it is rectangular) ; - a visible display area; this is a smaller display area that can move like a window on a wider display area, in the simplest case coinciding with the actual display area (for example a display area of a web browser that encloses a whole web page typically contains a smaller visible display area corresponding to the visible portion of the web page, that can be moved on the whole page through scrollbars) ; - a structure of pages, namely a sub-division of the display area into a fixed or variable number of pages, namely smaller display areas (in the simplest case the display area has a structure of pages limited to a single page) .

2. Graphical elements

Graphical elements are the minimum abstract entities managed by the layout generation method of the invention, whose data can be acquired from the input database 12.

Every graphical element includes a universal or neutral representation of parameters and attributes of the element or element "model", independently from the destination display device.

They are used to make a model of the geometric layout of real widgets used in graphic interfaces. Each element is defined by: - geometric shape (for example the rectangular shape commonly used for widgets such as buttons, text fields, labels, graphic panels, ..., or the circular shape traditionally used for widgets like the radio buttons) ; - a set of geometric parameters (for example width and height of a rectangular widget) ; - a pair of coordinates pointing out the relative/absolute position of an element on the display area; - a set of non-geometric attributes representing, for example, the graphical element type (for example, a model interface element of a button widget can have the' Typology = "Push Button" attribute) .

3. Constraints

Graphical elements are subjected to constraints that limit the possible positions and sizes on the display area and with respect to other elements, in addition to possible intrinsic constraints (for example the minimum size) .

Data related to constraints can be acquired from input databases 14 and 16, depending on what is required by the application or by the module 10.

The types of constraints include, for example: - ordering constraints, that determine a spatial ordering of widgets on the display area (for example, a "OK" button must appear at the end of a data entry form, a title label must appear at the beginning of the form, a group of three text entry fields "First Name", "Last Name" and "Age" can appear in any position in the form, but only in a predefined order) ; - adjacency constraints, that constrain two or more elements of the interface to be represented next to each other (for example a check button must appear near the corresponding label, horizontally or vertically but not diagonally) ; - size constraints, that impose limits to a widget size (for example, the width of a button must be included between 50 pixels and 200 pixels, the width of a text field must be at least equal to a number of pixels corresponding to ten times the number of field characters) ; - overlapping constraints, that impose a policy of space overlapping to elements of the interface (for example, widgets must not overlap) ; - repetition constraints, that impose the repetition of a structure in the layout of a group of widgets (for example, in a group of widgets that logically represent a table, the widgets corresponding to every row must have the same layout) ; - containment constraints, that impose that some widgets appear inside of other widgets (for example, two pairs of text field - label and a button must appear inside a frame widget) .

4. Moves

A move is an action that modifies position and/or geometric parameters of one or more elements of the GUI interface.

Data related to moves can also be acquired from input databases 14 and 16, depending on what is required by the application or by the module 10.

A set of moves is adapted to transform a GUI layout into another layout, changing the geometric configuration of the graphical elements, as will be described in detail below. Moves can be explicitly defined and can be set as preferred moves or moves having precedence with respect to other moves attempted by the layout generation module 10. The types of moves include, for example: - position modifying moves, that change the position of an element on the display area or the relative position of two or more elements (for example, the horizontal movement of a button on the display area by 100 pixels, the horizontal centring of a button on the display area, the movement of two horizontally aligned buttons in order to be vertically aligned, transforming a 4x2 grid of pairs of label field - text field into a 2x4 grid) ; - size varying moves, that modify the size of one or more elements (for example, widening of a text field by 30%, reducing a button till its adaptation to the display area) ; - alignment moves, that modify position and/or size of one or more elements, so that they are taken to alignment (for example, vertically align, both on the right and on the left, a group of text fields, vertically align on the left a button with the same group of text fields) ; - pagination moves, that divide the display area into different pages (minor display areas) and assign the elements to each of them (for example, dividing the display area containing a table of widgets into a number of pages that is equal to the number of table rows and assigning the widgets included in each row to the corresponding page) ; - replacement moves, that replace one or more elements with a different group of elements (for example, replacement of a widget of the listbox type with a widget of the dropdown list (cascade) type, or replacement of a table widget with a text label grid) ; - complex moves, that apply a complex transformation to a layout; in other words, a complex move can be any layout algorithm, which allows the dynamic layout generation method to exploit existing layout algorithms as its own components (for example, application of "FlowLayout" Swing layout method in Java language to modify position and size of a group of elements) .

5. Rules

A rule is a structure composed of a set of conditions and of an action (set of moves) .

Data related to rules can also be acquired from input databases 14 and 16, depending on what is required by application or module 10.

A rule specifies that all action moves can be performed only if all conditions defined in the set of conditions are satisfied.

Examples of rules are as follows : - if (condi tion) a group of widgets shows an alternating pattern of labels and text fields then (action) move: align the group of widgets exactly on a row

^• if (condi tion) a group of widgets shows an alternating pattern of labels and text fields then (action) move: arrange the group of widgets on two columns ^■ if (condi tion) there are two nearby pairs of checkbox-label then (action) move 1 : arrange the two pairs one above the other move 2 : vertically align the two checkboxes move 3: vertically align the two labels

• if (condi tion) a frame containing some widgets has been resized then (action) move: apply a layout generation method FlowLayout of the "Java Swing" type to these widgets.

When a rule is applied, the execution of the action that it defines, usually modifies the current layout, creating conditions to apply other rules, and so on.

With respect to the optimisation context (that will be explained in detail below) rules can be divided into: - mandatory rules : if the set of conditions is satisfied, the action MUST be executed; - exploring rules: if the set of conditions is satisfied, the action CAN be executed; the actual action execution then depends on the optimisation process, as will be described in detail below.

6. Potentials and potential components

A potential or potential function is a function that allows performing a quantitative evaluation of the validity of a specific layout. This function can be used in an optimisation context to evaluate and assign a score to alternative layouts in order to be able to select the best one, namely the one having, conventionally in the following description, the lowest score.

A potential function can be decomposed into a set of components of a global potential, in which each potential component can be represented with a potential function and is responsible for the assignment of a score to some specific layout characteristics.

Data related to potentials can also be acquired from input databases 14 and 16, depending on what is required by the application or module 10.

Types of global potential components include, for example: - preferred size potential, that favourably evaluates a predetermined range of sizes for a graphical element or a typology of graphical elements (for example, a button has a preferred width of 150 pixels, and its potential of P (width) defined as P (width) = abs (width-150) reaches its minimum when the actual button width is equal to the preferred width and increases when it goes away from the preferred width) ; - adjacency potential, that evaluates the adjacency between types of elements of the interface (for example, the buttons should appear one near the other, then this potential is numerically determined as P (layout) = 1/ (1+p) where p is an integer that represents the number of adjacent buttons) ; - magnetisation potential, that evaluates aligned groups of elements., since an interface layout in which widgets are aligned, at least partially, is often more usable and aesthetically pleasing (for example, the magnetisation potential of a given layout is computed as P (layout) 1/ [l+<number of labels and text fields that are vertically aligned leftward> + <number of buttons aligned horizontally towards the top>] ) ; - compactness potential, that evaluates the compact arrangements of elements, since an interface layout in which widgets are arranged in a compact geometric configuration can often be more usable and aesthetically pleasing; a specialised version of the compactness potential is the space filling potential, that evaluates the arrangement of elements of the interface that efficiently fill the display area minimising the visible empty space (for example, a compactness/space filling potential applied to the following two configurations

A ABC BC DEF DE F

will more favourably evaluate the right-hand configuration) ; - scrolling potential, that more favourably rates higher (equal to one maximum) ratios (visible display area) / (display area) , for example assigning them a value corresponding to the ratio inverse; purpose of this potential is minimising necessary scrolling operations to move across a display area.

It must be noted that, in general, the same layout can be favourably evaluated by some potential components, whilst it can be unfavourably evaluated by other components. For example, a layout that is favourably evaluated by the magnetisation potential could not be very compact and therefore be unfavourably evaluated by the compactness potential. Moreover, a rather compact layout with reduced sizes will be favourably evaluated by the compactness potential and by the sliding potential, but could excessively compress the width of all widgets, and thereby be unfavourably evaluated by the size potential.

The method according to the invention optimises a global layout potential, which determines a compromise among all potential components. In order to control such compromise, the potential components can be conveniently weighed in a global potential formula.

For example, the global potential of the graphic interface can be defined as : global ⁼ W_si_ze size + W compactness compactness +™magnetisation Pmagnetisation

m Which Psize / ^compactness and Pmagnetisation are the three potential components and W_size , W_COm_Pactness and W_mag_netisation are the respective weights .

Alternatively, non-linear potential functions can be provided .

As an output of the layout generation module 10, the GUI layout includes, for example: - a display area with a set of values identifying all related geometric parameters; and - a set of graphical elements, including for each graphical element of the set a set of values related to position and geometric parameters. In other words, the generated GUI layout is a specific geometric configuration of a set of graphical elements on a display area.

The embodiment example of the invention that follows refers to a GUI presentation on a device, for example a communications network device, by a data entry (form) application.

In the example, the layout generation module 10 receives interface-related information through a document of the XML type, however realised.

The <GEs> section allows defining the list of elements of the interface to be displayed (in this case a title, four entry fields with their respective labels and two buttons) :

</GEs >

The <Constraints> section allows defining the constraints that must be observed by the elements of the interface .

The <Ordering> sub- section allows specifying the displaying order of the graphical elements (elements of the GUI interface) :

<0rdering> <Element ge=title /> <Element ge=labell /> <Element ge=textFieldl /> <Ξlement ge=label2 /> <Element ge=textField2 /> <Element ge=label3 /> <Element ge=textField3 /> <Element ge=label4 /> <Element ge=textField4 /> <Element ge=buttonl /> <Element ge=button2 /> </0rdering>

Each constraint <Next-to> in the <Proximity> sub-section allows specifying that two elements of the interface must be displayed one near the other (namely one above the other, or one on the left of the other) .

</Constraints>

The <Moves> section allows specifying the available moves by the layout generation module for the interested interface. <Moves >

The "one_row" move allows aligning a group of widgets exactly along one row.

The "in_rule" value of the "usage" parameter allows specifying that this move can be used only as action of a rule, namely it cannot be applied for itself.

The " two_cols " move allows arranging a group of widgets on two columns :

</Moves >

The <Rules> section allows specifying the rules available for the layout generation module for the present interface (under process GUI interface) .

<Rules>

The "One_row_l_t" rule allows verifying whether a group of widgets has an alternating pattern of labels and text fields (<Condition>) and, if this is the case, applies the above- defined "one_row" move aligning the group of widgets exactly along one row (<Action>) :

The "Two_cols_l_t " rule allows verifying whether a group of widgets has an alternating pattern of labels and text fields (<Condition>) and, if this is the case , applies the above- defined " two_cols " move arranging the group of widgets on two columns (<Action>) .

_<Rule name=Two_cols_l__t> <Condition name=l_t_2 pattern= (label. textField)n /> <Action move=Two_cols on=l_t_2/> </R le>

</Rules>

The <Potential> section allows specifying the potential components and the global potential used by the layout generation module to apply the optimisation process to the present interface.

The "VScroll" potential component allows measuring the amount of vertical scrolling necessary for a user of the graphic interface to see all elements of the interface of a given layout .

The "GE" value of the "measure" parameter allows specifying, for a given layout, that the value of such component is equal to the number of elements of the interface outside of the vertical border of the visible display area:

<Component name=VScroll> <Scrolling axis=vertical measure=GE </Component> The "HScroll " potential component allows measuring the amount of horizontal scrolling necessary for a user of the graphic interface to see all elements of the interface for a given layout .

The "GE" value of the "measure " parameter allows specifying, for a given layout , that the value of such component is equal to the number of elements of the interface outside of the horizontal border of the visible display area :

The "Global_potential " potential is defined as the weighed sum of "HScroll " and "Vscroll " components .

The "minimize" value of the " task" parameter allows specifying that the layout generation module will search layouts that minimise such potential , namely the lower the global potential value related to a given layout , the better the layout itself will be considered .

In the present example , it is decided to assign more importance to the amount of horizontal scrolling, so the related weight will be greater than the vertical scrolling component weight : <Global name=Global_jpotential compos ition=weighted_sum task=minimize> <Part comp=HScroll weight=3 /> <Part comp=VScroll weight=l /> </Global>

</Potential> </GUI>

When actuating the method of the invention, the layout generation module 10 is invoked by the application that contains it or to which it is associated every time it has to display data on a certain display device.

The module 10 collects the following input data, respectively from one or more above-described databases 12, 14, 16, 18, such as, for example: - a list of graphical elements adapted to compose the graphic user interface; - information about the specific graphic user interface to be displayed, such as for example the specific relationships between the elements of the interface; - information related to all application interfaces that can be displayed; and - information related to display area and widgets available on the display device for which the graphic user interface must be generated.

These input data define, the transforming capability of layouts available to the layout generation module 10, which uses them for performing the method of the invention and providing the graphic presentation module 20 with the optimum layout for GUI .

With reference to figure 2 , the method comprises the operations performed by the module 10 and described herein below.

In step 100 input data are acquired, that are comprised, according to the present embodiment, of a determined set of information.

Such information comprise, for example, the list of graphical elements, the definition of the constraints, of the moves, of the rules and potentials, and the definition of the additional constraints for the graphical elements imposed by the display device.

Then, in step 200 an initial layout L(0) is generated, compatible with the above-stated constraints.

Afterwards, in step 300, the layout optimisation procedure is started by building an initial set of sequences of actions, containing a single empty sequence of actions and an initial empty set of fitness values. In the context of the present description, a sequence of actions is a sequence of moves, exploring rules and mandatory rules .

The execution, on a layout, of moves and rules (exploring and/or mandatory) in the order defined in a sequence, transforms the layout into another layout.

Step 400 performs the transformation of the set of sequences of actions , according to at least one of the following operations : - adding new empty sequences to the set ; - deleting existing sequences from the set; or - modifying existing sequences.

In particular, the modification of an existing sequence of actions can be obtained through one of the following operations.

i) Inserting moves in the sequence Sj . Moves can be inserted only: if the resulting sequence, applied to the initial layout L(0), creates a layout compatible with constraints; if all moves in the sequence Sj after the insertion operation can still be applied after insertion (for example, after insertion of a paging move that implies the division of the display area into two pages, a possible alignment move performed on widgets in different pages would become unable to be applied) ; ii) Deleting moves from the sequence Sj

Moves can be deleted only: if the resulting sequence of actions, applied to the initial layout L(0), generates a layout compatible with the constraints; if all moves in the sequence Sj after the deletion operation can still be applied.

iii) Inserting exploring rules in the sequence Sj

Exploring rules can be inserted only if: the resulting sequence of actions, applied to the initial layout L(0), generates a layout compatible with the constraints; if all moves in the sequence Sj after the insertion operation can still be applied; if the sub-sequence defined from the beginning of the sequence to the insertion point, applied to initial layout

L(0) , generates a layout in which the set of conditions of the inserted rule is valid (in other words, as can be expected, an exploring rule can be added only in a point in which its own pre-requisites are satisfied) ; if the set of conditions of all exploring rules in the sequence Sj after the insertion operation are still valid after the insertion (otherwise some of the above rules would become unable to be applied) .

iv) Deleting exploring rules from the sequence Sj

Exploring rules can be deleted only if : the resulting sequence of actions, applied to initial layout L(0), generates a layout compatible with constraints; if all moves in the sequence Sj after the deletion operation can still be applied after deletion; if the sets of conditions of all exploring rules in the sequence Sj after the deletion operation are still valid after the deletion operation (otherwise some of the above rules would be unable to be applied) .

v) Every time a sub-sequence of a sequence Sj , applied to initial layout L(0), generates a layout in which the set of conditions of a mandatory rule is satisfied, such mandatory rule must be inserted in the sequence Sj after such subsequence .

Any convenient strategy can be performed in the actual creation, deletion and modification of the sequence of actions, starting from ad hoc strategies up to strategies that are commonly used in classical optimisation schemes.

In order to provide an example of this latter case, a strategy of the evolutionary type would modify the sequence of actions with casual insertion and deletion of elements and would create new sequences of actions by duplication and crossover of existing sequences. Alternatively, a "hill-climbing" search strategy (limited to the rules) would start with the single empty sequence of actions and would systematically create all sequences of actions that can be obtained by adding a different applicable rule.

After performing the transformation of the set of sequences of actions, step 500 applies the global potential function to all layouts Lj obtained by applying the sequence of actions Sj to the initial layout L(0), to obtain thereby for each layout Lj (and for the corresponding sequence of actions) , a fitness measure Gj = P(Lj) .

All values Gj obtained are added to the set of fitness values.

Then, in step 600, the number b of the best sequences of actions in the set of _. sequences of actions and the corresponding best b fitness values in the set of fitness values are selected and kept, discarding the other sequences and the other values .

The b value can be chosen by following any convenient strategy. Referring to the previously proposed examples of strategy, the "hill -climbing" search strategy would choose b=l (a single sequence of actions is selected for each iteration) , while the evolutionary strategy would choose b equal to the current size of the set of sequences of actions.

Finally, in step 700, a check is made about the satisfaction of the optimisation conclusion criterion.

Examples of conclusion criteria are: one of values Gj is lower than a predefined threshold; the difference in absolute value between the best Gj of the current iteration and the best Gj of the previous iteration is lower than a predefined threshold or within a predefined interval; a predefined maximum number of iterations has been performed; a predefined interval of time has elapsed.

The conclusion criterion can be determined both by the software application and by the module 10.

If the conclusion criterion is not satisfied, the optimisation cycle is continued returning to step 400. If one of the conclusion criteria is satisfied, the best sequence of actions is applied to the initial layout L(0) and the resulting layout is considered as the final result of the described method.

An embodiment based on the previously-described method is included herein below.

The example deals with a data entry form on a mobile device; in particular, the Graphic User Interface or GUI must be displayed on the visible display area of the mobile device shown in figure 3a and that can be made horizontally and vertically slide on a wider display area.

In the example, modules and databases involved in the layout generation (figure 1) are partly residing on the mobile device and partly residing on a remote server, for example of the HTTP type, associated with a radio-mobile device. More precisely, on the mobile devicethere are the layout presentation module 20 and the information database on the display area 18 representing the device characteristics, while on the remote server there are the layout generation module 10 and, as XML documents as previously described, the information databases about graphical elements 12, 14 and 16 and the relationships between them.

Mobile device and remote server mutually communicate through the radio-mobile communication network.

In the example, the mobile application user requests the display of the data entry form on the mobile terminal . The mobile application receives [extracts] from the database 18 all information related to the display area and sends to the remote server, through the communication network, a http protocol message that contains the display request of the data entry form and information received (extracted) from the database 18.

The remote server receives the http protocol message and provides the layout generation module 10 with received information related to the display area. The layout generation module 10 receives from databases 12, 14 and 16, as XML documents as previously described, the list of graphical elements related to- the data entry form requested by the user and information about relationships between graphical elements .

In the example, the layout generation module 10 uses a rather simple optimisation strategy, trying to apply all available rules that can be applied (namely the two rules defined in the above recalled XML file) , one at a time, and choosing the rule that determines the layout with the smallest global potential value.

The layout generation module takes first into account the initial layout in figure 3b.

For such initial layout, the potential component "HScroll" is equal to 0 (the number of elements of the interface outside the horizontal border of this configuration is in fact equal to 0), while the potential component "VScroll" value is equal to 3 (the number of elements outside the vertical border is in fact equal to 3) .

The global potential is thereby equal to: 0*3 + 3*1 = 3.

The layout generation module tries then to apply the "One_row_l_t " rule to the initial layout.

There is a group of widgets ("Label1 textFieldl ... Label4 textField4") that corresponds to the pattern "(label textField)n" in the rule condition portion, and therefore this rule can be applied, reaching the presentation shown in figure 3c.

After the application of the present rule, the potential component "HScroll" is equal to 5 (the number of elements of the interface outside the horizontal border of this configuration being in fact equal to 5) , while the potential component "VScroll" value is equal to 0 (the number of elements of the interface outside the vertical border being in fact equal to 0) .

The global potential is thereby equal to: 5*3 + 0*1 = 15.

The layout generation module then tries to apply the "Two_cols_l_t" rule to the initial layout.

There is a group of widgets ("Label1 textFieldl ... Label4 textField4") that corresponds to the pattern "(label textField) n" in the rule condition portion, so that this rule can be applied, reaching the presentation shown in figure 3d.

After having applied this rule, both potential components

"HScroll" and "VScroll" are equal to 0 (the number of elements of the interface outside horizontal and vertical borders for the present configuration is in fact equal to 0) .

The global potential is thereby equal to: 0*3 + 0*1 = 0.

The obtained layout has the lowest global potential value (value 0) . Therefore, this layout will be selected as the optimum one.

The optimum layout is then sent through the radio-mobile communication network from the layout generation module 10 on the remote server to the layout presentation module 20 on the mobile device, which takes care of displaying the data entry form on the mobile device display.

Advantageously, the method of the invention can be extended to the creation and optimisation of layouts for three- dimensional user graphical interfaces. This is important, for example, for devices of the virtual reality type or holographic display devices.

In fact, though the complexity and the computation burden added by the third dimension must be taken into account, the essential characteristics of the above-discussed method can also be applied in a 3D environment, with the following differences. The 3D display areas will replace the 2D display areas, the elements of the interface will have a three- dimensional shape, moves and rules will manage layout transformations in the 3D space, subjected to three- dimensional constraints, and the optimisation process will be applied, similarly to what has already been described, following the definition of a potential in the three- dimensional space.

The method can further be extended to the creation and optimisation of layouts for graphical displays different from graphical user interfaces, such as for example document paging, technical diagrams, etc., in addition to the management of some not merely geometric characteristics related to graphic user interfaces layouts, such as for example text formatting, text font setting, line tracing styles, etc.

The layout generation system for a graphical interface has been described as a set of programs, but could be realised also through any physical computational device, such as for example an integrated circuit that performs the previously- described logic functions for the system.

Obviously, the principle of the invention remaining valid, the embodiments and the realisation parts can be widely changed with respect to what has been described and shown merely as a non-limiting example, without departing from the scope of the present invention, as defined in the enclosed claims .

Claims

1. Method for dynamically generating a layout for a Graphic User Interface or GUI starting from an application aimed to be shown in a destination display context, characterised in that it comprises the steps of: arranging a collection (12) of graphical elements to be shown, including a representation of said graphical elements independently from the destination display context; acquiring (100) said graphical elements and information (14, 16, 18) indicating the display context; generating (200) an initial layout - configuration depending on said graphical elements and said information; transforming (300, 400, 500, 600, 700, 800) the initial layout configuration into alternative configurations; and computing, through a potential function, respective fitness degrees to be associated to said alternative configurations .

2. Method according to claim 1, wherein the step of computing said respective fitness degrees comprises the steps of: evaluating (500) by means of the potential function, the alternative configurations by computing respective score values of the alternative configurations; and selecting (600, 700, 800) and keeping a set of alternative configurations whose score values satisfy predetermined fitness criteria.

3. Method according to claim 1 or 2 , characterised by the further step of ending the process of dynamically generating the layout when at least one of the following conditions is verified the potential function has assumed for a certain alternative configuration a determined value suitable to comply with a predefined threshold; - the difference between values assumed by the potential function in following iterations assumes, for a set of alternative configurations, values within a predefined interval ; - a determined threshold of processing times or number of iterations has been reached.

4. Method according to claim 3 , wherein said step of ending the process comprises the definition of a related display area and a set of graphical elements wherein to each graphical element it is associated a set of values that specify the position and the geometric parameters within the display area of each graphical element .

5. Method according to claim 1 or 2 , wherein the step of transforming (300, 400, 500, 600, 700, 800) the initial configuration into alternative configurations comprises a sequence of : moves, namely actions adapted to modify the spatial configuration of a group of graphical elements; and rules, namely moves adapted to be applied if a predetermined set of conditions is satisfied by a group of graphical elements .

6. Method according to claim 5, wherein the moves include geometric and/or topologic transformations.

7. Method according to claim 6, wherein said moves include at least one among: - position modifying moves, adapted to change the position of a graphical element on the display area or the relative position of one or more graphical elements; - size changing moves, adapted to modify the size of one or more graphical elements; - aligning moves, adapted to modify the position and/or the size of one or more graphical elements in such a way asto align them; - pagination moves, adapted to divide the display area into smaller display areas and to assign the graphical elements to each one of them; - replacing moves, adapted to replace one or more graphical elements with a different group of graphical elements .

8. Method according to claim 5, wherein the moves include the execution of a pre-established method for generating a pre-existing layout.

9. Method according to claim 5, wherein said rules include: - mandatory rules, according to which the related move must be executed if said set of conditions is satisfied; - exploring rules, according to which the related move can be executed if said set of conditions is satisfied.

10. Method according to claim 5, wherein said alternative configurations can be obtained starting from a sequence of initial moves and rules and through one of the following operations : - adding new moves and rules ; - deleting existing moves and rules; or - modifying existing moves and rules.

11. Method according to any one of the previous claims, wherein said information indicating the display context include information (18) about the display area.

5 12. Method according to any one of the previous claims, wherein said information indicating the display context include information (14, 16) about relationships among the graphical elements to be displayed.

10 13. Method according to claim 11, wherein the display area is defined by at least one of the following characteristics: - the geometric shape; - a set of geometric dimensional parameters describing the geometric shape;

15 - the visible display area; - a structure of pages .

14. Method according to any one of the previous claims, wherein said collection (12) of graphical elements includes a

20 representation of parameters and attributes of each graphical element .

15. Method according to claim 14, wherein said parameters and attributes comprise at least one among the following

25 characteristics: - the geometric shape of the graphical element; - a set of geometric dimensional parameters describing the geometric shape; - coordinates indicating the position of the graphical 30 element on the display area; - a set of typologic attributes.

16. Method according to claim 15, wherein said graphical elements are subjected to constraints that limit their possible positions and sizes on the display area and with respect to other elements .

5 17. Method according to claim 16, wherein said constraints include at least one among: - ordering constraints, aimed to determine a space ordering of the graphical elements on the display area; - adjacency constraints, aimed to constrain two or more 10 graphical elements to be shown one adjacent to the other; - size constraints, aimed to impose limits onto the size of a graphical element; - overlap constraints, aimed to impose a policy of partial overlapping to the graphical elements;

15 - repetition constraints, aimed to impose the repetition of a structure in the layout of a group of graphical elements ; - containment constraints, aimted to impose the inclusion of some graphical elements inside other graphical

20 elements.

18. Method according to claim 1 or 2 , wherein said potential function includes a plurality of components, each component being responsible for the assignment of a score to

25 predetermined layout characteristics.

19. Method according to claim 18, wherein said potential function includes a plurality of weighed components.

30 20. Method according to claim 18 or 19, wherein said potential function components include: - a potential with a preferred size, aimed to assign a vote related to the size of a graphical element; - an adjacency potential, aimed to assign a vote related to the vicinity between graphical elements; - a magnetisation potential, aimed to assign a vote related to the alignment of groups of graphical elements; - a compactness potential, aimed to assign a vote related to the compact arrangement of graphical elements; - a space filling potential, aimed to assign a vote related to the arrangement of graphical elements within the display area; - a sliding potential, aimed to assign a vote related to the relationship between visible display area and total display area.

21. System for dynamically generating a layout for a Graphic User Interface or GUI starting from an application, comprising a processing layout generation module (10) arranged

- for acquiring: a collection (12) of graphical elements; and information (14, 16, 18) indicating a display context, and

- for performing a method according to claims 1 to 20 in order to generate a final layout configuration suitable for the display context on which the graphical elements must be shown.

22. Set of modules adapted to create a layout for a Graphic User Interface or GUI, comprising: at least one module aimed to acquire (100) a collection (12) of graphical elements to be shown in a destination display context, including a representation of said graphical elements independent from the destination display context; at least one module aimed to acquire (100) said graphical elements and information (14, 16, 18) indicating the destination display context; at least one module aimed to generate (200) an initial layout configuration depending on said graphical elements and said information; at least one module aimed to transform (300, 400, 500, 600, 700, 800) the initial configuration into alternative configurations; and at least one module aimed to compute, through a potential function, respective fitness degrees to be associated with said alternative configurations.

23. Set of modules according to claim 22, wherein said module aimed to acquire (100) information (14, 16, 18) indicating the display context is aimed to acquire information about relationships among graphical elements to be displayed (14, 16) and information indicating the display area (18) .

24. Set of modules according to claim 23, wherein said display area is defined by at least one of the following characteristics : - the geometric shape; - a set of geometric dimensional parameters describing the geometric shape; - the visible display area; - a structure of pages .

25. Set of modules according to claim 22, wherein said at least one module aimed to compute through a potential function respective fitness degrees to be associated with said alternative configurations comprises at least one sub-module aimed to evaluate (500) by means of the potential function the plurality of alternative configurations by computing respective score values of the alternative configurations; and at least one sub-module aimed to select (600, 700, 800) and keep a set of alternative configurations whose score values satisfy predetermined fitness criteria.

26. Set of modules according to any one of claims 22 to 25, wherein said at least one module (300, 400, 500, 600, 700, 800) aimed to transform the initial configuration into alternative configurations comprises sub-modules aimed, respectively: to operate moves, namely actions aimed to modify the space configuration of a group of graphical elements; and to apply rules, namely to operate moves depending on the fact that a predetermined set of conditions is satisfied by a group of graphical elements.

27. Set of modules according to claim 26, wherein the moves include geometric and/or topologic transformations.

28. Set of modules according to claim 27, wherein the moves that can be operated by said sub-module aimted to operate moves include at least one among: - position modifying moves, aimed to change the position of a graphical element on the display area or the relative position of one or more graphical elements; - size changing moves, aimed to modify the size of one or more graphical elements; - aligning moves, aimed to modify the position and/or the size of one or more graphical elements in such a way as to align them; - pagination moves, aimed to divide the display area into smaller display areas and to assign the graphical elements to each one of them; - replacing moves, aimed to replace one or more graphical elements with a different group of graphical elements . 5

29. Set of modules according to claim 26, wherein said sub- module aimed to operate moves includes a sub-module for generating a pre-existing layout.

10 30. Set of modules according to claim 26, wherein the rules that can be applied to said sub-module aimed to apply rules include : - mandatory rules, according to which the related move must be executed if said set of conditions is satisfied;

15 - exploring rules, according to which the related move can be executed if said set of conditions is satisfied.

31. Set of modules according to any one of claims 22 to 30, wherein said at least one module aimed to compute through a

20 potential function respective fitness degrees to be associated with said alternative configurations is aimed to compute said potential function depending on a plurality of components, each component being responsible for the assignment of a score to predetermined GUI layout

25 characteristics.

32. Communications network comprising at least one user terminal and at least one system for dynamically generating a layout for a Graphic User Interface or GUI according to claim

30 21.

33. Computer program product or group of computer program products loadable and executable into a memory of at least one computer comprising one or more code modules for actuating a method for dynamically generating a layout for a Graphic User Interface according to any one of claims 1 to 20.