WO2001086516A1

WO2001086516A1 - A decision support system

Info

Publication number: WO2001086516A1
Application number: PCT/DK2000/000229
Authority: WO
Inventors: Preben ALSTRØM
Original assignee: Stig Jørgensen & Partners
Priority date: 2000-05-05
Filing date: 2000-05-05
Publication date: 2001-11-15
Also published as: AU2000242866A1; EP1299828A1

Abstract

The invention relates to a method for determining a change in index relating to a state and a modified state and relates in a specific embodiment to a neural network based decision support system. The invention analyzes the consequences of a decision with respect to given measures of success, and compares and orders the considered decisions according to the gain in index, thereby unveiling the most profitable decisions. In addition, the invention may provide ordered estimates on which parameters among those considered being most important or significant with respect to the given measures of success. In a specific aspect a method for determining a change in index relating to a state and a modified state is provided. The state and modified state are each being a single state or a number of sub-states and the modified state is being a modification of said state. The method is utilizing at least one evaluation means and the method determines aggregation values, (v) (w), of the single states or of each sub-states and combines the aggregation values, (v) (w), thereby determining a change in index, (G), relating to the state and the modified state.

Description

A DECISION SUPPORT SYSTEM

The present invention relates to a method for determining a change in index relating to a state and a modified state and relates in a specific embodiment to a neural network based decision support system of the type stated in the claims and accompanying description.

BACKGROUND FOR THE INVENTION AND INTRODUCTION TO THE INVENTION

A decision support system aids the decision-maker in his decision. Usual decision support systems are build as pure statistical analyzers, providing only statistical information of the possible parameters that are taken into account before a decision is made. The invention described herein has as its purpose primarily to analyze the consequences of a (the) decision with respect to given measures of success (indices), and to compare and order the considered decisions according to the gain or in general a change in index, unveiling the most profitable decisions.

In addition, the invention can provide ordered estimates on which parameters among those considered that are most important or significant with respect to the given measures of success (indices).

Typically, a decision-maker have to rely on a mere intuitive method in order to figure out how his decision will influence, change - or modify, the evaluation that lies behind his decision.

A problem in connection with such intuitive based decision support systems is that no easy way of measuring the result of a decision is available. This problem is enhanced in situations wherein an optimized state of the system is looked for as this optimization will require evaluation of the consequences of many decisions.

BRIEF DESCRIPTION OF THE INVENTION

In a first aspect of the present invention these and many more problems has been solved by a method for determining a change in index relating to a state and a modified state, said state and modified state are each being a single state or a number of sub-states and the modified state is being a modification of said state, and said method is utilizing at least one evaluation means, the method comprising

determining aggregation values, (v) (w), of the single states or of each sub-states and ,

combining the aggregation values, (v) (w), thereby determining a change in index, (G), relating to the state and the modified state.

In the first aspect of the present invention the terms state and modified state have been introduced. A state may preferably be understood as being an expression of a present state of a system and the modified state may preferably be understood as an expression of a fictitious state of said system in the sense that the fictitious or modified state is a state used for elaborating on changes in the system.

Depending on the nature of the system in consideration a state might be a single state or a number of sub-states or a combination of those and a state of a system might be the conjunction of the sub-states of the system. Each of the sub-states comprised in the number of sub-states may preferably have the same or similar characteristic but different characteristic values.

In order to ease the understanding of this way of dealing with states only, this concept may be illustrated by the following example: a state of a system may for instance be obtained by a set of questionnaires. Such a set of questionnaires comprises a number of questionnaires wherein the answers to each questionnaire each represent a sub-state of the system when answered. In addition, a question not being asked also contributes to the sub-state, for instance if one question consequently is not answered it could be the result of a to private question. Accordingly, the set of answers to these questionnaires represents the state of the system and aggregation values, (v) (w), are determined for each questionnaire.

In other situations, sub-states may not easily - or may not at all - be discriminated from the state of the system and in this case the state of the system may very advantageously be construed as a single state. Furthermore, the method according to the present invention may - of course - be applied in the situation where only one sub-state of a system is considered and in this case the sub-state considered might be construed as a single state.

According to the broad aspect of the present invention aggregation values of the state (the single state or each of the sub-states) are being determined; each aggregation value may preferably be construed as being a measure, such as a number, for the corresponding state. An aggregation value may therefore preferably be considered as being an overall value representing a sub-state or a single state depending on the way the system is construed.

Determination of an aggregation value of a state is very relevant as a measure for the state is thereby provided. Such a measure renders it possible to compare different states which is particular relevant when aggregation values of a state and a modified state are determined because these values very advantageously may be applied to render the result of a modification of a state visible and interpretable.

As the method according to the present invention render the result of a modification of a state visible and interpretable, the method is especially applicable for assisting in decision-making systems, as a decision may advantageously be considered as a process comprising changing the state of a system.

Typically, when decisions are considered it is very advantageously to measure the total effect of a change in state of a system and this total effect is provided as a change in index relating to the state and the modified state. The change in index is provided according to the broad aspect of the invention by combining the aggregation values relating to the state and the modified state.

The term "index" is used so as to relate a specific modification of a state to the measure of the effect that modification causes, which is particular useful when more than one modification is considered in a decision system. (It should be noted that no actual linking between the modification and the effect necessarily has to be performed).

When more than one modification of the system is considered the method according to the broad aspect of the invention is applied repeatedly for providing a change in index relating to each of the state and the modified state. Thereby, a number of change in index are provided and as each of these changes in index represent the effect of the modification for instance the most profitable modification may be rendered visible (the most profitable modification may be determined based on a user defined criterion).

This most profitable modification may in turn be used as a basis for a decision in the sense that a decision may be made relating to changing the state of the system in such a manner that the modified state becomes the state of the system.

In cases where a number of sub-states are considered, the method according to the present invention provides an aggregation value for each sub-state and modified sub- state. The modification of each sub-state is preferably performed so that each sub-state is modified in a similar manner.

Typically, a state is represented by a number of conditions and in a preferred embodiment of the method a state is accordingly represented by state variable(s), (x), wherein each of the state variable(s) represents a discrete condition of the state. A discrete condition of a state might furthermore comprise a number of sub-conditions (such as answers to questions). When a condition is represented by a state variable a quantification of the condition has been rendered possible.

Typically, the aggregation values of states are being determined on the basis of the state variable(s). Preferably, the state variable(s) are inputted to the evaluation means and this means evaluates, or these means evaluate, measure(s) of the state providing the basis for determining the aggregation values.

In a preferred embodiments of the method determination of the aggregation values comprises determination of value variables being measures of a state, (z), by use of the evaluating means, said value variables being a measure of a state and may preferably serve as the basis on which aggregation value(s) is/are to be determined.

These value variables, (z), may preferably be answers to specific questions. In a typical situation, the states are answers to questionnaires and depending on what is to be obtained by using the method a selection of answers being state variables, (x), and answers being value variables, (z), is performed. In practical use of the method according to the present invention the system to be examined with respect to determining aggregation values may be defined by a number of variables which are to be classified as being state variables or value variable. Classification of the variables into the two different classes may preferably be performed based on a selecting criteria set up by the user of the method. Typically the user of the method might want to investigate the effect on one or more of the variables caused by changing one or more of the remaining variables. In this case the variables being the ones to be examined for changes are classified as value variables and the remaining variables are classified as state variables.

This classification may be viewed upon as a step of extracting a share of a state, being used as a state in the present method, and defining the remaining part of that state to be the value of that state. This value or these values is/are then to be determined by the method according to the present invention.

Accordingly, said value variables being determined by the evaluation means are preferably determined on the basis of the state variables.

Typically, the determined value variables comprise more than one number and therefore determination of the aggregate value of a state comprises in a preferred embodiment of the method, assessment of value variables being determined by the evaluation means on the basis of the state variables, said assessment may preferably provide a single number for the aggregate value of a state.

In a preferred embodiment of the method according to the present invention, determination of the aggregate values of the state and the modification thereof comprises executing a value-function (h) on the value variables corresponding respectively to the single state and the modification thereof or - to each of the sub-states and the modifications thereof thereby providing the aggregate values of the state and modification thereof.

The value function is preferably a function which as input takes a number of values, preferably the value variables, and provides as output a single number being the aggregate value corresponding to the input. The value function may, accordingly, comprise a set of sub-functions, in which case each sub-function is adapted to take as input a certain set of value variables.

In a preferred embodiment of the method according to the present invention the modified state is provided by executing a modification-function (f) on the state, the modification- function being adapted to modifying the single state into a modified single state or being adapted to modifying each of the sub-states into modified sub-states.

The type of the state in consideration is preferably reflected in the type of modification function, in the sense that in case a single state is considered to be modified the modification function may be adapted to modify that state and in case a set of sub-states is considered the modification function may be adapted to modify each of the sub-state. The later case may preferably be provided by a set of sub-modification functions constituting the modification functions.

Preferably, the modification-function is mapping at least one of the state variable(s) into a constant, C. In case the state is a number of sub-states the modification function, preferably being a set of sub-modification functions, maps at least one of the state variables of each of the sub-states into a constant, C. Preferably, the modification-function is either pre-defined, such as coded during the implementation of the method, or externally supplied, such as being supplied by the user of the method.

In a preferred embodiment of the method according to the present invention the combining of the aggregation values is based on integration of the aggregation values, which in a specific preferred embodiment is a weighted integration. The term integration may in some cases be construed in a strict mathematically sense, in which case integration comprises a discretization of the integration operator. In general the term integration may preferably be construed to denote a mathematical operation, such as an operation comprising summation, providing as output a number.

In embodiments of the method in which input to the evaluation means is required in a binary form, the state variables are preferably discretized and each element of the discretization is represented by two states, such as 1 or 0, and these variables are accordingly inputted to the evaluation means. Furthermore, the evaluation means may preferably output(s) discretized value variables. In a preferred embodiment of the method, the method utilises one evaluation means evaluating the value variables of a state and a modification thereof. Alternatively, the method may utilise more than one evaluating means, such as at least one for evaluating the value of a state and such as at least one for evaluating the value variables of the modified state.

Preferably the evaluation means is/are of a neural network type, said network(s) preferably is/are a back-propagation trained fully connected feed forward network having an input layer, an output layer and one hidden layers.

In another aspect, the present invention relates to a system for determining a change in index, (G), relating to a state and a modified state, said state and modified state are each being a single state or a number of sub-states and the modified state is being a modification of said state, said system comprises at least one evaluation means, the system determines by use of the at least one evaluation means aggregation values, (v) (w), of the single states or of the sub-states and combines, the aggregation values, (v) (w), thereby determining a change in index, (G), relating to the state and the modified state.

Preferably the system is adapted to perform any of the steps comprised in the method according to the present invention in which case the system is typically adapted by comprising/utilizing means, such as software means, designed to perform the operations comprised in the steps.

BRIEF DESCRIPTION OF THE DRAWINGS

In the following the invention, and in particular preferred embodiments thereof, will now be described in greater details in connection with the accompanying drawings in which:

Fig. 0 shows, schematically, a fully connected feed-forward neural network with one hidden layer, characterized by activation function and synaptic weights, Fig. 1 shows, schematically, determination of an aggregation value corresponding to a state and determination of an aggregation value corresponding to a modified state, said modified state being a modification of the state provided by a modification function, f,

Fig. 2 shows, schematically, determination of aggregation values corresponding to a number of sub-states and determination of aggregation values corresponding to a number of modified sub-states; said modifications being provided by a set of modification functions. Fig. 2 also shows, schematically, determination of a change in index relating to the modifications, said determination being performed by an integration of the aggregation values,

Fig. 3 shows, schematically, the processes involved in the determination of aggregation values of Fig. 2,

Fig. 4 shows statistical information, histograms of overall distribution, related to an example of utilizing the method according to the present invention on fictitious data,

Fig. 5 shows statistical information, distribution breakdown, of the example according to fig. 4,

Fig. 6 shows statistical information, casual readers only, of the example according to fig.

4,

Fig. 7 shows statistical information, casual reader breakdown, of the example according to fig. 4,

Fig. 8 gives the figures of the statistical information presented graphically in fig.s 4-7,

and Fig. 9 shows the questionnaire relating to the example according to fig. 4. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

Neural networks are used for data processing purposes on the basis of a plurality of complexly related input parameters to give the best possible response thereto without necessarily knowing the relation between the individual input parameters. This is extremely advantageous when no such, linear, relation exists.

The starting point of neural networks is the ability of the human brain to identify the most important parameters in a decision-making process and to draw correct conclusions by experience. Neural networks are therefore constructed according to the same basic principles as the human brain, comprising a multitude a decision making cells or neurons as well as connections or synapses between these.

In order to make the best possible decisions artificial neural networks therefore go througha comprehensive learning procedure before they are used in practice, and the experience thus acquired is utilized for adjusting the control parameters for the neurons and the synapses.

As far as the neurons are concerned, these control parameters comprise a threshold value and describe an activation function, which determines the output from a neuron as a function of the input received from other neurons. The output from a neuron is transferred via one or more synapses to other neurons. A synapse reduces or enhances the output by a factor called the weight of the synapse. The weight of a synapse is one of the adjustable control parameters in the network.

In the present context, the neural network could typically be a fully connected feedforward neural network (FIG. 0) with an input layer, an output layer, and one or two hidden layers, and neurons with sigmoid activation functions (output = (1 + e^~2 ""^pu' )^~1 ). The network is typically back-propagation trained The rate of change of the weights Wy of the synapse from neuron j to neuron /^', \N_ή (t + 1) = W_v (t) + ΔW_tj (t) + aά\N_sj (t - 1) , where

AW,; = η error, output , , and error _} = T output ,-(1 - output ,)l l _? error, except at the

/ neural network output layer, where error, = target, - output, and target, is the desired output. The desired output is user-provided for the examples constituting the training basis for the neural network. The weights may initially be set to random small values close to zero or to previous attained values if desirable. The training of the neural network stops when the output error is sufficiently small (for instance less than 1%), or the training has gone through a maximum number (for example 500) of training cycles (each say 10 times through the example base).

In the invention a classification-type neural network (FIG. 1 , for instance . of the type above) is applied to generate an input-output function g : (x x₂,...,x_m) -> (z z₂,...,z_n) from a set of user-provided paired examples of input x^(,) = (x₁ ⁽'⁾,x₂ ⁽'⁾,...,x_n ⁽'⁾) and output z^{( )} = (zf' z₂ ^{l),...,z '⁾) , /^' = 1,2,..., N . The variables (x₁,x_{2 >}...,x_n ) are in the invention called the state variables, and the variables (z₁,z₂,...,z_m) are in the invention called the value variables. The map g (input-output function) is expressed in terms of the control parameters of the neural network, including the weights of the neural network. The numerical values of the weights provide useful information about correlations between state variables and value variables.

The state and value variables could be available as system data or from questionnaires. A state variable describes a well-defined system parameter that is influenced by the state. It could be the temperature, the pressure, or the concentration of a chemical, or it could be the number of employees or the sales price of a product, or it could be a measure of how much a given service is used. A state variable is an object for decision - a variable that the user of the invention believes can be changed by a well-described decision. In industrial reactors, the chemical conditions as temperature, pressure, and concentrations can be changed, in business the number of employees as well as the sales price of a product can be reduced or increased, or a given service can be improved or removed.

The value variables characterize the goals for the decision. A value variable could be a measure for optimal production, a measure for customer satisfaction, or a measure for knowledge value or innovation depth. It could also be plain economic values or a measure for public importance (energy, environment, etc.).

In a typical situation, each variable x_y [j = \2,...,n ] has a binary form [S ,a_2J,...,a_kU)J ] of y^'-dependent length k(j) , and with all a_/7 being 0 or 1 , and each variable z_y [j = 1,2,..., m ] has a binary form [by,b_2j,...,b_l(j)j ] ofy-dependent length /(/) , and with all by being 0 or 1. (See FIG. 1. ) Typically, for a given j [j = 1,2,...,π ], only one a_j} equals 1 , and for eachy^' ty = \2,...,m ], only one b,_j equals 1.

To obtain a binary form, the variables are discretized. It could for example be in the form [very small, small, medium, large, very large], or [never, very little, little, sometimes, often, very often, always], or [very bad, bad, reasonable, good, very good], or [very negative, negative, indifferent, positive, very positive], or [know, don't know], or it could simply be a separation of numbers into intervals.

Accordingly, a discretization of a state, described by the variables x_} [j = 1,2], where Xi = quantity could be few or many (i.e. / (1) = 2), and x₂ = quality could be bad, good or very good (i.e. k(2) = 3), would have the following binary form x_ή = [a^,a₂^ ] = [a_few ,a_many ], and x₂ = [a₁₂,a₂₂,a₃₂ ] = [a_bgd,a_good ,a_verygood ] with each a being 0 or 1. For example x₁ = [1 ,0] and x₂ = [0,1,0] describe the state with quantity = few and quality = good.

When the values of the weights in the neural network have been determined (learned), the strengths S (importance, significance) of every path from a given state input site a ( for instance customer state feature) to a given output site b (for instance loyalty level) may be determined and used to identify features characterizing segments (for instance customer segments) associated with the different value variables (for instance loyalty levels). The strength S_ba from input site a to output site b is in the above mentioned

example of a neural network defined as S_bα

which for example may be sorted by size for every value output site b.

A modification is characterized by a transformation f : (x_vx₂,...,x_n ) -=► (y₁,y₂,--.,y_π ) that maps input (x₁ , x₂ , ... , x„ ) into another (y, , y ₂ , ... , y „ ) , see FIG. 1. The user will provide a selected set of transformations f ...,f_κ , corresponding to a selected set of possible modifications. However, in a standard form of the invention, a standard set of transformations will be automatically available, including the transformations f(x ...,X_j x„ ) = (x_v...,C_j,...,x_n) , where one state variable x_y is mapped into a constant c_y , while the rest remains unchanged. If, for example, the state variable x_} corresponds to the use of a service discretized in the form [never, little, sometimes, often, always], and the modification considered is the removal of the service, then the relevant C_j would have the form [1 ,0,0,0,0] corresponding to the 'never-state'.

The invention also uses a value function h : (z₁,z₂,...,z_m) -> v or a set of these (FIG. 1).

In the invention, v^U) = h(z^U)) = h(g(x ⁾)) [i = 1,2,...,/V] is called the value of the input x^(/) , and a reasonable weighted integration / V⁽¹⁾,v⁽²⁾,...,v^(W) ) defines an index (FIG. 2).

The user will provide the value function h as well as the index function /. In a standard form of the invention, a standard set of value functions will be automatically available, including the value functions b(z_Λ z_; , ... , z_m ) = (b_υ + 2b_2j + 3b_3J + ... + l(j)b_IU)J ) I l(j) , which only depend on the value variable Z_j . A first choice of index function is

The purpose of the invention is to give the change G in index / under a modification f,

G = l(w^m ,w⁽² ...,w ^N)) -l(v ⁾ ,v^(z ...,v^m) , where w^{!) = h(z'^{i) ) = h(g(y^U))) is the value of the transformed input y ^{( )} = f (x^{( )} ) , see FIG. 2. For a set of transformations f ,..., f_κ , the gains G₁ , ... , G_κ obtained are sorted after size, in order to unveil the most profitable modifications (FIG. 3). In the case of more than one value function or index, a sorted gain list is produced for each index.

EXAMPLES SHOWING USE OF THE METHOD

When the functions g and h have been defined, every state is characterised by its value. The variation of the value in state space is called the value landscape.

As a specific example of the invention, consider a bank that wishes to improve their advising to their business customers that in turn wishes to improve their businesses, more specifically, say their consolidation. In this case, the state variables (x) are the descriptors of the customer profile (the state), for instance such as age, sex, place of residence and flexibility (see below) and the value variables (z) are various economic data for the customer, for instance such as income, fortune and assets, the value (v) being the consolidation - a well-defined quantity obtained from the economic data. An example- based neural-network trained function (g) associates every customer profile with a given set of economic data, and from these the consolidation is calculated (function h). The neural network training has formed the consolidation landscape, i.e. the consolidation variation with customer profile.

To identify features characterising customer segments associated with the various consolidation variables, the strength (S) from a given state input to a given consolidation output may be determined and sorted by size.

A simple business customer index (/) for the bank, measuring its performance, may be defined as the average consolidation over all their business customers. Assume that one descriptor (state variable) is how the business customer finds the banks flexibility when he experiences a temporary crisis due to lack of available funds. Is the flexibility considered high, medium, or low. If the bank contemplates measures to increase their flexibility, they should consider the modification (/) of the customer profiles to modified profiles where all business customers are assumed to find the flexibility high (nothing else is changed). The gain (G) now determines the growth in index (average consolidation) under the above modification. In other words, the invention through the above consolidation analysis gives the bank manager an idea of the expected gain in average consolidation if he decides to improve flexibility.

As another example of the invention, consider a newspaper that wishes to improve their sale by improving the customer loyalty, or their working environment by improving the employee loyalty. In this case, the state variables (x) are the descriptors of the customer profile or employee profile (the state), obtained for example from questionnaires. The profiles have information for instance on age, sex, place of residence. The customer profile will also contain information of how the customer judges the various sections of the newspaper, how available they are and so on. The employee profile will contain information on how the employee judges the working environment, how his development possibilities are and so on. The loyalty (see below) and the value variables (z) are various loyalty measures for the customer or employee. In the customer case the loyalty measures are for instance repurchase, satisfaction, or recommendation-level. The value (v) is the loyalty - a well-defined quantity obtained from the loyalty measures. An example-based neural-network trained function (g) associates every customer profile (employee profile) with a given set of loyalty data, and from these the loyalty is calculated (function /?). The neural network training has formed the loyalty landscape, i.e. the variation in loyalty with customer profile (employee profile).

Again the strengths (S) from every state input to loyalty output can be determined and used to help identifying features characterizing customer or employee segments.

A simple loyalty index (/) for the newspaper may be defined as the average loyalty over all their customers (employees). Assume that one descriptor (state variable) is how the customer judges one particular section, e.g. the IT section. Is the quality considered high, medium, or low. If the newspaper contemplates measures to improve the IT section, they should consider the modification (/) of the customer profiles to modified profiles where all customers are assumed to find the quality of that IT section high (nothing else is changed). The gain (G) now determines the growth in loyalty index under the above modification. In other words, the invention gives the newspaper editor an idea of the expected gain in loyalty index if he decides to improve the IT section. Considering all section, and related modifications, the invention through the above loyalty analysis supports the editor in finding which section that is most profitable to improve in term of loyalty.

A Loyalty Simulator Example - A Fictitious Example

This, fictitious, example relates to a newspaper that wishes to increase customer loyalty (repurchase). The newspaper therefore provides its readers with a questionnaire, as shown in Fig. 9, containing a loyalty question regarding if they read the paper regularly or only casually and questions regarding whether or not they are satisfied with the sections of which the paper contains 8 referred to as section 1 to section 8.

A total of 1287 answers is received and a table (data file) is made where the answer frequently and the answer occasionally are listed as 2 and 1 , respectively, and the answer dissatisfied and the answer satisfied are listed as 1 and 2, respectively.

The paper now uses a statistic tool in order to analyse the data. A histogram for each section is thus printed, shown in Fig. 4, which divides the customers into satisfied and dissatisfied, respectively. This simple analysis shows that most readers, i.e. 49 per cent, are dissatisfied with section 3. This figure is 10 per cent more than the readers dissatisfied with section 6. The paper therefore considers to make an effort to improve this section.

An employee goes one step further and compares the satisfaction statistics with how often a customer reads the paper, regularly and casually, respectively (a typical correlation study). The result of this study is shown in Fig. 5. The study clearly shows that the regular readers (i.e. a total of 589 readers or 45.8 per cent) are satisfied with all sections (please confirm with the figures in fig.8).

The paper now concentrates on the customers that only read the paper casually (i.e. a total of 698 readers or 54.2 per cent). Again histograms for each section are printed which divide the customers into satisfied and dissatisfied, respectively. The result is shown in Fig.7. The result is now more clear. 90 per cent of these readers are dissatisfied with section 3, 18 per cent more than the readers dissatisfied with section 6. Trusting the result of the statistics software, the paper implement an effort to improve section 3.

New questionnaires are sent out after 6 months. The paper is frightened to discover that nothing has changed! Not a single paper more has been sold.

After having come to terms with the situation, the paper contacts the loyalty expert at X- Act who offers a Loyalty Simulator based on the method according to the present invention which takes into consideration all connections between the data. The simulator clearly shows that an improvement of section 3 alone has no effect on loyalty. The same goes for the other sections with the exception of section 6! It is worth noting that the statistic tool showed that section 6 was the section with which fewest was dissatisfied!

The Loyalty Simulator has thus revealed patterns in data that is a combination of various questions that the statistic tool cannot reveal. Naturally the paper immediately changes its strategy and transfers the effort to section 6. Six months later, sales have increased by approx. 8 per cent as predicted by the Loyalty Simulator (49.7 percent regular readers now / 45.8 per cent regular readers before (the figures for the casual readers are ignored). The paper is now the largest in the country. The explanation to the above-mentioned example is as mentioned before the connection between various questions. It is no use only to look at the connection between two questions. The elaboration of histograms and calculations of the connections (pair correlation) that relate the satisfied/dissatisfied of one section to the satisfied/dissatisfied of another section is no efficient tool. Besides, there are a total of 112 of such relations (28 of which concern section 6) which in itself seems impossible to survey! And even though the statistic tool is able to handle triple correlations between the sections (of which there are 448!) it is of no use.

The fact is that the special feature of section 6 is that there is a small group of 50 of the casual readers (type 1 readers) that are only dissatisfied with section 6 while the remaining 648 casual readers (type 2 readers) are dissatisfied with all sections but one.

Claims

1. A method for determining a change in index, (G), relating to a state and a modified state, said state and modified state are each being a single state or a number of sub- states and the modified state is being a modification of said state, said method is utilising at least one evaluation means, the method comprising

determining aggregation values, (v) (w), of the single states or of each sub-state and ,

2. A method according to claim 1 , wherein a state is represented by state variable(s), (x).

3. A method according to claim 1 or 2, wherein the aggregation values of states are being determined on the basis of the state variable(s).

4. A method according to any of the preceding claims, wherein determination of the aggregation values comprises determination of value variables, (z), by use of the evaluating means.

5. A method according to claim 4, wherein said value variables being determined by the evaluation means are determined on the basis of the state variables.

6. A method according to any of the claims 2-5, wherein determination of the aggregate value of a state comprises assessment of value variables being determined by the evaluation means on the basis of the state variables.

7. A method according to claim 6, wherein determination of the aggregate values of the state and the modification thereof comprises executing a value-function (h) on the value variables corresponding respectively to the single state and the modification thereof or - to each of the sub-states and the modifications thereof thereby providing the aggregate values of the state and modification thereof.

8. A method according to any of the claims 2-7, wherein the modified state is provided by executing a modification-function (f) on the state, the modification-function being adapted

5 to modify the single state into a modified single state or being adapted to modify each of the sub-states into modified sub-states.

9. A method according to claim 8, wherein the modification-function is mapping at least one of the state variable(s) into a constant, C.

10

10. A method according to claim 9, wherein the modification-function is either pre-defined or externally supplied.

11. A method according to claim 10, wherein combining the aggregation values is based 15 on integration of the aggregation values.

12. A method according to claim 11, wherein the integration is weighted integration.

13. A method according to any of the preceding claims, wherein the state variables are 20 discretized and each element of the discretization is represented by two states, such as 1 or O.

14. A method according to claim 13, wherein the discretized state variables are inputted to the evaluation means and wherein the evaluation means output(s) discretized value

25 variables.

15. A method according to any of the preceding claims utilising one evaluation means evaluating the value variables of a state and a modification thereof.

30 16. A method according to any of the claim 1-14, utilising more than one evaluating means, at least one for evaluating the value variables of a state and at least one for evaluating the value variables of the modified state.

17. A method according to any of the preceding claims, wherein the evaluation means 35 is/are of a neural network type.

18. A method according to claim 17, wherein the neural network is a back-propagation trained fully connected feed forward network having an input layer, an output layer and one hidden layers.

19. A system for determining a change in index, (G), relating to a state and a modified state, said state and modified state are each being a single state or a number of sub- states and the modified state is being a modification of said state, said system comprises at least one evaluation means, the system determines by use of the at least one evaluation means aggregation values, (v) (w), of the single states or of the sub-states and combines, the aggregation values, (v) (w), thereby determining a change in index, (G), relating to the state and the modified state.

20 A system according to claim 19, which system being adapted to perform any of the steps according to any of the claims 1-18.