5.2 Player Model Optimization with Evolutionary Computation
5.2.4 Optimization of Player Model Parameters
The optimization process produces sets of player model parameters from DE algo-rithms. One set of player model parameters specifies one player model. The player
model makes decisions in games according to its parameters. It generates all game commands and creates game actions. When these actions interact with the video game environment, a variety of game states emerge. At the end of the game, we select some game statistics and evaluate the fitness scores for the given set of player model parameters. DE algorithm uses sets of parameters with high fitness scores to reproduce new sets of parameters. We repeat the selection and reproduction pro-cedures until the set of optimal player model parameters, denoting the best game player, is found.
5.2.4.1 Optimization of FS Membership Function Parameters
During the inference process in FS decision making, an FS membership function maps an FS input variable and its corresponding DOM to a numerical value. The shape of a membership function and its DOM computation are controlled by the membership function parameters. With different sets of membership function pa-rameters, various numerical interpretations of the FS input can be achieved, result-ing to a variety of game decisions.
A large number of player model parameters belong to FS membership function parameters. The size of membership function parameters is proportional to the number of DOM states in an FS variable that the function represents. In addition, its size also depends on the shape of the membership function. Different types of function shapes require different numbers of the function parameters. For example, the parameter size of a simple triangular-shaped function is smaller than the size of a trapezoid-shaped.
5.2.4.2 Optimization of FS Multi-output Decision
To implement the multi-output optimization, DE algorithm assigns a weight for each multi-output decision. When calculating the weight of a rule with multiple outputs, we proportionally share the calculated rule weight according to the weight of each decision. We then accumulate the shared weights into the output weight of the corresponding decision. Finally, the output with the maximum accumulated weight is selected as the final decision. With our implementation of modular FS table feature, this final decision may point to another FS table for further reasoning computations or to a decision callback for action planning and game command generation.
The number of gene elements, required for multi-output optimization, is pro-portional to the number of FS rules, containing multiple outputs, and the number of output choices, provided in such rules. The number of encoding representations for multi-output optimization tends to be small when compared to the one of other types of representation. This is because only a few rules require this supplementary feature even though it is very handy in necessary cases.
In a representation of a gene element, we encode the multi-output decision with the focus on maintaining the unity weight distribution among multiple outputs. The number of gene elements we required is one degree of freedom less than the number of output choices in the rule. Assuming a rule with three possible decisions, we require two gene elements to encode their weights. The last weight is calculated from the first two values to satisfy the unity of their total weights. In the same example of the three optional decisions, suppose we have two encoding elements of 0.4 and 0.5, respectively. The weight of the first decision is straightly 0.4. The weight of the second decision is 0.5 of the rest, i.e. 0.5×(1.0−0.4), that equals to 0.3. To maintain the unity of total weights, the remaining weight, i.e. 1.0−(0.4 + 0.3) = 0.3, belongs to the last decision. Hence, the two encoding values of (0.4,0.5) are equivalent to (0.4,0.3,0.3) of three-output weights. The summation of the three weights is one, which conserves the unity weight distribution we focus on.
5.2.4.3 Optimization of Game Command Arguments
A command argument(s) is required in some game commands as supplementary data to enhance the functionality of the commanding action. An argument of a game command is normally in a numerical form, but some other forms, e.g. a character or a symbol, are also possible. There are times when it is difficult to indicate the command argument only with FS rules. In this case, we have to supply the command argument with some other methods such as a constant value, a random value, a computed value, or an optimized value.
In command argument optimization, we assign a value of a specific gene element for a specific command throughout the game. The optimized command argument is used in either one of the following ways.
• Preset command argument: We use the optimized value directly as a command argument. This is usually for a simple game command.
• Callback function argument: We use the optimized value as a function
param-eter of a decision callback. The callback function calculates an actual value for the command argument. This is an indirect use of the optimized value, usually for a more complex game command.
An example of our implementation in command argument optimization for Star Trek game is presented in subsection 5.2.5.
The number of gene elements, required for game command arguments, depends largely on the input commands to the game environment, as well as the design of the KB component in our player model. The game command generator module acquires prospect command arguments, then generates the actual game commands according to the given decision. The prospect command arguments come from several methods, depending on our KB design for each specific game. The possible methods are static values from the predefined constants or random number generator, adaptive values from decision callbacks or optimized gene elements, or the mixture of both. Hence, the gene number varies drastically among game commands and game developers.