5.2 Player Model Optimization with Evolutionary Computation
5.2.5 Example Implementation of Player Model Optimization in
param-eter of a decision callback. The callback function calculates an actual value for the command argument. This is an indirect use of the optimized value, usually for a more complex game command.
An example of our implementation in command argument optimization for Star Trek game is presented in subsection 5.2.5.
The number of gene elements, required for game command arguments, depends largely on the input commands to the game environment, as well as the design of the KB component in our player model. The game command generator module acquires prospect command arguments, then generates the actual game commands according to the given decision. The prospect command arguments come from several methods, depending on our KB design for each specific game. The possible methods are static values from the predefined constants or random number generator, adaptive values from decision callbacks or optimized gene elements, or the mixture of both. Hence, the gene number varies drastically among game commands and game developers.
5.2.5 Example Implementation of Player Model
Table 5.1: DE encoding representation for simple FS rules in Star Trek game. Three types of our player model parameters in type column consists of FS membership function parameters (MFn), multi-output decision (Outs), and game command ar-guments (Cmd), where Cmd1 denotes a preset command argument and Cmd2 de-notes a callback function argument.
gene min max type representation
g0 0.0 3000.0 MFn LOW membership of SHIELD ENERGY g1 0.0 3000.0 MFn HIGH membership of SHIELD ENERGY g2 0.0 3000.0 MFn LOW membership of ENERGY LEFT g3 0.0 3000.0 MFn HIGH membership of ENERGY LEFT g4 0.0 100.0 MFn LOW membership of TIME LEFT g5 0.0 100.0 MFn HIGH membership of TIME LEFT
g6 0.0 5.0 MFn LOW membership of UNKNOWN STARBASE g7 0.0 5.0 MFn HIGH membership of UNKNOWN STARBASE g8 0.0 8.0 MFn LOW membership of DISTANCE TO STARBASE g9 0.0 8.0 MFn HIGH membership of DISTANCE TO STARBASE g10 0.0 1.0 Outs Output move to show klingon()
from rule no.1 ATTACK table (see section B.3) g11 0.0 1.0 Outs Output fire torpedo()
from rule no.5 ATTACK table (see section B.3) g12 0.0 3000.0 Cmd1 argument of SHE command
g13 10.0 1000.0 Cmd1 argument of PHA command when fighting 1 Klingons g14 10.0 1000.0 Cmd1 argument of PHA command when fighting 2 Klingons g15 10.0 1000.0 Cmd1 argument of PHA command when fighting 3 Klingons g16 1.0 8.0 Cmd2 radius factor to calculate arguments of NAV command
when exploring the galaxy
g17 -1.0 1.0 Cmd2 distance weight to calculate arguments of NAV com-mand when exploring the galaxy
g18 -1.0 1.0 Cmd2 new exploration weight to calculate arguments of NAV command when exploring the galaxy
g19 -1.0 1.0 Cmd2 distance weight to calculate arguments of NAV com-mand when travel to fight the Klingon
g20 -1.0 1.0 Cmd2 new exploration weight to calculate arguments of NAV command when travel to fight the Klingon
We implement FS membership function with a triangular function as illustrated in Fig. 5.4 (a). Thus, each DOM state requires only one function parameter. Ac-cording to the simple FS rules in section B.1, all five FS input variables are two-state DOM: LOW and HIGH. Hence, we use ten gene elements in this part, i.e. two gene elements for each FS input variable.
Our encoding contains two gene elements for multi-output decision, the least number of gene elements among all three types of key parameters. This is because, out of 42 rules in 5 tables, only two rules in the ATTACK table implement this feature, both with two-output decision. As we need one degree of freedom less than the number of output options for the encoding of the feature, one gene element is used for each two-output rule. Hence, we use two gene elements in this part.
For game command arguments, there is no exact formula to compute the number of gene elements. It varies greatly due to our design of game command generator module. In our simple player model parameters, we use preset command arguments for SHE and PHA commands, and callback function arguments for NAV command.
For SHE command, the value ofg12 is issued as an expected shield energy. For PHA command, where the efficiency of the phaser attack depends mainly on the number of Klingons in the quadrant, we assign three gene elements for different numbers of Klingons. For example, when there are two Klingons in the quadrant, the player model will fire the phaser with an energy of g14. For NAV command, we categorize the navigating actions into two groups: NAV to explore the galaxy when we don’t know where the Klingons are and NAV to reach the Klingons when we know its location. To explore the galaxy, we calculate the best quadrant to travel to by three factors. Each factor is represented by each gene element. All three factors are: the radius of how far we would like to move (g16), the preference to explore the galaxy nearby or far away (g17, and how many unknown quadrants can be uncovered with the move (g18). Once we know the location(s) of the Klingon(s), we examine all their discovered locations. To find the best location to travel to, the last two factors also apply to the NAV command, except for the radius factor. Both factors are represented by g19 and g20.
5.2.5.2 Game Score Evaluation
Previously, we suggest a general guideline to design the game scores with Eq.(5.2) in section 5.2.2. Here, we show our implementation of the game scores for Star Trek
in Eq.(5.4).
game score= 0.60∗#destroyed enemies +0.30∗#f ound enemies
+0.10∗(win game? remaining time: 0.0)
(5.4)
When the game is over, the game scores for fitness evaluation are calculated as the weighted sum of three normalized game statistics. We clarify all three statistics terms denoting to the game scores as below:
1. The number of destroyed Klingons is selected to motivate destroying more Klingons, which eventually leads to winning a game. Due to its highest im-portance, we assign the largest weight to this term.
2. The number of Klingons discovered in the galaxy is chosen to encourage better exploration. We cannot win a game unless we find all Klingons. This is the second most important factor to win the game.
3. The remaining time after we destroy all Klingons is also a helpful measure. It is to distinguish the best winner from average winners. If two player models win the same game, we prefer the one spending less time. When a game become more difficult with more Klingons, the extra remaining time could be used to search for and destroy the added Klingons. This measure is less significant than the above two because it is not directly related to winning the game. Therefore, we assign the smallest weight to this term. Unlike the above two terms where there are no conditional statements, we apply this term only when the model wins the game. This bonus point is also an incentive to win the game.
All three statistical terms above must be normalized first before applying game score equation. This is to assure that the game scores are rational over varying game parameters.
