Experimental Settings

7.3.1 Experimental Procedure

Gugeek, a KJ-Method support groupware, is used for the experiment. The total number of par-ticipants in the experiment is 90. Parpar-ticipants consist of master and doctoral students who study

Figure 7.2: Team members perform the KJ-Chart and discuss the problem by using Chart Inter-action and Face-to-Face (FF) communication methods during an experiment

at the School of Knowledge Science and School of Information Science of Japan Advanced In-stitute of Science and Technology. All participants must have English testing scores equivalent to a TOEFL score of 550 or more. The experiment consists of ten sessions. In each session, nine participants are divided into three teams of three participants each.

All members in a team were seated at the same table and were asked to access the KJ Method creativity support groupware via their own portable computers (one computer per participant).

Before the experiment, every participant is informed of the following:

1. KJ Method’s purposes, procedures, and rules.

2. Three communication methods and Gugeek instructions.

3. Examples and Q&A.

For each session, every team performed the group KJ method on the same set of three problems with different problem types. The problem types are listed as follows:

1. Artifact Synthesis/Unusual used (AS): “Imagine the new kitchenware used in the fu-ture” and “Apart from avoiding rain, write down any other uses of an umbrella”.

2. Problem Solving (PS):“How can we solve political corruptions?”.

3. Knowledge Gathering on a Topic (KG):Members contribute their knowledge of a given topic and help each other to construct a KJ chart representing the team’s knowledge of that topic, for example, “Fast foods”.

Three team communication methods were used as described in Section 7.2. A log of their chart interaction and text chats was collected. The face-to-face conversations among members were recorded by video camera. To avoid the effects of tool experiences, the group commu-nication methods and problems were assigned to teams in a different order as shown in Table 7.1.

For evaluation purposes, the ten-minute and five-minute time-outs were set for the divergent and convergent thinking phases respectively. Once the time-out is reached, the current thinking

Table 7.1: Group communication methods and problems types assignment.

Team Problem 1 Problem 2 Problem 3

1 CI TC FF

2 TC FF CI

3 FF CI TC

phase moves forward to the next automatically. In the evaluation stage, instead of using the original voting system, we used “Participant-cross evaluation” to prevent the bias in the experi-mental result described in Section 7.3.3. At the end of the session, the questionnaire was shown on their computer screens. Every participant input his/her opinions and was dismissed.

7.3.2 Quantitative Evaluation

The effectiveness of a communication method is measured by the degree of creativity of KJ charts constructed by teams by that communication method. As proposed by Guilford [70]

and being used in Torrance test of creativity performance [90], the degree of creativity can be measured by fluency, flexibility, originality, and elaboration of ideas. These are described as follows:

• Fluency of Ideas:The total number of non-redundant ideas.

• Flexibility of Ideas: The breadth of ideas.

• Originality of Ideas: The statistical rarity of the ideas.

• Elaboration of Ideas: The amount of detail in ideas.

We decided to evaluate the degree of creativity in divergent and convergent thinking sep-arately. It allows us to understand the effects of three communication methods in each way of thinking. The following measures adapted from Guilford’s work [70] and that of Jack A.

Goncalo’s work [89] are proposed to evaluate the effectiveness of divergent and convergent thinking in the experiment.

Divergent Thinking

• Fluency of Ideas: The total number of labels excluding those judged to be off-topic, redundant, impossible, and/or useless.

• Originality of Ideas: The total number of labels that did not occur in other charts under the same problem generated by other teams.

• Flexibility of Ideas: The total number of viewpoints appearing in labels.

• Elaboration of Ideas: The average length of text (in characters) of labels.

The fluency of ideas, flexibility of ideas and originality of ideas are decided after the experiment by the majority vote of three human evaluators.

Convergent Thinking

• Category Fluency:The total number of categories (sub-categories are counted).

• Within-Category Fluency: The average amount of labels per category.

• Average Depth of Top Categories: The average depth of top categories.

The within-category fluency is proposed for solving some problematic cases. If titles of categories are defined too abstractly, the amount of labels in such categories tends to be high, and the number of categories in that chart tends to be small, they compensating breadth of ideas with greater detail. Such detailed categories should be considered as a product of quality convergent thinking. The within-category fluency indicates the strength of derived solutions (categories) in terms of the amount of support ideas (labels), which also reflects the quality of convergent thinking.

The average depth of categories is proposed. This is an average number of induction steps in deriving a solution from the collection of ideas under a top category in a KJ chart. This number reflects the depth of derived solutions, which is intuitively proportionate to the performance of convergent thinking.

7.3.3 Qualitative Evaluation

Participant Cross-Evaluation

In each problem, after three teams finish the convergent thinking stage, the participant cross-evaluation stage is performed. Instead of evaluating their own KJ chart, all members of a team are asked to evaluate two other charts created by the other two teams in the same session.

All participants are guided to vote for their favorite labels on the aspect of their creativeness by announcing: “Please click on bulb icons of ideas that you do not expect to see”. They are also guided to vote for their favorite categories on the aspect of their completeness by an-nouncing: “Please click on bulb icons of categories that almost/completely contain all possible viewpoints”. Note that a user cannot repeatedly vote on the same label or category. There are no lower and upper limits in the number of votes per user.