Methods

In the field of corpus linguistics, Zanettin (2012, p. 7) has defined a corpus a collection of texts in electronic format which are processed and analyzed using software to enable linguistic research. Technological advances of computers and the wider availability of computer-readable texts are what made corpus linguistics possible.

The purpose of building a corpus of JA CSR reports is (1) to establish the frequency of each NTI identified in Chapter 4 in a large collection of texts which is representative of CSR reports in general, and (2) to extract samples that contain the NTIs in order to conduct an experiment to assess whether eliminating these NTIs can improve translation quality, answering RQ5 and RQ6 respectively.

The corpus is a collection of 13 JA CSR reports published by 13 different Japanese companies that have earned the Year 2017 Environmental Communication Awards hosted by the Japanese Ministry of the Environment. Thus, they are representative, all the more because they are the only award-winning companies that published ZH-CN CSR reports translated from their JA CSR reports in 2017, based on their parallel layout, content structure, and texts. In short, these reports constitute a principled and valid corpus for the present study.

1 This chapter is based on He (2019).

124

All of the JA and ZH-CN CSR reports can be freely downloaded from each company’s website; some are locked PDFs, while others are not. After unlocking the locked PDF files using a web-based freeware, the PDF files were converted to text files encoded as UTF-8 using Adobe Acrobat Pro (a PDF tool that allows for converting scanned files into editable PDFs in addition to creating, signing, sharing, editing, converting, and exporting PDFs). At first, an attempt was made to use an OCR (Optical Character Recognition) software—which allows for converting images of typed, handwritten, or printed text into machine-encoded/computer-processable data—to convert the files from the PDF format to text format. However, an alternative method was chosen for two main reasons. First, a significant volume of text is enclosed in text boxes, which becomes an issue because the text in boxes cannot, by default, be recognized by either CAT tools or corpus tools (e.g., Sketch Engine, a web-based corpus manager and text analysis software developed by Lexical Computing Limited since 2003). Secondly, a great number of JA characters could not be recognized properly via OCR, and most text included in graphics was garbled, which would have required too much time in fixing corrupted characters. However, line breaks were surprisingly not an issue despite the fact that when texts are copied from a PDF file and pasted into a text editor, unnecessary line breaks are often inserted at the end of each line, ending up by breaking sentences. As a result, it was decided to convert PDF files directly into text files using Adobe Acrobat Pro, because after conversion no text is boxed, and no characters are garbled. However, it still took some effort to clean up the converted text data by deleting unnecessary line breaks and spaces, fixing punctuation issues, and so forth.

After the 13 text files of CSR reports were cleaned manually using regular expressions to speed up the process, they were uploaded to Sketch Engine, which

125

annotates texts using the JA MeCab² part-of-speech (POS) tagset³. As an extension of JUMAN System⁴ and Chasen⁵, MeCab is the current state of the art JA language POS and morphological analyzer⁶. It first segments (tokenizes) words—since JA is an unsegmented language—and then lemmatizes them to provide annotations of base forms (lemmas) with predefined POS categories, inflection type, and inflection form.

MeCab is open source, and there is a number of free tools that can annotate texts with the MeCab POS tagset. These include, for example, a web-based morphological analysis tool named “UniDic-MeCab,” ⁷ which was developed by a project called yasashii nihongo (Easy-to-understand Japanese). However, very few corpus tools facilitate searching JA linguistic patterns using a sequence of MeCab POS tags, and Sketch Engine is one of the few. Being able to use the MeCab POS tagset is essential to this study because although some of the NTIs formulated in Chapter 4 are lexical items which can be searched by word form or lemma, many others are syntactic patterns such as “N + ga + Vpassive” (Vpassive means the passive form of a verb), which feature a combination of lemma and tags.

In addition, Sketch Engine can provide statistical information about the frequency of each search result, to help answer RQ5 (frequency of NTIs). Knowing how frequently each NTI occurs in a large corpus can make clear how far-reaching the influence can be when eliminating a particular NTI.

2 MeCab: Yet another part-of-speech and morphological analyzer. Retrieved June 29, 2019, from http://taku910.github.io/mecab/

3 Sketch Engine Japanese MeCab part-of-speech tagset. Retrieved June 29, 2019, from https://www.sketchengine.eu/tagset-jp-mecab/

4 JUMAN Information. Retrieved June 29, 2019, from http://nlp.ist.i.kyoto-u.ac.jp/nl-resource/juman/juman-eng.html

5 ChaSen—A Morphological Analyser. Retrieved June 29, 2019, from http://chasen-legacy.osdn.jp/

6 Morphology is the part of linguistics that deals with the study of words, their internal structure and partially their meanings. It refers to identification of a word stem from a full word form. A morpheme in morphology is the smallest units that carry meaning and fulfill some grammatical function.

Morphological Analysis is the process of providing grammatical information of a word given its suffix.

7 Morphological analysis web application UniDic-MeCab. Retrieved June 29, 2019, from http://www4414uj.sakura.ne.jp/Yasanichi1/unicheck/

126

Another useful function of Sketch Engine is that Corpus Query Language (CQL) can be used to search for complex grammatical or lexical patterns in combination with regular expressions or regex (i.e., a formalism specifying sequences of characters that define a search pattern) and wildcards (i.e., a character that will match any character or sequence of characters in a search used within a regular expression to generalize the search for patterns). For example, the regular expression “.{20,}^において” searches strings that contain “において” (nioite) before which there are 20 or more characters as the antecedent following the end of the previous sentence. In practical terms, this allows the identification of all instances of infringements of the NTI that says “Phrase before においてis too long (length limit: 20).” As examples of wildcards, an asterisk (*) stands for zero or more characters, and a question mark (?) stands for exactly one character.

Figuire 5.1 illustrates the displayed screen when searching for a JA word in Sketch Engine.

Figure 5.1 Search result for the Japanese word “^イメージ”

127

Following the methods described above, a JA-monolingual corpus of 555,135 tokens/425,302 words/29,162 segments was built via Sketch Engine. Tokens are occurring word forms, while words indicates the number of different tokens in the corpus (i.e., discounting all duplicates). Appendix 3 shows the CQLs or regular expressions formulated for searching all NTIs identified in Chapter 4. It establishes the frequency of each NTI, answering RQ5. Table 5.1 shows part of Appendix 3.

Table 5.1 Sample of CQL/regex and frequency of search result

NTI CQL/Regex Query type Frequency

(per million)

支援 支援為る Simple query 108(194.55) 評価 評価為る Simple query 121(217.96) N1やN2、N3

など [word=".+"] [word="や

"] [word=".+"]{1,3}

[word="、"]

[word=".+"]{1,3}

[word="など"]

CQL 41(73.90) N1に向けたN2 に 向け た Simple query 159(286.42) Sentence is too

long (Limit: 120) [¥n。][^。

¥n]{121,}?[¥n。] RegEx 723(1302.12) に向けて に 向けて Simple query 153(275.61) Vsahen-stemを

進める [tag="名詞-普通名詞-サ変可能"] [word="を

"] [lemma="進める"]

CQL 198(356.67)

※ is used to indicate a superscription or note

[word="※"] CQL 901(1,623.03)

A centered dot (・) is used to connect two nouns

[^¥n ]*[・･] RegEx 3603(6489)

Phrase beforeと して/において is too long (Limit: 20)

[¥n。、][^¥n。、]{21,

}[^目][^的]として RegEx 271(488.07) End a sentence

with a particle [tag=".* 助 詞 .*"]

[word="。"]{0,1} </s> CQL 498(897.08)

After corpus creation and query formulation, the next step is to extract samples from the corpus for the corpus-based experiment detailed in Section 5.2. Saldanha and

128

O'Brien (2014, p. 207) state that a sample must be representative of the whole group it belongs to if studying the sample is to allow the researcher to make valid inferences about the group as a whole. On this basis, Saldanha and O'Brien (2014) recommend that careful consideration must be given to the sampling method (in order to make sure that the selected sample enables the researcher to answer the research question) and the size of the sample (which could impose significant limitations on the generalizability of the findings). This research adopts random sampling, which corresponds to the ideal scenario where every member of the population being studied has an equal chance of being selected as a participant (Saldanha & O'Brien, 2014, p. 207). For each NTI, this study developed a CQL or a simple query that can be used in Sketch Engine to search for the target expression (see Appendix 3). For each search, a random sample of 20 results was extracted by Sketch Engine using an in-built function.

Although only four random samples were deemed necessary for each NTI, 20 samples were extracted in case the results contained noise. It was assumed that 20 samples would ensure four valid samples, counting from the beginning of the list. On the other hand, when it was necessary to use a text editor (rather than Sketch Engine) to extract samples using regular expressions, the search result was saved as a text file, and all samples copied to an Excel sheet and randomized using the inbuilt RANDOMIZE function. Again, the first four valid samples are used for the experiment described in Section 5.2. All samples and data were managed in a master Excel file.

Translation and error identification/labeling were also performed using Excel.

The experiment employed four translators—all of whom meet the requirements for qualified translators defined by ISO 17100:2015 (see Section 1.1.4)—to translate the selected sample, one rewriter who is a Japanese native speaker working as a professional EN to JA translator in Japan’s translation industry for more than 15 years, two evaluators (who were previously employed for developing the CSR scheme as described in Chapter 3) for accuracy evaluation, and three evaluators for fluency

129

evaluation (native Chinese speakers who majored in ZH-CN or English and are currently working as high school teachers in China). Although the present author is fit to be an accuracy evaluator, two other evaluators were employed for the accuracy evaluation since, as stressed by Saldanha and O'Brien (2014, p. 105), it is sensible to separate the evaluation from the data analysis, so that the researcher’s own expectations do not influence the evaluation results.

The measures of translation quality adopted in this study are accuracy and fluency, i.e., the two fundamental attributes used to assess translation quality in both academic-based and industry-based TQA models, as discussed in Section 2.2. For accuracy evaluation, instead of evaluating the translation based on a detailed error typology, evaluators were asked to assess if the meaning of the STs is conveyed in the TTs, disregarding the readability of the translation. They were also given the option that it cannot be decided if the translation is accurate because the ST is incomprehensible.

As for fluency evaluation, ranking (comparing different versions of translation and ranking them for fluency in the target language) was adopted as the quality evaluation model. Ranking translations is a well-tried method for quality assessment. However, as Saldanha and O'Brien (2014, p. 101) have pointed out, it is more common in MT quality assessment where the evaluators are asked to rank a number of candidate translations produced by one or more MT systems according to specific criteria (e.g., fluency, adequacy, comprehensibility). The advantage of comparing different versions of translation by ranking is that, compared to evaluation based on an error typology, it is less time-consuming because evaluators do not have to identify and classify specific errors, and it also gives immediate information on which translation is better (Saldanha

& O'Brien, 2014, pp. 101-102). Moreover, as pointed out by ALPAC (1966), monolinguals are more representative of potential translation users. Although the methods for evaluating accuracy and fluency are less informative in that the results do not provide any insight into the nature of the errors encountered, they were adopted in

130

the experiment since the nature of the errors is not important. The key point is that the results can be used to diagnose whether avoiding an NTI can improve accuracy and/or fluency.