楽曲を用いた情報伝達システム Tone Code

(1)

ᴦ᭤ࢆ⏝࠸ࡓ㏻ಙࢩࢫࢸ࣒ 7RQH&RGH

ßâ ÷

ǉ

ǊƱŉ Û

ǉ

ǊUdaya PARAMPALLI

ǉǉ

7RQH &RGH$&RPPXQLFDWLRQ6\VWHP%DVHGRQ0XVLFDO3LHFHV

7HWVX\D .2-,0$

㸨

.DQ .$0$'$

㸨

8GD\D 3$5$03$//,

㸨㸨

Tone Code is a communication scheme based on musical pieces. In Tone Code, the message is embedded into the components of the musical pieces, such as chord progressions, distortions, rhythms, and so on. The embedded message is considered as one of the components of the musical pieces and harmonizes with other components. It can be heard by the human ears as the sound played by the additional musical instruments while it can transmit the secret messages. If the recorded musical pieces are stored in smart phones or tablets, the transmitted message can be referred to anytime and anywhere. In this paper, the concept and future applications of Tone Code are introduced.

ǇKeywords : Tone Code, communication system, musical piece, data hidingǈ

1. QR コードと Tone Code

4(C7 QR Qw_,ÓŔ @ QR Qw _&,Á 1 +; (7+%Ǌ?BĉåƽƉ('+ Kmq%ċñ>&ǊGHhRF^+ URL 8mws D_tVǊ++üÅ('BƎ4¬>&% >7&7&,ãÆ)>Ƭº+ÃëšņǊĸƤš ņ('+6)ƴŎ?TV\l%"ǊŅ Ã%,ǎİdwQw_+xś&$xŶ)é ň?$> QR Qw_+7+Bœ6$4$7Ǌ?B þ·$>,Ƶ),A<(Ǌċñ ?Qw_,, ĉåƽƉ('+Dir%þ·+ >m[XwU)ô¯>&%>QR Qw_,Ǌ ŘÚ+üÅBm]ED)ƹ]wYcF]EvP& Á 1. QR Qw_+ǋ

1. QR Code and Tone Code

You may know what QR code is. An example of QR codes is shown in Fig. 1. If you capture such a code by your smartphone camera, you can get the information such as URL of a website, e-mail address, and so on. QR codes were originally developed for the purposes such as stock control in factories or logistic management, and so on. They have been widely used as generic 2-dimensional barcodes for general purposes these days.

If you look at QR code itself, you cannot get what it means. However, the code captured by a camera can be decoded into the original message by smartphone applications. QR code is one of the data hiding techniques which are technologies to embed the secret information in multimedia data1). More specifically, it can be called a visible data hiding since the code carrying the secret message is visible for human eyes (see the next section for the details). QR code itself is not beautiful as an image, but it is not annoying in the printed media or webpages since the printed area is not very large in general. Besides, there are some trials to print some photo images or illustrations behind QR codes. It can improve the appearance of QR codes.

Figure 1. An example of QR codes. The authors have proposed a new technology called 㸨᝟ሗᕤᏛ⛉ 㸨㸨7KH8QLYHUVLW\RI0HOERXUQH

(2)

ĤãĩǅŠÝƳÔĥŕŜÅµĜ Ǉş49¯ǈ ¸-?>ăŹ 1) _{+xś%=Ǌm[XwUB´C} Qw_+7+Ő)ž>&<®ſŏ]wY cF]EvP&7¸-?>ǇòƠªŁǈQR Qw_ ,žŐ),ĵ$ū&,ƃ(ǊƤæ3 =Í,¨¡?(6Ǌ¶À+ŋ('+ƫǆ )(>; (&,(3ǊĝƟ%,ǊQR Qw _+ŲĚ)Œ8FqV^('BƭŪ>&%Ǌ QR Qw_+7++žŐBČ½>ãÎ7Ĉħ ?$> ŷŭ<,Ǌ+ QR Qw_+ĬúBƿī]wY) ùň%(&ŬǊTone Code (>ăŹBĈħ 2)_{Tone Code ,ŘÚ+m[XwUBīě}

+ĭĀŽŤ+x#&$Ş¯¦>ăŹ%>ī ěBű$$7Ǌ+})ŘÚ+m[XwU´ 3?$>&,Ƶ),A<(ǊÝň DirBň>&%Ǌưƿ?īě<þ·+ >m[XwUô¯%>& 7+%>m [XwU,ǊQw_ƥŸ8rWlǊNYwRGv_+ ĳ4Ǉ]EV^wTpvǈ('+ƿīŽŤ&$Ş¯ ¦?>6ǊƵ+ů),Ŵŀ(īě+ĭĀŽŤ &$ƋƑ?Ǌ#Ǌ+ƿīŽŤ&7Ŵŀ)Ɛ¹ >; )ãÎ?$>QR Qw_®ſŏ(] wYcF]EvP%"+)ÜǊTone Code ,Ş ¯¦?m[XwUƿīŽŤ&$ű> 6Ǌ®űŏ]wYcF]EvP&¸/&%> ]wYcF]EvP)#$,İ+Ţ%Ɗť >&);

2. データハイディングと Tone Code

£Ơ+Ƥ=ǊTone Code ,®űŏ]wYcF]E vP+xś%>&ƃ>]wYcF]EvP&,Ǌ ŋ8ƿǀ¯ǊĘ('+ksZm]ED]wY+ })ǊƵ+Ŕƀ),A<(; )ŘÚ+m[X wUBÄ6ƞ5ăŹ+&Bć]wYcF]Ev P+&$,ǊƽÑƣ8ƽÑćţ('Ŕ<? $>1) ƽÑƣ,ǊksZm]ED]wY+āğį8 ŷįBƒ>6)Ǌŷŭ('+üÅBm]E D)Ä6ƞ5ăŹ+&%>xĒǊƽÑćţ&,Ǌ ]UYsm]ED+|ĲQfw+Ʒı8ơƚBŐŏ &7+%>-ǊFvYwb[^z%Œ 8ƿīBÉƔ>Ƹ)ǊƗŭ)Âğ+üÅBm]E D)Ä6ƞC%ƓÉ>Ɨŭ+ƽÑm]ED B|Ĳ)QfwÆ±ǊƤæǊ+Ɨŭ+üÅ

Tone Code2)_{, where the concept of QR code is extended to}

the music data. Tone Code is a technology to encode secret messages as one of the components of a musical piece. If you listen to the musical piece composed by Tone Code, you cannot find out if it conveys a secret message similarly as QR code case. However, the message can be decoded from the recorded musical piece by the designated smartphone application. In Tone Code, a message can be encoded into the musical components such as chord progressions, rhythms, or distortion effects for the guitar sound, and so on. Such encoded musical components can be naturally recognized by human ears as well as they can harmonize with other components included in the musical piece. As described above, QR code can be called a visible data hiding while Tone Code can be called a hearable data hiding since the encoded message can be recognized as a musical component. In the next section, the details of data hiding technologies are introduced.

2. Data Hiding and Tone Code

As described above, Tone Code can be considered as a hearable data hiding. Data hiding is a technology to embed secret messages into the multimedia data such as images, audio and video signals without being recognized by human recognition systems. The technologies such as digital watermarking or digital fingerprinting are included in data hiding.

Digital watermarking is a technology to embed the copyright or ownership information into the multimedia data to protect the intellectual property rights. On the other hand, digital fingerprinting is a technology to avoid or track the illegal copies or distributions of the multimedia data. For example, when photo images or musical pieces are purchased through the internet, the consumer information is embedded into such multimedia materials. If the consumer illegally copies the purchased material, there are plural copies which convey the identical consumer information though there should be only one copy with his/her information. Therefore, such illegal copies or distributions can be disclosed by digital fingerprinting.

Data hiding technologies are, in general, used for the security purposes such as protecting the copyrights or avoiding the illegal copies. However, they can be used for the secret communication means between two parties. For

(3)

Ä6ƞ3?Qfw,ǍÓÃ()7ƶA <ǊżĎ+QfwÓÃ>&&(=Ǌ|ĲQf w+Ŏž8ĊŎ®ų&(> +; )Ǌ]wYcF]EvPăŹ,ƤæǊŷ ŭ+į +ƒ8|Ĳ ňƷı('+XMor\ EŐŏ%ň<?>&ÌǊşyŭ)A <(; (ĒĶ%ǊŘÚƤ) &7®ų% >-Ǌ\tečƢ8ĉåƽƉ,Ǌm[XwU B>Ð&$ƽķB ň$>?)ÜǊ ƽķ+A=)Ę8ƿī('+ƽÑm]EDBm [XwUƧ) ň>&7®ų(+%>Tone Code ,3)?B×Ņ7+%=ǊBGM (' +ƿīBVfwK<čƢ> %ǊüÅƧ® ų&(>ăŹ%>

3. Tone Code のしくみ

ġŢ%,ǊTone Code B×Ņ>ĒĶ+x#& $Ǌďń 2)%Ĉħ?7++ĬŽBť>+ ĒĶ,Ǌƿī+ĭĀŽŤ%>mu]EǊcwnawǊ rWl+ !Ǌcwnaw)őï>Qw_ƥŸBň $Ƨ>m[XwUBźŅ>7+%> Qw_Ǌ>,¹ƿ&,ǊƿŚō(>żĎ+ ƿ²ę)ǀƿ+&%>y#+ƿ);>¹ƿ ,y¹ƿǊ¿#+Æ±,¿¹ƿ('&¸-?> %,Ǌy¹ƿB ň$ƣüÅBÄ6ƞ5ĒĶB ť>Qw_+ĭĀƿ&+¶ķĎ+Bź 1 ) ŖQw_ƥŸ&,Ǌīě+Qw_+ř=ÊA=B ź7+%>u[O8j[iVǊEDM (Electronic Dance Music) ('ǊŅƿī%,Qw_ƥŸ)¢" ě=©¢%>&ƃ> x#+y¹ƿ,ź 1 )Ŗ; )¶ķĎ+ō(> y#+ƿ+Ŧ4±A%ĭĀ?>Tone Code %,Ǌ īě+Qw_ƥŸBĭĀ>y¹ƿ+°ĭĀƿ)Ü ù>¶ķĎĀ)m[XwU¯BƭŪ>& %Ǌm[XwU¯+4%Qw_ƥŸ)Üù>ƿǀ ¯BĭĀ>?,!: 'ǊTvXRFS(' +ƽÑī¾%Qw_BļÏǊĔÓ+ě)Ʈ*±A 7+)őï>6Ǌm[XwU=ƿ,, "=&ű>Ǌ+ƿīŽŤ&Õ)Ɛ¹& ?$>6ǊŴŀ(Ǎě+īě&$ƋƑ%>+ %>ďń 2)%,Ǌ~Ɓ×ǄBƤ$Ǌm[XwU Ä6ƞ3?īěŴŀ)ƋƑ%>&Ŗ ?$> 3ǊQw_ƥŸË+ŽŤBňĒì7Ĉħ ź 1. Qw_&+ĭĀƿ+. Table 1. Examples of the chord structures and the frequency components corresponding to each note. Chord Three notes and

the corresponding frequency components [Hz] C C / 261.73 E / 329.63 G / 392.00 F F / 349.23 A / 440.00 C / 523.25 A A / 440.00 C# / 523.25 E / 659.26 Dm D / 293.66 F / 349.23 A / 440.00 Bm B / 493.88 D / 587.33 F# / 698.46

example, electro-magnetic wave is used for broadcasting, wifi networks, smartphones, and so on. In fact, the electronic media such as photo images, music or video can be used as a communication means instead of the electro-magnetic waves. It can be realized by Tone Code. Tone Code can transmit the message only by broadcasting the musical pieces from loud speakers.

3. How Tone Code Works

In this section, we introduce one of the methods to realize Tone Code2)_{. In this method, a message is encoded into the}

chord progressions of the musical piece. There are three major components of musical pieces, that is melody, harmony and rhythm. The chord corresponds to one of them, harmony.

A chord consists of multiple different harmonic notes which are made simultaneously. A chord which has three different notes is called as a triad. Here we introduce a way to embed the secret message into the musical piece by using triads. Table 1 shows some examples of the chords and frequency components of each note constructing those chords. The chord progression represents the transition of the chord in a musical piece. It is inevitable to take the chord progression into account when one composes musical pieces in almost all genres of contemporary music.

A triad consists of three different notes corresponding different frequency components shown in Table 1. In Tone Code, a message is divided into three parts in advance. Such divided part of messages is encoded and located at three different frequency components corresponding to the chord progression of the musical piece. In other words, the audio signal corresponding to the chord progression can be composed only by the encoded messages. Such audio

(4)

ĤãĩǅŠÝƳÔĥŕŜÅµĜ Ǉş49¯ǈ ?$>ďń 3)%,Ǌu[Oƿī)>NYw RGv_('%ň<?>ƽÑŏ(ĳ4Ǉ]EV^w TpvǈBň$m[XwUBƧ>ĒĶĈħ ?$>?,Ǌśǂ+]EV^wTpv¯B ňǊ+??)ǎƥŞ¯¦?m[XwU+ ǌ&ǍBÜù>&%Ǌm[XwUBźŅ>& 7+%><),Ǌm[XwUBŞ¯¦ ¯%VbD_ql+ƿBźŅǊīě+rWl%m [XwUBƢ>ƈ47Ÿ(A?$>

4. Tone Code の応用と関連技術

z%Ơ1Ƥ=ǊTone Code ,ƿīBň$m [XwUBƧ>7+%>6ǊBGM ('+ð %ƿīĸ?$>Æƾ),ùň®ų%> -Ǌ»ĩēƇ('%ƿÈD`GvV+A=) ň >&Ŭ<?>ƿÈD`GvV,Ǌ,"=& Ű¬>&Ƽ&7>zǊů+|ŴŊ( ),üÅBƧ>&%(Tone Code Bň ?-Ǌƿī)Ä6ƞ3?m[XwUVkw^g Iv8Yht[^('%ſƀŏ)źŖ%>6Ǌ Ƨ>üÅ+ĦäBƂĹ>&7%>3Ǌū Źǃ8§łǃ('%Jw]EJLF_+A=& $Þ>&7%>óģ+Jw]EJLF_% ,ǊÝň+ŝĠz%ćÖ?Ō¯B¤>&% áŖº+ƍĖŠBű&%?)Ü Tone Code %,ǊáŖº+¶Ɲ% BGM čƢ? $?-ǊĂĆ!+Vkw^gIv('B ň$ƍ ĖBű=ǊƎC =>&%> Tone Code %īě)Ä6ƞ5m[XwU,ďÒü

Å %,(ǊƿÈ¯8ŋ7ǎƥ+Ş¯&$ źŅ?$?-ǊÄ6ƞ4®ų%>6Ǌīě BčƢ> %33(üÅBƧ>& ®ų%>×Ƹ+ùň)Ƹ$,ǊVfwK <īěBčƢVkw^gIv('%> &BýÖ>6Ǌ»ĩēƇ('ÌƺƿÌ ŇÇ{%,ǊĲƤŸ(&7Ŭ<?> 3Ǌƿǀ¯,ƙƻ);=ĻŻǊ«ǀ('+ñǀ 7>6ǊƲƙƻ%+Ƥ7Ùė%,(xĒǊ ūŹǃ('%, BGM BčƢ%&$7ƿƯB ß#øŽ>6Ǌõí(¯<%7m[ XwUBô¯%>ăŹøŽ%=Ǌò+ŕŜƴ Ŏ)>Əǁ7Ì ŷŭ<,ǊTone Code )$ǊRFtvƿ) ĽØüÅ8ƪƼŧƛŠ+üÅBÄ6ƞC%ǊWiFi (

signals can be recognized as the sound played by additional musical instruments such as digital synthesizers. Therefore, the chord progression conveying the message is hearable for human ears, and it also can naturally harmonize with other musical components. It has been shown that the musical piece conveying the secret messages can be naturally recognized as a kind of digital music by human ears through subjective assessments2)

.

It has also been proposed that the musical components other than the chord progression can be utilized to send the secret messages. For example, it is shown that the messages can be embedded into the distortion effects for the sounds of electric guitar used in rock music3). In this method, the two different distortion signals are employed to represent 0’s and 1’s in the binary-encoded messages. On the other hand, there is another idea to embed the message into the rhythm of the musical piece. Specifically, the message is expressed by the snare-drum sounds in this method.

4. Tone Code and the Related Topics

As mentioned above, Tone Code is a technology to send the message by using the musical piece as a communication medium. Therefore, it can be applied into various scenario where the music pieces are broadcast as the form of back ground music, and so on. For example, it can be used in the market complexes instead of the conventional audio announcements. It is often difficult to catch the audio announcements clearly. Especially, such announcements cannot be recognized by hearing-impaired people. In Tone Code, such information-gap problems can be solved since the message embedded into the musical piece can be decoded and displayed on the screen of smartphones or tablets. Tone Code can also be employed in the art galleries or museums instead of the audio-guide. In the conventional audio-guides, the visitors can listen to the explanations on the exhibits as they input the numbers of the items on the designated mobile terminals. Such terminals are not required if Tone Code is applied for this purpose since the visitors can hear or see the explanations on their smartphones by recording the back ground music broadcast from the speakers.

In fact, the messages transmitted by Tone Code can be given in various forms. For example, images, videos or another audio signals as well as text messages can be

(5)

'ƽķ);>ĿũƤBňǊRFtvƿ+4% ƤBŸ ƷĽĿũTV\lBƴŎ$4)_×Ƹ +ĽØ)$,ǊĉåƽƉ8FvYwb[^('+ ƤŨƩĐ?>&7ý?ǊRFtvƿBč Ƣ> %üÅƧ®ų%>ăŹ,Ī6$Ʈ Žêǅ&Ŭ<?>

+ăŹ7 Tone Code &²Į)ƽķBňǊƿ ǀ¯+4%ĿũƤB×Ņ>7+%>Ǌƿǀ ¯,3%7RFtvƿ%>&@ ō ("$>+Æ±ǊƿƖ$7¼ǁ(&Ǌ ň>¶ķĎå)7ŴŊê>&<ǊƲƙ ƻ%7ĲƤBŸ &8ƺƿ0+Ůû&" ăŹŏƏǁBƂĵ>&7ĴƜŏÙė%>& ƃ>à)$VfwK<RFtvƿBčƢ ǊVkw^gIvBň$>×Ǆ%,Ǌ10m BƘ>ƙƻ%"$7Ǌ2&C'ƌ=+(Ƥ ®ų%>&Åµ?$>5)_{Tone Code +Æ}

±,ǊīěBčƢ>øŽ>ƶzǊƿƖ)7 >Śêƭÿ>øŽ=ǊƤºƖ+³z),< (>ãÎøŽ%>

5. 今後の展望

Tone Code ,ǊĿũƤăŹ&$7]wYcF

]EvPăŹ&$7Ǌ?3%)(đŃöB Ć"ăŹ%>Ǌ£Ţ%Ơ1Ƥ=ǊƂĵ1 ăŹŏƏǁ7Ì»ĩēƇ8ūŹǃ('Ǌùň> Æƾ);"$øŽ&?>Ģ7ō(>Ǌ?+ Æ±)$7ƤƙƻB>Śê)#&&Ǌƌ =+(ƤBŸ( &,ĝ%Ƃĵ1Ə ǁ%>?3%,ƌ=ƄĲ('Bň)Ƥ× Ǆ('BŸ("$Ǌ×Ƹ+ùň)Ƹ$,Ǌƨ (ƌ=ƄĲŞ¯Bň>&øŽ%> 3ǊQw_ƥŸ8]EV^wTpvǊrWl+ 4(<Ǌƿī+=&<9>ŽŤ)m[XwUB Ä6ƞ5&%Ǌīě+xƬ)ÍƯ+üÅBÌƮ¦ $Ä6ƞ5&7ŵ·ĺ¼ǁ%><),Ǌ +ăŹBĘ(')ùňǊĘ&$Ŵŀ)ſű %>ǊVkw^gIv%ưŋ>&%m[Xw UB%>; (TV\l0&ąî>&7 Ĩƅ?$>(A!Ǌ]UYsRFbwU(' )Ąñ?ĘBưŋ> %Ǌ)Ä6ƞ3 ?m[XwUBæ)Ć!Ʀ/&%>; ( ĕ8$Ɔ?>7?(

encoded and embedded into the musical pieces. However, it is sometimes difficult to decode the transmitted message correctly under the noisy environments such as the market places crowded with many people since the musical pieces are broadcast from the loud speaker and received by the smartphones through quite noisy communication channels. In addition, it is not easy to send the messages for long distance because the audio signals are attenuated with the distance, and affected by echoes or ambient noise in general. In the art galleries or museums, the volume of the back-ground music should be suppressed, which implies that it is required to decode the transmitted message successfully even if the recorded audio signals are weak. These problems should be solved in the future studies.

The authors have proposed a disaster prevention broadcasting system using audio data hiding4)_before

proposing Tone Code. In this system, the messages including the detailed disaster information or the evacuation route are embedded into a siren sound and broadcast from the loud speakers. It does not require any wifi connections or internet access, so it can be used under the large-scale disasters such as earthquakes or tsunamis.

This system also transmits the messages only with the audio signals in a similar way as Tone Code. However, it is different from Tone Code since siren sounds are employed as the communication media in this system. In general, it is easier to transmit the messages by using siren sounds than using musical pieces since the quality of the siren sounds is not required to be very high, and various frequency components can be used to embed the message. Therefore, the performances of such broadcasting system using siren sounds are better than those of Tone Code. In our recent studies5)_{, it has been shown that the messages embedded into}

the siren sound and broadcast from the loud speaker can be correctly received and decoded by the smartphones through the distance larger than 10 meters under an indoor environment. In the case of Tone Code, the quality of the musical pieces should be maintained at high definition, which implies that there are many problems to be solved to improve the performances of the communications.

5. Future Studies

Tone Code is one of the novel technologies as data hiding techniques as well as wireless communications.

(6)

ĤãĩǅŠÝƳÔĥŕŜÅµĜ Ǉş49¯ǈ 謝辞 ġŕŜ+xƬ,ǊJSPS ŗŕƕ 15K01109Ǌ;.Ĥ ãĩǅŠÝƳÔĥçĀ 29 èêƮľƭŧƕ+¥ĀB $Ÿ(A?3 References (参考文献)

1) F. A. Petitcolas, R. J. Anderson and M. G. Kuhn, “Information Hiding — A Survey,” Proc. IEEE, vol. 87, no. 7, pp. 1062−1078, Jul. 1999.

2) K. Kamada, T. Kojima and U. Parampalli, “Tone Code: A Novel Method for Covert Communications Based on Musical Components,” Proc. ISITA 2016, Monterey, CA, USA, pp. 340−344, Nov. 2016.

3) Y. Matsunaga, T. Kojima, N. Aoki, Y. Dobashi and T. Yamamoto, “A Digital Watermarking Technique for Music Data Using Distortion Effect,” Acoustic Science and Technology, vol. 39, no. 1, pp.37−39, Jan. 2018. 4) T. Kojima, T. Tachikawa, A. Oizumi, Y. Yamaguchi

and U. Parampalli, “A Disaster Prevention Broadcasting Based on Data Hiding Scheme Using Complete Complementary Codes,” Proc. ISITA 2014, Melbourne, Australia, pp. 45−49, Oct. 2014.

5) Y. Sada and T. Kojima, “Improvement of Emergency Broadcasting System Based on Audio Data Hiding (in Japanese),” IEICE Tech. Rep., vol.117, no.476, EMM-2017-88, pp.55−60, Mar. 2018.

However, there are many problems in the future study as mentioned above. For example, it is quite important to realize the error-free communications for a long distance regardless of the applied scenario such as market complexes or museums. In the previous studies, any error correcting codes have not been employed, but they are strongly required to be used in the practical applications.

It is interesting to embed the messages into various musical components other than the chord progressions, distortion effects or the rhythms. It is expected that a large amount of message is multiplexed and embed into various musical components such as melody lines, chords, rhythms, and so on. It can be considered that the concept of Tone Code can be applied into the video signals like short movies. In this case, messages are embedded into various components of the video signals. Such signals conveying the messages can be recognized as a short movie by human eyes. The messages can be decoded from the recorded video signals by the smartphone applications in a similar way as Tone Code. In our future, for example, the messages embedded into a short movie displayed as a digital signage can be carried anywhere if such a movie is recorded by your smartphone.

Acknowledgments

This work is partly supported by JSPS KAKENHI Grant Number 15K01109, and also funded by National Institute of Technology, Tokyo College.

㸦ᖹᡂᖺ᭶᪥ཷ⌮㸧