Protection and Utilization of Privacy Information via Sensing ∗

(1)

INVITED PAPER

Special Section on Enriched Multimedia

Protection and Utilization of Privacy Information via Sensing ^∗

Noboru BABAGUCHI^†a),Fellow andYuta NAKASHIMA^††,Member

SUMMARY Our society has been getting more privacy-sensitive. Di- verse information is given by users to information and communications technology (ICT) systems such as IC cards benefiting them. The information is stored as so-called big data, and there is concern over privacy violation. Visual information such as images and videos is also considered privacy-sensitive. The growing deployment of surveillance cameras and social network services has caused a privacy problem of information given from various sensors. To protect privacy of subjects presented in visual information, their face or figure is processed by means of pixelization or blurring. As image analysis technologies have made considerable progress, many attempts to automatically process flexible privacy protection have been made since 2000, and utilization of privacy information un- der some restrictions has been taken into account in recent years. This paper addresses the recent progress of privacy protection for visual information, showing our research projects: PriSurv, Digital Diorama (DD), and Mobile Privacy Protection (MPP). Furthermore, we discuss Harmonized Information Field (HIFI) for appropriate utilization of protected privacy information in a specific area.

key words: privacy information, sensing, visual abstraction, privacy pro- tection, information disclosure and utilization

1. Introduction

In recent years, our society has been getting more privacy- sensitive or privacy-aware. We are facing various information and communications technology (ICT) systems, e.g., an IC card system, in our daily life, and receiving a lot of benefit and convenience from them. At this moment, we are unconsciously giving diverse information such as personal data and movement history to them. The information is stored as so-called big data, which leads to concern over privacy violation. Visual information such as images and videos is also considered privacy-sensitive. As systems and services using visual information have been developed, privacy issues of subjects in the visual information have at- tracted considerable attention [1]–[4]. In Japan, the lawsuit about infringement on the privacy right by Google Street View triggered the public attention. Video surveillance is also an essential ICT system to maintain safety at surveillance areas; however, it can be viewed as an critical source of privacy disclosure.

Manuscript received March 13, 2014.

†The author is with the Graduate School of Engineering, Osaka University, Suita-shi, 565–0871 Japan.

††The author is with the Graduate School of Information Sci- ence, Nara Institute of Science and Technology, Ikoma-shi, 630–

0192 Japan.

∗This work is supported in part by a Grant-in-Aid for scientific research from the Japan Society for the Promotion of Science.

a) E-mail: [email protected] DOI: 10.1587/transinf.2014MUI0001

In fact, visual information distributed via TV broadcasts and the Internet contains privacy-sensitive information (PSI) [5], [6], e.g., subjects’ faces, which may cause privacy violation in its acquisition, distribution, and share. There- fore, some protection procedures for the PSI are strongly re- quired. In Google Street View, regions of subjects’ faces are blurred to prevent their identification. In surveillance video, whichever the camera is static or mobile, various protection methods are proposed. In TV broadcasts, not only subjects’

region but also the background region is sometimes blurred so that the viewer cannot guess where the video was taken.

Formerly, this kind of protection procedures was man- ually carried out, which was very time-consuming. Nowa- days, automated procedures are tried using image analysis technologies. After detecting PSI regions in an image, an image processing operator, such as pixelization and blurring, is performed onto all the regions so that they cannot be seen. This can be viewed as exhaustive privacy protection. For Google Street View images, all the regions of human faces and car license plates are blurred. This automated method [7] has drawn attention as a practical example of visual privacy protection. A similar method is also provided on YouTube [8]. In social network services (SNSs), such as Twitter, Facebook and Flickr, the number of users who upload images are surprisingly increasing. A system is proposed that alerts users to potential privacy violation due to uploaded images based on image content analysis [9]. In this way, the technologies of visual privacy protection and those of image analysis are mutually correlated.

In this paper, we address the recent progress of privacy protection for visual information, showing our research projects: PriSurv, Digital Diorama (DD), and Mobile Pri- vacy Protection (MPP). They aim at flexible privacy protection in contrast to exhaustive privacy protection. Fur- thermore, we discuss the importance of utilizing privacy- protected information appropriately in some special situatu- ions.

The rest of the paper is organized as follows. Section 2 overviews the related work. Section 3 discusses the PSI in detail. Section 4 presents our research projects about visual privacy protection. Section 5 presents our new attempt to build an information field to realize both protection and utilization of privacy information. Section 6 concludes the paper.

Copyright c2015 The Institute of Electronics, Information and Communication Engineers

(2)

2. Related Work

In the last decade, research eﬀorts have been dedicated to privacy protection for visual information. For removing PSI such as faces, entire bodies, or other specific objects [10], wide variety of image processing techniques, e.g., blurring, pixelization, blocking out, and edge detection, are applied to them, and some of which capability of privacy protection was evaluated [11]–[16]. Some research attempts were made for preserving some specific visual content such as facial details [17] and body motion [18]. An eigen-space filtering-based technique [19] determines visual content to be preserved based on a preliminarily calculated set of image bases. With these techniques, PSI is permanently lost so that the visual information is viewed without infringing on others’ privacy. In other words, they are irrecoverable abstraction of the image/video content.

Recoverable abstraction is another interesting research direction, which allows authorized entities to view the original visual content recovered from an abstracted one. This type of techniques also involves the problem of privacy protected images/video delivery to designated entities. Recov- erable abstraction is beneficial for video surveillance systems that require further investigation once an abnormal event happens. Dufaux and Ebrahimi [20] proposed to en- crypt some PSI regions in a compressed domain. This technique was adapted for compressed video streams [21]–[24]

and uncompressed ones [25], [26]. Instead of using encryption techniques, information hiding is also employed to em- bed PSI into the abstracted images/video [27]–[29].

With the development of these abstraction techniques, various applications with image/video privacy protection have been developed. Privacy protection for video surveillance is one of such applications that has been widely studied [30]. The basic approach is locating PSI regions in surveillance video, e.g., using background subtraction, and applying such visual abstracting techniques as blurring, blocking out, and pixelization as in [31], [32]. Some systems provide removed PSI through a diﬀerent data chan- nel for later investigation. Since the main purpose of video surveillance is security, recoverable abstraction techniques have been employed, which are based on encryption and data hiding to allow authorized entities to access PSI as mentioned above, for example, [33]–[38]. An important idea of privacy is that people have the right to control information on them, leading to video surveillance systems that allow people who are captured in surveillance video to choose whether their appearance is disclosed [39]–[41], which often use RFID or other techniques for identification of people in video. A new attempt for video surveillance potentially provides various additional values by selectively disclosing PSI to public [42].

Compared with video surveillance systems that can as- sume fixed cameras, mobile camera-based applications con- front the diﬃculty in locating PSI regions in images/video because of the inapplicability of background subtraction,

and an object detection technique such as face or pedestrian detection is used instead. For mobile surveillance, which captures suspicious people using mobile cameras, some privacy-aware systems are summarized in [43], e.g., [44], [45], and a more recent work focused on automatically discriminating people of interests based on the videographer’s intention [46]. Besides mobile video surveillance, privacy protection has been designed for various applications. Google Street View is a well-known example of such applications, and some researchers focus on removing pedestrian regions from the images with less visual arti- facts [47] or on improving the recall of PSI detection (i.e., face or pedestrian detection) [7]. Some systems are proposed for other applications such as life-log [48], video conferencing [49], and consumer generated videos [50], [51].

Some techniques do not rely on visual abstraction to protect privacy in images/video. For example, systems for knowledge discovery from unpublished videos have been proposed for medical purposes [52], [53]. PicAlert [9] automatically finds images from an image collection that potentially contain privacy sensitive contents and notify the user to prevent their disclosure to public. Privacy Visor [54] is another interesting approach to privacy protection, which physically masks face regions by goggles with a tailored pat- terns to hinder face detection.

Privacy protection is generally considered as a sub- discipline of information security and often adopts techniques of information hiding, encryption, secret sharing, and access control for abstraction and identification (e.g. [20], [29], [37], [55]).

3. Privacy Sensitive Information

Privacy sensitive information (PSI) can be defined as information that may invade somebody’s privacy in case it is extracted from the entire information acquired from sensors located in the real world [5]. We classify the PSI into 1) information accessible to personal ID, 2) information related to personal ID, and 3) information on one’s private area.

The information accessible to personal ID is the information with which a corresponding individual is distin- guishable from others. This kind of PSI includes person’s appearance, such as faces, full figures, clothes, gaits, ges- tures, and expressions in images, as well as utterance in audio signals. Behavioral information, such as locations and trajectories, is also included. As a special case, text on nameplates or car license plates is included as well.

The information related to personal ID is not essen- tially privacy-sensitive in itself, but turns into PSI once it is associated with personal ID. This kind of PSI includes one’s belongings and possessions. For example, an animal is usually not concerned with privacy; however, if it is pos- sessed by someone as his/her pet and is known to be his/her possession, it turns into PSI.

The information on one’s private area is the third kind of PSI. The private area implies where most people feel private. For example, the inside of one’s house, a clothes-

(3)

Fig. 1 An example of visual privacy protection.

drying place, and displays of PCs or smartphones are in this case.

Let us think about a scene in an image as indicated in Fig. 1(a). Which region in the image is privacy-sensitive?

Figure 1(b) shows the regions corresponding to three kinds of PSI, i.e. 1), 2), and 3) as stated above. If we take ad- vantage of exhaustive privacy protection, most parts of the image cannot be seen, and the image becomes meaningless as shown in Fig. 1(c). Therefore, a framework that flexibly deals with the PSI depending on the subject and the viewer will be needed.

4. Privacy Protection for Visual Information

In the following, we proceed to describe our research projects about visual privacy protection.

4.1 PriSurv

PriSurv (Privacy Protected Video Surveillance System) project was an attempt to construct a pilot system of video surveillance with protection of subjects’ privacy. The purpose of PriSurv is to make video surveillance a social system oﬀering both safety and security. Our prototype of PriSurv [41] works in spatially local community such as a school zone while preserving community members’ privacy.

PriSurv is capable of generating a surveillance video that dynamically changes the appearance of subjects in order to protect his/her privacy. In contrast to the existing systems by exhaustive privacy protection, PriSurv is characterized by selective privacy protection [56].

To control disclosure of the subject’s visual PSI, PriSurv provides multiple image processing operators named visual abstraction. The subject’s appearance can be changed via visual abstraction operators. PriSurv oﬀers 12 operators: transparency, dot, bar, box, silhouette, border, edge, mosaic, blur, monotone, see-through, and as-is.

One of the main characteristics of PriSurv is that the content of generated video is changeable based on the relationship between the viewer and the subject. Figure 2 shows

Fig. 2 Privacy control in PriSurv.

privacy control in PriSurv. From the original imageI, visual abstraction operatorafor the subjectsand the viewervpro- duces an privacy protected imageI_<^a_s_,v> that is diﬀerent for each viewer. PriSurv works in a community where there are preliminarily registered members and non-members. Each member can describe in PriSurv’s privacy policy how his/her appearance should be presented to various viewers: family, neighbors, strangers, etc. The relationship between the visual abstraction and the human sense of privacy was in- vestigated from a psychological viewpoint [57], which was reflected on design of PriSurv’s privacy policy. It should be noted that PriSurv is embodiment of self-information control, which is modern interpretation of the privacy right.

4.2 Digital Diorama

In Sensing-Web project, we addressed how appearance of each person should be represented in surveillance video that captures a scene of a public space when the video is open to public through the Internet. Our main focus was to protect privacy of subjects in a strict way. In this project, we built a real-world content named Digital Diorama (DD) [58] that displays real-time information acquired from various sensors. In DD, movement of people in the 3D environment can

(4)

Fig. 3 Privacy control in DD.

be observed from an arbitrary viewpoint. Location of each person is estimated from the video stream from surveillance cameras, and is presented in a texture mapped 3D model of the target scene.

Because DD deals with a public space, privacy of the people should be strictly protected. We think that the appearance of all the subjects in surveillance video should not be presented to viewers. In DD, a subject is represented as a human-shaped stick figure. Furthermore, DD displays specific subjects in a diﬀerent color: If a viewer is a member of a certain group registered to DD and is authenticated, he/she can find other members easily by showing them in a color diﬀerent from other people. This function is very useful for locating family or friends. DD realizes identification of the subjects, authentication of the viewer, and registration of group members using RFID tags. Figure 3 shows privacy control in DD. The human-shaped stick figure in pink be- longs to the same group as the viewer, and the other people are colored blue.

DD’s privacy control [42] is similar to PriSurv’s one, because it is based on the relationship between the viewer and subject. DD was demonstrated at a shopping mall ‘Shin- Pu-Kan’ in Kyoto for actual visitors. The questionnaire sur- vey indicated that DD’s privacy protection was favorable to most visitors.

4.3 Mobile Privacy Protection

Our mobile privacy protection (MPP) project aims at automatically generating privacy protected videos captured by videographers using mobile cameras. One of the essential diﬀerences between surveillance videos and ones captured by videographers is that the videographers have their cap- ture intentions, e.g., to record a child’s growth. A video captured by a videographer contains important persons, without whom the video becomes meaningless and thus the videographer captures them intentionally (intentionally captured persons; ICPs), as well as objects that are not important for the intentions (non-ICPs). For this, we developed a method for automatically detecting ICPs and their abstraction [46],

Fig. 4 Overview of MPP.

Fig. 5 Examples of original video frame (left) and privacy protected one by [50] (right).

Fig. 6 Our prototype of real-time mobile privacy protection system (left) and its usage example (right).

[51], [59]–[61]. Figure 4 shows an overview of privacy protection for mobile cameras.

One of the fundamental techniques in MPP is ICP detection [59], [61], which automatically locates ICPs in video. After locating all people using face/pedestrian detection [62], [63], assuming that videographers’ intention is reflected in how they move their mobile camera, ICP detection extracts visual features such as each detected persons’

position, size, and similarity between the person’s motion and the camera motion. Using these features, a machine learning technique-based ICP model, e.g., a support vector machine and a Markov random field (MRF)-based model, classifies each detected person into ICP or non-ICP.

Our MPP then obscures non-ICPs identified by ICP detection. Considering that most videos captured with mobile

(5)

cameras are watched by others, possibly through video sharing services, MPP must provide various types of abstraction so that its users can choose appropriate one for their video contents. In [60], we implemented blurring, blocking out, and removal by seam carving [64]. We also developed a background estimation technique for mobile cameras and adopted it for abstraction as shown in Fig. 5 [50], which can be a preprocessing for various abstraction operators used in PriSurv [41]. In addition, we proposed a mobile system that automatically obscures non-ICPs in real-time as in Fig. 6.

5. Protection and Utilization of Privacy Information

Based on the research results on sensing and privacy, we

Table 1 Comparison of our projects.

Fig. 7 Conceptual sketch of HIFI.

have deepen our ideas, and launched Harmonized Informa- tion Field (HIFI) project [65]. Table 1 summarizes the comparison of our projects in terms of sensors, subjects, and sensing targets.

HIFI is an information infrastructure that underlies a field, which is a certain area in the real world, and provides its inside people with timely recommendations based on privacy information intentionally released by the people. In our project, the field is a bounded space for specific purposes, for example, shopping malls, stations, and theme parks. We regard, as privacy information in a broad sense, various information that links to a person’s ID: face, figure, expres- sion, motion, location, trajectory, preference, interest, and so on. A user, who is a visitor to HIFI, will intentionally release his/her privacy information and gain profit in exchange.

Let us describe HIFI’s goal in the following scenario.

Everybody knows that he/she can get useful information and premium services by giving, to a some extent, his/her privacy information to trusted organizations and shops. An ex- pert salesclerk in a shop will recommend some goods that suit the preference or interest of his/her familiar guest. This comfortable recommendation can be derived from privacy information released by the guest. In other words, it is very diﬃcult for the guest to obtain premium services without giving his/her privacy information. Note that the disclosure of privacy information and the obtained profit in exchange for the disclosure will be balanced in HIFI.

Figure 7 shows a conceptual sketch of HIFI. A visitor who comes into the field submits his/her privacy infor-

(6)

mation to HIFI. The privacy information acquired from ac- tive or passive sensors is securely stored and structuralized in the spatio-temporal database (STDB). Mining the STDB produces useful information that will be fed back to the visitor as recommendation of useful or premium information and services tailored for him/her. HIFI’s privacy information processing consists of four steps: 1) collection and disclosure, 2) protection, 3) storing and structuring, and 4) utilization.

We are now pursuing the following research issues:

• Fusing environmental and social sensing

• Acquiring spatial and temporal attributes of users to be stored in the STDB

• Data cleansing for privacy protection

• Recommendation by utilizing privacy information

• Information entry at HIFI’s entrance and exit gates

• Evaluation of the balance between the disclosure and profit

Among them, we here describe the information entry system (IES) placed in entrance/exit gates of HIFI. At the gate, the visitor will be able to enter his/her privacy information and to determine its disclosure level. Then HIFI will obtain his/her consent on privacy management in HIFI. Namely, the visitor can decide, according to his/her own intention, whether his/her privacy information is stored or discarded at the entrance and exit. Thus, IES plays an important role in HIFI.

As illustrated in Fig. 8, IES exists at the boundary of HIFI and the external world (EW), and has two sides: the HIFI- and EW-sides. IES consists of a camera, Kinect sensors, and a touch-screen monitor for interactively collecting visitors’ privacy information, e.g., their face, clothing, age,

Fig. 8 Entrance gate of HIFI.

gender, and so on, with a less load on its visitors. In IES, a visitor first passes through the EW-side gate and moves to the front of the monitor. Next he/she chooses the ID type during his/her stay at HIFI, interacting with the monitor.

Then he/she comes into HIFI through the HIFI-side gate.

The ID type is divided into three classes: autonym, pseudonym, and anonym, which mean that the visitor is identifiable in both HIFI and EW, identifiable in HIFI but not in EW, and not identifiable in both HIFI and EW, respectively. With the ID type, the visitor can determine the level of his/her privacy information disclosure by himself/herself.

The autonym ID will make all privacy information in EW disclosed in HIFI. The pseudonym ID will make abstracted or partial privacy information disclosed. The anonym ID will make no information disclosed.

While the visitor passes through IES, Kinect sensors continually detect his/her body region on depth images using background subtraction. His/Her clothing information is represented as a color histogram based on the extracted body region. The camera installed on the top of the monitor extracts his/her frontal face region by a face detection technique during the behavior in front of the monitor when he/she chooses his/her ID type. The face information is represented as facial images. The facial images are used for visitor identification and age/gender estimation for autonym ID, and are immediately discarded for pseudonym and anonym IDs. IES tries to collect privacy information with interaction in order to facilitate the visitor’s manipula- tions.

6. Conclusion

The trend of privacy information processing has been chang- ing from exhaustive protection and selective protection to appropriate disclosure and utilization. In HIFI, we focus on ‘profit’ as the opposed concept of ‘disclosure’ of privacy information, and try to harmonize them. It is impossible for everybody to disclose his/her privacy information at any time, at any place. We therefore consider ‘disclosure’ at the bounded space called the field. HIFI’s usefulness depends on how much benefit the visitor can obtain in it. It should be noted that ‘protection’, ‘disclosure’ and ‘profit’ are related to human factors, subjectivity, and situation/context- awareness. Human-centered perspective is indispensable for system development. In addition, we point out that not only technological approaches but also psychological, social, and legal approaches are of great importance in privacy information processing.

References

[1] A. Senior, ed., Protecting Privacy in Video Surveillance, Springer, 2009.

[2] Special Issue on Enhancing Privacy Protection in Multimedia Sys- tems, EURASIP Journal on Information Security, vol.2009, 2009.

[3] Special Issue on Privacy-aware Multimedia Surveillance Systems, Multimedia Systems, Springer, vol.18, no.2, 2012.

[4] Special Issue on Intelligent Video Surveillance for Public Security &

(7)

Personal Privacy, IEEE Trans. Information Forensics and Security, vol.8, no.10, 2013.

[5] N. Babaguchi, “Visual privacy sensitive information and its processing,” ISCIE Journal ’Systems, Control and Information’, vol.54, no.6, pp.242–247, 2010. (in Japanese)

[6] N. Babaguchi and S. Nishio, “Sensing and privacy protection of network robots,” J. IEICE, vol.91, no.5, pp.380–386, May 2008. (in Japanese)

[7] A. Frome, G. Cheung, A. Abdulkader, M. Zennaro, B. Wu, A.

Bissacco, H. Adam, H. Neven, and L. Vincent, “Large-scale privacy protection in google street view,” Proc. IEEE Int. Conf. Computer Vision, pp.2373–2380, 2009.

[8] YouTube, Face blurring: when footage requires anonymity, http://youtube-global.blogspot.com/2012/07/face-blurring-when- footage-requires.htmlGG, accessed Nov. 21, 2014.

[9] S. Zerr, S. Siersdorfer, and J. Hare, “Picalert!: a system for privacy- aware image classification and retrieval,” Proc. ACM Int. Conf. In- formation and Knowledge Management, pp.2710–2712, 2012.

[10] A. Cavallaro, O. Steiger, and T. Ebrahimi, “Semantic video analysis for adaptive content delivery and automatic description,” IEEE Trans. Circuits Syst. Video Technol., vol.15, no.10, pp.1200–1209, 2005.

[11] Q.A. Zhao and J.T. Stasko, “Evaluating image filtering based techniques in media space applications,” Proc. ACM Conf. Computer Supported Cooperative Work, pp.11–18, 1998.

[12] M. Boyle, C. Edwards, and S. Greenberg, “The eﬀects of filtered video on awareness and privacy,” Proc. ACM Conf. Computer Sup- ported Cooperative Work, pp.1–10, 2000.

[13] C. Neustaedter, S. Greenberg, and M. Boyle, “Blur filtration fails to preserve privacy for home-based video conferencing,” ACM Trans.

Computer-Human Interaction, vol.13, no.1, pp.1–36, 2006.

[14] N. Babaguchi, T. Koshimizu, I. Umata, and T. Toriyama, “Psycho- logical study for designing privacy protected video surveillance system: PriSurv,” in Protecting Privacy in Video Surveillance, pp.147–

164, 2009.

[15] F. Dufaux and T. Ebrahimi, “A framework for the validation of privacy protection solutions in video surveillance,” Proc. IEEE Int.

Conf. Multimedia and Expo, pp.66–71, 2010.

[16] P. Korshunov, S. Cai, and T. Ebrahimi, “Crowdsourcing approach for evaluation of privacy filters in video surveillance,” Proc. ACM Mul- timedia 2012 Workshop on Crowdsourcing for Multimedia, pp.35–

40, 2012.

[17] E. Newton, L. Sweeney, and B. Malin, “Preserving privacy by de- identifying face images,” IEEE Trans. Knowl. Data Eng., vol.17, no.2, pp.232–243, 2005.

[18] D. Chen, Y. Chang, R. Yan, and J. Yang, “Tools for protecting the privacy of specific individuals in video,” EURASIP J. Advances in Signal Processing, vol.2007, pp.1–9, 2007.

[19] J.L. Crowley, J. Coutaz, and F. B´erard, “Things that see,” Commun.

ACM, vol.43, no.3, pp.54–64, 2000.

[20] F. Dufaux and T. Ebrahimi, “Video surveillance using JPEG 2000,”

Proc. SPIE, pp.268–275, 2004.

[21] F. Dufaux and T. Ebrahimi, “H.264/AVC video scrambling for privacy protection,” Proc. IEEE Int. Conf. Image Processing, pp.1688–

1691, 2008.

[22] K. Martin and K. Plataniotis, “Privacy protected surveillance using secure visual object coding,” IEEE Trans. Circuits Syst. for Video Technol., vol.18, no.8, pp.1152–1162, 2008.

[23] F. Daia, L. Tonga, Y. Zhanga, and J. Lia, “Restricted H.264/AVC video coding for privacy protected video scrambling,” J. Visual Communication and Image Representation, vol.22, no.6, pp.479–

490, 2011.

[24] Y. Wang, M. O’Neill, and F. Kurugollu, “Privacy region protection for H.264/AVC by encrypting the intra prediction modes without drift error in I frames,” Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, pp.2964–2968, 2013.

[25] P. Carrillo, H. Kalva, and S. Magliveras, “Compression independent

object encryption for ensuring privacy in video surveillance,” Proc.

IEEE Int. Conf. Multimedia and Expo, pp.273–276, 2008.

[26] S.M.M. Rahman, M.A. Hossain, H. Mouftah, A.E. Saddik, and E. Okamoto, “Chaos-cryptography based privacy preservation technique for video surveillance,” Multimedia Syst., vol.18, no.2, pp.145–155, 2012.

[27] W. Zhang, S. Cheung, and M. Chen, “Hiding privacy information in video surveillance system,” IEEE Int. Conf. Image Processing, pp.868–871, 2005.

[28] P. Meuel, M. Chaumont, and W. Puech, “Data hiding in H.264 video for lossless reconstruction of region of interest,” Proc. European Sig- nal Processing Conf., pp.2301–2305, 2007.

[29] G. Li, Y. Ito, X. Yu, N. Nitta, and N. Babaguchi, “Recoverable privacy protection for video content distribution,” EURASIP Journal on Information Security, vol.2009, 9 pages, 2009.

[30] S. Moncrieﬀ, S. Venkatesh, and G. West, “Dynamic privacy in public surveillance,” Computer, vol.42, no.9, pp.22–28, 2009.

[31] A. Cavallaro, “Adding privacy constraints to video-based applications,” European Workshop on the Integration of Knowledge, Se- mantics and Digital Media Technology, 8 pages, 2004.

[32] S. Fleck and W. Straßer, “Towards secure and privacy sensitive surveillance,” Proc. ACM/IEEE Int. Conf. Distributed Smart Cam- eras, pp.126–132, 2010.

[33] A. Senior, S. Pankanti, A. Hampapur, L. Brown, Y.L. Tian, and A. Ekin, “Enabling video privacy through computer vision,” IEEE Security & Privacy, vol.3, no.3, pp.50–57, 2005.

[34] T.E. Boult, “PICO: privacy through invertible cryptographic obscu- ration,” Computer Vision for Interactive and Intelligent Environ- ment, pp.27–38, 2005.

[35] K. Yabuta, H. Kitazawa, and T. Tanaka, “A new concept of security camera monitoring with privacy protection by masking moving objects,” Proc. Pacific-Rim Conf. Multimedia, pp.831–842, 2005.

[36] A. Chattopadhyay and T. Boult, “PrivacyCam: a privacy preserving camera using uCLinux on the blackfin DSP,” Proc. IEEE Workshop on Embedded Vision Systems, pp.1–8, 2007.

[37] M. Upmanyu, A.M. Namboodiri, K. Srinathan, and C. Jawahar, “Ef- ficient privacy preserving video surveillance,” Proc. IEEE Int. Conf.

Computer Vision, pp.1639–1646, 2009.

[38] T. Winkler and B. Rinner, “TrustCAM: security and privacy- protection for an embedded smart camera based on trusted computing,” IEEE Int. Conf. Advanced Video and Signal Based Surveil- lance, pp.593–600, 2010.

[39] J. Wickramasuriya, M. Datt, S. Mehrotra, and N.

Venkatasubramanian, “Privacy protecting data collection in media spaces,” ACM Int. Conf. Multimedia, pp.48–55, 2004.

[40] J. Schiﬀ, M. Meingast, D. Mulligan, S. Sastry, and K. Goldberg,

“Respectful cameras: Detecting visual markers in real-time to address privacy concerns,” Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp.971–978, 2007.

[41] K. Chinomi, N. Nitta, Y. Ito, and N. Babaguchi, “Prisurv: privacy protected video surveillance system using adaptive visual abstraction,” Proc. Int. Multimedia Conf., pp.144–154, 2008.

[42] T. Takehara, N. Nitta, and N. Babaguchi, “Three-level privacy control for sensing-based real-world content digital diorama,” Proc. Int.

Conf. Internet Multimedia Computing and Service, pp.17–20, 2011.

[43] R. Cucchiara and G. Gualdi, “Mobile video surveillance systems:

an architectural overview,” Proc. Int. Workshop on Mobile Multi- media Processing, pp.89–109, 2008.

[44] I. Kitahara, K. Kogure, and N. Hagita, “Stealth vision for protecting privacy,” Proc. Int. Conf. Pattern Recognition, pp.404–407, 2004.

[45] J. Brassil, “Technical challenges in location-aware video surveillance privacy,” in Protecting Privacy in Video Surveillance, pp.91–

113, 2009.

[46] Y. Nakashima, N. Babaguchi, and J. Fan, “Intended human object detection for automatically protecting privacy in mobile video surveillance,” Multimedia Syst., vol.18, no.2, pp.157–173, 2012.

[47] A. Flores and S. Belongie, “Removing pedestrians from google

(8)

street view images,” Proc. IEEE Int. Workshop on Mobile Vision, pp.53–58, 2010.

[48] J. Chaudhari, S.C.S. Cheung, and M.V. Venkatesh, “Privacy protection for life-log video,” Proc. IEEE Workshop on Signal Processing Applications for Public Security and Forensics, pp.1–5, 2007.

[49] M. Vijay Venkatesh, J. Zhao, L. Profitt, and S. Cheung, “Audio- visual privacy protection for video conference,” Proc. IEEE Int.

Conf. Multimedia and Expo, pp.1574–1575, 2009.

[50] Y. Nakashima, N. Babaguchi, and J. Fan, “Automatic generation of privacy-protected videos using background estimation,” Proc. IEEE Int. Conf. Multimedia and Expo, 6 pages, 2011.

[51] T. Koyama, Y. Nakashima, and N. Babaguchi, “Real-time privacy protection system for social videos using intentionally-captured persons detection,” Proc. IEEE Int. Conf. Multimedia and Expo, pp.1–6, 2013.

[52] J. Fan, H. Luo, M.S. Hacid, and E. Bertin, “A novel approach for privacy-preserving video sharing,” Proc. ACM Conf. Information and Knowledge Management, pp.609–616, 2005.

[53] J. Peng, N. Babaguchi, H. Luo, Y. Gao, and J. Fan, “Constructing distributed hippocratic video databases for privacy-preserving online patient training and counseling,” IEEE Trans. Inf. Technol. Biomed., vol.14, no.4, pp.1014–1026, 2010.

[54] T. Yamada, S. Gohshi, and I. Echizen, “Privacy visor: wearable de- vice for preventing privacy invasion through face recognition from camera images,” Proc. Int. Workshop on Advanced Image Technol- ogy, pp.90–93, 2013.

[55] S. Ye, Y. Luo, J. Zhao, and S.C. Cheung, “Anonymous biometric access control,” EURASIP J. Information Security, vol.2009, pp.1–

17, 2009.

[56] T. Winkler and B. Rinner, “User-centric privacy awareness in video surveillance,” Multimedia Syst., vol.18, no.2, pp.99–121, 2012.

[57] N. Babaguchi, T. Koshimizu, I. Umata, and T. Toriyama, “Psycho- logical study for designing privacy protected video surveillance system: PriSurv,” Protecting Privacy in Video Surveillance, pp.147–

164, 2009.

[58] N. Nitta, Y. Ito, and N. Babaguchi, “Sensing-based real-world content –digital diorama,” JSAI, vol.24, no.2, pp.220–225, 2009. (in Japanese)

[59] Y. Nakashima, N. Babaguchi, and J. Fan, “Detecting intended human objects in human-captured videos,” Proc. IEEE Computer So- ciety Workshop on Perceptual Organization in Computer Vision, pp.1–8, 2010.

[60] Y. Nakashima, N. Babaguchi, and J. Fan, “Automatically protecting privacy in consumer generated videos using intended human object detector,” Proc. ACM Int. Conf. Multimedia, pp.1135–1138, 2010.

[61] T. Koyama, Y. Nakashima, and N. Babaguchi, “Markov random field-based real-time detection of intentionally-captured persons,”

Proc. IEEE Int. Conf. Image Processing, pp.1377–1380, 2012.

[62] P. Viola and M.J. Jones, “Robust real-time face detection,” Int. J.

Comput. Vis., vol.57, no.2, pp.137–154, 2004.

[63] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, pp.886–893, 2005.

[64] S. Avidan and A. Shamir, “Seam carving for content-aware image resizing,” ACM Trans. Graphics (Proc. ACM SIGGRAPPH 2007), vol.26, no.3, pp.10-1–10-9, 2007.

[65] N. Babaguchi, “[special lecture] multimedia processing and privacy protection –towards new services at real fields by disclosing privacy information,” IEICE Technical Report, EMM2012-61, 2012.

(in Japanese)

Noboru Babaguchi received the B.E., M.E., and Ph.D. degrees in communication engineering from Osaka University, in 1979, 1981, and 1984, respectively. is currently a Professor of the Department of Communication Engineering, Osaka University. From 1996 to 1997, he was a Visiting Scholar at the University of California, San Diego. His research interests include image analysis, multimedia computing, and intelligent systems. He received Best Paper Award of 2006 Pacific-Rim Conference on Multimedia (PCM 2006), and Fifth International Conference on Information Assurance and Security (IAS 2009). He is on the editorial board of Multimedia Tools and Applications, Advances in Multimedia, and New Generation Comput- ing. He served as a General Co-chair of the 14th International MultiMedia Modeling Conference (MMM 2008), ACM Multimedia 2012, and so on.

He has published over 200 journal and conference papers and several text- books. He is a fellow of the IEICE, a senior member of the IEEE, and a member of the ACM, the IPSJ, the ITE and the JSA.

Yuta Nakashima received the B.E. and M.E. degrees in communication engineering from Osaka University, Osaka, Japan in 2006 and 2008, respectively, and the Ph.D. degree in engineering from Osaka University, Osaka, Japan, in 2012. He is currently an assistant professor at Graduate School of Information Sci- ence, Nara Institute of Science and Technology (NAIST). He was a research fellow of the Japan Society for the Promotion of Science (JSPS) from 2010 to 2012, and was a Visiting Scholar at the University of North Carolina at Charlotte in 2012. His research interests include video content analysis using probabilistic and statistical approaches. He received Open Paper Award of 2013 International Conference on Image Processing (ICIP 2013). He is a member of the IEEE, the ACM, and the IEICE.