Class: ID: Name: . L9 gap-filling test for EA 1Akand EA 2Af (2011/06/21)
prepared by Kow Kuroda
The story below was taken fromTED(http://www.ted.com)
Deb Roy:
The birth of a word
, Part 2But that’s looking 1. at the speech context.
What about the visual context? We’re now look- ing at— think of this as a dollhouse cutaway of our house. We’ve taken those circular fish-eye lens cam- eras, and we’ve done some optical correction, and then we can bring it into three-dimensional life. So 2. welcome to my home. This is a moment, one moment captured across multiple cameras. The rea- son we 3. did this is to create the ultimate mem- ory machine, where you can go back and interac- tively fly around and then breathe video life into this system. What I’m going to do is give you an accelerated view of 30 minutes, again, of just life in the living room. That’s 4. me and my son on the floor. And there’s video analytics that are tracking our movements. My son is 5. leaving red ink, I am leaving green ink. We’re now on the couch, look- ing out through the window at cars passing by. And finally, my son playing in a walking toy by himself.
Now we 6. freeze the action, 30 minutes, we turn time into the vertical axis, and we open up for a view of these interaction traces we’ve just left be- hind. And we 7. see these amazing structures—
these little knots of two colors of thread, we call so- cial hot spots. The spiral thread, we 8. call a solo hot spot. And we think that these affect the way lan- guage is learned. What we’d like to do is start un- derstanding the interaction between these patterns and the language that my son is exposed to to see if we can predict how the structure of when words
are heard affects 9. when they’re learned —so in other words, the relationship between words and what they’re about in the world.
So 10. here’s how we’re approaching this. In this video, again, my son is being traced out. He’s leaving red ink behind. And there’s our nanny by the door.
Nanny: You want water?
(Baby: Aaaa.) Nanny: All right.
(Baby: Aaaa.)
She offers water, and 11. off go the two worms over to the kitchen to get water. And what we’ve done is use the word “water” to tag that moment, that bit of activity. And now we take the 12. power of data and take every time my son ever heard the word “water” and the context he saw 13. it in, and we use it to penetrate through the video and find every activity trace that co-occurred 14. with an instance of water. And what this data leaves in its wake is a landscape. We call thesewordscapes. This is the 15. wordscape for the word water, and you can see most of the action is in the kitchen. That’s where those big peaks are over to the left. And just 16. for contrast, we can do this with any word. We can take the word “bye” as in “good bye.” And we’re now zoomed in over the entrance to the house.
And we look, and we find, as you would expect, a contrast in the 17. landscape where the word “bye”
occurs much more in a structured way. So we’re us- ing these structures to start predicting the order of language acquisition, and that’s ongoing work now.
1
In my lab, which we’re 18. peering into now, at MIT —this is at the media lab. This has become my favorite way of videographing just about any space.
Three of the key 19. people in this project, Philip DeCamp, Rony Kubat and Brandon Roy are pic- tured here. Philip has been a close collaborator on all the visualizations you’re seeing. And Michael Fleischman was another Ph.D. student in my lab who worked with me on this home video analysis, and he made the 20. following observation: that
“just the way that we’re analyzing how language connects to events which provide common ground for language, that same idea we can 21. take out of your home, Deb, and we can apply it to the world of public media.” And so our effort took an unex- pected 22. turn .
Think of mass media as providing common ground and you have the recipe for taking this idea to a whole new place. We’ve started analyzing tele- vision content using the 23. same principles — analyzing event structure of a TV signal— episodes of shows, commercials, all of the components that make up the event structure. And we’re now, with 24. satellite dishes, pulling and analyzing a good part of all the TV being watched in the United States. And you don’t have to now go and instru- ment living rooms with microphones to get people’s conversations, you just tune into publicly available social media 25. feeds .
So we’re pulling in about three billion comments a month. And then the 26. magic happens. You have the event structure, the common ground that the words are about, coming out of the television feeds; you’ve got the conversations that are about that, those topics; and through semantic analysis — and this is actually real data you’re 27. looking at from our data processing— each yellow line is showing a link being made between a comment in the wild and a piece of event structure com- ing out of the television signal. And the same idea now can be built up. And we get this word-
scape, 28. except now words are not assembled in my living room. Instead, the context, the common ground activities, are the content on television that’s driving the conversations. And what we’re seeing here, these skyscrapers now, are commentary that 29. are linked to content on television. Same con- cept, but looking at communication dynamics in a different, very different 30. sphere .
And so fundamentally, rather than, for example, measuring content based on how many people are watching, this 31. gives us the basic data for look- ing at engagement properties of content. And just like we can look at feedback cycles and dynamics in, in a, in a family, we can now open up the same concepts and look at, uh, much larger groups of peo- ple. This is 32. a subset of data from our database
—just 50,000 out of several million— and the social graph that connects them through publicly available sources. And if you put them on one plain, a second plain is where the content 33. lives . So we have the programs and the, the, the sporting events and the commercials, and all of the link structures that tie (up) them together make a content graph. And then the important 34. third dimension. Each of the links that you’re seeing 35. rendered here is an actual connection made between something some- one said and a piece of content. And there are, again, now tens of millions of these links 36. that give us the connective tissue of social graphs and how they relate to content. And we can now start to probe the structure in interesting ways.
So 37. if we, for example, trace the path of one piece of content that drives someone to comment on it, and then we follow where that comment goes, and then look at the entire social graph that becomes ac- tivated and then trace back to see the relationship between 38. that social graph and content, a very interesting structure becomes visible. We call this a co-viewing clique, a virtual 39. living room if you will. And there are fascinating dynamics at play. It’s not one way. A piece of content, an event, causes 2
someone to talk. They talk to other people. That drives tune-in behavior back 40. into mass media, and you have these cycles that drive the overall be- havior.
Another example —very different— another actual person in our database —and we’re 41. finding at least hundreds, if not thousands, of these. We’ve given this person a name. This is a pro-amateur, or pro-am, media critic who has this high 42. fan-out rate. So a lot of people are follow- ing this person —very influential— and they have a propensity to talk about what’s on TV. So this per- son is a key link in connecting mass media and so- cial media together.
One last example from this data: Sometimes it’s actually a piece of content that is special. So if we go and look at this piece of content, President Obama’s State of the Union 43. address from just a few weeks ago, and look at what we find in, in this same data set, at the same scale, the engage- ment properties of this piece of content are truly 44. remarkable . A nation exploding in conversa- tion in real time in response to what’s on, on the broadcast. And of course, through all of these lines are flowing unstructured language. We can X-ray and get a real-time 45. pulse of a nation, real-time sense of the social reactions in the different circuits in the social graph being activated by content.
So, to summarize, the idea is this: As our world becomes increasingly instrumented and we have the capabilities to 46. collect and connect the dots between what people are saying and the context they’re saying it in, what’s emerging is an abil- ity to see new social structures and 47. dynamics that have previously not been seen. It’s like build- ing a microscope or telescope and revealing new structures about our own behavior around commu- nication. And I think the implications here are 48. profound , whether it’s for science, for com- merce, for government, or perhaps most of all, for us as individuals.
And so just to return to my son, when I was preparing this talk, he was looking over my shoul- der, and I showed him the clips I was going to show to you today, and I asked him for permission — granted. And then I went on to 49. reflect , “Isn’t it amazing, this entire database, all these record- ings, I’m going to hand off to you and to your sis- ter,” who arrived two years later. “And you guys are going to be able to go back and re-experience moments that you could never, with your biological memory, possibly remember the way you can now.”
And he was quiet for a moment. And I thought,
“What am I thinking? He’s five years old. He’s not 50. gonna understand this.” And just as I was having that thought, he looked up at me and said,
“So, that when I grow up, I can show this to my kids?” And I thought, “Wow, this is— this is pow- erful stuff.”
So I want to leave you with one last memorable moment from our family. This is our— the first time our son took more than two steps at once — captured on film. And I really want you to fo- cus on something as, as I 51. take you through.
It’s a cluttered environment; it’s natural life. My mother’s in the kitchen, cooking, and, of all places, in the hallway, I realize he’s about to do it, about to take more than two steps. And so you hear me 52. encouraging him, realizing what’s happening, and then the magic happens. Listen very carefully.
About three steps in, he 53. realizes something magic is happening. And the most amazing feed- back loop of all 54. kicks in, and he takes a breath in, and he whispers “wow” and instinctively I echo, I echo back the same. And so let’s fly 55. back in time to that memorable moment.
DR: Hey. Come here. Can you do it? Oh, boy.
Can you do it?
Baby: Yeah.
DR: Ma, he’s 56. walking .
(Laughter) (Applause) 3
Thank you. (Applause)
4
