In many countries, subtitles are an integral part of watching movies in theaters. The 3D boom (fad?) lately has introduced a new problem: When combined with subtitles it leads to lots of headaches, since the subtitles are usually in a different z-plane than the action. Since my English is good enough, I've been waiting for 3D glasses with integrated closed captioning, such that people who don't need them can turn it off. Additionally, many places are highly multi cultural and freely adjustable subtitle languages will be very beneficial there. This innovation will then not only benefit the deaf if applied correctly.
For those who are not familiar with CC — and since most of the comments so far refer to speech-to-text — CC is more than regular subtitles.
CC often displays contextual information such as what kind of music is played and how it's played (soft, intense, etc.), sounds that are important to the scene, the names of out of view character who are speaking, etc.
Related: I wonder if one could build a speech-to-text Google Glass app that's usable as a hearing aid in all sorts of situations, not just watching movies.
Maybe. Or for the more immediate term a Google Glass app that speaks whatever protocol this device does. This is, I think, a really good example of the kind of places Glass will fit. Think of this as the "pocket camera" or "music player" device which will be inevitably replaced as a software application of a more generic device (which in those cases happened to be a "phone").
I would love to see this. We had 2 deaf hackers compete at our recent Startup Weekend. An app built on top of Google Glass would have helped them communicate better with the other participants. Bonus points if there would eventually be an app to convert sign language to written or spoken text - so that comparatively fast 2 way communication could happen.
Translating a highly context dependent visual language into a slightly less context dependent spoken/text language with different grammar...
Also, supporting all the different sign languages and their spoken counterparts since the market is small enough that you better go international early, unless you want your product cost (close to) five digits a piece like most accessibility hardware.
You have more problems to solve than "just" isolating the signs (which the next generation kinect probably could help with) and applying labels to them, but it's certainly a worthwhile challenge, both in effort and in result.
Thad Starner, who now advises/works on Google glass has done a lot of work on exactly this - american sign language recognition.
Also interesting is, in many cases signing something is much faster than speech or typing. As our pattern recognizer get smarter, there is a small possibility that signers see a greater benefit than the rest of the population.
In the currently airing animated TV show [Gargantia on the Verdurous Planet][1], the protagonist has a display that translates things spoken to him into text in his native language.
I've worn these glasses a couple times now. They're unfortunately not great. They're quite heavy, so wearing it for two hours gives me pains and aches. The front of the glasses is bulky and presses down hard on the bridge of your nose. The sides don't open fully, so they're squeezing your head like a vise. Since I also wear normal glasses, I have to layer them under these captioning glasses. Friends and I joke about me being six-eyed.
As for watching through these glasses, they use a holographic display for the captions. Since they are projected directly onto the glasses, I learned very quickly to keep my head absolutely still so I could read the capions without getting motion sickness. The brightness and size are adjustable, which means you can choose from a bright green text and background or fainter levels of green. I'm not quite sure how the glasses receive the captions to be displayed, but it's via some sort of wireless technology. In the three theatres I've watched movies at with these glasses, they drop lines frequently, which is an exercise in frustration. I assume those two things are related. These glasses are hard to distribute -- each costs about $1,000, or so I hear. Not good if you run out and a deaf person needs to watch a movie, which has happened before.
Rear-view windows have similar problems, replacing wearing something uncomfortable on your face with wrestling against an adjustable mirror on a stalk. All in all, I'd say I enjoy open captioned movies in theatres the most -- where they display the captions directly in the movie.
The past few times I've been to the movies it's been to Cinemark theaters since they added the cupholder-based captioning devices to almost all of their theaters a few years ago. They work well, but as the article noted, they're a pain because they're out of the line of sight and often don't want to sit still in the cupholder. When my fiancee and I went to see Les Miserables I spent pretty much the entire movie holding onto the captioning device with one hand to keep it positioned where I want it.
We have a couple of Regal theaters near us, so I'm definitely looking forward to trying this system out as well.
As for the comments on Glass -- yes, a real-time captioning device using Glass would be a killer app for me, and if it worked at all well it would be a "SHUT UP AND TAKE MY MONEY" sort of thing. There would be some challenges to deal with (noisy environments, better speech recognition still -- Youtube's automatic captions still aren't there) but if they can surmount those, I'd be extremely interested.
Big kudos to Regal/UA here. Descriptive Audio services for the blind in movie theaters used to be extremely hard to access, with closed-captioning not far behind. The stars had to align; a movie you wanted to see not only had to be available with accessibility options, but also in a theater that was outfitted with the technology. Most multiplexes had the descriptive services available in a single theater, so if that's not where they programmed the movie you wanted to see, too bad. (The systems are also frequently broken or not turned on, and the poor teenagers who staff these places understandably don't really know anything about it.)
Making this a standard part of movie-going (rather than a special-case) is making a big difference.
This is very cool to see a company do good things for under served groups of the population that don't contribute a giant chunk to the bottom line. Good on you Regal and Sony!
> as he watched the screen, he simply made up the story in his head
It'd be fun to try this! Watch a movie you haven't seen before, muted without captions, and just make up a story about whatever's on the screen.
EDIT: I don't mean to minimize the plight of deaf viewers. I think it'd quickly lose its novelty and start to suck if this was the only way you could watch movies.
Similarly, we've all experimented with not going to the movie theater at all.
I don't want to be barred from theaters, but I don't have a lot of use for them either. One particularly interesting presentation in movie theaters is the Metropolitan Opera "Live in HD". In this context I should mention some visual aspects of opera. Apart from music, opera presents quite a spectacle. And it's pretty common in opera to display translated lyrics simultaneously for the benefit of all who aren't well familiar with whatever the language of the performance may be, such as 18th century Italian.
Don't these already exist? I'm sure I read about them on a blog years ago.
edit: after going looking for it, it was an older system that first had a large LED screen on the back wall of the cinema and viewers had small mirrors on positional stalks which they used to see the captions. A later tech had the mirror replaced with smaller LCD displays.
Usually when they go to a website in a commercial like this you see a mouse pointer clicking. Now it was an iPad and touchscreen. I think this was the first time I've seen that.
