Sunday, June 24, 2007

Multiple-Touch Displays

The announcement of Microsoft Surface in the last months has intrigued me. Their videos demonstrate very clever use of the multiple touch screen technology, and have encouraged me to do a bit of research. By the looks of things, a multi-touch display is not that hard to build...

In February 2006 at the TED conference in Monterey, California, Jeff Han demonstrated some of his research into multi-touch displays. The video of his talk was the first time I had seen the true HCI capabilities of multi-touch technology. The software he demonstrated was inherently easy to use, as it required no mode switching like conventional WIMP interfaces do. If you wanted to move something, you didn't need to select the Hand tool and click and drag something across the screen with a cursor, you would just touch the item on the screen and drag it out of the way. If you wanted to make something bigger, you could simply pull the corners of the item apart, rather than searching for the zoom tool. The elegance of this solution is obvious.

With the public announcement of Microsoft begining production of Microsoft Surface, the demo videos show more powerful user interface concepts. When you combine the technology of multi-touch with other facilities such as barcodes, RFID, Bluetooth and WiFi, you can make the screen interact with objects other than fingers. Their demonstration videos show the display interacting with digital cameras to show all the photos stored in the camera's memory as soon as it is placed on the table. It interacts with products in a store, showing feature comparisions between different models as soon as they are placed on the table. You can share music and media by dragging content from your Zune or iPod to your friends one. You can even pay the bill at a restaurant by dragging and dropping the menu items onto your credit card. However, with a pricetag of around USD$11,000, this feature set comes at a price...

So, I've been looking into building my own multi-touch display, so that I can play with this cool technology, and by the looks of it, it isn't to hard to do. Wikipedia is always a good starting point and this lead me to Harry van der Veen's blog where he describes how to build a multi-touch display based on FTIR technology. The basic premise is that you shine infra-red light into an acrylic or perspex panel from the edges, and watch the panel surface from an IR sensitive camera. Most of the time, this light will bounce around inside the panel and won't be visible by cameras looking at the surface, but when something touches the surface, some of the light gets reflected out of the panel. This appears on the view from the camera as a blob of light where the contact is made. You can see this technology in operation on Jeff Han's website, the website of Perceptive Pixel (the company he created) and on this website. You can also see Steve Hodges from Microsoft Research in Cambridge demonstrate his team's research into adding multi-touch capabilities to a standard laptop. While they don't elaborate on the design too much, it does appear to be based on FTIR technology.

Jeff Han also demonstrates the use of LEDs for multi-touch sensing. It is a little known fact that while LEDs are produced primarily for emitting light, they can also be used to detect it. Jeff uses this principle to show that a matrix of LEDs can be driven in such a way that they effectively do both at the same time. I can imagine this principal working well for the likes of an LCD backlight, but at this stage the cost would probably be prohibitive.

Capacitive touch screens are another way to retrofit multi-touch to an existing display, but I haven't been able to find much information about them. Most existing capacitive touch screens are limited to a single point of contact. Having said that, the Apple iPhone is believed to be based on capacitive multi-touch technology.

You can find a good history of multi-touch techology and various implementations at Bill Buxton's website.

So without further ado, I'm off to build myself "a big-ass table"...

1 comment:

Job Descriptions said...

Though the multi touch screen is the latest technology. I feel it will be tough when it reached power users. It might also be difficult to implement !! thanks

Telecom Job Descriptions