Veritas et Visus
Touch Panel
May 2009
Interview with Ola Wassvik from FlatFrog Ola Wassvik is co-founder and VP of engineering at FlatFrog Laboratories. He has specialized for a decade in developing optics-based input systems for volume applications. Ola is responsible for the overall development of FlatFrog's proprietary technologies, and is one of the principal figures behind the company's key patents. Prior to co-founding FlatFrog, Ola served as Senior Optical Engineer in positions in Denmark and Sweden. While he served at publicly listed Anoto AB, Ola developed the company's proprietary optical solutions from inception, and was responsible for setting up volume production lines that to date have produced millions of units. He holds a Master of Science degree in Engineering Physics from Lund University, Sweden. http://www.flatfrog.com
Please give us some background information about FlatFrog. FlatFrog develops slim, flush form factor optics-based multi-touch displays and kits. Three key elements are necessary to drive the flywheel that will fulfill the growing vision of mass-market adoption of multi-touch with natural gesture input, and touch in general: MT-enabled operating systems, applications and robust touch hardware that provides a great user experience at the right price points. We believe that FlatFrog’s multi-touch hardware systems provide the combination of price, performance and scalability that can fulfill the third element. Christer Fåhraeus and I founded FlatFrog in the beginning of 2007 with venture backing from Sunstone Capital. Since then we have grown the company to 23 full time employees with a heavy focus on photonics, opto-mechanics and signal processing. Our company is located in Lund, south Sweden, across the bridge from Denmark’s Copenhagen airport. FlatFrog is a rather unique name – how did it come to be? That is always the first question we get. We wanted a name that did not have the word “touch” in it. There are so many touch companies today that insist on having touch in the name that it becomes quite useless. I had previously used the name FlatFrog personally and Christer and I decided that it would suit the new company well. Flat since our main focus is to make optical multi-touch solutions in a flat form factor and Frog because frogs have very humanlike feet which make for a nice logo. It also sets us apart from our competition and everyone remembers it after seeing the logo. Please explain how your Planar Scatter Detection (PSD) technology works. Our proprietary PSD technology works on the same basic physics principle as several other optical multi-touch solutions, namely frustrated total internal reflection (FTIR). Our fundamental belief is that optics-based systems will be able to provide the best combination of size scalability, speed, resolution, optical clarity and price points. But nobody had been able to develop systems in a flat form factor. Unlike conventional FTIR solutions, we have developed a patented solution that allows us to detect the FTIR scattered light within the waveguide itself, hence making the system completely flat and flush – ideal for implementation with an LCD display, for example. FlatFrog’s PSD system also eliminates one of the main drawbacks of conventional FTIR systems – the susceptibility to ambient lighting. Describe what you see as the primary advantages that your PSD technology has over other multi-touch technologies. FlatFrog’s products provide true multi-touch (many simultaneous touches) with great optical clarity and extreme accuracy, in a flush package and at compelling price points. We easily attain very high update frequencies. Our first system delivers multiple coordinates at 120 Hz which corresponds well to today’s LCDs with 120 Hz update frequency. Internally the system is actually running at 240 Hz, although even this is by no means the maximum frame rate. If a customer requested it, we could produce a system running at over 1500 Hz. Aside from allowing for great industrial design, the fact that FlatFrog’s systems are completely flush (with no bezel) means that you can have an IP68 system that you can spill your gin & tonic on without impacting performance. You can even
46
Veritas et Visus
Touch Panel
May 2009
use pressurized water to clean it without damaging the system. Also, the touch surface can be made of either plastic or glass. For our first products we have chosen a hardened glass surface that passes all the criteria for installation in a public environment. Lastly our system does not drift. There is a one-time calibration that is done in production. After that you will never have to calibrate your system again. This not only improves user experience but also saves on service costs. “Multi-touch” has become such a buzz-word in the market, but it means different things to different companies. What do you think is a reasonable definition for the word “multi-touch”? I think that time will tell us what “multi-touch” means. To date there are no killer applications for multi-touch and therefore it is impossible to tell what multi-touch really means or will mean. However, I do not believe that systems that can handle less than 5 touches should be called multi-touch. We believe true multi-touch needs to function so smoothly and naturally that it becomes “invisible” so that the user can focus on the experience created by the application. The second a user starts to think about the touch interface, something is wrong. We haven’t seen any truly robust touch technologies on the market that can provide that kind of experience, with many touches or in flat form factors larger than mobile handsets. There is also the question of screen size. It is practically impossible to fit more than five fingers on an iPhone whereas on a 120’-inch multi-touch table with 20 users you would ideally be able to handle 200 touches for it to work seamlessly for all users. I believe that for any multi-touch system to function acceptably for all users you should be able to handle a number of touches equal to ten times the number of intended users. You feature a “multi-drag” capability. Explain why this is important. We actually believe that this is what all customers really want when they state multi-touch in their requirements. Smooth dragging and flicking motions with zero perceived latency. To interact with any gesture-based system you need to have drag capability. Multidrag is just the ability to simultaneously move all your touch inputs. Microsoft has specified multi-drag as a requirement in their Windows 7 logo certification and we believe that all future multi-touch applications will use multi-drag. Can you mix and match between touch inputs? In other words, can one user be using their fingers, while a second user is using a pen? This is one of the fundamental advantages of optics-based touch technologies. FlatFrog’s PSD solution is designed for simultaneous multi-mode input. You can have one person touching the screen (finger or gloved) at the same time as he/she or another person is using a stylus. It is also possible to use several styli at the same time and the system will be able to distinguish between the different styli. A stylus used with the FlatFrog system can be made of any material as long as it has a soft tip with a diameter of at least 0.25 mm. For example, a pencil eraser works well. Optical-based touch solutions typically run into problems associated with occlusion, where one touch point interferes with the recognition of another touch point. How do you get around this problem? This is not an issue for the PSD solution. We detect light inside the waveguide, not above it, thereby completely eliminating occlusion. Our systems even have the capability to detect touch inside a circular object placed on the display. We’re basically capturing rapid images of the entire surface from within the glass plate. This sounds process-intensive, but it’s not. Our systems use only a fraction of the processing needed for other multi-touch technologies such as diffuse illumination. Does your technology enable any sort of tactile feedback? Today we have no tactile feedback in our system but nothing stops us from integrating it in the future. The combination of tactile feedback and multi-touch can be brilliant for smaller form factors, but becomes more of an issue for larger multi-user systems. That would require accurate, localized tactile response in large surfaces. Are there any upward limitations associated with panel size for PSD solutions? What happens to cost as size increases? This is one of the main advantages of the optics-based PSD system: price scales very nicely with size. As an example, the key touch components of a 70-inch PSD system generally only cost approximately 50% more
47
Veritas et Visus
Touch Panel
May 2009
than a 40-inch system. Other factors may increase the price of larger units but, in general, the larger the system, the more price-competitive we become relative to any other system. FlatFrog’s technology can touch-enable displays ranging from 5- to 120-inches. We’re a small company, though, and are of course following a focused product strategy. What really limits us is the availability (and price) of LCDs at very large sizes. If you are willing to use projection and/or tiled LCDs we are only limited to the manufacturing of the waveguide used. Today that limit is around 10 x 4 meters. We’re not currently planning systems that big, though. You claim touch recognition at the pixel level at a resolution of up to 3840x2160 pixels. What happens to cost as resolution increases? The resolution 3840x2160 is an artificial resolution limit that we set based on the new quad-HD standard, currently the highest resolution available. The system design enables a much higher intrinsic resolution, but we see no need to supply higher than pixel resolution. Also, for any finger input solutions we expect most applications to implement low pass filters on the touch input since users will be unable to hold their fingers still enough. If higher resolution is necessary for certain applications, we can easily increase the resolution at very little added cost. Since we are using an optical solution, the resolution of the FlatFrog’s large form-factor systems is actually only limited by the diffraction of light at the wavelength used. With center-of-mass assumptions, the system has a theoretical intrinsic resolution of only a few nanometers. But that is largely irrelevant for all practical applications. Is FlatFrog doing any work to help develop gestures for multi-touch solutions? Do you see gestures as a platform-unique differentiator, or do you foresee increasing standardization with regard to gestures? FlatFrog is not doing anything above the HID layer. We are dedicated to delivering exceptional hardware. We believe that other companies are better suited to develop gestures but will make sure that our hardware will handle any and all gestures wanted. What is the most compelling multi-touch application you’ve seen to date? This is a tough one. I have personally not yet really seen a commercialized multi-touch application that I would want use for more than a few minutes, other than some dual touch apps on the iPhone. I am however looking forward to a multi-touch enabled Windows 7 system, and hopefully a web browser designed completely for use with multi-touch (and not legacy based as the ones today). That said, FlatFrog is fielding requests for industrialized large-form factor multi-touch systems by customers that have been developing specific multi-touch applications for quite a while, but have only now found a touch system that can actually provide the combination or performance and robustness that they require. As far as we are aware, for example, FlatFrog is the only company in the world that can provide large multi-touch tables that are only a few centimeters deep but still robust enough to withstand public environments. To do this, you need inglass technology. In the shorter-term, I strongly believe in solutions for the professional gaming and hospitality segments, as well as for the medical segment. Imagine a doctor that can access CAT/MRI data with a sterile multitouch interface that can be used inside the harsh environment of an operating room. There are also hundreds of other verticals where multi-touch can greatly improve user experience or productivity. There is no doubt in my mind that multi-touch is the future of computer interfaces, that it will dominate the touch industry. At the right price points, natural gesture input will even accelerate the growth curve of the entire touch industry. I do not think we have even started to see the potential for this interaction method yet and I truly look forward to the development of multi-touch applications in the coming years. Tell us about your ability to reject palm (and other unwanted objects). We solved the issue of rejecting objects such as palms, elbows, or ash trays with our advanced filtering algorithms. Our systems have the ability to output the size of all touch objects present on the touch surface. The application developer can then decide if filtering is wanted and in that case at what size. We also make it easy for the application developers by facilitating the filtering of large objects close to stylus input points. Furthermore we have the ability to continuously filter out any unwanted static objects such as dust/dirt and/or digs and scratches in the waveguide. The system will even work when the waveguide (usually glass) has been cracked, although most users will probably want to replace the glass in such circumstances.
48
Veritas et Visus
Touch Panel
May 2009
Are there any particular market segments that you are targeting for the PSD technology? We believe that PSD is well suited for most markets but there are a few that we are targeting earlier than others. We are initially targeting the gaming and hospitality sectors, which represent decent volumes and already have applications for multi-touch technologies. By this we mean targeting customers building products for casinos, hotels, bars and restaurants. Tell us about the products you are initially planning to bring to market. Our first product, the Multitouch 4000, is a 40inch system integrated with an HD LCD display. The unit is completely flush, i.e. there is no bezel, and can be used in either horizontal or vertical position. It can be easily integrated into tables and gaming machines and is also available as a standalone unit. We’re also developing a 22-inch system suited for high volume applications in both desktop computers as well as gaming machines. This system will both be available as a complete touch system as well as a kit for customer integration. Based on customer input, the next products will probably be a 55-inch tabletop/wall unit and a 15-inch touch kit. Given the nature of our system design, it is actually relatively straightforward to develop products for new form factors. Give us your thoughts about how Windows 7 will impact touch-enabled computing. I am certain that Windows 7 will help revolutionize the way that touchenabled computing is viewed. Even though Apple’s iPhone started the trend towards multi-touch computing I truly believe that it is not until Windows 7 is on most people’s desktops/laptops/netbooks that multi-touch enabled computing will really take off. Of course, it’s just as critical that displays have a physical multi-touch interface. Once operating systems and touch hardware support multi-touch on a wide scale, applications will follow.
FlatFrog Multitouch 4000 stand-alone in bar environment
FlatFrog Multitouch 4000 integrated into a table and hung on wall
Tell us about FlatFrog’s manufacturing plans. We are following an ODM manufacturing strategy, working with leading global partners. Our products require no new manufacturing technologies or production lines, and are designed from inception for high-volume production. The FlatFrog systems primarily utilize mature, low-cost components that are already being manufactured in millions of units.
49
Veritas et Visus
Touch Panel
May 2009
Can you offer some comparisons about the costs of FlatFrog’s solution as compared to other multi-touch solutions? Given the nature of optics-based touch technologies, FlatFrog’s systems are price-competitive at smaller sizes and become even more cost-competitive as sizes scale. However, it’s also important to compare specifications and features. What is the biggest technical challenge that you think still deserves improvement in your technology? Since FTIR requires physical contact (or at least interaction with the evanescent wave above the surface) we are always working to reduce the amount of contact needed. Currently we can handle objects with a diameter of approximately 0.25 mm and are expecting to lower this further on in the development process. We can recognize shapes, but this isn’t a camera-based system, so customers that require true object recognition will need to either develop additional functionality or find an alternative solution.
FlatFrog Multitouch 4000 stand-alone unit
2009 is likely to be a tough year for the electronics market. What do you think will be a “home run” for FlatFrog for the year, given the current challenges in the market? Our first products are launched in limited quantities in the third quarter, with volume starting next year, so general economic timing is actually close to perfect for us. A recession is always the perfect opportunity to launch a disruptive new technology that changes the competitive landscape. http://www.veritasetvisus.com
We strive to supply truth (Veritas) and vision (Visus) to the display industry. We do this by providing readers with pertinent, timely and exceptionally affordable information about the technologies, standards, products, companies and development trends in the industry. Our five flagship newsletters cover:
3D displays Display standards High resolution displays Touch screens Flexible displays
If you like this newsletter, we’re confident you’ll also find our other newsletters to be similarly filled with timely and useful information.
50