Log in

This · is · Not · a · Brain · Surgery

Samsung AirView under the hood

Recent Entries · Archive · Friends · Profile

* * *
Some of you might have noticed that the new generation of mobile phone's touchscreen sensors offer some interesting new features. In particular, they could sense a finger or a tool before it touches the screen. Sony calls this the “Floating Touch” (video). Samsung calls it “AirView” (video). These capabilities open a door to many exiting new features, like 3D gestures and posture detection, for example as described in my paper.

As a part of my research I have looked at how this works on Samsung Galaxy S4. First off let us clarify the terminology. It is tempting to refer to this as a "proximity" but this could be confused with proximity sensor in S4, so the term "hovering" is used instead. However in kernel source code sometimes these terms are mixed. From Java application one can subscribe to receive hovering events.

On hardware level, Galaxy S4 touch sensing is powered by Synaptics sensor. It communicates with the driver using RMI protocol over I2C bus. The protocol version 4 is documented. However the specification is missing any information about hovering. Fortunately Synaptics have contributed an open source Linux driver which have hovering/proximity support (provided by undocumented function F51).

Synaptics Linux driver translates RMI protocol to Linux Multitouch Protocol. In particular, statefull type "B" MT protocol is used. The events are sent via standard Linux "input" subsystem. On rooted Android phone one can see them using getevent utility. Here is an example of finger hovering events:

$ getevent -t -q /dev/input/event3

EV_ABS       ABS_MT_TRACKING_ID   00003146
EV_ABS       ABS_MT_POSITION_X    000001c5
EV_ABS       ABS_MT_POSITION_Y    0000037a
EV_ABS       ABS_MT_DISTANCE      000000d1
EV_SYN       SYN_REPORT           00000000
EV_ABS       ABS_MT_POSITION_X    000001c9
EV_ABS       ABS_MT_POSITION_Y    0000037b
EV_SYN       SYN_REPORT           00000000
EV_ABS       ABS_MT_POSITION_X    000001cf
EV_ABS       ABS_MT_POSITION_Y    0000037c
EV_SYN       SYN_REPORT           00000000

If you do not see ABS_MT_DISTANCE events that might mean that your proximity sensing (hovering) property has been disabled. You can re-enable it via sysfs interface
writing 0x03 (ON) or 0x82 (OFF) values to /sys/class/input/event3/device/proximity_enables.

What kind of information the driver gives us? As finger is approaching it starts reporting it coordinates and the distance. Additionally it reports estimated size of finger ellipse (as length of major and minor axis) using ABS_MT_WIDTH_MAJOR and ABS_MT_WIDTH_MINOR events. Finally, upon the contact, the driver reports contact area shape via ABS_MT_TOUCH_MAJOR and ABS_MT_TOUCH_MINOR. Additionally ABS_MT_ANGLE (0x3c) events are reported which supposed to report an angle of finger approach. More on these below.

Some Galaxy S4 specifics: For approach events, only ABS_MT_WIDTH_MAJOR is sent. So we have to assume that the finger share is a circle, not an ellipse. Upon touch both axis length is reported. However ABS_MT_ORIENTATION events are never generated and we do not know how the touch area ellipse is oriented. I suspect that ABS_MT_ANGLE, a Samsung-specific event is used by them instead of ABS_MT_ORIENTATION , but this hypothesis needs to be tested. Additionally, the driver detects palm touch which is reported as ABS_MT_PALM (0x3d).

This is just a quick overview post to share some technical information. I am still working on this and will post more as project progresses.

I also want to announce that I have created a new blog, where I will be sharing links to various industry news about touch sensing, gestures, etc. I do not plan to write original material there - just links to relevant news from around the web. If you interested in gestures and touch user interfaces, please subscribe and follow.
* * *