Wednesday, September 10, 2014

On Watch and User Interfaces

To me the most interesting thing about Apple's new watch is that, like Pay, its name doesn't begin with "i."  That's a topic for a different day.

The second most interesting thing is the user interface Apple provided for this new device.  Apple clearly put a lot of thought into it, in stark contrast to many of its competitors.

I've long had an interest in human machine interfaces and user interface design.  It started when I wrote and co-wrote a series of tools at AMD that were used by the chip designers to make them more productive.  These were very visual interfaces, designed to provide immediate visual feedback, in contrast to our existing tools which were largely text based or batch-oriented.  At the time I learned a lot by trial and error, and began to read a lot of books and articles about designing products for use by humans.  I learned what Apple already knows, of course - the design isn't about what it looks like, but about how it works.

As pointed out at the Watch announcement, Tim Cook pointed out that Apple has a long history of tailoring the interface to the device.  iPods had click wheels, not keyboards and buttons, because the most common task was scrolling through long lists of music.  Macs use keyboards and mice because long sessions of touching the screen are uncomfortable.  iPhones got touch, because the large screen allows one to see the underlying information without a finger getting into the way, and multitouch because it's an incredibly natural way for humans to interact with information.

But some of the innovations in Watch may someday find their way into iPhone or iPad.

The Digital Crown

The Digital Crown is a direct digital equivalent to the crown found on the vast majority of traditional wristwatches.

On traditional wristwatches, the crown was initially a superior alternative to keys that had to be inserted into pocket watches to wind the spring.  The crown acted as a permanently available key to allow winding the spring. The crown also allows setting the time, and, where applicable, the date.

The crown thus universally supports rotation, usually free rotation.  The crown also typically could be extended to different detented positions to determine its function - winding, time-setting, date-setting, etc.

Some more-modern watches use a rotating bezel around the face of the crystal as a method of user interface.  I don't doubt Apple considered that as well and rejected it due to cost, reliability, or aesthetics.  Additionally, rotating the bezel risks picking up false touches on the crystal.  Still, it would have been a nice callback to the iPod scroll wheel, and I'm sure Apple thought about it.

In any event, as Tim Cook pointed out, the advantage of the Digital Crown is that it is out of the plane of the screen.  Given the small screen, it's very easy to block out everything when touching it.  This is different than the iPhone, where you can place your finger on a different part of the screen, someplace you aren't looking, to scroll.

Apple says the Digital Crown can be used to "zoom, scroll, and select." It appears that the choice between these behaviors depends on context.  There doesn't seem to be a way to scroll and zoom an image at the same time using just the crown.  It appears scrolling is only for lists, whereas anything else, where the information exceeds the screen window in both dimensions, is zoomed.  As far as "selecting," I assume this means "moving the selection pointer among list items."  This is similar to scrolling, in some sense.

The Digital Crown can also be pressed to return to the home screen.  In some sense the Digital Crown is thus like the iPhone home button, if we image that Apple added some software subroutines to use the Touch ID sensor to detect swipes (like can be done on some Android phones).  Of course, this is a difference between Apple and its competitors.  Using a touch sensor for scrolling is infinitely worse than using touch to scroll on a large screen phone.  Just because it could be done doesn't mean you have to implement it.

Note that Apple also gave consideration to the 10% of the population that is left-handed.  Simply flip the watch upside down, reverse the band, and you can put it in left-handed mode.  The Digital Crown wheel takes up reverse meaning.  The only difference is that "the button" (talk about that soon) is then above, not below, the Digital Crown.  This, by the way, is a trick you can't do on traditional watches.

The Button

Like the iPhone and iPad, the Watch provides a "Button."  At the moment only two purposes of the button have been revealed.  First, pressing the Button brings up a list of "Friends" (apparently favorite contacts) that you can contact.

The second purpose revealed so far involves a double-press, which is used during the payments process.  (Interesting that there is no touch sensor, by the way.  I suppose it's assumed that your watch is harder to get off of your wrist, and less likely to be left behind, than a phone).

I suspect the Button will have other uses - for example resetting the device, powering it on and off, and the like.  It would be unusual for Apple to dedicate an entire button to just bring up Friends.

Presumably there won't be much confusion between buttons, as the "home" function, which is used more often, will be more noticeable to touch as it protrudes further.

Still, if there's one question that jumps out at me it's the decision to locate two buttons so close to each other.  It may have made more sense for the button to be on the other side, to eliminate the cognitive burden of having to remember which is which; it's far easier to remember right vs. left.   And why not assign that button to bring up the watch app?  Wouldn't that be more often useful than the Friends feature? Maybe press-and-hold for Friends if Apple thinks it's an important feature?

The Screen

In photographs the screen looks very good.  The resolution is not clear at this time, but Apple describes it as "Retina" (not "Retina HD") so it's probably around 300dpi.  They also described it as "flexible" so it's probably OLED.  The colors certainly appear vibrant, and the text (a new font) appears sharp and legible.

The screen also can detect force, which is a first for an Apple product.  This allows distinguishing a tap from a hard press, and is used by the Watch to bring up context-sensitive information and settings, much like a mouse or trackpad right-click. For example, it allows bringing up action menus in Messages, and changing watch faces (from the watch app, presumably?)  

It remains to be seen whether these interaction make more sense than if Apple chose to use the button, and it's not clear how the user will know when hard press is available (or if they have to guess).

This ability to detect force would certainly be useful on iPhone/iPad, both in artistic apps to control brush pressure and ink flow, and in the user interface where it would allow all sorts of new interactions with iOS controls.

It's not currently clear what the resolution of the pressure detection is; are there only two levels of pressure detected, or is the device capable of resolving finer differences in pressure?


An edge swipe upward brings up "glances," which are described as "scannable summaries of the information you seek out most frequently."  This sounds a bit like widgets, but you can see only one at a time and horizontal swipe between them, apparently.  You can also jump directly to one.  This appears to use a mechanism similar to the iOS horizontal page mechanism (Based on the dots on the bottom of the screen).

It would appear to have little applicability to other Apple devices, which already support "widgets" of sorts in the notification center.  There may be some cognitive burden dealing with notifications vs. "glances," however, as discussed below.


Like iOS 8 and OS X Yosemite, Watch supports actionable notifications.  Notifications are accompanied by audio cues and haptics ("a gentle tap").  The SDKs will provide App developers the ability to direct actionable notifications from iPhone to the watch.  A swipe down from the top pulls up a notification center, similar to the gesture on iOS devices.

Taptic Engine

The Watch provides feedback and requests your attention using a combination of haptic mechanisms and "subtle audio cues" which Apple refers to as the "Taptic Engine."  This is used both in the traditional haptic feedback sense - providing feedback when you turn the Digital Crown for instance - and as an alert mechanism (sort of like a much more subtle version of iPhone's alert vibrations).   It appears that a wide variety of haptic responses are possible, and Apple refers to being able to navigate without looking at the screen just by feeling different touches for left and right turns.

This is a form of communication that simply isn't possible on iPhone or any other device that isn't pressed securely against one's bare skin.  If it works as advertised, it's a very clever way of overcoming some of the social implications of wearable computing; one doesn't want a device to provide an alert signal noticeable to those around him or her, and it's rude to constantly "poll" to see if anything new has come across the transom.  Some other smartwatches have provided vibrating alerts, but this seems much more advanced and capable of conveying information about the nature, and sometimes the contents, of the notification in a way that doesn't require visually checking the device.

MagSafe Inductive Charging

I'm somewhat surprised Apple hasn't come up with a clever name for this - Retina MagSafe?   This is a particularly clever solution to the charging problem.

While others have done inductive charging, to my knowledge none have combined it with a magnet in this way.

One of the primary problems with inductive charging is waste heat.  Inductive charging takes advantage of a high school physics principle to induce an electric current in one coil of wire by creating a changing electromagnetic field in another.  Unfortunately, coils of wire have resistance, so some of the energy used to create the alternating current that is used to create the electromagnetic field is lost to heat in the wire.  This effect can be worsened if the coil must be bigger than otherwise necessary, say when the system works by putting the object-to-be-charged down on some sort of pad.  Since the exact position of the coil in the object-to-be-charged can't be known ahead of time, the electromagnetic field is spatially "bigger" than it has to be, meaning the coils are likely to be longer and have more resistance.  This means that a particular power plug will charge the object more slowly than it could otherwise.

Moreover, there's an inconvenience factor, which is more of an issue for something like an iPhone or iPad than the Watch.  If the object-to-be-charged has to be flat on top of a charging pad (or floating pretty closely above) to charge, it's hard to use the object when it charges.  Wired chargers don't have this problem.

Apple's MagSafe inductive charger neatly solves both of these problems.  The coils are cleanly lined up and minimally-sized.   You can play around with the watch while it continues to charge because the wire will be firmly attached at all times.

Because Apple has solved these problems, I would expect iPhones and iPads to adopt the same technology eventually.  The primary hold-up is likely to be the higher charging current that must be generated for these devices, particularly the iPad.  This may require a bigger connection just to handle the appropriate current levels in the coils.

Home Screen

The Watch home screen is in many ways the spiritual successor to "springboard," the home screen on iPads and iPhones.

It consists of a grid (this time non-Manhattan) of app icons.  The icons are round, here, which is all the rage these days (just look at the avatars in your favorite social network).  There are several other differences from springboard, however.

First, the grid is, it appears "infinite."  One doesn't swipe between pages (I think) but rather sweeps along a space containing all these icons.  The icons are positioned in consecutive rings, with outer rings being smaller than the inner ones.  One pans along using one's finger on the screen, and zooms in and out using the Digital Crown.

In some sense the icons can be thought of being arranged on a grid superimposed over a geodesic dome, with the center of the screen being the point on the dome closest to you and the outer ring being furthest away.  (This isn't to say that the animations associated with scrolling adopt that paradigm, though. They don't.)

In some sense this may make more sense on iPhone than on iWatch; organization (as by assigning to pages) seems more important on a device with such a small screen.  It's not clear whether "folders" exist, but I sense they currently do not.  Icons can be arranged using the traditional press-to-jiggle method, it looks like.

I suspect Apple started with the idea of an infinite plane, with the concept that sizes of icons would provide spatial cues, and quickly realized that round icons allow more per-screen than square.  Maybe they were inspired by honeycombs, or by hexagonally-shaped icons that have appeared in various devices from time-to-time.

It's interesting to consider whether this system could be adapted for iPhone/iPad (which is probably overdue for improvements to the home screen) and, if so, whether it would provide any advantages over the Manhattan grid.


There are a lot of questions left unanswered about Watch, and time will tell.  Pebble, for example, changed its button functionality via software updates along the way (which I found very confusing at the time, by the way), and Apple may change many things between now and release day.  And it's hard to judge the effectiveness of a user interface without actually using the device.  Still, Watch presents several intriguing possibilities for future iPhone and iPad iterations, and shows possible directions Apple may take with other types of devices in the future.  As we learn more, I'll dive deeper into some of these aspects and let you know what I think.  (Apple, how about a simulator in Xcode?! I'm waiting...)  One day soon I also expect to provide my thoughts on Apple's design decisions - almost as interesting as what the Watch is is what it's not.


Please see also Watch: Design is What You Leave On the Cutting Room Floor, for my analysis of what features Apple left off and why, what they will add in the future and why, and what the purpose of the Watch really is.