Posted by: digdug2k | May 2, 2011

Touch Events in Fennec Nightly Builds

This past Friday we checked in the initial frontend patches to Fennec to turn on single-touch TouchEvents in the mobile browser. As a result, you can now pan around on sites like Google or Bing Maps in Fennec! Apple added support for touchevents in mobile Safari early in the iPhone’s lifetime, but the W3C TouchEvent specification has only recently starting coming together. Mozilla’s mozTouchEvents will likely be deprecated in favor of the W3C standard specification.

Touchevents are necessary on browsers designed to be used with your fingers like Fennec. Our panning and double tap detection can make it difficult to forward typical mousedown, mouseup, and mousemove events to the client in a timely manner. For instance, in Fennec, we currently set the active pseudoclass on elements when they are first tapped (and use it for tap highlighting). If the user lifts their finger, without moving it very far from the initial mousedown, we wait a short time to make sure there isn’t a second (‘double’) tap, and if not we fire mousedown, mouseover, mouseup, and click simultaneously on the element. The delay makes it impossible to create pages that quickly respond to user input.

To help with this, in Fennec 4 we removed the time spent waiting for a double tap if the page has disabled user scaling in the metaviewport tag. This allows the page to quickly receive click events, but since we pan the page as your finger moves, we still couldn’t send mousemove events to the client. Touchevents provide the bridge between the two modes of interaction. Instead of registering for mouse events pages can register for touchstart, touchmove and touchend events. These seem very similar to typical mouse events, but have a few key differences. Touchevents don’t contain clientX/Y, pageX/Y etc coordinates directly on the event. Instead, the mouse coordinates are stored as an array of touchpoints and added to the touches property of the event. So, for instance, to use just the first touch or an event, you can call:

window.addEventListener("touchstart", function(aEvent) {
  var x = aEvent.touches[0].clientX;
  var y = aEvent.touches[0].clientY;
}, true);

In addition, touchevents also target themselves differently than regular mouse events. While touchdown is fired on the initially touched element, touchmove events will always remain targetted at that same target, even if the user moves their finger onto a different element. So if you want to write touchevents to drag an object around the screen, you can register a touchdown and touchmove listener on the object, and you will always receive the move events on the object, even if the user moves their finger onto a different element on the page. That’s how this example from quirksmode works.

When a page registers touchevents we attempt to detect it in the Fennec chrome and disable panning until the page has had  its chance to handle touchmove events. If the page author decides to call preventDefault on either touchstart or on the very first touchmove event, it will prevent Fennec from panning the page (we still allow panning the sidebars and urlbar off of the page, but will not allow you to pan them back on). It will also prevent any clicks that would normally be fired. This can pose a problem for Fennec where panning the page is necessary in order to gain access to the urlbar, and to the tab- and control-sidebars.

In order to continue to allow you to access the UI, and still allow pages to allow complex touchevents, we’ve currently implemented two escape hatches into Fennec:

  1. Pressing the menu button will show the urlbar, allowing you to move to a different page
  2. Pans starting at the very top of the page (within the top 1/2 inch) can not stop the panning of chrome

These will always bring in the chrome, and so you can trust that they are showing valid content.

Single-touch events are supported in nightly builds of Fennec (Androidother platforms) and should ship (fingers crossed) with Fennec 6. Support for multi-touch on Android is currently targeted at Fennec 7. Feedback is welcome. In particular, we’d like to ensure that turning these on hasn’t introduced noticeable regressions in panning for anyone, or if you can find places where our handling of preventDefault doesn’t agree with what is present in other modern mobile browsers. Test them out. Enjoy! Google Maps is likely the primary place where you will notice the work (NOTE: Maps has display problems partly due to scaling issues in Fennec, and partly because of Google’s use of webkit prefixed css without providing fallbacks for other browsers). I wrote a quick (and buggy) finger painting program to demo as well, but we’re excited to see what kind of nifty web applications and games people use this stuff for!

Special thanks to Matt Brubeck and Olli Pettay for their hard work within the WebEvents work group. Olli (smaug) also implemented the backend pieces necessary for Fennec to support touchevents. Matt has been hard at work documenting how touchevents work in different mobile browser (hint: they’re not all the same), and filing/fixing bugs to help us continue to bring Fennec’s implementation inline with as many as possible.

About these ads

Responses

  1. Are gestures (e.g. pinch-to-zoom, swipe, etc) not in the W3C spec because of patent concerns?

    • I’m not involved in the process directly, but I have heard rumblings to this effect. Having access to the events means you can do gesture support yourself though, which is what the Fennec frontend does as well!

      • Except implementing gesture support manually will work differently than the platform itself. You’d have inconsistent sensitivity, etc.

        The irritating thing is that the native Android browser has much of the same limitations.

        Could Mozilla announce why it does not support gestures, and take up an evangelist position? The same as done for H.264 and MP3?

      • From what little I have read, most gesture support in the Android platform seems to mirror the TouchEvents APIs. For instance, there is no pinch/magnify gesture event I know of. Each individual app determines the size and rotation for themselves. There really isn’t a “platform” to match that I know of (although I’ve only done a few quick searches on the topic when I was curious about this too).

        That’s a bit of a bikeshed though. Certainly worth talking about exposing either Mozilla’s own SimpleGestureEvents or Apple’s variant to content. In addition, we have been talking about the best way to expose things like the current zoom level to pages as well. Once multi-touch is available in the product, I imagine we’ll start focusing more on those issues.

  2. Do you know if jquery.ui’s drag-n-drop supports this? http://jqueryui.com/demos/draggable/ and http://jqueryui.com/demos/droppable/ ?

    • A quick test says not yet. I have heard John Resig say that jQuery mobile (http://jquerymobile.com/) supports some touch events, typically for things like swiping between pages.

  3. [...] for tools that help developers create rich, compelling mobile Web apps and websites. With the single touch events API, developers can build Web experiences that detect touch events and gestures. Imagine mobile Web [...]

  4. [...] for tools that help developers create rich, compelling mobile Web apps and websites. With the single touch events API, developers can build Web experiences that detect touch events and gestures. Imagine mobile Web [...]

  5. [...] for tools that help developers create rich, compelling mobile Web apps and websites. With the single touch events API, developers can build Web experiences that detect touch events and gestures. Imagine mobile [...]

  6. [...] hinaus bietet Firefox 6 für Android neue Funktionen für Entwickler an: Über das Single Touch Events API lassen sich Touch-Ereignisse und Gesten erkennen, so dass Webapplikationen darauf gezielt reagieren [...]

  7. [...] browser can be summed up in it's the APIs that come with the new Firefox for Android build. The single touch events API can help developers build Web experiences that detect touch actions, like a swipe or a pull to pan [...]

  8. [...] the single touch events API, developers can build Web experiences that detect touch events and gestures. Imagine mobile [...]

  9. [...] shade introduces facilities accessible in a mobile browser. Developers can make use of a “single hold event” API to capacitate web apps to respond to swipes; multi-touch support is also betrothed for [...]

  10. [...] mobile browser can be summed up in the APIs that come with the new Firefox for Android build. The single touch events API can help developers build Web experiences that detect touch actions, like a swipe or a pull to pan [...]

  11. [...] is making a big deal out of new developer tools: With the single touch events API, developers can build Web experiences that detect touch events and gestures. Imagine mobile [...]

  12. [...] Entwickler für detailreiche und unwiderstehliche mobile Web-Apps und Webseiten benötigen. Mit der Single Touch Events API können Entwickler Webseiten bauen, die Berührungen und Gesten automatisch registrieren. Stellen [...]

  13. [...] apresenta os recursos do navegador. Desenvolvedores também já podem fazer uso da API para “eventos de um toque” para permitir que aplicativos web respondam ao deslizar de dedos (swipes), o suporte à [...]

  14. [...] for tools that help developers create rich, compelling mobile Web apps and websites. With the single touch events API, developers can build Web experiences that detect touch events and gestures. Imagine mobile [...]

  15. [...] ciekawych, zaawansowanych mobilnych aplikacji i stron internetowych. Dzięki API zdarzeń “single-touch” deweloperzy mogą budować aplikacje, które wykrywają zdarzenia dotykowe i gesty. Teraz [...]

  16. Have you considered adding a few social bookmarking links to these sites. At least for flickr.

  17. [...] for tools that help developers create rich, compelling mobile Web apps and websites. With the single touch events API, developers can build Web experiences that detect touch events and gestures. Imagine mobile Web [...]

  18. [...] ciekawych, zaawansowanych mobilnych aplikacji i stron internetowych. Dzięki API zdarzeń “single-touch” deweloperzy mogą budować aplikacje, które wykrywają zdarzenia dotykowe i gesty. Teraz [...]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: