Combining HTML Hypermedia APIs and Adaptive Web Design

tl;dr

HTML is the lowest common denominator in a world where the number of devices are steadily growing. In such a world, our API should be hypermedia driven and we should consider choosing HTML as the media type for our APIs. If we choose this, we can enhance the API with CSS and JavaScript, making the API human-friendly as well. Further, we can use Adaptive Web Design to make the best possible experience for all types of browsers and devices.

This kind of solution has some risks and technical issues related to it though. And it is not a silver bullet.

Introduction

First, I would like to point out that this article is quite high-level. You won’t see much code here, but I hope to do some follow-up posts in the future with more detailed scenarios and some code examples.

Second, it might be a good thing if I list some of my assumptions

  • we build a product that should be available on mobile devices.
  • we have control over the client development, either directly (in-house) or indirectly (via partners)
  • it’s not a product that is based on real-time web, i.e. trading or (some) games

From my perspective, a common pattern for building a product for mobile devices is:

1.Build a JSON API (using Pragmatic REST) and a “first native app”, i.e. iPhone.
2. Build a “second native app”, i.e. Android
3. Build a “third native app”, i.e. WP7, iPad, etc
4. Build a mobile web site

I propose that we should consider the somewhat opposite order:

1. Build a mobile web site, which IS your hypermedia driven API.
2. Build a native app which fulfills “the main usecase” using the native platform’s technology, consuming the API produced in (1). For the rest of the application, embed a browser and point it to the corresponding mobile web page.
3. Repeat (2) for the other devices.

Background

For some time now, I’ve been interested in these two topics:

  • Hypermedia APIs
  • HTML5/JavaScript

About two and a half years ago I read “Sammy.js, CouchDB, and the new web architecture“, and I became very interested in the idea of building a fat client in the browser – what is now called a Single Page Application.

Then, more recently, I read the Resource-oriented Client Architecture Style, and it really made me start thinking about if I was going in the right direction. For example, one requirement of ROCA is that the application should be “in line with the principles of progressive enhancement, JavaScript is used unobtrusively and the application remains usable (albeit with a decrease in usability and convenience) if JavaScript is disabled. [must-jslimits]“. Thus, Single Page Applications are not allowed.

I had (previously) realized that the “Pragmatic REST” architecture would produce clients with lots of duplicated code, i.e. logic for

  • what information I should see at a particular time
  • what actions I can perform on an entity, both due to its internal state and due to the user’s role/state
  • meta-data for forms, including validation rules
Clients should be DRY

“Clients should be DRY”

I also knew that JSON has no built-in support for hypermedia, but also that hypermedia types like HAL and collection+json exist for JSON.

Before ROCA, I had been a supporter of the graceful degradation style, but I now realized that I had not put enough energy understanding progressive enhancement, so I started to read more about it.

I found out that there exists another name for progressive enhancement: Adaptive Web Design (AWD). The more I started to read about it, the more I thought: “Yes! This is how web should be built. A web that works in every browser, how useful!”

All these concepts and ideas made me realize the possibility of another way of building solutions that supports multiple clients, based on HTML as the hypermedia type for your API – where your desktop web is your mobile web and your mobile web is your API.

Using HTML as the Media Type for your API

Let’s back up a couple of steps. If we want to start to use hypermedia in our API, we have an important design decision to make: what media type(s) should we use for our API? This is an important decision to make. A good place to start is to look at what media types already exist, to see if one could fit our scenarios. Mike Amundsen has created a good summary of the different alternatives, using what he calls “hypermedia factors”. As we see, JSON is not a hypermedia type – instead we can use HAL and collection+json. We also see that HTML has a lot of these factors built in. And, as we know, there are lots of existing clients (browsers) that already understand HTML, compared to the JSON based media types.

A while ago, Jon Moore posted an excellent article where he described the benefits of using HTML as the media type for an API. I can really recommend reading that article – it’s excellent. Even if you don’t want your API to be consumed by a browser, there are still lots of benefits of using HTML as your media type. But a very nice side-effect of choosing HTML is that it can be consumed by a browser.

It’s beyond the scope of this article to describe how to design the HTML for your API. In summary, very briefly, use as much semantic markup from HTML5 as possible and use the micro format style for your domain specific information.

A web for everyone

If you build your API using only HTML, what you get is something that (in a browser) visually looks worse than “the web á la 1996″. But that is not a problem, since we primarily wanted our API to be consumed by machines (API clients). On the contrary, it’s a huge benefit for us if we want to support lots of different browsers – from old IEs to Kindle’s (experimental) browser to small Android phones to a freshly updated Google Chrome on a laptop. What you have exposed with your HTML-based hypermedia API is probably the /core service of your product/, in a way that is similar to the adaptive web design approach, which we’ll touch upon soon.

The number of devices is increasing

“The number of devices is increasing.”

What to do now is to make a basic typographic CSS in order for the site to look “ok, but not more”, which is a good place to start.  Voilà – we have a web site for all devices with a browser, old and new! Now, we’ll turn to more advanced techniques, making the experience as good as possible for browsers with lots of modern capabilities.

Adaptive Web Design

At this step of the development process, our site is very simple, which fits well for browsers with limited capability and for machines. But we need to think about modern mobile browsers and desktop browsers, to enable a service that is both as visually attractive and as usable as possible for the user. Enter adaptive web design.

Here is a presentation by Nicolas Zakas, called “Progressive Enhancement 2.0″, that describes AWD much better than I could ever do. And here, Aaron Gustafson describes the difference between responsive web design and adaptive web design, to make it more clear what the terms mean.

Two AWD techniques that look promising for our scenario are delayed and conditionally loaded content. Another example shows that this kind of content loading can be done more effectively if multiple requests are bundled into a single request, which needs to be supported on the server.

What could also be interesting to notice here is that the different requests could have different caching policies – this is in fact the most probable scenario – so it’s important to think through the impact on caching carefully. Probably you’ll need different URIs for content following different cache policies.

Before and after

“Before and after.”
Note: An ‘application component’ is mandatory in the first design, but not in the second.

For modern browsers, we could also make conditional loading based on the page load time (caniuse page). Today, it’s not always true that a desktop browser has a good connection behind it – it could be run on a laptop connected to a mobile device with a 2G/3G connection. In this case, the best user experience could be to not load the “extra content”, even though it’s a desktop browser.

Detect and act

“Detect and act.”

One important thing to remember with AWD is that every stakeholder should be informed that the experience will be different for different browsers – and that the difference of experience is a good thing! It also puts some pressure on the UX crowd to deliver dynamic design sketches (perhaps in collaboration with the developers?), rather than static image files.

But it is standing on one leg!

A possible future scenario of this architecture is that there is too much pressure on it from both the API side and the browser side. The reason for this pressure could for example be that the structure of the HTML for the particular solution needs to be different for the different perspectives, or that the impact on performance is too high from one of the perspectives. But another possible future scenario is that there will /not/ be too much pressure from both sides. And we don’t want to build something that’s not useful.

Here we can create an option: the possibility for us to split the solution into two parts – one for machines and one for humans. This could be by creating a proxy, www.example.com, which proxies all its requests to api.example.com. That way, if we want to split the solution in the future, it will not be a problem for us to replace the proxy with a web server containing software for your web site.

An option for the future

“An option for the future.”

This way we could perhaps move some delayed content loading back to the server, i.e. letting the server render a basic header and footer instead of loading it with AJAX on each request.

Remaining problems

Both using HTML as the media type for your hypermedia API and Adaptive Web Design are fairly new techniques, so we have some things to investigate when designing and developing this type of architecture.

  • Patters for content loading are starting to emerge, but how about patterns for “service loading”. Can we construct a Single Page Application built adaptively from a mobile web, and do we want to?
  • Without JavaScript, browsers can only GET and POST from forms. At the same time, it’s nice to expose all the other verbs, following Uniform Interface. So it seems that we need to use both the “_method workaround“.
  • The structure of application/x-www-form-urlencoded data is flat, so conventions need to be agreed upon if we want to send non-flat data.
  • How can we effectively build hybrid native apps? How do we integrate the embedded browser and the native part of the app?
  • Not all app stores may allow “an embedded browser” app, or require that a certain percent of your app should be built with native technology.
  • It could be hard for native clients to map the meta-data in HTML forms to native forms.
  • We need “HTML API parsers” for each client platform, that make it as easy as possible to consume the exposed service.
  • As mentioned earlier, caching could be a challenge.
  • Off-line support could be a challenge.
  • At which point in time should we split the solution into two parts, if ever?

Bonus: A strategy for existing products

If you already have an existing product on the web and in some app stores, one way forward could be to start building a hypermedia API using HTML, operating in parallel with your JSON API, and having that API as a new mobile web site. Then, gradually start to use the API from the native apps, analyzing what use cases that should still be native and which will use an embedded browser.

Summary

As Jon Moore points out, using HTML as the media type for your hypermedia has some benefits. If you choose to go this route, you can let your API be your mobile web and let your native clients use the HTML – either by parsing it or by embedding it. Using Adaptive Web Design makes both your mobile web and your desktop web a better experience and it lets older browsers view and interact with the content/service. And ROCA can teach us a lot.

I should also point out that I have no practical or professional experience of many of these ideas. Nonetheless, I think they look very promising, especially together, and I would be very thankful if you could share your thoughts and opinions on them with me.

Resources

Video: Progressive Enhancement 2.0
http://www.webmonkey.com/2012/03/video-progressive-enhancement-2-0/

Using HTML as the Media Type for your API
http://codeartisan.blogspot.de/2012/07/using-html-as-media-type-for-your-api.html

support vs optimization
http://bradfrostweb.com/blog/mobile/support-vs-optimization/

Don’t use HTML as the Media Type for your API
http://blog.stateless.co/post/26898344742/dont-use-html-as-the-media-type-for-your-api

An Ajax-Include Pattern for Modular Content
http://filamentgroup.com/lab/ajax_includes_modular_content/

Resource-oriented Client Architecture (ROCA)
http://roca-style.org/

“native vs web” is total bullshit
http://bradfrostweb.com/blog/notes/native-vs-web-is-total-bullshit/

The Coming Zombie Apocalypse [of devices] http://designmind.frogdesign.com/blog/the-coming-zombie-apocalypse-small-cheap-devices-will-disrupt-our-old-school-ux-assumptions.htm

Responsive Design and Progressive Enhancements for the Web (workshop)
http://www.slideshare.net/lucasalvini/responsive-design-and-progressive-enanchements-for-the-web-workshop

Microformats
http://microformats.org/about

Building hypermedia APIs with Node
http://www.amazon.com/Building-hypermedia-APIs-HTML5-Node/dp/1449306578

Responsive Design
http://www.alistapart.com/articles/responsive-web-design/

Adaptive Web Design – Crafting Rich Experiences with Progressive Enhancement (book)
http://easy-readers.net/books/adaptive-web-design/

Designing Hypermedia APIs (e-book site/blog/wiki)
http://designinghypermediaapis.com/

A Hypermedia API Reading List
http://blog.steveklabnik.com/posts/2012-02-27-hypermedia-api-reading-list

Hypermedia Types, H-Factor
http://amundsen.com/hypermedia/hfactor/

Creating a Mobile-First Responsive Web Design
http://www.html5rocks.com/en/mobile/responsivedesign/

Speed Up Your Site with Delayed Content
http://24ways.org/2010/speed-up-your-site-with-delayed-content

Conditional Loading for Responsive Designs
http://24ways.org/2011/conditional-loading-for-responsive-designs

JavaScript is not hypermedia
http://fingernailsinoatmeal.com/post/306541120/JavaScript-is-not-hypermedia

Responsive Web Design
http://en.wikipedia.org/wiki/Responsive_Web_Design

Progressive enhancement
http://en.wikipedia.org/wiki/Progressive_enhancement

Sammy.js, CouchDB, and the new web architecture
http://www.quirkey.com/blog/2009/09/15/sammy-js-couchdb-and-the-new-web-architecture/

HAL – Hypertext Application Language
http://stateless.co/hal_specification.html

Collection+JSON – Hypermedia Type
http://www.amundsen.com/media-types/collection/

Stack Overflow, Measuring Site Load Times via performance api
http://stackoverflow.com/questions/7606972/measuring-site-load-times-via-performance-api

When can I use Navigation Timing API?
http://caniuse.com/nav-timing

Measuring Page Load Speed with Navigation Timing
http://www.html5rocks.com/en/tutorials/webperformance/basics/

The _method workaround, for Sinatra and Rails
http://www.humbug.in/docs/sinatra-book/the-put-and-delete-methods.html
http://guides.rubyonrails.org/form_helpers.html#how-do-forms-with-put-or-delete-methods-work

HAL-HTML
http://blog.stateless.co/post/27262566522/hal-html-html-for-automatons

Unobtrusive JavaScript
http://en.wikipedia.org/wiki/Unobtrusive_JavaScript

Uniform Interface
http://en.wikipedia.org/wiki/Representational_state_transfer#RESTful_web_services

10 Comments

  1. Christian Hedin

    Very interesting. My perspective is the mobile client one and intuitively it doesn’t feel so good to get rid of the trusty JSON responses but I realize the many advantages of using HTML. I have long thought about delegating the tedious server communication in mobile apps to a WebView. It could be accomplished completely with JavaScript. Now, with HTML as data carrier it would be even simpler and JQuery could be utilized to extract the wanted information. This would then be bridged to the native app layer and normal native model objects could be created. Or, as you suggest, it would be so easy to just show the data as it is to the user for where it’s useful.

  2. Hi Gustaf,

    This is a very nice article. At my company we moved from json based api to html5 based api, thanks to the must read book from Mike Amundsen.

    By the way, there were some commercial and attractive negative aspects of this : The 3rd party clients are poorly hypermedia api’s aware, so having only html5 as export format for the api was a no go from the manager. But a super easy workaround to this was to mark the api part of the html markup with a @api tag, this way we could parse the html and convert it to vendor+xml or hal+json and you have 3 formats and the need to update only one.

    Regards,

    Chris

  3. Link exchange is nothing else except it is simply placing the other person’s web site link on your page at appropriate place and other person will also do same in favor of you.|

  4. Howdy very nice website!! Man .. Excellent .. Wonderful .. I’ll bookmark your site and take the feeds also?I am happy to find so many helpful information right here in the put up, we need develop more strategies in this regard, thank you for sharing. . . . . .

Trackbacks for this post

  1. Combining HTML Hypermedia APIs and Adaptive Web Design | Dev Breakthroughs | Scoop.it
  2. Combining HTML Hypermedia APIs and Adaptive Web Design | nodeJS and Web APIs | Scoop.it
  3. Today in APIs: Advanced Twitter Filtering, OAuth Voice of Reason and 18 New APIs
  4. This Week in #REST – Volume 44 (Apr 4 2012 – Aug 24 2012) « This week in REST

Leave a Reply