Tuesday, September 20, 2011

Resource-oriented URLs with dojo.hash and HTML5 history

Web development is so exciting! Every new browser release is shipped with a whole bunch of shiny new features to play with. And now that Firefox and Chrome are releasing new versions every couple of months, the dream of HTML5 is getting closer and closer to becoming "real". One of these new features is History API, which allows a web page to programmatically update the browser's URL without causing a page transition. Holy ajax goodness, Batman!  We've been stuck manipulating hashes for years because hashes were the only part of the URL that could be changed without causing the page to reload... until now.

In HTML5, the history object has been augmented with two new functions: pushState and replaceState. pushState adds a new entry to the browser back history and replaceState replaces the current entry in the browser history, both without reloading the browser.  This is very powerful, and makes it extremely easy for ajax applications to use resource-oriented URL schemes.  What's a resource-oriented URL and why is it so desirable to use them?  I'm glad you asked!

In resource-oriented architecture (ROA), an application must expose one URI per resource, and operations on each resource are performed using standard HTTP operations (GET, POST, PUT, DELETE, etc). For the purpose of this post, I'm going to focus on GETs, which is what web browsers do when you type a URL into the address bar. When a browser performs a GET on a resource, content negotiation is performed (by looking at the request's accept header), and the server responds with the resource in the format that was requested. The browser expects an HTML response, but other consumers may want a different representation (RDF+XML, JSON, etc). This concept is extremely powerful, because it serves different types of consumers in the way they wish to be served via the exact same URL - the URL of the resource.

The trouble starts when you want to combine these resource-oriented URLs with ajax applications (i.e. web apps that enable transitions between different resources without reloading the page). This is a fairly common practice these days, with big players like Google, Facebook and Twitter all using an ajax approach to UIs. Ajax applications are desirable because it means you only have to load common parts of the UI once. How annoying would it be if your Facebook chat windows along the bottom had to disappear and reload everytime you navigated to some other page in Facebook? (Answer: very). It would be a jarring user experience and wasteful of bits over the wire, even with heavy caching and optimized performance. How can we have our cake and eat it too? Enter history.pushState.

Before, because we were unable to manipulate the URL's path without a page reload, we used the following hash-based URL design:
https://example.com/myapp#/resource
Under the covers, we would register a servlet at /myapp/* that sends down an empty page with some JavaScript. The JavaScript would inspect the hash, figure out what needs to be displayed, fetch it, and then render it.  There are a few problems with this approach:
  1. The servlet is unable to see the value of the hash because the browser does not send it with the request. This makes it impossible to do any pre-processing of the hash on the server-side (forces reliance on a second round trip to fetch the resource payload).
  2. The page is blank until the JavaScript parses the URL and loads the appropriate resource. Making users wait is bad.
  3. The URL is ugly.
  4. It's impossible to do content negotiation to send other resource representations, since the server can't see what resource you're trying to fetch.
Now, with history.pushState, we can use the following URL scheme:
https://example.com/myapp/resource
We still register a servlet at /myapp/*, but this servlet can read the full path, so it can a) do content negotiation, and b) send more meat down with the initial HTML payload. Within the UI, links to other resources follow the same format (i.e. https://example.com/myapp/anotherresource), but we use a body click listener to intercept clicks to links that are within the servlet's namespace, and use history.pushState to update the URL instead of allowing the click to reload the page. Some JavaScript detects that the URL has changed, parses it, and then fetches the resource.

This adds some new considerations: a) how do I detect that the URL has changed? b) how do we gracefully degrade for browsers that don't support history.pushState (IE9-, FF3.6-)?  This is where dojo.hash comes in.

A while back, I contributed a little utility to the Dojo Toolkit for manipulating hashes, called dojo.hash (with help from Bill Higgins and John Ryding).  dojo.hash shims the HTML5 onhashchange event for browsers that don't support it, and provides convenience methods for getting and setting the hash value.  Before history.pushState existed, we lived in this strange dystopia where all useful information that we needed for building up the UI was part of the hash, so we took dojo.hash() for granted and used it everywhere; for triggering transitions to other resources, for subscribing to hash changes, and for reading the current hash for determining context. Rather than reinvent the wheel and come up with new API for these things, I decided to add history.pushState support to dojo.hash.

First off, I had to come up with a convention for distinguishing between dojo.hash() setter calls that should invoke history.pushState versus those that should set the hash. This was easy. Anything that starts with a forward slash is intended to be part of the path, so use history.pushState (if the browser supports it).  Otherwise, set the hash. Next, I had to augment the dojo.hash() getter to return the resource specific part of the path. Lastly, I had to define the concept of a context root, so that the getter reads and the setter sets only the part of the path that would have previously been part of the hash.

I had to implement a few other little nitpicky things: 1) the body click listener that listens for clicks on links that begin with the context root and call dojo.hash() instead, 2) a first-load redirect rule that strips off redundant hashes.

In the end:
  1. All of our URLs are fully qualified and resource-oriented.  
  2. The servlet is able to see the full URL, so it can do content negotiation.
  3. The first load experience doesn't have to be a blank page anymore!
  4. Consumers continue to use the same API they always used to get and set hashes and it transparently handles pushState if it can.  
  5. URLs are RESTful and pretty.
  6. Graceful degradation for older browsers.
Visit Jazz personal dashboards on Jazz.net (registration required) for a look at the (almost) finished product. We're doing content negotiation and everything! (if you request a dashboard resource with an application/rdf+xml accept header, you'll get what you asked for). We haven't implemented (3) yet though, but that'll be along shortly.

Note: these dojo.hash changes have not been contributed to dojo. They're implemented as overrides in Jazz. Stay tuned.

Friday, August 14, 2009

My real feelings about Macs

I've recently been involved in a bit of a flame war regarding Macs vs. PCs on twitter. Unfortunately, 140 characters isn't quite enough to explain my real feelings. I am definitely on the Windows side of the argument, but I'm not as staunchly opposed to the Mac platform as my tweets have made me sound.

So let me set the record straight. I don't hate Macs. On the contrary, they're quite beautiful. Also, the operating system is solid and straightforward to use.

The main reason I don't care to own a Mac is that I like to build my own machines. I spend weeks researching and plotting every single component in my machine. I wouldn't own a Dell or a Toshiba or an HP for that very same reason.

I think it all boils down to comfort. The C drive comforts me. Unless Microsoft really disenfranchises me somehow in the near future, I'm perfectly satisfied with the performance and UI of Windows. Especially now that Windows 7 is out in the wild. I've been running Windows 7 RC on my home PC and I love it. And I can't wait for IBM to upgrade my workstation to Win7 (although I'm not holding my breath).

The Eastern Townships School Board, the public school system in the region where I grew up, provides MacBooks to most of their student population (grades 3 thru 11). I had access to these MacBooks, and was involved in informally troubleshooting problems with them at Sunnyside Elementary School. I ran website design workshops with the students, and struggled with providing good instructions for the kids because of a lack of good WYSIWYG web design freeware. I ended up installing Nvu and Paint.NET on some old Windows desktops. Granted this can now be solved by VMWare Fusion or Parallels, but neither of these are freeware either.

So I've eaten the green grass on the other side of the fence, and it's not quite as green as advertised. I've never seen a pinwheel of death in Windows. There are pros and cons to both platforms, and I feel strongly that the none of the various pros of Mac OS are compelling enough to lure me away from my comfort zone.

Feel free to enjoy whatever OS you like best. If you need to be coddled by Apple to enjoy your computing experience, go for it! Essentially, I don't care to evangelize or be evangelized. I will make my own operating system and computer platform decisions.

Wednesday, August 12, 2009

Browser Upgrades

Dean Hachamovitch, I have a bone to pick with you.

You just wrote a (mostly hand waving) blog post on the IEBlog outlining how "the choice to upgrade belongs to the person responsible for the PC." This post was in response to a Digg.com survey regarding why people are still using IE6, a 9 year old browser that is essentially holding the entire web development community hostage.

I applaud you for approaching the topic openly and I understand that your POV is grounded in pragmatism, but I think you've missed the point.

Yes, I am an enthusiast, and enthusiasts love living on the bleeding edge. I remember dutifully installing IE 5.5 Beta 1 on my Windows 98 SE box back in 1999. In those days, I wasn't really a web developer (beyond my Geocities "personal homepage"). I don't remember noticing any differences between IE 5 and IE 5.5, aside from a few little UI tweaks and a fancy Print Preview feature. This is how non web developers perceive new web browser releases - It's just a chore; change for the sake of change. Why should I upgrade to the latest version when the version I'm on works just fine?

This is precisely why upgrading shouldn't be left up to the user or system administrator, because the status quo is perceived to be easier even if it's counter-productive to the global community. In order for the web to evolve, the lowest common denominator needs to evolve with it.

I can hear the retort now: "But we don't care about the global community. We care about the bottom line. And the bottom line means we want to keep using the same crusty Windows XP image that we created back in 2004 that we subsequently wrote tools for using non-standard IE6-specific code that we're stuck with because the tool developers have since moved on to other jobs and can't update them to work with new browsers."

Cry me a river.

I work on the IBM Rational Jazz Foundation. We upgrade our servers every 4-6 weeks to the latest code, forcing everyone to upgrade their clients on their systems. This chore has just become part of the job. I have no choice, I just have to do it. And I've become quite fond of this monthly dance. Small jumps are much less painful than big jumps.

IE6 people are going to start noticing that more and more pages are broken for them because developers can't justify supporting them any longer. If users were forced to update their browsers, the world would be a better place. It's as simple as that.

The solution to this problem doesn't need to be as draconian as I'm suggesting. Just prod the user more forcefully. Do a better job of providing incentives to upgrade. Release more incrementally. Back port new features into old browsers (instead of relying on the community to write hacky shims). Decouple IE8 from the core OS and allow two versions to coexist. Just do everything in your power to raise the bar.

Tuesday, January 27, 2009

Pragmatism

I've recently been working on implementing a two column layout for Jazz web UIs (reg req'd). I could easily update our current layout's images and styles and be done with it, but while I'm restructuring, I want to clean up some of our current layout's problem areas. Currently, we've got the following "features":

1) Two column layout with equal height columns (one fixed width, one fluid width)
2) Drop shadow and gutters on either side of the layout
3) Sticky footer; make the layout stretch the full height of the window with a footer at the bottom.

CSS hackers everywhere know that these three requirements are tricky to achieve together. The sticky footer was achieved by adding a page wrapper div with 100% height, followed by a footer div with a negative top margin equal to its height. The columns inside the page wrapper were then stretched down to the footer using the One True Layout hack (large negative bottom margin with large positive bottom padding).

Unfortunately, this hack required overflow: hidden on the wrapper, which killed off the possibility of enabling horizontal scrolling on the layout (i.e. what if a table has a zillion columns and is wider than the screen? Sorry, it gets chopped at the edge).

To make things worse, even if I used another mechanism for the equal height columns, DIVs don't expand to fit their contents, so the wide table would just overflow out of the layout, which is ugly.

Another problem area had to do with floats. To achieve the two column layout, I floated one of the columns left, and allowed the other to fill the remaining space (with overflow: hidden to give it its own block formatting context so that the columns drew beside each other instead of overlapping). Unfortunately, anytime the right-most column was forced wider than its auto width, it would wrap below the left column. There were various ways to mitigate this wrapping, but they all precluded horizontal scrolling.

As far as I know, there is no CSS only solution to achieve all of my requirements while mitigating the aforementioned problems (if there is, I'd love to hear it). So I set out to find a happy medium, which led me to tables.

I can hear the groans from the CSS elite (What's your CSS level?). Table layout!! NOOO!! But hear me out.

Tables are expand-to-fit! They're the only HTML element that does this. So I'm using this property to achieve my horizontal scrolling. The great news is that tables expand even if they are set to overflow: hidden, so my One True Layout hack will still work for the equal height columns. I lose my sticky footer, but I can easily achieve that with some simple onload javascript logic that sets the height of the table's one and only cell.

But that still leaves my column wrapping problem. Horizontal scrolling is moot if the column wraps before it scrolls. So how do I fix this? I could split my table into two columns, but my willingness to rely on tables only goes so far. What other ways are there to put divs side by side? For that, I turned to display: inline-block.

Display: inline-block causes the element to generate a block box (like display: block), but to flow relative to its siblings using an inline formatting context (i.e. side by side like display: inline). To make this work on IE, I had to use spans instead of divs, but that's easy enough. By adding white-space: nowrap to the parent of these divs, all wrapping is nixed, and the table cell expands as it should when content forces out the side.

Sounds too good to be true, and it is... Inline-block elements do not fill their available width as block elements do, so inner divs would only be as wide as their contents force them to be.

To fix this, I had to apply a fixed width to this column using JavaScript, and register a resize handler to update the width when the window changes. This is no problem in Jazz Web UI land since these pages require JavaScript to even work (async code loading and dojo-based UI widgets). Even with the fixed width, the table continues to encapsulate overflowing contents.

Take a look at the final result: http://twitpic.com/17wdi

So is it bad that I'm using a table? I don't think so. It's important to be pragmatic in web development. If an element has a property that you need, use it!

Thursday, December 4, 2008

User Interface Programming: Why does it takes 80% of the time to do 20% of the work?

I'm working on a decorated box widget, with a drop down menu, expand/collapse, header icon, and more. No matter how close I get to finishing it, another little feature or tweak pops up!!

In half a day, I was able to get from nothing to a polished widget that satisfied almost all of my requirements. Now I'm two more days into hacking CSS and squashing edge cases, and I'm still not quite finished!

This boggles my mind. Yesterday evening when I left work, I said to myself "What a great day of work! Just a few more little tweaks and I'll be able to deliver this code!" Hah, yeah right :)

Why does it take 80% of the time to do 20% of the work? Maybe I'm just horrible at estimating. Or maybe it's a law. So I googled and found this:

http://en.wikipedia.org/wiki/Pareto_principle

That's not quite right, although one could draw parallels. The 20% of my time that accomplished most of the work could be considered "the vital few". Sounds like nature is screwing with everyone.. even the economists.