« Rock City Band March 2011 Appearances | Home | Why the New York Times paywall is not like public radio »

Documents in applications

By Noah | March 8, 2011

It has become fashionable to divide Web resources into two broad categories: each resource is either a document, rendered primarily in HTML, or an AJAX-style  Web application that uses Javascript to facilitate very dynamic interaction, navigation and information retrieval.  My purpose here is to argue that we need to be more careful, that many AJAX applications in fact provide access to documents after all, and that the Web would be much more robust if we took some care to identify and access those documents using the same sorts of URIs that we use for other Web documents.

Background: the #! fuss

I’ve been thinking about all this for quite some time, but the recent fuss over so-called “hash-bang” (#!) URIs raises the stakes considerably. As previously noted, Jeni Tennison has posted a terrific overview, with links to contributions from many others. If you’re not up on all that, please read those postings first.

Three examples

Consider three Web applications, each of which makes extensive use of Javascript.

1. The simulator is an interactive environment that simulates, we’ll say, driving a race car. You launch the application by navigating to its URI, and then Javascript takes over, providing an interactive simulation of the car driving around a race course, while giving you control over the car’s acceleration and direction. Just to make the example more clearly Ajax, we can assume that the application periodically goes out to various Web sites to retrieve data to be integrated into the simulation (weather information, etc.)
2. The résumé browser is similar in style to AJAX implementations of GMail or Yahoo mail, but its purpose is to let the user browse through collections of job application résumés. The initial screen provides a list with one line per applicant, but the user can request to see individual résumés as well.  Under the covers, all the usual Ajax tricks are being used to pre-cache résumés, to provide fluid interactivity for the user, etc. Users require the ability to email and post links to the individual résumés.
3. The final example is the full (as opposed to mobile) implementation of Google Maps. Like the others, this is an Ajax application. Users can retrieve maps for most any part of the world, zoom, pan, and annotate the maps with points of interest or driving directions. Crucially, an interface is provided that gives a URI (with no # in the syntax!) for the current location, zoom level, annotations, etc. As with the résumé application, these URIs can be emailed, linked from other Web pages, etc., and indeed this is a very valuable capability that is often exploited by users.

Resource identification

With respect to URIs and resource identification, the résumé and simulator applications are quite different in character. Indeed, if we ignore implementation concerns, the résumé browser closely resembles a traditional, non-Ajax, document-based Web site. Users have every reason to want each résumé to be integrated into the Web as a first class resource with its own stable URI, per the Architecture of the World Wide Web‘s mandate to Identify with URIs. It’s very important that people be able to email links to these résumés, put links to them on other Web pages, etc. Ideally, non-Javascript user agents should show the same résumés as Javascript-enabled browsers (albeit with less fluid navigation). Furthermore, it’s very useful if crawlers and other server-side agents can find and retrieve the documents, and for the FWD/BACK buttons in the browser to work as they would in a non-Ajax implementation.

Even though the implementation is very similar in its use of Javascript, the simulator is very different in character. There is no sense in which the user is navigating through a succession of pages.  Maybe or maybe not the application would find some use for FWD/BACK, or could provide links to intermediate simulation states that might usefully be emailed, but in any case there seems little need that such links be crawlable, or that non-Javascript user agents do much with them.

My main concern here is that too few Ajax applications take the trouble to meet the goals set out for the résumé browser, even when they could. With some hand waving about how the Web has moved past its old static past, and making some quite valid points regarding what happens when you update the address bar in typical browsers (you tend to kick off a page retrieval if anything other than the fragment identifier is touched), the designers of these applications often punt on providing useful URIs for the individual documents managed by their applications. Furthermore, if such identifiers are provided, they are often distinguished only in the fragment identifier, resulting in all the disadvantages noted by Jeni, Tim Bray, and others. For example, the resources tend not to be accessible from non-Javascript  agents, rather ugly kludges are required if crawling is to be enabled at all, etc.

There are two main points I’m trying to make in this note. The first is:

Use of AJAX implementation technology is not a sufficient excuse for failing to provide first class URI identification for documents on the Web

If you’re building a simulator, this advice may not apply; if you’re building the résumé browser, then provide URIs for each résumé, and probably for each sorted collection of résumés, etc. Ensure that those URIs work, at least to display the referenced document, when accessed with non-Javascript enabled user agents.

Documents when possible

Now let’s consider the Google Maps example. What’s interesting about it is that the Google engineers could have easily made the claim that the maps application was like the simulator, provided a single URI for the entire application, and stopped there. They might, to go a bit further, have done some local kludging to enable FWD/BACK to work, while still not exposing URIs for the different views of the map (or if you prefer, for the different maps.)

In fact, Google Maps is valuable in part because Google did take the trouble to model their application as a collection of document-like Web resources, and in fact to create URIs that work equally well in Javascript- and non-Javascript enabled user agents. To do this, they created URIs with no fragments (I.e. no #) for each view of the map; the compromise, at least for now, is that these URIs do not show up automatically in the address bar, but are available from a separate link button supported by the Javascript.  These URIs have some interesting characteristics:

• They are understood by both server and client: when a request is received from a non-Javascript enabled user agent (often from a mobile device), the server interprets the entire URI and sends a relatively static image of the map as a response; when the same URI is received from a Javascript agent, the Ajax implementation is used. Interpretation of query parameters is (potentially) distributed between the server and the client, and that division of labor can change without affecting the URIs.
• Because the same URI is used in all cases, a link can be created on a Javascript-enabled client, emailed to a non-Javascript client, and the right thing happens. Contrast that with http://twitter.com/#!/noahmendelsohn vs http://twitter.com/noahmendelsohn,  which are provided by Twitter for Javascript and non-Javascript clients respectively. The former does nothing useful if emailed to a non-Javascript client.  Pointing again to the advice given in Web Arch  Avoid URI aliases: “The problem with aliases is that if half of the neighborhood points to one URI for a given resource, and the other half points to a second, different URI for that same resource, the neighborhood is divided” The #! forms also break the Self-describing Web.

Google Maps shows the great value of using the document abstraction when you reasonably can. It wasn’t the easiest choice for that application, but the benefits are very significant. In spite of being very fluid and interactive, each state of the Google Map is modeled as a document with its own URI. You can email links to particular maps, to driving directions for your house, or to maps annotated with places of interest. Each such document is available, using the same URI, from the non-Javascript implementations, and in principle, server agents can crawl links to the different maps (e.g. to find maps that are popular).

So, using Google Maps as my poster child:

Where practical, model your application as a collection of documents, each with its own URI

Syntax: # or ? or neither

Note: this section was updated on 10 March 2010 based on the important observation from Jeni that the query component isn’t necessarily more important than the hierarchical path as an alternative to # fragment syntax.

I continue to have a general feeling that fragment syntax (#) has been over-emphasized in all this, and that identification using other parts of the URI, such as the path for hierarchical identification and/or the query string (with ?) for non-hierarchical may be preferable. Yes, for that to happen, we’ll have to get changes like pushState() and changeState() widely deployed, or maybe we’ll need other changes too. Still, from an architectural perspective, # seems somehow more fragile: until we ran unto address-bar update problems, fragments were used with documents to identify, well, fragments. Yes, with new media types we can do what we want, but in this case we don’t have a new media type, and the normative specifications make clear that for text/html, fragments refer to anchors within the document.

To be clear, I don’t think there’s any outright architectural barrier to server-side interpretation of fragment ids in cases where agents like crawlers happen to get hold of them, but of course, typical clients don’t send them to the server at all. Maybe we can find a way to use # that meets all the goals, and update the normative specs to support going that way. My guess is that, in the long term, avoiding fragments by using path and/or query is a better solution. Either way, we need to align the specs. with common practice.

Anyway, my main point is to model things as documents where possible, and to give each document a unique, crawlable URI that works consistently at client and server.

Finally, I hope it’s clear that I am in no way discouraging Ajax implementations, whether for résumés, simulators, or maps. I don’t want to let Ajax be an excuse for building a Web full of documents that can’t be properly linked.

Documents in applications by Noah Mendelsohn is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Topics: Web, Internet, Computing | No Comments »

Submit a comment: