Friday, September 21, 2007

Asynchronous JavaScript And XML (AJAX) -- And Why It's Not Enough

Asynchronous

AJAX applications typically use XMLHTTPRequests, a communication feature built in to modern browsers such as Mozilla/Firefox and IE.

Ironically, the real value in XMLHTTPRequest isn't that communication can be asynchronous. Form submissions, the previous way of handling server communication, were and still are asynchronous. XMLHTTPRequest actually makes it possible to support _synchronous_ calls and provides a single unified way to make either type of call.

Most writings on AJAX would have you believe that's enough -- make an asynchronous call, get an AJAX application. Unfortunately, as anyone who's done much programming with asynchronous calls (aka threads) can tell you, making the call is the easy part -- it's coordinating the results that's hard. A simple real world example highlights "the sync problem".

The Sync Problem

Its a common client-side requirement to fetch a template containing static data along with dynamic data from a web service call, blend them into a single UI element, and then display that element in a portion of the user's currently displayed page. In this common case you've got two separate data sources meaning you have two separate calls whose results have to be blended only after both complete. When confronted with this reality most developers switch to making synchronous calls (taking advantage of XMLHTTPRequest's truly new feature to "cheat"). If you decide to hang in there and stay with the asynchronous call format that's great, but we've yet to see a single AJAX toolkit that offers you any kind of multiple-request coordination -- so you'll end up writing the synchronization logic yourself.

If you use TIBET its simple:

// set up two requests to make the async calls (you can have more if you like)
request1 = ServiceRequest.create(dc('async',true,'other','parameters'));
request2 = ServiceRequest.create(dc('async',true,'some','parameters'));
// create a function to observe the completion signal we'll be sending below
response = function(aSignal) { alert('The requests are done'); };
// create an "And Join" saying both need to complete (TIBET also supports "Or Joins")
join = TPAndJoin.create('AllDone');
join.addObservation(request1);
join.addObservation(request2);
// tell our response function to observe the signal our join will send
response.observe(join, 'AllDone')
// activate the requests...TIBET handles the rest
request1.activate();
request2.activate();

Since TIBET was built before you could "cheat" via XMLHTTPRequest it's been designed to support asynchronous event coordination from the start. TIBET uses event signaling and a set of types based on the Workflow Management Coalition's petri-net models for workflow event coordination to handle event synchronization. Because it uses events rather than XMLHTTPRequest callbacks you can use the previous example's pattern to join responses from both the server and the user since interactions with the user are asynchronous as well.
Simply put you won't find a better library for managing the full spectrum of asynchronous browser behavior than TIBET.

The URI Problem

One of the core problems in building web applications is the fact that URI paths often have to be fully-specified. While relative URI addressing solves some of the issues, it's all too common to find yourself working with pages and source code that use full URI paths...and that's a real problem.

Whether you're relocating an application, deploying a demonstration copy, trying to develop against a test server, pushing a release to production, or simply supporting developers on multiple platforms if you can't support a flexible URI specification model you'll end up with maintenance headaches.

TIBET solves the URI problem through a number of mechanisms.

First, TIBET supports a unique ~ (tilde) syntax reminiscent of Unix paths (think "home"). TIBET will automatically adjust ~ paths to reference the current application or library "home" so that all your URI references resolve _without code changes_ -- even when you switch between file and http-based launch models. You can also create your own variables as reference points. For example you can define XSLT_HOME and then reference ~XSLT_HOME/somefile.xsl as a way of keeping your URI paths organized yet flexible.

Second, TIBET lets you map URIs using a configuration file that supports a variety of important features including user and role-specific aliasing, development-mode (devl, test, prod) aliasing, and custom load and storage handlers. Using TIBET you can write code that references ~/foo.html and map that URI to different concrete URIs based on user, role, and mode. Then you can define different load and storage handlers to let you customize how data is managed, including building server simulations so you can develop in complete parallel while the server team builds out your web services support layer.

Third, TIBET supports local cache management built on top of standard HTTP header information combined with industry-standard duration syntax. Using TIBET you can define local stores for common lookup tables, leveraging the standard HTTP expires header data or overriding it with a duration. For example, you might say that ~/lookup_codes.xml should be stored locally at /usr/local/dat/lookup_codes.xml and updated with a duration of P7D (every 7 days). You'd combine this with an alias for ~/lookup_codes.xml that says read from that local store. Now your application only loads that data over the network once a week. Want to change it? Just update the cache entry for the cache file itself and a new set of rules will be delivered.
We can't imagine a web framework that doesn't include types for such fundamental web constructs as URIs, but again, we've been surprised to find most AJAX toolkits don't make much use of OO, and when they do they neglect fundamental types like URI. TIBET's TPURI type and it's various subtypes build on top of our cross-platform URI primitives to give you a complete set of Object-Oriented types and methods for URI management.

The HTTP Problem

Most writings on AJAX limit their discussion to HTTP's two main verbs: GET and POST, but there are 5 others (PUT, DELETE, HEAD, OPTIONS, TRACE) and they can have real value for serious application developers. Unfortunately few AJAX toolkits offer support for HTTP's full verb set. In fact, GET and POST are supported via Form submission so if you're just going to do GET/POST in an asynchronous format you've gained almost nothing over what browsers have been able to do for a decade.

To get full value out of XMLHTTPRequest you need to be able to control both the command verb and the HTTP headers that are sent so you retain control over how calls are made and how the server is likely to respond to them. You also need complete control over data encoding so you can adapt as needed to the various services you may have to integrate with. As with "The Sync Problem" this is another area where most AJAX toolkits fall down on the job.
Dealing with the full spectrum of HTTP and HTTPS options, as well as things like redirects is critical to any Enterprise-Class solution.

TIBET Low Level Protocols

TIBET leverages XMLHTTPRequest as do most true AJAX offerings, however TIBET adds a set of fully-instrumented cross-browser primitives for "low level" web and file-based data communication including support for two protocols not normally associated with AJAX, namely WebDAV and local file access.

This gives TIBET a core set of transport protocols that include:

HTTP(S)
WebDAV
FILE://

Since you may need your applications to run from either HTTP or file URIs TIBET layers a second set of primitives on top of the standard HTTP and file primitives so you can invoke a common "load", "save", "lastModified" API that insulates you from the specifics of where the application was hosted. These are our URI primitives and they're the foundation of our solution to "the URI problem".

TIBET's low level protocols can be further enhanced by leveraging TIBET's built-in SHA1, MD5, and 256-bit AES encryption capability, allowing you to secure your most sensitive data end-to-end or to support authentication protocols such as digest-based authentication which can help avoid putting passwords or other data in clear text over the wire.

TIBET Web Service / Enterprise Protocols

Layered on top of those core transport protocols TIBET adds support for today's common web service and Enterprise service protocols. While several AJAX offerings offer SOAP support, TIBET adds support for several protocols which are equally valuable including:

SOAP
XML-RPC
REST (CGI)
XMPP (Jabber)
RSS
DSML (Directory Services Markup Language)
NNTP (Network News Transport Protocol)
With TIBET you can build applications that can speak to virtually any data source, not just those with a SOAP API.

TIBET Prebuilt Service Proxies

Low-level function calls are all well and good and TIBET provides some of the most comprehensive around, but for ease of development TIBET encapsulates access to each of our supported data formats behind Service, Request, and Response types you can subclass for specific web services for maximum reuse and maintainability. With TIBET's support for Web Services Description Language (WSDL) you can even generate Service, Request, and Response types from a WSDL file to help your development go even faster.

TIBET ships with sample service wrappers for:

Google
GMail
Zoe
Amazon
Blogger
Syndic8
Meerkat
XIgnite
More...

Of course, TIBET's Request and Response types are designed so they work identically regardless of whether you're making a synchronous or asynchronous call. You implement a simple result handler for your request and TIBET will make sure it's invoked at the proper time.

TIBET Real Time Collaboration / Workflow

The full scope of TIBET's support for asynchronous data is most visible in our XMPP (Jabber) module. TIBET includes a complete set of types for interacting with XMPP servers. That means you can seamlessly integrate presence information, instant messaging, or distributed events with your client-side applications.

With TIBET you can build applications that send and receive XMPP packets in near-real-time. Arrival of new packets triggers automatic notification of XML Event-based listeners, allowing you to create data-driven, collaborative, workflow-oriented applications using W3C and IETF approved XML standards. Sending a packet to an XMPP targeted resource allows you to message other services or users in real time.

TIBET's development tools provide an example of this functionality by including an XMPP "shell", an interactive chat console that makes use of the XMPP library components to instantly connect you with other TIBET developers world-wide. That's just one example of the power of XMPP and TIBET.

In short, if you're looking to build Enterprise-Class applications that leverage asynchronous data you can't do better than TIBET.

JavaScript

At the center of AJAX is JavaScript but, as many of us learned the hard way, doing Enterprise Class code in JavaScript is difficult and time-consuming. Simply giving DHTML programming a new name didn't magically upgrade all our web browsers ;). It's still hard work.
In addition to well-known issues of cross-browser consistency (or lack of it), JavaScript's support for object-oriented programming is limited and features and tools necessary to support fast development while ensuring quality such as error logging (ala Log4J), unit testing (ala JUnit), and interactive development tools (ala Eclipse) are often non-existent, even among commercial AJAX and "rich client" offerings.

TIBET -- Fixing JavaScript In JavaScript

To make JavaScript truly Enterprise Class TIBET adds a number features built from the bottom up in a layered architecture that includes:

200+ compatibility "primitives" for JavaScript, XML, and XSLT processing, giving you a single, cross-browser API foundation

a powerful XML logging layer and component-specific filtering and leveling, allowing you to constantly monitor activity

encapsulation including set() and get() calls with state change notification, giving you support for full data-driven design

object-oriented support including an XML-based meta-object system with reflection, maximizing reuse and flexibility

unified DOM Level2 and XML Events-compliant event handling, removing complexity and adding true MVC support

powerful URI, Node, and Service/Request/Response types, unifying how you access both data and UI elements

using XInclude, XMLBase, XSLT, and XPath, supporting extensible modular development
a powerful object-oriented scripting layer that lets you script all of this power using Unix-like pipes and redirection

While this may sound like a lot of overhead, TIBET application packages are typically less than 256K in size, load once on browser startup, and can launch in as little as 5 seconds -- faster than many desktop apps. Your users experience a short delay at the start of their work day, then get the performance benefits of enhanced caching and data management along with insulation from network and server failures all day.

TIBET Development Tools

To support the requirements of Enterprise Class development schedules, TIBET is augmented with powerful development tools for creating browser-based applications including:
a fully-interactive browser-based IDE, the TIBET Application Portal (TAP™), giving you point-and-click development interactive xslt and xpath visualization tools to support your XML-based development processes a powerful UNIX-like command shell with full runtime code access, letting you manipulate your running applications reflection-based unit testing as well as a complete port of the xUnit APIs, helping you ensure quality at every step If you're sick of hitting Reload then you need TIBET. Found a bug on page 3 of your application? With TIBET you simply ESCAPE to bring in the TAP tools, fix the bug interactively, then save your fixes back to any WebDAV server or your local file system. With any other approach you have to exit, edit the source file, recompile the page, relaunch the app, re-enter data on pages 1 and 2, then hope you fixed it. Lather, rinse, repeat. We call it "Reload Hell" and with TIBET it's a thing of the past along with those dreaded "Preview in Browser" steps.

Using TIBET's tools is like working in an environment combining the best of NeXT's (Apple's) Interface Builder, the best of Squeak, and the best of Unix.

Perhaps best of all, you can get most of the advantages of the TAP even when building non-TIBET applications. Your native JavaScript, DOM, CSS, and XHTML are all accessible to TIBET's tools regardless of whether you're including code from the TIBET Development Kit (TDK).

and XML

The final element of AJAX is "and XML". TIBET supports XML "and how!". While modern browsers like Firefox and IE, and hence most AJAX toolkits, provide baseline XML support they fail to provide consistent support for that baseline and often don't support important standards for Enterprise Class authoring like XML Events, XInclude, or XMLBase that let you modularize your development.

TIBET provides cross-browser support for the following XML standards without applets, plugins, or server add-ons:

XInclude
XML Base
XML Events
XML Schema (all 46 built-in types)
XPath (plus extension functions)
XSLT (including parameterized sheets)
XForms
SOAP
XML-RPC
XMPP
DSML
RSS

With TIBET, XML support is built in from the bottom up. TIBET takes internal advantage of XML in these areas:

TIBET Boot System - "Ant" formatted XML files describing application modules, scripts, and inline code to load

TIBET Metadata - XML formatted data suitable for XSLT into OMG-standard XMI format for OO diagramming

TIBET Logs - stored in XML suitable for XSLT filtering and formatting for debugging or other uses

TIBET Signals - W3C standard XML Events-compliant XML including fast-loading persistent registrations in XML

TIBET XMPP - IETF standard messaging and presence for pub/sub, distributed events, workflow, and IM

TIBET Shells - XML-based history, snippet, and other environment data storage for sharing / manipulation

On top of this foundation, TIBET allows you to create your own XML-based custom tags and XSLT chains, meaning you can model common application elements or blocks of complex XForms/XHTML markup in simple reusable "macros" that are processed client-side. When you're trying to build complex Enterprise Class applications this can mean hundreds of hours of saved effort.

There's no other product, open or closed source, that can match TIBET for XML support in a web browser.

SUMMARY

Thanks to "AJAX" JavaScript and XML are getting a lot more attention -- and that's a good thing. Unfortunately, our experience has shown that unless you're able to spend a lot of time fighting cross-browser issues, browser bugs, and a severe lack of interactive development tools you'll find your AJAX projects failing to meet their delivery goals on time and under budget.
If you leverage TIBET in your projects you'll have the benefit of the web's most interactive development lifecycle and the web's most comprehensive JavaScript/XML framework. We've spent the past 6 years "writing JavaScript so you don't have to" -- and while others may still want to call it AJAX we'd rather just call it TIBET.

No comments: