new Module(‘patience.js’);

A while ago I came up with my own javascript ‘require’ command, which I recently spun off into it’s own repository and test suite.

Testing

I had previously done a little with QUnit. The first thing I did after pulling (most of) the require history out of the Naked Javascript repository was set up some tests. The require mechanism had picked up some non-trivial behavior, and more than likely the logic would become even more convoluted as I tried to solve my outstanding problems – all without breaking existing features. (In the end I totally renamed things, but I’ll still want the same result.)

Since the next file can’t load until the current one completes, I first had to set up a mechanism to wait and come back. Immediately after putting this in place, I discovered that QUnit already had an async test feature.

After characterizing the existing behavior – a process that continued as I discovered new cases to examine and often correct – I started working in a more test driven fashion. I would still often write the test after – sometimes with intentional breakage to make sure the test worked – but one test in particular, “out-of-order mutual dependencies” was failing for a large part of the project as I built up the infrastructure to handle my challenges and make it pass.

Challenges: Not Invented Here and Creeping Featurism

I made my own mechanism because I didn’t really like any of the existing solutions. Some used AJAX, others were huge packages that included lots of other features. Of course, as I’ve added features, my own solution has grown steadily larger, and seems well on it’s way to becoming a similar monstrosity.

The first generation worked well enough on local files with a manifest, but failed to achieve true modularity of individual files and fell down on real web servers. The new version should fix these problems. The solutions required more code of course – about 100 new lines, (from ~130 before the latest iteration) of which I can only account for about 75 attributed to particular features.

Challenges: Network Delays

The old system worked on my local filesystem, but I upload some demos to my website only to discover that it took several reloads before things started working.

The new system passes it’s test suite online, provided the tests are given a more generous time delay before they declare success or failure. I could streamline the delays by including the test restart in the modules being loaded, but deciding which test files should include a restart sounds like a potential tar pit of tweaking, and the timeouts useable on the local filesystem are small enough for practical flow.

Challenges: Reloading

Both problems – failures due to network delays, and localized dependencies, have a related solutions – being able to stop and resume a particular file.

I know of two methods to force a runtime dependency system down Javascript’s throat. Probably the more popular one is using AJAX to download and eval the script at run time. Two things I don’t like about this is that makes all the required code exist in a different state (evaled vs. script tag) which can make debugging more difficult. It is also a stop-the-world method which fully serializes file downloads. I suppose you could arrange to place the continuation in a closure, but this affects the mechanism’s syntax significantly.

The other method in the wild is inserting script tags at run time. This technique allows dependencies to be downloaded in parallel and preserves the script source information.

One major fault of the script tag technique is that it requires the whole file be aborted and re-run a later time upon encountering an unmet dependency. This can be wasteful, but it doesn’t appear that actual duplicate downloads occur. Once I’d chosen this technique, I had a few sub-challenges: aborting a script, running it again, and detecting when dependencies were actually loaded.

Challenges: Aborting a Script

It actually turns out this is pretty easy, although I’ve only tested in two browsers so far. Throwing an exception aborts the current script file, but later script files will still be attempted by the browser. I defined a special exception for unmet dependencies and set up an error handler to keep it from getting thrown in the user’s face.

Challenges: Running a Script Twice (or more)

This also turns out to be fairly easy: just add another script tag. The browser will download the file once and run it multiple times, at least in Firefox’s case.

Challenges: When Is a File Loaded?

This isn’t so easy. There is an event fired when script tags are complete, but ‘complete’ includes failure due to an exception. This took a little closure slight-of-hand. The event that fires on load, if it is to communicate with the outside world, has to be a closure referencing some data. My event references a one-member object. One other reference to this object exists. When a script file gets aborted, the ‘other reference’ gets thrown away and replaced with newly minted object, attached to the new event handler. The old event handler updates a unreferenced object, and at some point the both of them get garbage collected.

Challenges: Pathing

One feature I wanted was the ability to reach across my filesystem and pull in Naked Javascript so I could poke around in the structures of my other projects. This required a method of modular pathing that I wrote about earlier.

Between tracking the paths and file-loaded data, the somewhat ad-hoc collection of persistent arrays and object I was using to track things slowly formed themselves into into two new objects: Dependency and Module.

Path to Self Improvement

Before, combining relative paths and other forms of aliasing could be handled sufficiently well by recording a hash from relative path to absolute path. However, with the need to track load status, I had to have some place to record that information amongst all possible aliases. That place is the Dependency object.

One of the useful things I discovered is that once you create a script tag, even with a relative path, you can read out it’s fully qualified path. This offers a last chance to spot identical files and enforce require-once. The canonical path is recorded with the Dependency, and the file mapping is stored with the Module.

Module collects not only the map from paths to files, but also the information on current relative and absolute or root path. Both of these last were stacks in the previous iteration, but now are properties of the module. Descending into a sub-directory creates a prototypal derived object with an updated path property, but re-using the rest of the information; the sub-directory is a very real sense a virtual sub-module. Since the module passes itself to it’s constructor, all operations are properly scoped, without having to deal with the exception catching I was using previously to unwind my path stacks.

It also leads to a novel syntax for defining modules:

new Module('script/blarg.js', function(m) {
	m.require('bleep.js');
	m.require('foo.js');
})

Passing the script name is what what allows us to know which file to reload if a dependency is missing. There is a potential for error here, but since Javascript and the browser don’t offer us any help, we have to help ourselves. I’ve favored the short ‘m’ so far, but I’ve mainly been writing tests. Naming the parameter after the module, e.g. ‘blarg’, may offer better readability.

Challenges: Browser Portability.

I’ve done limited testing so far. In the long run I’m not too concerned: the dynamic version is targeted for development, so it really only has to work on my browser. A companion method, yet to be developed, will read the module declarations and concatenate all the dependencies into one file. The build process could also incorporate compression if desired.

Posted Tuesday, November 17th, 2009 under Devlog.

Comments are closed.