The best JavaScript stack I have ever used

December 23, 2011 6 Comments by Richard

JavaScript as a real language

Like many, the suggestion that JavaScript could be used in a fully architected system initially left me doubting the sanity of the person with whom I spoke. Over the years I have done several different types of Web Development; I am best known for my work with browser plugins in general and the FireBreath framework in particular. What most do not know is that I actually started out doing a lot of PHP and JavaScript development and that it was actually my work with JavaScript that eventually led me to learning about NPAPI and ActiveX plugin technologies.

Throughout all this I experimented with many different types of javascript code; I wrote AJAX applications long before I’d heard the term coined using a system called JSRS — which is surprisingly still around, or at least the page I got it from. I worked for Move Networks on their JavaScript SDK and I wrote several AJAX heavy websites, such as HamStudy.org. Many reading this will scoff when I admit that while I considered myself fairly well versed in JavaScript I still wrote it as if it were a simple scripting language. Others may wonder what is wrong with that? JavaScript more than perhaps any other language I have used gives you just enough rope to shoot yourself in the foot with. I have learned over the last few months just how little I knew — and didn’t know that I didn’t know.

My upward climb started off simply enough — I wanted to learn about some of the new technologies that I had heard about; Backbone, MVC frameworks, client-side template libraries… I had written my own “frameworks” on which to base my websites, all of which were side projects, but I found myself spending much more time than I liked updating and maintaining those frameworks. I thought perhaps I could use someone else’s system and save myself some trouble.

I purchased a book called JavaScript Web Applications which claimed to teach how to write an actual application with JavaScript. The table of contents looked interesting, and besides teaching about the principles used it also had instructions on several different MVC frameworks that I wanted to know about, including Backbone.

This book was the beginning — as a result of this purchase I soon came to grips with just how comparitively incomptetent my old code had been, and started to actually understand that JavaScript can indeed be a real language. Over the course of a few posts I would like to share with you the stack that has grown out of this system: What the components are, how they help, and why they are integral to the overall solution. I will start out in this post with the basics and my next post will detail the specific tweaks that we have made to them that have made our system so successful.

Dependency Management

Having been so long-winded with my earlier explanation, I will now spare you the details of all of the things I tried, researched, or experimented with during my search. One thing quickly became clear to me, though: Perhaps the #1 reason that my previous code had ended up so ugly was the need to avoid creating lots of JavaScript files. These can be difficult to deploy cleanly as it is not a good idea to have too many script includes in your production site and combining them can be an annoying process. The end result of this problem is that I had about 10 JavaScript files — and felt guilty about it — and all of my functionality was stuffed in those files. I had page-specific files, site-specific files, and general reuse library files, and all of them tried to register themselves cleanly in the global scope without clobbering each other. In short, it was a mess.

Enter require.js. Require.js is a system in which all javascript files are wrapped in a function. This is actually a really common pattern, but with require.js that function is passed into the “define” function call along with a list of dependencies. Require will first resolve these dependencies and then when they are all resolved it will call the function you provided, passing in the exports of each dependency as arguments in the order that the dependencies are listed in the define call.

The question left unanswered, then, is what are the exports of the other dependencies? Simply put, each “module” exports some value by simply returning that value from the function passed into the define call (See file1.js and file2.js).

[gist id=1500557 file=reqfile1.js]
[gist id=1500557 file=reqfile2.js]

The natural result of so much flexibility is a lot of small files; this is really good for code organization because it makes it easy to seperate things into logical pieces and even to include whole modules consisting of multiple files (because one can depend on multiple others and combine them in its return value). The downside is that with lots of small files your page load time starts to get a bit out of control. Fortunately, require.js has an answer for this problem as well!

Require.js has an optimize phase that will examine the dependencies of your main file and combine all of them into a single file so that they are all loaded at once. The require.js system still makes sure that the functions are called in the correct order, but makes it very easy for your production site to be seamlessly updated to use a combined and minified single javascript file instead of the potentially dozens of small other files.

To me, this was a nearly life-changing WIN.

JavaScript Templates

If dependency management became the #1 most useful change in my architecture, a close #2 would have the be JavaScript Templates. The more I did things on the client the more client-side HTML I created in JavaScript; I experimented with several methods but still ended up with really hard to read html mixed in with my JavaScript as strings. When the time came to update one of those sites it was painful; not all of the code was in the same place, it wasn’t easy to read, it wasn’t easy therefore to change.

I considered different types of templates but had previously rejected the idea. The problem with templates, you see, is that you still have to include the template code somehow. The most “traditional” method would be to pretend you’re on a backend and just store them one template per file. Of course, then if you have any real number of these files you find yourself making a multitude of AJAX requests to download all of these templates; it’s clean from a file structure perspective and fairly easy to use, but to me the added complexity of all those requests is not acceptible. Other options include storing the template in javascript strings (same problem as I had before without templates) and storing them in a special script tag on the html page, which had the (to me) serious disadvantage of being nearly impossible to reasonably use on multiple pages. It was a problem.

Enter require.js and the text plugin. The text plugin of require.js is an idea both simple and brilliant; it allows you to specify files that should be loaded and the text of those files will be passed into your page function as the value of that dependency. With the text plugin you can save your templates each in their own file and have require load the text when the page loads. Of course, this doesn’t solve the problem we mentioned earlier! You still have potentially lots of AJAX requests being made to load those files. Problem? Not at all! Remember the optimize step in require.js; just like the normal dependencies are loaded and combined into one file the text plugin provides a mechanism to serialize those templates into javascript as strings and combine them into that same file. You get the best of both worlds! When you’re developing you have the flexibility of having each template in an easy-to-read file, but when you deploy those templates are packaged with your javascript.

[gist id=1500557 file=reqtpl.js]

Top all of this off with the fact that for several of the most popular template libraries there are already require.js plugins, such as requirejs-tpl. These plugins work like the text plugin, but they compile the template string into an actual template function before passing it back to resolve the dependency. Then, when you optimize, they actually compile your template code into javascript functions and minify them along with the rest of your JS code!

To me, this was a nearly life-changing WIN.

Deferred Objects

This may seem like it doesn’t fit with the other items on this page, but if templates and dependency management have revolutionized the organization of my code then deferred objects have revolutionized the way that I write JavaScript code. One of my coworkers at GradeCam, Robin, has already begun a small introductory series on how deferred objects work, so I’m not going to go into those details, but I do want to explain the conceptual shift in how pieces of the system interact that this led me to. It is both subtle and extremely powerful.

Most JavaScript operations anymore are asynchronous; in other words, they return immediately even though the operation in question hasn’t been completed. This can make it difficult to handle result cases; the traditional method is to pass a callback into the function to be called when the operation completes. You then perform some other operation once that function is called back and so forth.

The challenge with that is that it is a significant deviation from the patterns that we are all accustomed to. Granted, it’s important to understand what is actually¬†happening, but often you may want to call a function in a place where you don’t actually want to handle the results; a good example is when you are requesting data from a model in your controller code, but you will actually be rendering it withing your view and your controller doesn’t really care what the results are. The view, of course, should not need to know or care where the data came from. In fact, it shouldn’t really need to know if the data is immediately available or will be available later on — it should display it when it is available¬†and not worry about the other details.

Code speaks louder than prose, so here is an example of the difference between these two approaches; in both async1.js and async2.js three async operations take place. In async1.js they are handled in a traditional way, and in async2.js they are handled using jQuery.Deferred.

[gist id=1500557 file=async1.js]
[gist id=1500557 file=async2.js]

As you can see, the Deferred object method is slightly more complex at first; however, the complexity is where it belongs. Each method need know nothing about the others — they all assume that the input value *might* be Deferred. They return a Deferred object, and so they use the Deferred.pipe method, which makes it easy to chain deferred objects. Because each of the methods returns a Deferred object and is aware that it may receive a Deferred object, you can call one after the other as though they were happening synchronously — because indeed they will end up doing so, in all practical purposes. However, you shouldn’t think of this as hiding the fact that it is asynchronous, but rather as returning asynchronous return values.

Because we used $.when, you can pass a regular parameter in to any of those methods as well and chain them as much as you want. It will handle them all perfectly.

To me, this was a completely life-changing WIN.

This is just the beginning

I have a lot more to talk about here; this is just the foundation. If you know nothing more about our stack, you should understand these things. Next time, though, I’ll talk about how we tie this together using Backbone.js Routers, Views, and Models, how those tie into our templates, and how Deferred objects allowed us to add data cacheing to our data models completely transparently — without changing a line of code outside of the model classes.

6 Comments

  1. Nate
    5 years ago

    I’ve just been coming to the same conclusion when it comes to javascript stacks. Looking forward to the next installment when you talk about Backbone.

  2. Carlos Gant
    5 years ago

    Thank you for your article, I found it very useful

  3. Karl
    5 years ago

    Twenty years later I still code using JS as a script language. I choosed ExtJS over jQuery, then testing Backbone. I feel I’m always in the other side to the river…

  4. Peter Tseng
    5 years ago

    Nice post – good solutions to some of the major difficulties in JS coding.

    BTW, something seems to be wrong with the code sample views. I only see a tiny line for the content (“view raw” is fine as a workaround though).

  5. Tulsi
    5 years ago

    Very useful. Thank you.

  6. Scott Silvi
    5 years ago

    A few weeks down the require/backbone path in my first production app & I agree… it’s a beautiful thing to behold. No more callback hell. No more spaghetti dom manipulation. Makes me chuckle to think I used to say “yeah I’m a javascript developer”…

    Having the same experience as Peter – can’t see your gist’s unfortunately. Using latest chrome. Also able to view raw as workaround, but it takes away from the flow.

Post a Comment

Your email is never published or shared. Required fields are marked *