Tag Archives: Javascript

For us, the largest benefit of Javascript templating is reduced size

There are quite a few Javascript templates. In my projects however, there are very few cases where I would prefer using any of them in place of regular HTML being pushed out from the server (running Ruby-on-Rails). The same can be said of the Ponzu conference system.

As far as I understand, the benefits of using Javascript templates are 1) reducing the load on the server (generating JSON is less load than generating full HTML), 2) speed if used in combination with a single page design.

The downside is the additional work that browsers have to do, which can be a problem on mobile where the devices are not as powerful as their desktop counterparts.

I’ve touched this subject before in these two posts [1](https://code.castle104.com/?p=289), [2](https://code.castle104.com/?p=291).

As discussed by David Heinemeier Hansson, the same benefits can be achieved without Javascript templates by using a PJAX/Turbolinks/Kamishibai like system that eliminates reloading Javascript and CSS on each page transition, and the use of aggressive caching on the server side to reduce the load of HTML generation.

There is a real case however, where I feel a strong need for a Javascript tempting language.

That is when I try to cache responses in browser side cache. The issue is that HTML is extremely verbose, and is a real killer in terms of storage consumption when you are working with repetitive content.

For example, the following is a “social box” that we use in Ponzu for the like button and a voting button. It takes about 2,000 bytes. Each social box is associated with a presentation so we have hundreds to thousands of social boxes for each conference. This can easily fill up the limited browser side cache.

<div class="" id="presentation_326_social_box"><div class='like_box'>
<div class='like' id='like_button_326'>
<span class='social_controls'>
<!-- To invalidate Like related paths, we need a like object -->
<a href="/likes?like%5Bpresentation_id%5D=326" class="button icon like" rel="nofollow">like</a>
<div class='prompt_message'>
(
To add to your schedule, please &quot;like&quot; it first.
)
</div>
</span>
<div class='social_stats'>
<img alt="Like" src="/assets/like-c3719a03fc7b33c23fda846c3ccfb175.png" title="いいね!を押すと、応援メッセージになります。またあなたのスケジュールに登録されます。" />
<a href="/presentations/326/likes?user=1">15 people</a>
liked this.
</div>
<div class='likes_list' id='likes_presentation_326'></div>
</div>

</div>
<div class='vote_box'>
<div class='like' id='vote_button_326'>
<span class='social_controls'>
<form accept-charset="UTF-8" action="/likes/vote" class="new_like" id="presentation_326_new_like" method="post"><div style="margin:0;padding:0"></div>


<span>

<label for="presentation_326_like_score_1">Excellent!</label>
</span>
<span>

<label for="presentation_326_like_score_2">Unique!</label>
</span>
<span>

<label for="presentation_326_like_score_0">No Vote</label>
</span>
</form>
</span>
</div>

</div>
</div><div id="session_details_presentation_326"></div>

Most of this content is repetitive and will be identical for each “social_box”. In fact, the content that is unique to each individual social box can be summarized in the following JSON.

[{ 
    presentation_id:326,
    user_id:1,
    score: 0,
    liked: 0,
    scheduled: 0
}]

If we could use Javascript templating to generate the 2,000 byte HTML from this small set of JSON, local storage savings would be huge.

This is one feature that we will be adding to Kamishibai and Ponzu in the near future, to enable the ultimate goal of complete offline viewing.

On JavaScript MVC (part 2)

In Kamishibai and Ponzu, speed is a huge concern. This is particularly true for smartphones. In desktop web sites, it is possible to cram a lot of information and navigation elements into a single page to make up for slow page loading. You can see this in news websites like Asahi.com where 80% of the top page consists of navigation and shortcuts. The idea is that instead of asking the user to click a link and reload a new page (which is slow), the user simply can scroll down to see more content. With smartphones, cramming all this information is simply not a good idea and we have to reload pages.

One way to reduce the load time for pages and to update only the parts that you want is to use Javascript MVC or client-side MVC. With client side MVC, the pages are not reloaded as the content is switched. Instead of sending HTML pages, the server sends JSON data to the browser and browser-side javascript is used to construct the DOM from the JSON data. The advantage is that the client does not have to reload the whole page, and that it can intelligently update only the portions of the DOM that need to be redrawn.

In Kamishibai and Ponzu, we seriously contemplated using these Javascript MVC frameworks. However, we decided not to do so. Instead, our approach is similar to how GitHub handles updates with PJAX, and with how the new Basecamp uses Turbolinks. David Heinemeier Hansson gave a presentation describing why they did not use Javascript MVC extensively and a video is on YouTube.

There are other highly respected programmers who question the use of Javascript MVC. Thomas Fuchs, the author of script.aculo.us and Zepto.js has this to say in his blog “Client-side MVC is not a silver bullet” and his comments on a post about one of his projects, Charm.

I’ve come to the realization that this much client-side processing and decoupling is detrimental to both the speed of development, and application performance (a ton of JavaScript has to be loaded and evaluated each time you fire up the app). It’s better to let the server handle HTML rendering and minimize the use of JavaScript on the client. You can still have fast and highly interactive applications, as the new Basecamp shows—letting the server handle most stuff doesn’t mean that you have to cut back on cool front-end features and user friendliness.

We’ve spend a lot of time getting Backbone to work properly, as the easy-of-use quickly deteriorates when your models get more complex. It’s a great choice for simple stuff, but email is far from simple. We also had to add yet an other extra layer of processing to generate “ViewModels” on the server because the normal Rails serialization of objects wouldn’t cut it.

If you do any non-trivial resources, you’ll quickly end up with JSON objects that are just too large, especially for lists. For emails, imagine that you have nested threads, user avatar images, nested assigned cases, etc.
Because of this, you’ll need specialized JSON objects/arrays for different use cases (search, list view, detail view, and others). It follows that you’ll end up with this with more or less any front-end framework (if you care about performance!). Doing this adds complexity, which can be avoided by rendering HTML on the server where access to arbitrarily deeply nested data is relatively cheap (and can be highly optimized by keeping snippets of HTML around in memcache, etc).

In Ponzu and Kamishibai, we ended up following the traditional Rails route, enhanced with techniques that are seen in PJAX and Turbolinks.

What to we want to achieve

For us, the reasons for contemplating the use of a Javascript MVC framework were;

  1. Speed
  2. Use without network connection (offline apps)

A common benefit for Javascript MVC is interactivity, but this was not a concern for the types of web applications we had in mind which tend to be very read-heavy.

David Heinemeier Hansson has written a detailed post on how the new Basecamp dramatically increased performance through PJAX-like techniques and extensive server-side caching.

This technique basically nullifies the first benefit, because it makes it very easy to increase performance. Furthermore, the code will be almost the same as traditional Rails and hence very simple.

The second issue, offline usage, is something that even most Javascript MVC frameworks do not support very well. This is even more so when the data gets complex. Hence this in itself is not a compelling reason to go Javascript MVC; we would still have to figure out how to do offline usage effectively ourselves.

Why do people use Javascript MVC?

I came across this blog post describing why they used a Javascript MVC framework (Ember.js) instead of simple jQuery.

The author, Robin Ward, gives a specific example.

For example, on the bottom of every discourse post there is a button a user can click to like a post. When clicked, it vanishes and adds a footer below the post saying you liked it. If you implementing this in jQuery, you might add a data-post-id to the post. Then you’d bind a click event on your button element to a function that would make the AJAX call to the server. However, the click function passes a reference to the button, not the post. So you then have to traverse the DOM upwards to find the post the button belongs to and grab the id from there. Once you have it, you can make your XHR request. If the XHR succeeds, you then have to traverse the DOM downward from the post to the footer, and add in the text. At this point it works, but you’ve tied your implementation of the button click to a particular DOM structure. If you ever want to change your HTML around, you might have to adjust all the jQuery methods that accessed it. If this example seems simple – consider that in the footer we offer you a link to undo your like. When clicked, the footer text vanishes and the button appears again. Now you’re implementing the opposite operation against the DOM, only in reverse of what you did before. Discourse even takes it a step further – we know that 99% of the time when you click the like button the request is going to work, so we hide the button and show the footer text right away, even before waiting for the server to reply. In the infrequent event that request fails, we’ll show an error message and pop the UI back to the state it was in before. If we were doing that in jQuery, we’d have to have a callback on our AJAX request that knew how to put the UI back into the state it was in before. A prudent programmer might say, okay, well I’ll have a render function that can rebuild the DOM to the current UI state of whether the post is liked or not. Then both ‘undo’ and ‘like’ can call it. If ‘like’ fails it can call it again. Oh, and we have to store the current state of whether the post is liked somewhere. So maybe we add another data-liked=”true” attribute. ACK! Just typing this all out is giving me a headache!. Congratulations, your code is now spaghetti, your data is strewn out in the DOM and your logic is tied to a particular layout of HTML elements.

Although I understand Robin’s point and I have also experienced frustration when we want to update multiple elements that are in separate locations, I tend to think that using a full-blown Javascript MVC framework is an overkill. No doubt, DHH and Thomas Fuchs would send Javascript in the response to do the complex updates.

In fact, it is pretty difficult to find an Ember.js example on the web that does stuff that is complex enough that simple Javascript would not cut it.

A more intelligent RJS

Given that most of the use-cases of Ember.js and Javascript MVC frameworks seems to be stuff that could be done with regular Javascript, but might be a bit complex, the more pragmatic approach in my opinion, is to create a small library that would make updating the DOM through RJS-like methods simpler. It could also manage caching, expiration and updating of the response on the server side, so that subsequent requests do not have to go out to the network.

This is the approach that Kamishibai intends to persue.

Furthermore Kamishibai even handles the two-pane interface which is demonstrated in this Ember.js guide. This is possible because we have included a lot of “intelligence” in the library, whereas with the simple RJS approach, there is not library where complicated logic is stored for reuse.

More on this later when we open up the code.

On Javascript MVC

David Heinemeier Hansson gave a presentation on Backbone.js or rather how they are not using it too much. The video is on YouTube.

He mentions how he is using PJAX (or actually TurboLinks) and extensive caching on the server side to make the new Basecamp just as responsive as a Javascript MVC application would.

This is very similar to the Kamishibai approach.

There are a few things that I’m wondering about, and this post is a memo about these.

Why are Javascript MVC applications responsive?

Possibilities;

  1. Because all the Javascript and CSS files are not being reloaded and re-parsed.
  2. Because the whole screen is not being redrawn. Only a small portion is being rendered.
  3. The server is much faster at returning JSON responses compared to rendering HTML.

The first possibility is addressed by the PJAX-like approach.

The second possibility is not addressed by PJAX itself because it renders the whole page. Kamishibai on the other hand, can render segments of pages so Kamishibai addresses this.

As for the third possibility, I find it incredible that the server might be slower than the Javascript client at rendering HTML (or the DOM). Of course, Rails caching can solve this problem and all is well, but I wonder how a client, especially a mobile client can be that fast. Maybe the issue is that desktop view templates tend to be too complicated.

Thoughts on PJAX

Before starting work on Kamishibai, I was working with PJAX. PJAX was a nice and clean idea for implementing the HTML5 History API. The promises of PJAX was speeding up of web loading times by virtue of AJAX partial page loading, while at the same time, maintaining standard URLs and being web crawler friendly.

There were several problems with PJAX that made it difficult to work with and suboptimal for a flaky network environment.

For situations where the AJAX container changed between pages, PJAX simply reverted to re-issuing a full page load. This happened often when going backwards. For example, if we were moving forwards towards a page that changed in only a small fragment. Then we would set the AJAX container to the small fragment and this is all that is entered into the History state object. However, when we move back, then that would mean that we have to redraw more than the just the small fragment; we have to redraw the stuff surrounding it. PJAX simply balked in these situations and reverted to a full page load.

It’s great that with PJAX, there is always a way to simply reload the page and get a regular response that will always work. The issue is that it reverts to this too easily.

Also, this means that PJAX will send out two different types of requests depending on the situation it finds itself in. This makes it complicated and wasteful if we want to store the responses in localStorage. For example, if we do a reload of a deep-linked page, we would receive the whole HTML page. The question is, do we want to store all of it in localStorage under the deep-URL? When we request the same URL via PJAX, should we actually use the HTML body in localStorage? What if the case is reversed and we have an HTML fragment in localStorage, but the browser has requested a full reload. What should we show?

I was also unhappy with the way PJAX tended to waste the History memory. PJAX stores a copy of each AJAX body in the History state object. The problem is that this information can only be retrieved when going back. We can’t use this cache to load pages going forward and we have no way to check whether a certain fragment has already been stored. Given the low speed and reliability of mobile networks and the limited RAM of mobile devices, it doesn’t make sense to be so conservative with the use of precious memory.

Another factor is the lack of support of History API in the Android 4.0 and 4.1 stock browsers. Of course there is Chrome on these platforms, but it doesn’t make sense. The lack of History API on Internet Explorer up to 9.0 doesn’t help either, since it means that half of the browsers in the conference room will be gobbling up unnecessary amounts of bandwidth.

Since PJAX takes a one-page-at-a-time approach to the idea of URL resources, it is very compatible with a pure HTML document model, but is not really efficient for cases where bandwidth is limited. The issue that I had is with the use of the application cache manifest. The application cache manifest work with a full HTML document as the smallest unit (fragment). It does not work with Ajax, and requires a full HTML page reload. Therefore it doesn’t work at all with PJAX unless we cache all entry URLs. On the other hand, if we take a hash-based navigation approach, we will only need a single resource URL in the cache manifest. This URL will return the HTML header and body responsible for the “chrome” of all pages. To put it another way, if we wish to use the cache manifest with PJAX, the only way to do it would be to have entries for every single page that could be an entry-page. This would be extremely wasteful, and even if we could do this, we wouldn’t be able to navigate between these pages with AJAX since AJAX wouldn’t access the cached pages.

In summary, we decided against PJAX due to issues in the current implementation. It is also very likely that we will not use PJAX even if the library improves and browser support becomes more ubiquitous. This is because we feel that PJAX cannot use the cache manifest efficiently.

The need for speed (how Javascript slows things down)

Speed is a very important but often overlooked feature.

Regarding speed on mobile websites, there are several things to consider.

  • Web sites can be very fast to load.
  • Overuse of JavaScript can make mobile websites very slow, even if the resources are cached.
  • jQuery significantly slows down websites, just by being loaded and compiled.
  • Responsive web design is not the same as optimizing for mobile.

Discussion on the web

A detailed discussion about how jQuery can slow down websites was written by Martin Sutherland. I have confirmed this to be true, and this was the first reason I abandoned jQuery Mobile for use in Ponzu (there were many more important reasons, but this was the first show-stopper).

Arguments against responsive web design were put forward by Jason Zimdars of 37 Signals (Behind the speed: Basecamp mobile).

In Ponzu

In Ponzu, we ended up taking a similar route as Basecamp mobile by 37 Signals. Our setup is basically the same except that our Kamishibai Javascript library is much more complicated (but still much, much smaller than jQuery).

With Basecamp mobile, the resource sizes are as follows;

PC HTML:42.92KB, CSS: 515.79KB, JS: 273.80KB
Mobile HTML:16.90KB, CSS: 38.91KB, JS: 14.73KB

You can see that they cut down on the size of JS and CSS immensely. Since the browser has much less CSS and JS to parse and compile, the browser responds much faster after a reload (even when CSS and JS are cached).

With the MBSJ2012 Ponzu system, the sizes are the following;

PC HTML:12.20 + 13.61 + 3.09 = 28.90KB, CSS: 29.30KB, JS: 41.13KB
Mobile HTML:13.62 + 12.69 = 26.31KB, CSS: 18.19KB, JS: 40.96KB

Both Basecamp mobile and Ponzu keep the CSS and JS sizes very small.

Compare this to jQuery Mobile;

Mobile HTML: 2.73KB, CSS: 107.31KB, JS: 91.67 + 286.65 = 372.32 KB

The difference in JavaScript size is staggering.

Where you can see the difference

As mentioned in the articles that I linked to above, the difference in speed becomes apparent when the browser reads in the Javascript and parses it. Caching the response does not help.

When clicking on external links

Whenever browsers move to a new URL, they clear their memory. Therefore even common CSS and JS assets have to be reloaded and re-parsed. This can be prevented if the pages are loaded via Ajax. With Ajax, we keep using the same old CSS and Javascript.

jQuery Mobile mainly uses Ajax when links are clicked. Therefore, once you have loaded a jQuery Mobile page, you won’t notice slowness as long as you stay inside the same site. You will however notice slowness if you click a link to an external site (non-Ajax), and then come back. Coming back will require all the Javascript to be reloaded and re-parsed.

In other words, jQuery Mobile is OK if you stay if you never leave the site. It get’s terribly slow if you move in and out.

When you want to quickly look up something

When you are at the conference venue and you want to check up on you schedule, you want the page to show up fast. You don’t want to wait for seconds until your schedule shows up.

You want to first load to be fast. jQuery Mobile won’t provide that.

When you click on a link on twitter

The great thing about using the Web for the conference system as opposed to a native application, is that you can link to individual pages. For example, you could put a link on Twitter, telling the world how cool your presentation is and why they must come and see it. Anybody seeing that tweet could simply click on the link, and they will immediately see your abstract.

However with jQuery Mobile, it’s going to take a few seconds until the page shows up. Some people might decide that they don’t have enough time to wait and abort.

Conclusion

In Ponzu, we take speed very seriously. Without speed, many of the benefits of the system will be diminished. One of our strategies to achieve this goal, was to keep the size of our Javascript small. Other strategies including caching will be discussed later.

To jQuery or not (jQuery vs. raw JavaScript)

In Kamishibai, we decided to not use jQuery. Instead, Kamishibai is written in native Javascript although we might use CoffeeScript in the near future.

The reasons for not using jQuery are as follows.

  1. Modern browsers have much better JavaScript support and many utility functions are provided natively. For example, we don’t need jQuery map functions anymore since they are provided natively.
  2. jQuery is big. Although this is rarely an issue with desktop browsers, rendering and compiling the jQuery library alone can take something like 2-300ms on a mobile device. This is not acceptable.
  3. A lot of code in jQuery is actually support for legacy browsers. We don’t need that baggage.
  4. The jQuery library is bloated with lots of legacy functions. I’m always confused by the multiple ways of attaching an event to an object, for example.
  5. CoffeeScript solves a lot of the verbosity issues with plain JavaScript. There is no need to use jQuery to clean up the syntax (unless you really dig the functional style).

Regarding item 2., Martin Sutherland has written up a great post titled “jQuery Mobile doubts” which completely covers my concerns about using jQuery in a mobile environment. (I don’t agree with his “Problem 2: Ajax navigation” though, which I will probably cover in a different post). Take a look for a more detailed analysis.

On the other hand, jQuery is fine if we have a fast network connection and are using a PC. jQuery has a lot of great libraries and we want to use them, especially if we need to do complex editing on the desktop.

We also use jQuery as a compatibility layer. jQuery is good for providing support for IE and we code some of our IE-specific polyfills using jQuery.

We never intend to load jQuery on a mobile device though.