Tag Archives: Ponzu

For us, the largest benefit of Javascript templating is reduced size

There are quite a few Javascript templates. In my projects however, there are very few cases where I would prefer using any of them in place of regular HTML being pushed out from the server (running Ruby-on-Rails). The same can be said of the Ponzu conference system.

As far as I understand, the benefits of using Javascript templates are 1) reducing the load on the server (generating JSON is less load than generating full HTML), 2) speed if used in combination with a single page design.

The downside is the additional work that browsers have to do, which can be a problem on mobile where the devices are not as powerful as their desktop counterparts.

I’ve touched this subject before in these two posts [1](https://code.castle104.com/?p=289), [2](https://code.castle104.com/?p=291).

As discussed by David Heinemeier Hansson, the same benefits can be achieved without Javascript templates by using a PJAX/Turbolinks/Kamishibai like system that eliminates reloading Javascript and CSS on each page transition, and the use of aggressive caching on the server side to reduce the load of HTML generation.

There is a real case however, where I feel a strong need for a Javascript tempting language.

That is when I try to cache responses in browser side cache. The issue is that HTML is extremely verbose, and is a real killer in terms of storage consumption when you are working with repetitive content.

For example, the following is a “social box” that we use in Ponzu for the like button and a voting button. It takes about 2,000 bytes. Each social box is associated with a presentation so we have hundreds to thousands of social boxes for each conference. This can easily fill up the limited browser side cache.

<div class="" id="presentation_326_social_box"><div class='like_box'>
<div class='like' id='like_button_326'>
<span class='social_controls'>
<!-- To invalidate Like related paths, we need a like object -->
<a href="/likes?like%5Bpresentation_id%5D=326" class="button icon like" rel="nofollow">like</a>
<div class='prompt_message'>
(
To add to your schedule, please &quot;like&quot; it first.
)
</div>
</span>
<div class='social_stats'>
<img alt="Like" src="/assets/like-c3719a03fc7b33c23fda846c3ccfb175.png" title="いいね!を押すと、応援メッセージになります。またあなたのスケジュールに登録されます。" />
<a href="/presentations/326/likes?user=1">15 people</a>
liked this.
</div>
<div class='likes_list' id='likes_presentation_326'></div>
</div>

</div>
<div class='vote_box'>
<div class='like' id='vote_button_326'>
<span class='social_controls'>
<form accept-charset="UTF-8" action="/likes/vote" class="new_like" id="presentation_326_new_like" method="post"><div style="margin:0;padding:0"></div>


<span>

<label for="presentation_326_like_score_1">Excellent!</label>
</span>
<span>

<label for="presentation_326_like_score_2">Unique!</label>
</span>
<span>

<label for="presentation_326_like_score_0">No Vote</label>
</span>
</form>
</span>
</div>

</div>
</div><div id="session_details_presentation_326"></div>

Most of this content is repetitive and will be identical for each “social_box”. In fact, the content that is unique to each individual social box can be summarized in the following JSON.

[{ 
    presentation_id:326,
    user_id:1,
    score: 0,
    liked: 0,
    scheduled: 0
}]

If we could use Javascript templating to generate the 2,000 byte HTML from this small set of JSON, local storage savings would be huge.

This is one feature that we will be adding to Kamishibai and Ponzu in the near future, to enable the ultimate goal of complete offline viewing.

On JavaScript MVC (part 2)

In Kamishibai and Ponzu, speed is a huge concern. This is particularly true for smartphones. In desktop web sites, it is possible to cram a lot of information and navigation elements into a single page to make up for slow page loading. You can see this in news websites like Asahi.com where 80% of the top page consists of navigation and shortcuts. The idea is that instead of asking the user to click a link and reload a new page (which is slow), the user simply can scroll down to see more content. With smartphones, cramming all this information is simply not a good idea and we have to reload pages.

One way to reduce the load time for pages and to update only the parts that you want is to use Javascript MVC or client-side MVC. With client side MVC, the pages are not reloaded as the content is switched. Instead of sending HTML pages, the server sends JSON data to the browser and browser-side javascript is used to construct the DOM from the JSON data. The advantage is that the client does not have to reload the whole page, and that it can intelligently update only the portions of the DOM that need to be redrawn.

In Kamishibai and Ponzu, we seriously contemplated using these Javascript MVC frameworks. However, we decided not to do so. Instead, our approach is similar to how GitHub handles updates with PJAX, and with how the new Basecamp uses Turbolinks. David Heinemeier Hansson gave a presentation describing why they did not use Javascript MVC extensively and a video is on YouTube.

There are other highly respected programmers who question the use of Javascript MVC. Thomas Fuchs, the author of script.aculo.us and Zepto.js has this to say in his blog “Client-side MVC is not a silver bullet” and his comments on a post about one of his projects, Charm.

I’ve come to the realization that this much client-side processing and decoupling is detrimental to both the speed of development, and application performance (a ton of JavaScript has to be loaded and evaluated each time you fire up the app). It’s better to let the server handle HTML rendering and minimize the use of JavaScript on the client. You can still have fast and highly interactive applications, as the new Basecamp shows—letting the server handle most stuff doesn’t mean that you have to cut back on cool front-end features and user friendliness.

We’ve spend a lot of time getting Backbone to work properly, as the easy-of-use quickly deteriorates when your models get more complex. It’s a great choice for simple stuff, but email is far from simple. We also had to add yet an other extra layer of processing to generate “ViewModels” on the server because the normal Rails serialization of objects wouldn’t cut it.

If you do any non-trivial resources, you’ll quickly end up with JSON objects that are just too large, especially for lists. For emails, imagine that you have nested threads, user avatar images, nested assigned cases, etc.
Because of this, you’ll need specialized JSON objects/arrays for different use cases (search, list view, detail view, and others). It follows that you’ll end up with this with more or less any front-end framework (if you care about performance!). Doing this adds complexity, which can be avoided by rendering HTML on the server where access to arbitrarily deeply nested data is relatively cheap (and can be highly optimized by keeping snippets of HTML around in memcache, etc).

In Ponzu and Kamishibai, we ended up following the traditional Rails route, enhanced with techniques that are seen in PJAX and Turbolinks.

What to we want to achieve

For us, the reasons for contemplating the use of a Javascript MVC framework were;

  1. Speed
  2. Use without network connection (offline apps)

A common benefit for Javascript MVC is interactivity, but this was not a concern for the types of web applications we had in mind which tend to be very read-heavy.

David Heinemeier Hansson has written a detailed post on how the new Basecamp dramatically increased performance through PJAX-like techniques and extensive server-side caching.

This technique basically nullifies the first benefit, because it makes it very easy to increase performance. Furthermore, the code will be almost the same as traditional Rails and hence very simple.

The second issue, offline usage, is something that even most Javascript MVC frameworks do not support very well. This is even more so when the data gets complex. Hence this in itself is not a compelling reason to go Javascript MVC; we would still have to figure out how to do offline usage effectively ourselves.

Why do people use Javascript MVC?

I came across this blog post describing why they used a Javascript MVC framework (Ember.js) instead of simple jQuery.

The author, Robin Ward, gives a specific example.

For example, on the bottom of every discourse post there is a button a user can click to like a post. When clicked, it vanishes and adds a footer below the post saying you liked it. If you implementing this in jQuery, you might add a data-post-id to the post. Then you’d bind a click event on your button element to a function that would make the AJAX call to the server. However, the click function passes a reference to the button, not the post. So you then have to traverse the DOM upwards to find the post the button belongs to and grab the id from there. Once you have it, you can make your XHR request. If the XHR succeeds, you then have to traverse the DOM downward from the post to the footer, and add in the text. At this point it works, but you’ve tied your implementation of the button click to a particular DOM structure. If you ever want to change your HTML around, you might have to adjust all the jQuery methods that accessed it. If this example seems simple – consider that in the footer we offer you a link to undo your like. When clicked, the footer text vanishes and the button appears again. Now you’re implementing the opposite operation against the DOM, only in reverse of what you did before. Discourse even takes it a step further – we know that 99% of the time when you click the like button the request is going to work, so we hide the button and show the footer text right away, even before waiting for the server to reply. In the infrequent event that request fails, we’ll show an error message and pop the UI back to the state it was in before. If we were doing that in jQuery, we’d have to have a callback on our AJAX request that knew how to put the UI back into the state it was in before. A prudent programmer might say, okay, well I’ll have a render function that can rebuild the DOM to the current UI state of whether the post is liked or not. Then both ‘undo’ and ‘like’ can call it. If ‘like’ fails it can call it again. Oh, and we have to store the current state of whether the post is liked somewhere. So maybe we add another data-liked=”true” attribute. ACK! Just typing this all out is giving me a headache!. Congratulations, your code is now spaghetti, your data is strewn out in the DOM and your logic is tied to a particular layout of HTML elements.

Although I understand Robin’s point and I have also experienced frustration when we want to update multiple elements that are in separate locations, I tend to think that using a full-blown Javascript MVC framework is an overkill. No doubt, DHH and Thomas Fuchs would send Javascript in the response to do the complex updates.

In fact, it is pretty difficult to find an Ember.js example on the web that does stuff that is complex enough that simple Javascript would not cut it.

A more intelligent RJS

Given that most of the use-cases of Ember.js and Javascript MVC frameworks seems to be stuff that could be done with regular Javascript, but might be a bit complex, the more pragmatic approach in my opinion, is to create a small library that would make updating the DOM through RJS-like methods simpler. It could also manage caching, expiration and updating of the response on the server side, so that subsequent requests do not have to go out to the network.

This is the approach that Kamishibai intends to persue.

Furthermore Kamishibai even handles the two-pane interface which is demonstrated in this Ember.js guide. This is possible because we have included a lot of “intelligence” in the library, whereas with the simple RJS approach, there is not library where complicated logic is stored for reuse.

More on this later when we open up the code.

Deciding which smartphones to support

Deciding which platforms to support and to what extent we should support them is a complex decision.

On the desktop, the vast majority of problems reside in how we should support Internet Explorer (IE) prior to version 10 (which is a very standards-compliant browser). Both IE8 and IE9 are still widely used but have numerous deviations from the HTML and CSS standards. Many features are not supported, and those that are are often buggy. However, since the problem has persisted for such a long time on browsers that have historically been the most popular, the open-source community has constructed many libraries that fill in the holes. Furthermore, the issues and workaround have been extensively documented on the web. Hence the problems are for the most part contained and controllable.

There are also differences and bugs among the “standards-compliant” browsers. However, in recent years, as HTML5 and CSS3 have stabilized, most of the issues have been fixed. In addition, users tend to use the most recent versions of these “standards-compliant” browsers, meaning that we don’t have to deal with the old buggy versions.

On the smartphone and mobile platforms, the situation is very different. Although most mobile browsers are based on the standards-compliant WebKit rendering engine and aspire to be standards-compliant in their Javascript implementations as well, the reality is that there are still a lot of differences.

On the smartphone side, the issues are almost exclusively on the Android side rather than iOS. By far the major reason why these issues persist is because the Android OS is very often not updated by the manufacturers of the phones. As with the desktop browsers, early smartphone browsers contained numerous bugs or lacked many features. Although these bugs were resolved in future iterations and were included in subsequent Android OS releases, a huge number of Android devices did not receive the fixes. The manufacturers simply did not bother to adapt the new Android version to their devices, or, due to the fact that they skimped on RAM to develop “budget” phones, they could not update them due to insufficient hardware resources.

As a result, there remain a huge number of Android phones on old versions of the OS, and hence on old versions of the browser; versions which contains a lot of old bugs.

Bugs in the Android browser tend not to be as severe as the bugs in Internet Explorer. They mostly reside in the HTML5 and CSS3 implementations, both of which were not stable in the webkit code at the time the old Android browser was forked. Other bugs are in “touch interface” implementation, which also was not in the original webkit code. However, since web development in the HTML5/CSS3 era has evolved to emphasize animations, feedback and interactivity, these bugs are very significant.

A table of Android OS versions per phone

To decide which Android versions to support, we use this table (Android端末一覧) on Wikipedia that lists every smartphone model sold in Japan and which Android version it is upgradable to.

We can observe that almost all smartphones that were introduced up till September, 2011 have only been updated to 2.3.4. Up till April, 2012, many new phones ended up being stuck at 2.3.4-6. Only after April 2012, just a year ago, do we see the majority of phones being upgradable to Android 4.0. Hence support for Android 2.3 is inevitable.

Just for comparison, the current version of iOS, iOS6.1), is installable on even the iPhone 3GS, a model that was first sold in Japan on June 2009. June 2009 is a month before the very first Android phone was sold in Japan and that could only be updated to Android 1.6.

Assessing the current and future popularity of Chrome on Android

In developing websites, especially those like Ponzu/Kamishibai which make heavy use of Javascript and CSS3, it is extremely important to decide which browsers to support. Older browsers will tend to not support the features required to make advanced features on the website run, and so the decision has to be make whether to support that old browser at all.

In Ponzu/Kamishibai, we currently support the following platforms;

  1. Newer versions of Safari, Firefox and Chrome on desktop platforms. The decision to not support older versions is based on statistics that show that users of these browsers tend to update quickly to the newest version.
  2. Internet Explorer 10 on windows.
  3. Internet Explorer 8 and 9 are supported through browser-specific code modifications. More technically, we have separate CSS and Javascript files that are uses only on these platforms to make up for deficiencies. Hence testing tends to be less thorough compared to the more fully supported platforms. Supporting older versions of Internet Explorer is a necessary vice, due to the fact that Windows XP (which only supports up to Internet Explorer 8) is still prevalent, and that IT departments within corporations usually restrict updates.
  4. On iOS, we support the latest version only, with brief testing on older versions. This is due to statistics that show rapid adoption of newer versions of iOS. For example, iOS6 was found installed on 85% of devices after only 5 months. Furthermore, iOS6 can be installed on even the iPhone 3GS, a device released almost 4 years ago.
  5. On Android, we support the Android stock browser on Android version 2.3.6 and Android version 4.0. We also support the latest version of Chrome on Android.

Android fragmentation due to slow adoption of new OS versions

Android support is complicated due to two issues.

One is the fragmentation of the Android platform itself. This is well documented and data can be found on Google’s website. As of May 1, 2013, close to 40% of users are on “Gingerbread” (version 2.3.*), an OS version that was first introduced on December, 2010, and superseded by “Ice Cream Sandwich” (version 4.0) on October, 2011.

The Android stock browser is an integral part of Android and is updated together with the OS itself. Hence Android OS fragmentation directly corresponds to browser version fragmentation.

スクリーンショット 2013 05 13 9 35 30

Android fragmentation due to two different Google browsers

Android browser support is further complicated due to the fact that there exists two separate brands of browsers, both of which are developed by Google and both of which may be found as the default browser on even the newest versions of Android.

Up till Android version 4.0, the default browser on Android was always what is commonly referred to as the “stock Android browser”. However in June, 2012, Google released Chrome for Android. Since then, some but not all devices (e.g. Nexus 7) have Chrome as the default browser and do not have the “stock browser” installed.

Will Chrome become the new default browser for Android?

Because Chrome is developed by Google, the question is whether or not Chrome will be the default browser for Android. More practically, the real question is whether Chrome will become a significant proportion of the web audience.

Unfortunately, current statistics point to that not being the case in near future.

The below graph is taken from NetMarketShare which tracks global website usage. The graph is a breakdown of mobile browser usage on April 2013. Chrome usage (which is a combination of Chrome on Android and iOS) is 2.63% whereas Android Browser usage is 22.89%. If we make the assumption that Chrome usage is 100% Android, this calculates to 10.3% of Android users using Chrome. This compares to 28.4% of Android users on “Jelly Bean” (version 4.1.) and 27.5% on “Ice Cream Sandwich” (version 4.0.).

Chrome for Android cannot be run on versions lower that 4.0.*. The above numbers mean that 10.3 / (28.4 + 27.5) = 18.4% of “Chrome-capable” Android devices are running Chrome. If we restrict to “Jelly Bean”, the Android version from which Google removed the Android stock browser on Nexus devices, 36.2% are using Chrome at maximum (assuming that all Chrome users are on Jelly Bean, a rather unrealistic assumption).

These numbers suggest that Chrome for Android adoption is not particularly high, even among newer devices.

It does not look very likely that Chrome will become the number 1 browser on Android for at least a few more years.

スクリーンショット 2013 05 13 10 18 19

Non-Google branded smartphones do not use Chrome for the default

Whereas Google “Nexus” branded devices like the Nexus 7 come with Chrome as the default browser, and without the stock Android browser installed, the same is not true for devices from Samsung.

In this review for the recently introduced Galaxy S4, which comes with Android version 4.2, the default browser is mentioned to have Samsung-specific features which are not available for Chrome. Hence the stock Android browser is not only installed as the default, it has unique features which Samsung intends to use to differentiate from the competition.

It is therefore likely that Samsung has no plans to switch to Chrome as the default browser. Instead, Samsung will most likely add features to the stock Android browser to further differentiate itself. If Samsung were to switch to Chrome, that would mean the browsing experience would be almost identical to other smartphones. Poor differentiation means commoditization, and Samsung will fight hard to prevent that.

In fact, Chrome on Android is pretty badly implemented at this point, and using Chrome as the default browser is likely to worsen the user experience. As Google improves the Chrome code, this is likely to be less of an issue. However, there is a larger issue that will still stop Samsung from using Chrome aggressively.

Android is open-source, which Chrome is not

The comments in this article about Chrome for Android are mostly detailed and provide a lot of insight. Particularly interesting is the fact that while Android (including the browser) is open-source, Chrome is not. This is discussed in more detail here.

Hence for a manufacturer like Samsung, which has resources to modify Android to create a unique user experience, the Android stock browser is an obvious choice. They simply cannot customize Chrome. It is possible that Samsung will switch from the Android stock browser to their own browser based on webkit, but it is unlikely that they will move to Chrome. The same can be said of HTC and other major players.

For less capable manufacturers, they may chose Chrome, but they also might chose a third party which has a more capable browser or which will agree to customize.

Either way, the current situation is such that the first choice for the default browser has disappeared without anything to fill that gap. Chrome in the current licensing status will not fill in that gap, as Samsung will most likely not chose it. As a result, we might see extreme fragmentation in Android browsers.

Conclusion

Chrome is unlikely to become the dominant browser for Android in the near future. This is true even if we only consider new high-end phones since manufacturers seem reluctant to give up their own customized stock browser in favor of Chrome.

The result is likely to be that we will see even more fragmentation of Android browsers. The players will be customized stock Android browsers (from Samsung, HTC and larger manufacturers), Chrome, Amazon Silk and third party browsers (which will be endorsed by smaller manufacturers which cannot customize the stock browser themselves).

The situation is pretty grim for web developers which want to take advantage of cutting edge HTML5/CSS3 features on Android.

Time to rethink website navigation systems

On January 6th, 2013, I outlined my thoughts on how I wanted to design websites that worked for both tablets and PCs.

I made the following observations;

  1. We need to start designing websites with tablets in mind. Displaying PC websites on the 10-inch iPad worked quite well. However, with the smaller iPad mini, we need to make more optimized websites.
  2. Desktop websites tend to be loaded with insane amounts of junk and redundant material. 90% of the content on the Asahi.com website, for example, is either a) advertisements, b) navigation, c) recommended links. The real content of the top page, which we would expect to be the headline news, is only 10% of the page.
  3. Because desktop websites come with so much junk, you would actually lose very little by removing it and making the page fit comfortably on the iPad mini.
  4. Looking at newspaper applications designed for the iPad, it becomes apparent that even the top-navigation bar is not really necessary. If you provide a button that summons up a page dedicated to navigation, you can dispense with even the most basic web navigation elements.

Today, working on an updated version of Ponzu, I am now absolutely sure that top-navigation can and often should be removed. With even the most basic web navigation scheme being cast under doubt, I am now of the impression that website navigation as a whole needs to be completely re-imagined, with inspiration for the new generation most likely to come from native mobile applications.

We’ll update this blog with our progress as it happens.

In the meantime, here’s an excellent write-up of traditional navigation designs and patterns that are common on current websites. Unfortunately, as you can easily see, the vast majority of these designs require the precision of a mouse cursor and are very, very unsuitable for an iPad mini.

Participation Inequality in Social Networks and Ponzu

It’s an old post but still as relevant as ever. A great post by Jakob Nielsen on encouraging more users to contribute to a social network.

Some excerpts with my comments;

User participation often more or less follows a 90-9-1 rule:

  • 90% of users are lurkers (i.e., read or observe, but don’t contribute).
  • 9% of users contribute from time to time, but other priorities dominate their time.
  • 1% of users participate a lot and account for most contributions: it can seem as if they don’t have lives because they often post just minutes after whatever event they’re commenting on occurs.

At the MBSJ2012, we had better results. In the analysis of “likes”, we found that 45% of the participants used the “like” button more than once. That is many times more than the 1% + 9% = 10% you would except from the 90-9-1 rule.

Jakob Nielsen also gives some recommendations to increase participation.

Although participation will always be somewhat unequal, there are ways to better equalize it, including:

  • Make it easier to contribute. The lower the overhead, the more people will jump through the hoop. For example, Netflix lets users rate movies by clicking a star rating, which is much easier than writing a natural-language review.
  • Make participation a side effect. Even better, let users participate with zero effort by making their contributions a side effect of something else they’re doing. For example, Amazon’s “people who bought this book, bought these other books” recommendations are a side effect of people buying books. You don’t have to do anything special to have your book preferences entered into the system. Will Hill coined the term read wear for this type of effect: the simple activity of reading (or using) something will “wear” it down and thus leave its marks — just like a cookbook will automatically fall open to the recipe you prepare the most.

Our “like” system falls within the category of “make it easier to contribute”. It also incorporates “make participation a side effect”, because you need to “like” a presentation to put it into “my schedule”. These likely contributed strongly to the better participation ratios at MBSJ2012.

Thoughts

In scientific conferences, participation inequality is a very important issue. This is because the online community is not the end-product, but it only serves as a means to enhance the scientific community, the vast majority of which is not very active on social networks. More importantly, the more important members of the scientific community, i.e. the professors, are less inclined to participate in these activities compared to young researchers.

Hence we should restrain ourselves from going overboard with social features which will tend to make the whole system unfriendly towards less social active participants. The social features should be there, but not necessarily prominent. They should help participants subtly and not be intrusive. We should also measure the success by the ratio of participants and the median activity, rather than the total number of activities. Most importantly, I believe that we should refrain from using social network features as a “voting” system for awards, unless we augment that with more traditional methods.

I believe social networks are important for scientific networks, but it is only a tool that has to be used very carefully. Ponzu (A Japanese sauce) is great for some food, but not so for others. Even when it goes well with your dish, too much will spoil it.

“Jobs-to-be-done” in scientific conferences

Clayton Christensen often refers to “Jobs-to-be-done” as a way of thinking about innovation. “Jobs-to-be-done” is a concept that was brought forth by Bob Moesta and the example that is most often referred to is milkshake marketing.

In essence, “Jobs-to-be-done” means to focus on a product’s function and its job; to understand what job the customer wants to accomplish, and why they hired the product to help them do it. In contrast, traditional marketing is usually obsessed with product categories, customer categories, and competitors, etc. These are what 4P and 3C tells us to focus on, but they often steer us away from the job-to-be-done.

In this post, I want to look at the jobs-to-be-done for scientific conferences, what participants want to accomplish.

The way I see it, the job that conferences are hired to do are the following;

  1. Face-to-face discussions with your peers, especially with those whom you do not regularly communicate.
  2. Learn about other remotely related research (fields that you wouldn’t usually check up).
  3. Meet with old acquaintances.
  4. Training and encouragement for young researchers.

Conference systems other than Ponzu only address the second issue. They provide a way to search the abstracts and to schedule your time. Unfortunately, they tend to only have a primitive search interface which is useful only when you have a very good idea about which search keywords you should use. As such, their support for the jobs-to-be-done is very poor.

Ponzu was designed for the jobs above. Each of Ponzu’s features was envisioned as a way to assist either of the tasks.

We designed the “like” system to provide an initial connection which could bring together scientists who share a common interest, but have never met face-to-face. We have heard stories where researcher doing unrelated research have met and discussed after discovering each other through “likes”. The “like” system addresses the first job. In addition, getting “likes” is encouraging. We have heard stories of young researchers being delighted at receiving likes from renown scientists, and then sending emails to them.

Our search system uses Apache Solr as the backend, and as a result, our searches are ranked by relevancy. Furthermore, we provide excerpts of the text to assist discovery. More importantly, we list up all presentations by the same research group so that users can easily learn more about each author, and meet with them at the conference to discuss further. As a result of collaboration with DBCLS, we also display related presentations based on keyword data-mining. This empowers participants to complete the second job.

Ponzu provides private messaging. Additionally, at the MBSJ2012, we provided Yoruzemi (a BBS for nightly meet ups). These help with job number 3, meeting old acquaintances.

Other conference systems view conferences only from the perspective of a lone spectator. They consider conferences to be a vehicle for information transmission from the presenters to the the audience, similar to how books convey knowledge to readers. They borrow features from spectator-oriented media and websites, but do not understand the unique jobs that conferences help get done.

The difference between Ponzu and preceding conference systems is hence very conceptual and deep. The difference is not in the number of features. It is in how we view the role of scientific conferences in general and in our approach to the jobs-to-be-done.

Our future goals are to deepen our understanding of how conferences contribute to the advancement of science, and to design Ponzu to assist in these roles. We need to understand the jobs-to-get-done, and to get them done.

Balance between low-tech and high-tech

I just received a newsletter from Jakob Nielsen, a well know User Interface and User Experience expert. In it, he describes his experience a the National Museum of Singapore.

Sadly, the museum is an example of mobile technology run amok. All information about the exhibited objects is provided on a tablet that you borrow when entering the history exhibition. The good news is that this allows for more in-depth information than traditional museum labels, and it’s also easier to provide the text in multiple languages that would crowd a printed label.

But the tablet fails to support the actual museum-going experience. You pause by a wall or display case with maybe 10-20 objects, each only marked by a number. And then you’re supposed to enter all these numbers into the tablet to find out what you’re seeing. No way: much too slow.

Nothing beats the roaming human eye in terms of quickly taking in volumes of information, especially when glancing around a large physical space. In a second or two, you can scan a big wall and focus on the label for the object that interests you the most. A few more seconds suffice to scan the label for the most pertinent information (what is it? how old is it?), and then you can either read more or move your gaze to the next object. All in much less time than it takes to type in a 3-4 digit number on a tablet.

You can easily imagine a more efficient retrieval user interface: for example one that utilizes location and direction to display information about an object simply by being pointed in the direction of that object. While faster, this would still be slower than simply glancing at a label next to the object.

Let’s stick to old technology when it works better. Then for sure use tablets to provide supplementary information, multimedia, and other elements that utilize its strengths and provide sufficient added-value to justify the interaction cost.

The take away message is

  1. If basic information can be provided by a traditional museum label placed by the exhibited object, this is the superior way of providing it.
  2. Use tablets to provide supplementary information.
  3. Don’t force the users to use high-tech to gain simple information.

The same thing can be said of conferences. Basic information such as which talks are being given at a certain room should be displayed by the door on paper. There should also be a map of rooms hung on the wall of each floor. Don’t force participants to pull out their smartphones or tablets.

Are native apps a replacement for a mobile web site?

Jason Zimdars on Basecamp Mobile back on Sep 13, 2012.

But even if/when we offer native apps, we will still want this mobile web version. Apps aren’t a replacement for the mobile web.

Mark Zuckerberg, Disrupt SF, September 2012

One of the things that’s interesting is we actually have more people on a daily basis using mobile Web Facebook than we have using our iOS or Android apps combined. So mobile Web is a big thing for us.

So having a native application for the conference program does not mean that we can do without a mobile website.

Unfortunately, some conferences have apps while very few have mobile websites.

Choosing the browser-side data storage API

Kamishibai has limited offline support at present, and we plan to bring it to full support within 2013. By full support, we mean that despite Ponzu being a web-page, we want to make the vast majority of pages available even when offline. The reason we take this very seriously is because WiFi at conferences tends to be very sketchy and temperamental.

Choosing the data storage API

Although HTML5 has support for offline applications, it is still very much in its infancy.

The Application Cache API is relatively well supported on all new browsers (> IE9). However, this API, while easy to use for the most basic cases, tends to be difficult for more advanced uses. Most programmers reserve it for static assets.

To store each presentation, session and user profile, we need a cache that is more flexible. Hence, in these cases, we use a data storage API that allows us to create, update and remove records programmatically and at will (the Application Cache API can only update all entries or zero entries). The APIs for this have not been well developed. We have localStorage, webSQL and indexedDB APIs.

localStorage is the simplest to use but also has the least features. The storage capacity is also rather small and no mainstream browser allows more than 5MBytes of storage.

webSQL is basically a sqllite3 database wrapped in Javascript. Since it can use SQL statements, there is a lot that you can do with it. Unfortunately, it has been abandoned by the W3C working-group because it is too tied to the sqlite3 implementation. webSQL can be expanded to 50MBytes of storage even on mobile devices, and is well supported on both iOS and Android. Safari and Chrome desktop browsers also support this API.

indexedDB is the replacement for webSQL. It is currently supported on IE10, Chrome and Firefox. Safari does not support it on both desktop and iOS versions. Also, the Android stock browser does not support this. It is the newest standard, and will most likely become the standard in the future. However, it will take several years before it becomes mainstream enough.

We ended up combining localStorage and webSQL for Kamishibai. Our rationale is discussed below. In the future, we plan to add indexedDB support.

webSQL looks great on features, but falls short on some important details

Having the full power of an SQL database and also broad support on mobile devices, webSQL looked like the prime candidate for Kamishibai data storage. Furthermore, since the 5MByte storage limit for localStorage would most likely be insufficient for large conferences, we initially planned to go 100% webSQL. However, we hit some snags.

First of all, when we use the openDatabase command to create a new webSQL database on Safari, and databases with an initial size exceeding 5MBytes brings up a modal dialog to confirm with the user. This is perfectly OK if the user is already familiar with the Ponzu system. However, a large number of people would be afraid and alarmed. They might opt to not allow access. More seriously, they might be deterred from visiting the site.

Memcached like LRU (least recently used) management

Since localStorage can only store 5MBytes and WebSQL puts up a modal dialog to confirm usage of more than 5MB, we have to consider how we are going to manage with storage that is smaller than the conference scientific program (it will easily exceed 10MBytes).

One idea is to purge entries that are infrequently accessed. A good scheme is the LRU management in memcached. The idea is to detect when the storage quota has been exceeded, and then to delete old entries to make space. Old entries are those that have not been recently accessed.

Implementing an LRU-type scheme is possible in localStorage. localStorage fires a QUOTA_EXCEEDED_ERR when we try to store more than the quota (5MB). We can catch this error and delete old entries, then retry.

However, webSQL at least on Safari is different. A quota exceeded error fires, but only after the modal dialog has shown up. This means that we cannot transparently delete old entries. This makes it very difficult to use webSQL as a LRU cache. Hence webSQL is good if we can get the user to agree to using a large quota (like 50 MB). However, using webSQL within the 5MB limit is very difficult if the user rejects the request.

Final scheme

Our final scheme is to initially use localStorage, using an LRU-based storage scheme. Since we will not be able to store the whole scientific program, this is basically a history-like cache.

We will also provide a store data locally button that the user can click to download the whole scientific program. This will use 50MB of webSQL storage. A modal dialog will pop up, but the user is expected to approve it. If they don’t, then we will fall back to localStorage. If the user uses webSQL, then we have no further need for localStorage, and we work 100% in webSQL.