Time to rethink website navigation systems

On January 6th, 2013, I outlined my thoughts on how I wanted to design websites that worked for both tablets and PCs.

I made the following observations;

  1. We need to start designing websites with tablets in mind. Displaying PC websites on the 10-inch iPad worked quite well. However, with the smaller iPad mini, we need to make more optimized websites.
  2. Desktop websites tend to be loaded with insane amounts of junk and redundant material. 90% of the content on the Asahi.com website, for example, is either a) advertisements, b) navigation, c) recommended links. The real content of the top page, which we would expect to be the headline news, is only 10% of the page.
  3. Because desktop websites come with so much junk, you would actually lose very little by removing it and making the page fit comfortably on the iPad mini.
  4. Looking at newspaper applications designed for the iPad, it becomes apparent that even the top-navigation bar is not really necessary. If you provide a button that summons up a page dedicated to navigation, you can dispense with even the most basic web navigation elements.

Today, working on an updated version of Ponzu, I am now absolutely sure that top-navigation can and often should be removed. With even the most basic web navigation scheme being cast under doubt, I am now of the impression that website navigation as a whole needs to be completely re-imagined, with inspiration for the new generation most likely to come from native mobile applications.

We’ll update this blog with our progress as it happens.

In the meantime, here’s an excellent write-up of traditional navigation designs and patterns that are common on current websites. Unfortunately, as you can easily see, the vast majority of these designs require the precision of a mouse cursor and are very, very unsuitable for an iPad mini.

Participation Inequality in Social Networks and Ponzu

It’s an old post but still as relevant as ever. A great post by Jakob Nielsen on encouraging more users to contribute to a social network.

Some excerpts with my comments;

User participation often more or less follows a 90-9-1 rule:

  • 90% of users are lurkers (i.e., read or observe, but don’t contribute).
  • 9% of users contribute from time to time, but other priorities dominate their time.
  • 1% of users participate a lot and account for most contributions: it can seem as if they don’t have lives because they often post just minutes after whatever event they’re commenting on occurs.

At the MBSJ2012, we had better results. In the analysis of “likes”, we found that 45% of the participants used the “like” button more than once. That is many times more than the 1% + 9% = 10% you would except from the 90-9-1 rule.

Jakob Nielsen also gives some recommendations to increase participation.

Although participation will always be somewhat unequal, there are ways to better equalize it, including:

  • Make it easier to contribute. The lower the overhead, the more people will jump through the hoop. For example, Netflix lets users rate movies by clicking a star rating, which is much easier than writing a natural-language review.
  • Make participation a side effect. Even better, let users participate with zero effort by making their contributions a side effect of something else they’re doing. For example, Amazon’s “people who bought this book, bought these other books” recommendations are a side effect of people buying books. You don’t have to do anything special to have your book preferences entered into the system. Will Hill coined the term read wear for this type of effect: the simple activity of reading (or using) something will “wear” it down and thus leave its marks — just like a cookbook will automatically fall open to the recipe you prepare the most.

Our “like” system falls within the category of “make it easier to contribute”. It also incorporates “make participation a side effect”, because you need to “like” a presentation to put it into “my schedule”. These likely contributed strongly to the better participation ratios at MBSJ2012.

Thoughts

In scientific conferences, participation inequality is a very important issue. This is because the online community is not the end-product, but it only serves as a means to enhance the scientific community, the vast majority of which is not very active on social networks. More importantly, the more important members of the scientific community, i.e. the professors, are less inclined to participate in these activities compared to young researchers.

Hence we should restrain ourselves from going overboard with social features which will tend to make the whole system unfriendly towards less social active participants. The social features should be there, but not necessarily prominent. They should help participants subtly and not be intrusive. We should also measure the success by the ratio of participants and the median activity, rather than the total number of activities. Most importantly, I believe that we should refrain from using social network features as a “voting” system for awards, unless we augment that with more traditional methods.

I believe social networks are important for scientific networks, but it is only a tool that has to be used very carefully. Ponzu (A Japanese sauce) is great for some food, but not so for others. Even when it goes well with your dish, too much will spoil it.

Jakob Nielsen, Intranet Social Features report

Jakob Nielsen, the renown user experience consultant, has posted a report on Intranet Social Features.

Below are a few excerpts that I found especially relevant to Ponzu, together with my comments;

Similarly, there’s now no doubt that social features are even more useful inside the enterprise for supporting employee collaboration and knowledge management. We used the slightly hokey term “Enterprise 2.0” to summarize our early research, and the new research confirms that the real effect of social features on intranets is to change how organizations function by making communication more open.

We also believe that social networking features in a conference program system (Ponzu) has a real positive effect on scientific research.

Little training is required. Users take to social tools easily when they’re given them for the right reasons and in the right work context. It takes little training, transition time, or urging to get people on board. In general, you should design social tools that employees can easily use without special training. In addition to following usability guidelines, you can achieve this goal by emulating popular Internet designs, such as the 5-star rating system known from Amazon and Netflix.

Avoid advertising the new tools as “new tools.” Instead, simply integrate them into the existing intranet, so that users encounter them naturally. For example, you could turn an existing bookmarking (or “quick links”) feature into a socially shared bookmarking feature without great fanfare.

It is essential that Ponzu’s features are modeled after popular Internet designs. That is why our “like” system is modeled after Facebook. We should resist making features complex. In fact, the example that Jakob Nielsen mentions, “turning an existing bookmark into a shared bookmarking feature” is essentially what we did. In Ponzu, we turned an existing bookmark (my schedule) into a “like” feature.

Despite companies’ concerns about employees using social tools with impropriety, infractions remain rare.

As long as attribution is built in and required, communities police themselves.

As with Intranets, the users of Ponzu will also be professionally accountable for the comments that they make. There should be no concern about impropriety.

In user testing of intranet search, we’ve found that it’s essential to provide a single, unified search across all intranet resources. This finding was replicated for social intranet features: they should be searched as part of the overall intranet search, rather than having individual siloed search engines for each social tool. Depending on the implementation, the need for integrated search can be a strong argument against outsourced or hosted social software, because many SaaS services don’t support federated search.

This is a strong argument for integrating all the social features inside Ponzu, instead of relying on third-party SNS or commenting solutions. Using Facebook or Twitter badges to provide social features is not a good idea. Neither is using DISQUS for the commenting system. Everything should be within the same system so that we can provide integrated search.

The most important conclusion from both research rounds is that social intranet projects must be driven by business needs — that is, the problem or pain point you’re trying to solve.

The reverse process is common, but deadly. Don’t start by saying, “Twitter and microblogging are cool, and I’ve heard good things about Yammer.” Our report says good things about Yammer, too, but that’s no reason to throw it onto your intranet. Maybe your business needs something completely different.

The difficulty here is to understand what the “business needs” of scientific conferences actually are. If you think that the needs are only to learn about research, then you don’t need social features in the first place. Only by perceiving scientific conferences as a social event can we understand which social features we need, and how we should implement them.

Today, many companies see intranet information sharing and other social features as offering true competitive advantages. It’s not something to build because it’s fun — it’s a workday utility. Social tools are an expected part of a knowledge worker’s standard toolkit, and many executives recognize this.

Widespread use of internal social media breaks down communication barriers. Although that sounds good, it can threaten people accustomed to having a monopoly on information and communication.

So, before implementing intranet collaboration tools, you must consider your company culture. If people are strongly committed to the “knowledge is power” tenet and don’t want to share, then sharing technologies will obviously fail.

Also, there’s still concern that, given social tools in the workplace, workers will fritter away their days and not get any work done. What we actually find — in companies with vibrant social platforms — is that employees are no more inclined to fritter away their work hours on non work-related communication with social tools than they were likely to do before these tools existed.

Social sharing is not for all scientific conferences. The medical community tends to be more rigid then the scientific community, with a stronger hierarchy and more formality. Hence it might be less suited than the relaxed molecular biology scientific society.

“Jobs-to-be-done” in scientific conferences

Clayton Christensen often refers to “Jobs-to-be-done” as a way of thinking about innovation. “Jobs-to-be-done” is a concept that was brought forth by Bob Moesta and the example that is most often referred to is milkshake marketing.

In essence, “Jobs-to-be-done” means to focus on a product’s function and its job; to understand what job the customer wants to accomplish, and why they hired the product to help them do it. In contrast, traditional marketing is usually obsessed with product categories, customer categories, and competitors, etc. These are what 4P and 3C tells us to focus on, but they often steer us away from the job-to-be-done.

In this post, I want to look at the jobs-to-be-done for scientific conferences, what participants want to accomplish.

The way I see it, the job that conferences are hired to do are the following;

  1. Face-to-face discussions with your peers, especially with those whom you do not regularly communicate.
  2. Learn about other remotely related research (fields that you wouldn’t usually check up).
  3. Meet with old acquaintances.
  4. Training and encouragement for young researchers.

Conference systems other than Ponzu only address the second issue. They provide a way to search the abstracts and to schedule your time. Unfortunately, they tend to only have a primitive search interface which is useful only when you have a very good idea about which search keywords you should use. As such, their support for the jobs-to-be-done is very poor.

Ponzu was designed for the jobs above. Each of Ponzu’s features was envisioned as a way to assist either of the tasks.

We designed the “like” system to provide an initial connection which could bring together scientists who share a common interest, but have never met face-to-face. We have heard stories where researcher doing unrelated research have met and discussed after discovering each other through “likes”. The “like” system addresses the first job. In addition, getting “likes” is encouraging. We have heard stories of young researchers being delighted at receiving likes from renown scientists, and then sending emails to them.

Our search system uses Apache Solr as the backend, and as a result, our searches are ranked by relevancy. Furthermore, we provide excerpts of the text to assist discovery. More importantly, we list up all presentations by the same research group so that users can easily learn more about each author, and meet with them at the conference to discuss further. As a result of collaboration with DBCLS, we also display related presentations based on keyword data-mining. This empowers participants to complete the second job.

Ponzu provides private messaging. Additionally, at the MBSJ2012, we provided Yoruzemi (a BBS for nightly meet ups). These help with job number 3, meeting old acquaintances.

Other conference systems view conferences only from the perspective of a lone spectator. They consider conferences to be a vehicle for information transmission from the presenters to the the audience, similar to how books convey knowledge to readers. They borrow features from spectator-oriented media and websites, but do not understand the unique jobs that conferences help get done.

The difference between Ponzu and preceding conference systems is hence very conceptual and deep. The difference is not in the number of features. It is in how we view the role of scientific conferences in general and in our approach to the jobs-to-be-done.

Our future goals are to deepen our understanding of how conferences contribute to the advancement of science, and to design Ponzu to assist in these roles. We need to understand the jobs-to-get-done, and to get them done.

Balance between low-tech and high-tech

I just received a newsletter from Jakob Nielsen, a well know User Interface and User Experience expert. In it, he describes his experience a the National Museum of Singapore.

Sadly, the museum is an example of mobile technology run amok. All information about the exhibited objects is provided on a tablet that you borrow when entering the history exhibition. The good news is that this allows for more in-depth information than traditional museum labels, and it’s also easier to provide the text in multiple languages that would crowd a printed label.

But the tablet fails to support the actual museum-going experience. You pause by a wall or display case with maybe 10-20 objects, each only marked by a number. And then you’re supposed to enter all these numbers into the tablet to find out what you’re seeing. No way: much too slow.

Nothing beats the roaming human eye in terms of quickly taking in volumes of information, especially when glancing around a large physical space. In a second or two, you can scan a big wall and focus on the label for the object that interests you the most. A few more seconds suffice to scan the label for the most pertinent information (what is it? how old is it?), and then you can either read more or move your gaze to the next object. All in much less time than it takes to type in a 3-4 digit number on a tablet.

You can easily imagine a more efficient retrieval user interface: for example one that utilizes location and direction to display information about an object simply by being pointed in the direction of that object. While faster, this would still be slower than simply glancing at a label next to the object.

Let’s stick to old technology when it works better. Then for sure use tablets to provide supplementary information, multimedia, and other elements that utilize its strengths and provide sufficient added-value to justify the interaction cost.

The take away message is

  1. If basic information can be provided by a traditional museum label placed by the exhibited object, this is the superior way of providing it.
  2. Use tablets to provide supplementary information.
  3. Don’t force the users to use high-tech to gain simple information.

The same thing can be said of conferences. Basic information such as which talks are being given at a certain room should be displayed by the door on paper. There should also be a map of rooms hung on the wall of each floor. Don’t force participants to pull out their smartphones or tablets.

Are native apps a replacement for a mobile web site?

Jason Zimdars on Basecamp Mobile back on Sep 13, 2012.

But even if/when we offer native apps, we will still want this mobile web version. Apps aren’t a replacement for the mobile web.

Mark Zuckerberg, Disrupt SF, September 2012

One of the things that’s interesting is we actually have more people on a daily basis using mobile Web Facebook than we have using our iOS or Android apps combined. So mobile Web is a big thing for us.

So having a native application for the conference program does not mean that we can do without a mobile website.

Unfortunately, some conferences have apps while very few have mobile websites.

Using local data storage space efficiently

As described in my previous post, “Choosing the browser-side data storage API”, storing data locally is tricky. Despite the work that is being done for data storage in HTML5, there still isn’t a good storage option that allows us to easily scale from a small data set to a large one. Storage limitations differ with each browser and we cannot rely on them to provide us with sufficient space.

Therefore, it is important that we use the space that we have as efficiently as possible.

In Kamishibai, we store HTML fragments as is in local storage. The size of the HTML fragments range from a whole page to a small “like” box. Breaking up a page into multiple HTML fragments is advantageous for effective cache management (each fragment might require a different expiry date) and would promote code reuse. However, multiple HTML fragments mean multiple HTTP requests. Downloading the page as a single fragment is the most efficient in terms of HTTP requests.

This means that we have to strike a compromise. Either we chose to optimize cache management by breaking up into small HTML fragments, or we optimize for network speed with larger HTML fragments.

The choice depends on how many pages we want to store locally, and the level of redundancy in each page. If we want to store a large number of pages, then it will be more efficient to use small fragments. Especially if the level of redundancy is high, we will be able to reuse small fragments effectively. Hence the choice will tend towards smaller fragments in conference systems.

Let’s look at the level of redundancy in conference systems.

The main pages in a Ponzu conference system are;

  1. A list of sessions.
  2. A list of presentations within a session.
  3. A presentation page (with a list of related presentations).
  4. A list of presentations in the search results.

A lot of the pages show a list of presentations. It therefore makes a lot of sense to store each element (title, authors and author affiliations of a presentation) separately so that we can construct different lists simply by combining elements.

Furthermore, the title, authors and author affiliations section of each presentation is quite large. In addition to the text, the authors section is composed of links to user profile pages and we additionally have markup of superscript. The markup is significant, and we often see more than 1,000 characters per presentation for the heading alone (sans abstract text).

In MBSJ2012, we did not break up lists into fragments. In addition to the large size, we also observed that rendering took a lot of time when the number of presentations in a list were large. Rendering the author list required a lot of string manipulation and often triggered garbage collection, resulting in long response times (several hundred milliseconds).

In future versions of Ponzu and Kamishibai, we will break up presentation lists. Each presentation will have a long expiry time so that the version cached inside the browser will be used. Additionally, we will use caching on the server. Our current test show that it should improve responsiveness in most cases.

Choosing the browser-side data storage API

Kamishibai has limited offline support at present, and we plan to bring it to full support within 2013. By full support, we mean that despite Ponzu being a web-page, we want to make the vast majority of pages available even when offline. The reason we take this very seriously is because WiFi at conferences tends to be very sketchy and temperamental.

Choosing the data storage API

Although HTML5 has support for offline applications, it is still very much in its infancy.

The Application Cache API is relatively well supported on all new browsers (> IE9). However, this API, while easy to use for the most basic cases, tends to be difficult for more advanced uses. Most programmers reserve it for static assets.

To store each presentation, session and user profile, we need a cache that is more flexible. Hence, in these cases, we use a data storage API that allows us to create, update and remove records programmatically and at will (the Application Cache API can only update all entries or zero entries). The APIs for this have not been well developed. We have localStorage, webSQL and indexedDB APIs.

localStorage is the simplest to use but also has the least features. The storage capacity is also rather small and no mainstream browser allows more than 5MBytes of storage.

webSQL is basically a sqllite3 database wrapped in Javascript. Since it can use SQL statements, there is a lot that you can do with it. Unfortunately, it has been abandoned by the W3C working-group because it is too tied to the sqlite3 implementation. webSQL can be expanded to 50MBytes of storage even on mobile devices, and is well supported on both iOS and Android. Safari and Chrome desktop browsers also support this API.

indexedDB is the replacement for webSQL. It is currently supported on IE10, Chrome and Firefox. Safari does not support it on both desktop and iOS versions. Also, the Android stock browser does not support this. It is the newest standard, and will most likely become the standard in the future. However, it will take several years before it becomes mainstream enough.

We ended up combining localStorage and webSQL for Kamishibai. Our rationale is discussed below. In the future, we plan to add indexedDB support.

webSQL looks great on features, but falls short on some important details

Having the full power of an SQL database and also broad support on mobile devices, webSQL looked like the prime candidate for Kamishibai data storage. Furthermore, since the 5MByte storage limit for localStorage would most likely be insufficient for large conferences, we initially planned to go 100% webSQL. However, we hit some snags.

First of all, when we use the openDatabase command to create a new webSQL database on Safari, and databases with an initial size exceeding 5MBytes brings up a modal dialog to confirm with the user. This is perfectly OK if the user is already familiar with the Ponzu system. However, a large number of people would be afraid and alarmed. They might opt to not allow access. More seriously, they might be deterred from visiting the site.

Memcached like LRU (least recently used) management

Since localStorage can only store 5MBytes and WebSQL puts up a modal dialog to confirm usage of more than 5MB, we have to consider how we are going to manage with storage that is smaller than the conference scientific program (it will easily exceed 10MBytes).

One idea is to purge entries that are infrequently accessed. A good scheme is the LRU management in memcached. The idea is to detect when the storage quota has been exceeded, and then to delete old entries to make space. Old entries are those that have not been recently accessed.

Implementing an LRU-type scheme is possible in localStorage. localStorage fires a QUOTA_EXCEEDED_ERR when we try to store more than the quota (5MB). We can catch this error and delete old entries, then retry.

However, webSQL at least on Safari is different. A quota exceeded error fires, but only after the modal dialog has shown up. This means that we cannot transparently delete old entries. This makes it very difficult to use webSQL as a LRU cache. Hence webSQL is good if we can get the user to agree to using a large quota (like 50 MB). However, using webSQL within the 5MB limit is very difficult if the user rejects the request.

Final scheme

Our final scheme is to initially use localStorage, using an LRU-based storage scheme. Since we will not be able to store the whole scientific program, this is basically a history-like cache.

We will also provide a store data locally button that the user can click to download the whole scientific program. This will use 50MB of webSQL storage. A modal dialog will pop up, but the user is expected to approve it. If they don’t, then we will fall back to localStorage. If the user uses webSQL, then we have no further need for localStorage, and we work 100% in webSQL.

Why did we implement Facebook-ish “likes”?

In the MBSJ2012 system, we implemented a Facebook-ish “likes” feature. To our knowledge, this is the very first time that anyone has used a “likes” system in any kind of conference system, including areas outside of the life-sciences or even broader science.

Why did we do it? Why did we think that “likes” would be beneficial to a scientific conference? Who did we think would benefit from “likes” and in what situations? In the following, I would like to give my personal account.

Conferences are, almost by definition, social events

First of all, I would like to start by mentioning that we did not start by simply copying a popular feature on social network systems. Introducing a “like” system is the result of a team of researchers constantly thinking about how to make conferences better. It is a result of really bright people who are trying to understand the essence of scientific discovery, and how this process can be accelerated. A huge amount of thought went into this system.

That being said, there is a simple inevitableness about the “likes” system. Namely conferences, almost by definition, are social events. Conferences are where scientists come together to meet each other in person. For some people, getting together and seeing how each of us is doing is more important than the scientific program itself. Not because we’re lonely, but because getting together itself actually moves science forward.

Hence, combining concepts derived from social network systems have a good chance of being a good match. In the following, I would like to elaborate on this hunch and discuss this more specifically.

Innovation is often the result of people meeting each other

Innovation often stems from random ideas that bounce between people as they discuss an issue. These conversations can give us new perspectives or reinforce our own ideas. In fact, the idea of a new, vastly improved conference IT system itself (Ponzu) was borne out of lighthearted chatting between myself and some DBCLS people.

Steve Jobs himself regarded “random encounters” as key to innovation and creativity, and designed the Pixar headquarters so that people would constantly bump into each other.

There’s a temptation in our networked age to think that ideas can be developed by email and iChat. That’s crazy. Creativity comes from spontaneous meetings, from random discussions. You run into someone, you ask what they’re doing, you say ‘Wow,” and soon you’re cooking up all sorts of ideas.

The beauty of Facebook-ish “likes” is the extremely low hurdle; you simply push a button. You don’t have to think of any clever comments. You just push it. Because of its extreme simplicity, “likes” are probably the most effective way of creating a network between people.

However, as opposed to the easiness of pushing a “like” button, the recipients of “likes” are often genuinely delighted. They will go through the list of people who “liked” their presentations, try to find out who they are and why they may be interested in your research. They are your list of people who you might have an interesting discussion with. They are your potential collaborators.

“Likes” are as easy as saying “Hi!” when you bump into somebody in the toilet. They are the “random encounters” in Steve Jobs’ office design. Instead of “asking what they’re doing”, the Ponzu system allows you to look up their profile page and see what presentations they are going to give. If you want to “cook up all sorts of ideas”, you can go to their poster presentation and talk to them in person, or you could look up their email and send them a message.

We take “likes” very seriously. We think that they can be small catalysts of innovation.

“Likes” from high-profile researchers will encourage young people

As young researchers, we are often anxious of whether our research is meaningful; whether our peers find it interesting or not. We are worried that maybe nobody will come to our poster presentation. We want other people to recognize and congratulate us on our work.

Now imagine if you were young and a high-profile researcher “liked” your presentation. How would you feel?

I know how I would feel. I would be very happy. I’d check out what that researcher is doing and probably send him/her an email. I would very much like to ask that research what they liked about my research.

I would feel encouraged and gain confidence. Knowing that that person would likely be at next year’s conference, I would try make sure that I’d have even better results to show them. I’d put more effort into my work. I will be energized.

A simple “like” can encourage young researchers and maybe change their careers for the better. It’s worth thinking about.

“My Schedule” systems don’t get it

Although listening to presentations is a significant part of attending conferences, it is hardly the most interesting. The marginal benefit of listening as opposed to reading literature is hardly significant. The real value of conferences lies in the fact that the researcher is there in person, and waiting to reply to any question that you want to ask. Furthermore, you can discuss and you can chat. You can even go out with them for a drink.

The problem with “My Schedule” systems is that they only help you to listen. They make it easy for you to listen efficiently. They treat the presentations as books on a bookshelf. They are focused on benefitting the listeners but provide zero merit for the presenters. “My Schedules” are a one-way street. Presenters are not welcomed.

“My Schedule” systems might be beneficial if discussions are not necessary or welcome. They would be useful if each presentation was actually a product promotion where the presenters are on a prepared script. They however do not encourage bi-directional interaction and they do not promote discussions. They do not enhance the real joy of meeting researchers face-to-face, and do not heighten the magic of conferences.

In short, they don’t get it. They don’t appreciate the magic.

Conclusion

When devising the “like” system in MBSJ2012, we contemplated the meaning of conferences; what unique benefits they provide to the scientific community, where is the magic. The “like” system was simply one of the ways that we came up with to augment and enhance it.

Hence the “like” system is not an endpoint. Our goal is to make conferences wonderful and the “like” system is just a beginning. We will continue to draw inspiration from innovations in social interactions and integrate them. We will learn from people who, like Steve Jobs, created environments that nurtured innovation. There are still mountains of things to do.

Why Kamishibai uses hash-bangs for navigation

Kamishiba URLs look like

http://mbsj2012.castle104.com/ja#!_/ja/presentations/3366

We are using hash-bangs which are hated by some. I’ll like to outline why we use them despite their unpopularity.

  1. The most important reason is because we want to allow offline viewing. We thought over many alternative ways to do this, but our conclusion was that we needed a boot loader that we could keep in Application Cache. In the above URL, http://mbsj2012.castle104.com/ja is the bootloader. /ja/presentations/3366 is where the content resides.
  2. The stock Android browser for version 4.0 and above do not support popState. Neither does IE9 and below.

Obviously, the first reason is the most important for conference systems because the network connection is generally very unreliable. The only solution to do offline apps without a hash-bang scheme is outlined in Jake Archibald in A List Apart, and it’s a good solution. However, as Jake himself admits

The experience on a temperamental connection isn’t great. The problem with FALLBACK is that the original connection needs to fail before any falling-back can happen, which could take a while if the connection comes and goes. In this case, Gotcha #1 (files always come from the ApplicationCache) is pretty useful.

Unfortunately, a temperamental connection is exactly what you get at conferences. We need a solution that allows us to programmatically define when to FALLBACK using either a timeout or the results of previous requests.

Below our some of our thoughts about the hash-bang and how we are providing solutions to its ill effects.

  1. We think that the hash-bang is necessary to provide a better experience to users. Offline viewing is the most important but there are other smaller things.
  2. The argument against hash-bangs is mostly about the nature of the web and crawlability. Our solution is to provide a mobile website that uses regular URLs. Crawlers don’t have to come to our sophisticated, Javascript-heavy website. We make sure that the mobile website contains all important information and that URLs for the mobile website are automatically converted to our Kamishibai ones.
  3. Although a hash-bang webpage requires two HTTP requests for the HTML, the first request is for the boot loader and will be stored in Application Cache. Hence the first request is local if the user has visited this website before. As a results, only one HTTP request goes over the network.
  4. Although many people state that the hash-bang is temporary and that widespread popState support will make it unnecessary, I disagree. Hash-bang is the only way I know of that will support offline websites.
  5. GMail still uses hash-bangs for most of their stuff. Obviously, GMail doesn’t want bots crawling their website.

Interchangeability of regular URLs and hash-bang URLs

In Kamishibai, we support both regular URLs and hash-bang URLs. Regular URLs will be used if the browser is mobile (iMode) or unsupported or has Javascript disabled. Because we have two URLs, we have to provide mechanisms to reroute, etc.

From regular URL to Kamishibai URL

This will be done within Rails and will be simple rerouting on the server. 302 redirects generally honor the hash fragment, so everything should be OK.

  1. Rails will receive the regular URL.
  2. If the browser that requested it is supported in Kamishibai, then redirect to the corresponding Kamishibai URL via 302. If the browser is not supported, then it gets a regular HTML response. (this may be implemented with a Rails before filter because not all actions can be converted easily).

From Kamishibai URL to regular URL

iMode is Javascript capable, but in reality, a large number may have Javascript disabled. Other carriers do not have Javascript. We cannot rely on Javascript of mobile sites. However, rudimentary Javascript should be provided for our non-supported browsers.

  1. Rails will receive the Kamishibai URL.
  2. Rails will return the top page, not the bootloader. The top page will have a short Javascript snippet that detects whether the URL was actually a Kamishibai URL or not. If it was a Kamishibai URL, then we automatically redirect to the corresponding regular URL. This will work if Javascript is ON but not if Javascript is OFF.
  3. If Javascript is disabled, then there is nothing we can do.

Summary

All clients with Javascript enabled will be able to handle both Kamishibai and regular URL. If Javascript is disabled, they will be able to handle regular URLs but will only show the top page for Kamishibai URLs.

With regard to Googlebot, we try to make it index only the regular URLs. Requests to regular URLs return full iMode pages (if Googlebot). If Googlebot tries to crawl a Kamishibai URL, which might happen if it followed an external link, then it might get a bit confused but because the Javascript is complicated, we expect it to just find the top page. Finding the hash-bang, it might also try to index using the _escaped_fragment_ scheme. We should redirect any of these requests to the regular URL version (using 301 redirects).