We aim to misbehave

Evergreen: Progressive Web App

Dan Scott, Laurentian University

2017 Evergreen International Conference


Everyone wants an app

Let's improve the web app we have

Y'all got on this boat for different reasons, but y'all come to the same place. So now I'm asking more of you than I have before. Maybe all. Sure as I know anything, I know this - they will try again. […] They'll swing back to the belief that they can make [native apps]… better. And I do not hold to that. So no more runnin'. I aim to misbehave. Serenity, performance by Nathan Fillion, 2005-09-30.

Progressive Web Apps are... what?

From graceful degradation to progressive enhancement

Progressive Web Apps (PWAs)

  • Baseline - make web apps more like native apps:
    • Responsive: UI adapts to different devices
    • Fast to load and render
    • Secure: encrypted by HTTPS
    • Reliable under any network condition
    • Installable to your device home screen

Progressive Web Apps (PWAs)

  • Usually built on frameworks like or
  • Common model: app shell built with templated HTML, minified CSS and JavaScript, with REST calls to JSON data
  • Recent example: Mobile Twitter (2017-04-06)
  • Almost like the JSPAC of yore…
  • Let's see what we can do with the TPAC

Auditing PWA progress

  • Lighthouse audits web apps for quality
  • Focused on PWA baseline criteria
  • Offers advice for exemplary PWAs as well
  • Chrome extension & command line tool

Baseline: 37/100

  • Fresh Evergreen 2.12 install, without HTTPS
  • Praised for:
    • Page load performance is fast
    • Site is progressively enhanced
    • Design is mobile-friendly

Baseline: 37/100

  • Fresh Evergreen 2.12 install, without HTTPS
  • Punished for:
    • Network connection is not encrypted
    • User will not be prompted to Add to Homescreen
    • Installed web app will not launch with a splash screen
    • Address bar will just be default colour
    • App can't handle offline/flaky connections

Let's encrypt that connection

  • Use certbot to get a free TLS certificate from LetsEncrypt
  • Point the Apache config at the certificates
  • Enable TLS for all connections

Baseline with HTTPS: 50/100

Zoë: We mentioned that we were out of rations, and 10 minutes later, a bunch of apples rained into the trench.
Wash: And they grew into a big tree, and they all climbed up the tree into a magical land with unicorns and a harp. "War Stories." Firefly, performance by Gina Torres and Alan Tudyk, season 1, episode 9, 2002-12-06.

Manifest destiny

The purpose of the manifest is to install web applications to the homescreen of a device, providing users with quicker access and a richer experience. "Web App Manifest." Mozilla Developer Network

Manifest basics

  • A JSON file with metadata about the web app
  • A link to the manifest from the <head>
  "short_name": "Shiny Cap'n",
  "name": "Shiny Cap'n",
  "description": "A real shiny public catalogue!",
  "start_url": "/eg/opac/home"
<link rel="manifest" href="/manifest.json" />

Manifest icons

Multiple icons for multiple screen densities

  "short_name": "Shiny Cap'n",
  "icons": [{
      "src": "/images/icon48.png", "type": "image/png",
      "sizes": "48x48"
      "src": "/images/icon96.png", "type": "image/png",
      "sizes": "96x96"
      "src": "/images/icon192.png", "type": "image/png",
      "sizes": "192x192"

Manifest theming

While the app launches, give it nice colours

  "short_name": "Shiny Cap'n",
  "background_color": "#007a54",
  "theme_color": "#007a54",

Manifest display

  • When the app launches, set display to either:
    • Hide all user agent chrome: fullscreen
    • Launch as a separate application: standalone
  • Set orientation to either portrait or landscape
  "short_name": "Shiny Cap'n",
  "display": "standalone",
  "orientation": "portrait"

link and meta matching

<head> should contain <link rel="icon" /> and <meta rel="theme-color" /> elements that match the manifest values:

                        <meta rel="theme-color" value="#007a54" />
                        <link rel="icon" sizes="192x192" href="images/icon192.png" />

HTTPS + manifest: 85/100

'Yes. Yes, this is a fertile land, and we will thrive. We will rule over all this land, and we will call it… "This Land."' "Serenity." Firefly, performance by Alan Tudyk, season 1, episode 1, 2002-09-20. Netflix, https://www.netflix.com/watch/70133870?trackId=200257859.


  • /eg/opac/home first paint: 1648.1ms
    …opac/home (shiny)
      …opac/semiauto.css (shiny) - 719.2ms, 14.61KB
      …opac/simple.js (shiny) - 796ms, 16.4KB
      …tundra/tundra.css (shiny) - 1,060.6ms, 54.93KB
      …css/style.css (shiny) - 1,175.7ms, 60.35KB
      …dojo/openils_dojo.js (shiny) - 1,456.5ms, 12.82KB
      …opensrf/JSON_v1.js (shiny) - 1,541.4ms, 15.97KB
      …opensrf/opensrf_xhr.js (shiny) - 1,571.1ms, 17.78KB
      …themes/dijit.css (shiny) - 1,640.9ms, 36.78KB
      …images/small_logo.png (shiny) - 1,747.4ms, 15.37KB
      …opensrf/opensrf.js (shiny) - 1,840.6ms, 39.31KB
      …images/eg_tiny_logo.png (shiny) - 1,926.7ms, 14.96KB
      …images/main_logo.png (shiny) - 1,941.4ms, 21.25KB
      …dojo/dojo.js (shiny) - 2,106ms, 92.59KB

Dojo?!? Didn't we try to avoid Dojo?

Kaylee: Everything's shiny, Cap'n. Not to fret.
Mal: You told me those entry couplings would hold for another week!
Kaylee: That was six months ago, Cap'n. Serenity, performances by Jewel Staite and Nathan Fillion, 2005-09-30.


# Dojo is required to use the copy locations advanced search
# filter, therefore, it should always be enabled.
want_dojo = 1;

IF use_autosuggest.enabled == "t";
    want_dojo = 1;

IF ctx.google_books_preview;
    want_dojo = 1;

    want_dojo = 1;

The fix is in bug 1411699

…opac/home (shiny)
  …opac/semiauto.css (shiny) - 592ms, 11.48KB
  …opac/simple.js (shiny) - 642.3ms, 13.26KB
  …css/style.css (shiny) - 862.8ms, 58.53KB
  …images/eg_tiny_logo.png (shiny) - 1,041.5ms, 11.82KB
  …images/main_logo.png (shiny) - 1,071.5ms, 18.12KB
  • Still 85/100 , but first paint is down to 1011.9ms!
  • Most of the points are docked because we fail the App can load on offline/flaky connections test

Are we the Serenity?

Kept in the air by love (and Kaylee)

Or are we a Reaver Ship?

Belching smoke and uncontained radiation...

and Dojo 1.3

Handling offline/flaky connections:

Introducing service workers

Service workers

  • A separate JavaScript per origin scope
  • Runs on a separate thread
  • No access to the DOM
  • Can intercept and modify network requests for fine-grained caching strategies
  • Enables push notifications, too!

Service worker support

  • Supported by Chrome, Firefox, Opera today
  • Microsoft Edge is working on it
  • Safari… 🤷
  • But progressive enhancement means it's okay!

Service worker registration

if ('serviceWorker' in navigator) {
  window.addEventListener('load', function() {
    // Your service worker *must* be located at
    // the top-level directory relative to your site.
    // It won't be able to control pages unless it's
    // located at the same level or higher than them.
      .then(function(reg) { /* … stuff! */ })
      .catch(function(e) { console.error('Error:', e); });
Adapted from Google SW registration boilerplate

Service worker: Fetch API

  • More flexible than XMLHttpRequest()
  • FetchEvent handler + fetch() + Cache API + cache dedicated to a given web app:
    • Fetch network requests from the cache
    • Store network responses in the cache
    • Overcome network flakiness!

Let there be patterns!

serviceworke.rs offers examples of caching strategies


self.addEventListener('fetch', function(evt) {
  if evt.respondWith(fromNetwork(evt.request, 400)
  .catch(function () {
    return fromCache(evt.request);
function fromNetwork(request, timeout) {
  return new Promise(function (fulfill, reject) {
  var timeoutId = setTimeout(reject, timeout);
  fetch(request).then(function (response) {
    }, reject);

Cache and update

self.addEventListener('fetch', function(evt) {
function fromCache(request) {
  return caches.open(CACHE).then(function (cache) {
    return cache.match(request).then(function (matching) {
      return matching || Promise.reject('no-match');
function update(request) {
  return caches.open(CACHE).then(function (cache) {
    return fetch(request).then(function (response) {
      return cache.put(request, response);

Let there be tools!

  • We could roll our own service worker based on the serviceworke.rs patterns
  • But libraries provide a nice layer of abstraction:
    • sw-toolbox implements network handlers with five request/caching patterns
    • sw-precache builds on sw-toolbox to generate a service worker based on a configuration file


  "name": "evergreen-pwa-stuff",
  "version": "1.0.0",
  "description": "Evergreen PWA stuff",
  "author": "Dan Scott",
  "license": "GPL-2.0+",
  "dependencies": {
    "sw-precache": ">4.3.0"
$ npm install

sw-precache basic dumb config

module.exports = {
  staticFileGlobs: [
  stripPrefix: '/openils/var/web/',
  runtimeCaching: [{
    urlPattern: /^https:\/\/shiny.example.org\//,
    handler: 'networkFirst'

Enable our service worker

  1. Generate and install the service worker
  2. Install the boilerplate registration code
  3. Add registration code to templates/opac/parts/base.tt2

HTTPS + manifest + service worker: 100/100

Shut. Up.

100/100 PWA score!*

  • Specifically, no My Account pages are available offline
  • That was kind of the point of the exercise

*Score may not reflect real-world experiences

"My Account" headers

cache-control: no-store, no-cache, must-revalidate
expires: -1
  • no-store means "never cache this"
  • expires: -1 means "this expires immediately"
  • No network connection? You shall not see your checked out items!

Radical surgery

diff --git a/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader.pm b/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader.pm
index 6aa1f58..6548edc 100644
--- a/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader.pm
+++ b/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader.pm
@@ -184,8 +184,7 @@ sub load {
 return $self->redirect_auth unless $self->editor->requestor;

 # Don't cache anything requiring auth for security reasons
-$self->apache->headers_out->add("cache-control" => "no-store, no-cache, must-revalidate");
-$self->apache->headers_out->add("expires" => "-1");
+$self->apache->headers_out->add("cache-control" => "no-cache, must-revalidate");

 return $self->load_email_record if $path =~ m|opac/record/email|;

Rapid decay

If we remove those headers, Expires becomes + 5 seconds due to /etc/apache2/eg_vhost.conf:

<Location /eg/opac>
  PerlSetVar OILSWebContextLoader "OpenILS::WWW::EGCatLoader"
  # Expire the HTML quickly since we're loading dynamic data
  ExpiresActive On
  ExpiresByType text/html "access plus 5 seconds"

It's dynamic, but there are alternatives to Expires

Enter ETag

diff --git a/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGWeb.pm b/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGWeb.pm
index da18d7e..21d4715 100644
--- a/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGWeb.pm
+++ b/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGWeb.pm
@@ -129,11 +129,16 @@ sub handler_guts {
 $vhost_processor_cache{$processor_key} = $tt;
 $ctx->{encode_utf8} = sub {return encode_utf8(shift())};

-unless($tt->process($template, {ctx => $ctx, ENV => \%ENV, l => $text_handler}, $r)) {
+my $_out = '';
+unless($tt->process($template, {ctx => $ctx, ENV => \%ENV, l => $text_handler}, \$_out)) {
     $r->log->warn('egweb: template error: ' . $tt->error);
     return Apache2::Const::HTTP_INTERNAL_SERVER_ERROR;

+my $etag = md5_hex(Encode::encode_utf8($_out);)
+$r->headers_out->add('Etag' => $etag);
 return Apache2::Const::OK;

Whither Expires?

  • Given ETag and no-cache, the browser will always make a request to the server
  • If the server can't respond (is down, or has no network connection), the service worker can
  • Bonus: for other HTML pages, Expires can be turned off

<Location /eg/opac>
  PerlSetVar OILSWebContextLoader "OpenILS::WWW::EGCatLoader"
  ExpiresActive Off

Add to Home Screen: Shiny!

Add to Home Screen prompt
Android notification that the PWA has been added to the home screen
Android app drawer with PWA
Android PWA launcher splash screen
Android PWA launcher splash screen transition
Android PWA in full screen mode


  • Lighthouse recommends using HTTP/2
  • One TCP connection multiplexes requests and compresses headers
  • More efficient than four to eight separate connections per origin (http2 FAQ)
  • Nginx--which we're already using to proxy websockets--supports HTTP/2
  • Let's get this thing going

HTTP/2 on Debian Jessie

Jessie doesn't deliver a new enough version, so:

apt-get -t jessie-backports install nginx

HTTP/2: nginx config

diff --git a/examples/nginx/osrf-ws-http-proxy b/examples/nginx/osrf-ws-http-proxy
index d079230..2fbf832 100644
--- a/examples/nginx/osrf-ws-http-proxy
+++ b/examples/nginx/osrf-ws-http-proxy
@@ -19,7 +19,7 @@ server {

 server {
-    listen 443;
+    listen 443 ssl http2;
     ssl on;

     # Use the same SSL certificate as Apache.

HTTP/2: Are we good?

$ curl --http2 -H "Accept-Encoding: gzip" \
       -I https://shiny.example.org/eg/opac/home
HTTP/2 200
server: nginx/1.10.3
date: Sat, 01 Apr 2017 15:44:15 GMT
content-type: text/html; encoding=utf8
etag: 1735fc80e7ea268e656d53ac9126e438

nginx falls back to HTTP 1.1 for browsers that don't support HTTP/2

(Yes, I tested with w3m)

Brain dead approach PWA results

Not bad. Almost ready for prime time!

We've inadvertently enabled the web staff client--yay!?

But we're lacking a few things…

An offline fallback page

  • A search-driven app will lead to cache misses
  • Ideally, tell the user what they can access (perhaps via History API?)
  • This will require custom service worker code

Better control over GET params

  • Default behaviour is to treat the following pages as completely different:
    • https://shiny.example.org/eg/opac/myopac/holds?query=potter
    • https://shiny.example.org/eg/opac/myopac/holds?query=harry
  • For the /myopac/ pages, at least, GET params shouldn't matter
  • This will require custom service worker code

Separate the apps

  • /eg/opac/, /eg/staff/, and /eg/myopac/ are conceptually separate apps
  • But common JS, image, CSS assets under / require them all to use the same service worker
  • Instead, use /eg/opac/assets/, /eg/staff/assets/, and /eg/myopac/assets to serve up static assets from /openils/var/web/?

A modern /eg/myopac?

  • Precache a static app shell
  • Load fines, holds, lists, messages, etc as JSON over REST
  • Deliver native device notifications with the Push API
I got people with me, people who trust each other, who do for each other and ain't always looking for the advantage. There's good people in the 'verse. Not many, lord knows, but you only need a few. "Our Mrs. Reynolds." Firefly, performance by Nathan Fillion, season 1, episode 3, 2002-10-02.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License