Local Storage

Overview

Malasakit has been designed to gather feedback from developing areas where internet connectivity may be intermittent or nonexistent. This document proposes a scheme for caching user responses and assets from the server locally, and updating clients' and the server's state when network connectivity is available.

Resources

Clients store data locally using the localStorage API. localStorage is essentially a persistent key-value store created on a per-origin basis (analogous to a Python dictionary). Browser support for localStorage is fairly widespread, but storage capacity can vary from one to five megabytes.

client.js builds an abstraction layer to manage the serialization and invalidation of resources. Resources act like wrappers around regular JavaScript objects, and have the following attributes:

• name: A unique string identifier for a resource.
• timestamp: When this resource was last created or updated. This attribute is an integer of the number of milliseconds since the UNIX epoch (January 1st, 1970, 00:00:00 UTC).
• data: The desired object to store.

A resource is stored in localStorage by mapping the resource's name, prepended by malasakit-, to the resource represented as a JSON string. (The prefix can be used to prevent naming collisions when running multiple instances of Malasakit from the same origin, since all of these keys are stored in the same localStorage object.) Note that resources are not limited to these attributes--resources can define custom attributes as needed.

Response Storage

Responses by users are stored just like any other resource. The data object for a response resource has the following structure:

{
  "question-ratings": {
    <qid>: <score>,
    ...
  },
  "comments": {
    <qid>: <message>,
    ...
  },
  "comment-ratings": {
    <cid>: <score>,
    ...
  },
  "respondent-data": {
    "age": ...,
    "gender": ...,
    "division": ...,
    "sector": ...,
    "language": ...,
    "submitted-personal-data": ...,
    "uuid": ...
  }
}

Some notes:

• The language attribute of the respondent-data object is mandatory (this field is automatically deduced from the lang attribute of the html tag of the last page visited).
• The language determines what language we believe the comments are written in.
• The <qid> keys of the question-ratings and comments objects correspond to the id fields of the QuantitativeQuestion and QualitativeQuestion models, respectively.
• The uuid field is generated client-side for every new response (unique with very high probability). The backend uses this field to identify which responses are new and which are revisions. Unique identifiers cannot be assigned centrally by the server, since connectivity may not be available.

The name of a response starts with the prefix response-, followed by a UNIX timestamp (milliseconds since the epoch) of when the response was started. For instance, response-1499807909576. Due to the fine time resolution, this scheme practically guarantees unique names on a given client. It is acceptable for two clients to generate the same name, since the server never uses these resource names.

The current resource uses its data attribute to track the name of the response currently in use, or null if no response is currently being recorded. Once the user clicks the "Begin" button on the landing page, a new response resource is created, and pointed to by current.

All responses that are not current are capable of being pushed to the server. If connectivity is available and the response is successfully pushed (meaning the AJAX request did not time out, and the response returns a 200 HTTP status code), the response resource is deleted. Otherwise, the client continues to cache the response. This data synchronization is run once per page load (for any page).

Expiration Behavior

The questions and suggestions the user is presented with are cached as resources as well, and each has a lifetime associated before it expires. Once such a resource expires, the client will attempt to fetch more up-to-date data from the server. If the request is successful, the timestamp will be reset to the current time, and the data attribute of the resource will be updated with the question or suggestion data. Caching also helps reduce the number of computationally expensive requests that need to be served (the PCA projection of suggestions is a notable example).

On expiration, the current resource will have its data attribute set to null, which effectively frees up a stale response to be pushed to the server, even if the response never ended up being completed.