Skip to content

Developer OnBoarding Guide

Jump to bottom
Matt Sayre edited this page Jul 3, 2019 · 3 revisions

Overview

The following document details the development of SpeedUpAmerica(SUA).

M-Lab controls free Google BigQuery access using membership of our Discuss Google Group. Any user account subscribed to that should be able to query the M-Lab dataset / table: measurement-lab.release.ndt. If devs want to use GCP service accounts, M-Lab can subscribe them to the group manually.

Every GitHub issue/ticket has unique labels to help new developers contribute to the project: Good First Issue Enhancement Help Wanted Bug Urgent Technical Debt. Please see the Dropbox - GitHub Tickets document for an ordered list of tickets that are in the pipeline to be moved over to GitHub Issues when they are ready to be worked on.

Let's Code!

We're glad to have all experience levels to help with the project. You're here because you're looking to improve your development skills and/or have been affected by poor internet speed coverage.

  1. README - Setup - How do I run the code?
  2. GitHub Issues - What could I code?
  3. Slack #SpeedUpAmerica Channel - Who do I talk to about my code?

DISCLAIMER: All of our code is code-reviewed by trusted developers. After a code-review, you'll be able to merge your code into production (even our most experienced developers have code reviews). https://speedupamerica.com Send a message on the #speedupamerica channel on Slack and include a link your GitHub PR and a developer will help.

Roadwork Map

We outline four phases of development:

  1. Phase 1.5 - Vector Tilesets
  2. Phase 2 - Frontend Redesign in TypeScript and NodeJS
  3. Phase 2.5 - GIS Service
  4. Phase 3 - Consolidation

A detailed roadmap for our national solution to map Internet speed coverage is detailed in the Dropbox - Moving Forward document.

The current codebase downloads M-Lab NDT data Sunday evenings at 11:30pm. As the downloaded data is narrowed to a hard-coded list of Zip Codes, the amount of data downloaded is quite small. This doesn’t hold true for the nation, Chris from M-Lab reported in Slack that “we get ~2 million measurements per day globally, and the US is a high percentage of that”. While we don’t have hard numbers on San Jose and Louisville, a national amount of data is likely at least 3 orders of magnitude more data than the city-level implementations are handling.

Developing any relational database that can receive 1M records a day and update tens of thousands of aggregate records (~70k Census Tracts, ~40K Zip Codes, and ~3.1K Counties) is non-trivial. This is not a problem we are going to be able to solve overnight, or a weekend. Initially we will have to implement a Lane County level solution and iterate on performance from there.

Dropbox - Big Rocks

Tools Used

Google Big Query - query MLab data

MaxMind GeoLite2 - IP geolocation used for ASN

Geolocation API - enables the web browser to get user-consented location

Measurement Lab NDT - measurement tool for speed tests

Clone this wiki locally