Data bulkload

This page describes how to bulkload data into a database. The most common usage is to bulkload data into a VM.

Select and download the desired data

• Use OPS to digitize a polygon over the desired area and get the WTK coordinates.

• In MATLAB, type edit runOpsGetFramesWithinPolygon.m. Modify the User Settings section and then run the script. This script will output the segments that your polygon intersects.

• edit runOpsGetInitialData.m. The instructions are in the header of this script, but you will need to set the USER INPUT fields and update the parameter spreadsheets for the data that you want to download. Running this script will cause the server to create a datapack on the server that you can download.

• On your VM, download the datapack using Firefox

• Then run the following commands to move the data pack from your downloads folder into the datapack input folder:

sudo -i
mv /home/ops/Downloads/*.tar.gz /opt/ops/data/postgresql/` 

• If you have not run provisions.sh yet, you can choose to import this data when you run provisions.sh.

• If you have already run provisions.sh, you can import this data using sh /opt/ops/conf/bulkload.sh. Make sure that the datapack input folder, /opt/ops/data/postgresql/ contains only the datapacks you want to import since the script imports everything in the folder.