Skip to content

Deploying NVCL Data Custodian Systems

Jump to bottom
Peter Warren edited this page Oct 16, 2024 · 3 revisions

Introduction

These instructions describe the AuScope recommended deployment configuration/procedure for NVCL components within data custodian's networks. You may chose to follow these instructions exactly or modify them to suit your systems, however, diverging from the instructions will make it more difficult for us to assist you with any problems that arise. When using these instructions please ensure that you have informed Peter Warren that you are doing so and feel free to contact Peter Warren with any queries or bug reports.

NVCLWebServiceInfrastructure

Minimum Requirements

  • a database SQL Server, SQL Azure or Oracle

    • 500GB storage for the first year of operation

  • a widows server to host the GeoServer and NVCLDataServices apps

    • Read access to the NVCL database,
    • Read access to borehole metadata usually from another database
    • IIS installed with Application Request Routing
    • A Java Runtime Environment version 17
    • Minimum 8Gb memory, 4Gb minimum allocated to Geoserver
    • 200GB Storage

  • PCs for users to analyse and upload data

    • High performance desktop or laptops recommended
    • The Spectral Geologist Software installed and licensed
    • Database Drivers installed - Oracle instant client or MS SQL Server Native client

Installation Instructions

NVCL Database

Oracle

  • Install Oracle 11g or later.
  • Edit CreateTSs.sql and fill in the three data file's paths and file names (if you wish you may set a strong password for nvcl and grant connect so that the createnvcldb script can be run as nvcl)
  • Run CreateTSs.sql script as SYS or another account with DBA privileges
  • Run createNVCLDB11g.sql script as a DBA user or as nvcl The database is now installed but before you can use it you will need to create some users. Descriptions of the users levels available and how to set them up in Oracle can be found here OracleNVCLDBUsers.

SQL Server

  • Install Microsoft SQL Server 2008 or later
  • Edit createNVCLDB.sql and fill in both data and log file's paths and file names
  • Run createNVCLDB.sql
  • Run createNVCLDBObjects.sql (if you receive errors simply run the script again) The database is now installed but before you can use it you will need to create some users. Descriptions of the user levels available and how to set them up in SQL Server can be found here SQLServerNVCLDBUsers.

Web Services Machine

Geoserver Data Source

Geoserver will provide the interfaces to allow access to your borehole records. To do this it expects the borehole metadata to be available in a database table. To get you started here is a list of the basic columns expected. Providing more detail is always preferred so please refer to the complete list of available properties in the gsmlp:BoreholeView schema.

Column Name Data Type Content Format
BOREHOLE_ID any unique identifier for borehole  
NAME text text name of borehole  
DRILLED_DATE text date the core was drilled YYYY-MM-DD
DRILLING_DESCRIPTION text This is a text enumeration available values shown in template  
LOCATION_DESC text Text description of the area the core was taken from  
CORED_START_DEPTH number start depth of coring  
CORED_END_DEPTH number end depth of coring  
DRILLER_NAME text Name of the company that drilled the core  
ELEVATION number Elevation above mean sea level of the collar of the core  
DECLIN_FROM_HORIZ number Degrees declination from horizontal of the initial direction of coring  
AZIMUTH number Degrees from North of the initial direction of coring  
COLLAR_LOCATION geometry Latitude, longitude and CRS of collar location  
SHAPE geometry Down-hole survey shape  
GML_ID text Concatenation of 'gsml.borehole.' and BOREHOLE_ID 'gsml.borehole.' + BOREHOLE_ID
COLLAR_ID text Concatenation of 'gsml.borehole.COLLAR.' and BOREHOLE_ID 'gsml.borehole.collar.' + BOREHOLE_ID
SHAPE_ID text Concatenation of 'gsml.borehole.' and BOREHOLE_ID with '.linestring' suffix 'gsml.borehole.' + BOREHOLE_ID + '.linestring'
STARTPOINT text This is a text enumeration available values shown in template  
HYLOGGERDATA text Indicates the availability of HyLogger Data for this borehole 'true' or 'false'

GeoServer Installation

  • Download the windows installer
  • If upgrading, stop the geoserver service and take a copy of your application-schema.properties file, jetty.xml (or tomcat's server.xml and conf directory) and your geoserver data directory.
  • Run the installer
  • change the installation directory if you wish
  • Select the path to your Java 17 JRE
  • Select your existing geoserver home directory if you have one or use default
  • Set the administrator password
  • Set or accept the default port 8080
  • Select Install as a Service

Configuring Database Connections

Database connections can be configured as jndi resources in the geoserver/etc/jetty.xml file. examples:

  • Oracle:
<New class="org.eclipse.jetty.plus.jndi.Resource">
  <Arg>java:comp/env/jdbc/borehole</Arg>
  <Arg>
    <New class="org.apache.commons.dbcp2.BasicDataSource">
      <Set name="driverClassName">oracle.jdbc.OracleDriver</Set>
    <Set name="url">jdbc:oracle:thin:@oracle-prod6.it.csiro.au:1539:HYLOGNR</Set>
    <Set name="username">WEBSERVER</Set>
    <Set name="password">PASSWORD</Set>
    <Set name="maxTotal">200</Set>
    <Set name="maxIdle">10</Set><Set name="minIdle">0</Set>
    <Set name="maxWaitMillis">10000</Set>
    <Set name="minEvictableIdleTimeMillis">300000</Set>
    <Set name="timeBetweenEvictionRunsMillis">300000</Set>
    <Set name="numTestsPerEvictionRun">20</Set>
    <Set name="poolPreparedStatements">true</Set>
    <Set name="maxOpenPreparedStatements">100</Set>
    <Set name="testOnBorrow">true</Set>
    <Set name="validationQuery">SELECT 1 FROM DUAL</Set>
    <Set name="accessToUnderlyingConnectionAllowed">true</Set>
    </New>
    </Arg>
</New>
  • SQL Server
<New class="org.eclipse.jetty.plus.jndi.Resource">
    <Arg>java:comp/env/jdbc/borehole</Arg>
    <Arg>
        <New class="org.apache.commons.dbcp2.BasicDataSource">
            <Set name="driverClassName">com.microsoft.sqlserver.jdbc.SQLServerDriver</Set>
            <Set name="url">jdbc:sqlserver://mssql-prod.it.csiro.au:1433;database=Boreholes;</Set>
            <Set name="username">WEBSERVER</Set>
            <Set name="password">PASSWORD</Set>
            <Set name="maxTotal">200</Set>
            <Set name="maxIdle">10</Set>
            <Set name="minIdle">0</Set>
            <Set name="maxWaitMillis">10000</Set>
            <Set name="minEvictableIdleTimeMillis">300000</Set>
            <Set name="timeBetweenEvictionRunsMillis">300000</Set>
            <Set name="numTestsPerEvictionRun">20</Set>
            <Set name="poolPreparedStatements">true</Set>
            <Set name="maxOpenPreparedStatements">100</Set>
            <Set name="testOnBorrow">true</Set>
            <Set name="validationQuery">SELECT CURRENT_TIMESTAMP</Set>
            <Set name="accessToUnderlyingConnectionAllowed">true</Set>
        </New>
    </Arg>
</New>
  • Ensure each database resource has a reference in your geoserver\webapps\geoserver\WEB-INF\classes\app-schema.properties file. e.g.
# Databae connection parameters
borehole.dbtype = oracle
borehole.jndi = java:comp/env/jdbc/borehole
...
...
...

NVCL Data Services

Prerequisites

The NVCL DataServices can be installed as a standalone service or as a web application within a tomcat instance.  We recommend using the standalone package as it provides greater isolation from other services and makes management easier.  However, if you have tomcat running on your server then you can use it if its more convenient.

Install or Update the standalone NVCLDataServices package

  • If updating from a previous version take a copy of the old NVCLDataServices folder and then uninstall or remove the old app.
  • Download the Windows installer
  • Run the installer
  • If an old version exists it will ask you to uninstall it
  • Press Yes, your config will be preserved
  • The new installer will now continue
  • Once it completes the configuration file should be displayed for editing.  If not edit <INSTALL_DIR>\application.properties
  • Follow the instructions within the file (refer to your previous versions *.properties files)
  • Start the NVCLDataServices from windows services

Routing traffic to your GeoServer and NVCLDataServices applications using IIS

Using the IIS manager create an inbound rewrite rule using regular expressions for each app like this:

Propery Value
Action Type rewrite
Append query string true
Pattern ^geoserver(.*)
Rewrite URL http://localhost:8080/geoserver{R:1}
Propery Value
Action Type rewrite
Append query string true
Pattern ^NVCL(Data
Rewrite URL http://localhost:8081/NVCLDataServices{R:2}
  • Create a folder under your webservers document root folder called NVCLPreparedDownloads. e.g c:\Inetpub\wwwroot\NVCLPreparedDownloads.
  • Using IIS manager ensure the folder is not browsable by external users but files contained can be downloaded
  • Edit the NVCLDataServices/application.properties file and ensure the download.url and download.rootpath properties are set. e.g.
# download.url is the location of file downloads after they have been prepared and should map through your server to download.rootpath
download.url=http://<PUBLICHOSTNAME>/NVCLPreparedDownloads/
# download.rootpath it the servers directory path to served TSG files
download.rootpath=C:\inetpub\wwwroot\NVCLPreparedDownloads
  • Ensure proxy preserve Host is set in your IIS settings
  • Ensure the x-forwarded-proto request header is set by upstream appliances routing traffic to your server.
Clone this wiki locally