Rearranging

index.rst

@@ -18,22 +18,22 @@ THIS MATERIAL IS WORK IN PROGRESS, do not trust anything ! ;)
    materials/practicalities.md
    materials/welcome.md
    materials/csc.md
-   materials/support.md
+
 
 .. toctree::
    :maxdepth: 2
    :caption: Supercomputer:
 
    materials/supercomputer/supercomputing.md  
-   materials/supercomputer/data_storage.md 
+   materials/data/data_storage.md 
    materials/supercomputer/software.md
    materials/supercomputer/webinterface.md   
    materials/supercomputer/jobs.md      
    materials/supercomputer/htc.md
    materials/supercomputer/exercise_gdal.md     
    materials/supercomputer/exercise_python.md
    materials/supercomputer/exercise_r.md   
-   materials/supercomputer/isntallations.md
+   materials/supercomputer/installations.md
 
 
 .. toctree::
@@ -49,6 +49,7 @@ THIS MATERIAL IS WORK IN PROGRESS, do not trust anything ! ;)
    :caption: Wrap-up:
 
    materials/own_project.md
+   materials/support.md   
    materials/wrapup.md
    materials/where_to_go.md
 

materials/csc.md

@@ -26,17 +26,4 @@
 * Teaching materials: [Library of Open Educational Resources](https://aoe.fi/)
 * Sending data: [Funet FileSender](https://filesender.funet.fi/)
 
-* [CSC service catalog](https://research.csc.fi/en/service-catalog)
-
-
-
-Supercomputers
-How they differ from normal computer
-resources available
-OS and software
-resource allocation
-jobs
-Puhti/Mahti/LUMI
-differences
-Puhti software & data
-Skills needed and how to get them
+* [CSC service catalog](https://research.csc.fi/en/service-catalog)

materials/data/moving_data.md

@@ -1,35 +1,43 @@
-# [Moving files between a local computer and Puhti](https://docs.csc.fi/data/moving/)
+# Moving data
 
-## [Puhti Web Interface](puhti.csc.fi)
+## Moving files between a local computer and a supercomputer
+
+* [Moving files between a local computer and a supercomputer](https://docs.csc.fi/data/moving/)
+
+### Puhti Web Interface
 
 - Very easy, no installations needed.
 - For smaller amounts of data, < 10 Gb.
 - Upload, download, moving, creating folders.
+- [Puhti Web Interface](https://puhti.csc.fi)
+- [Puhti Web Interface in Docs](https://docs.csc.fi/computing/webinterface/)
 
-## [Graphical data transfer tools](https://docs.csc.fi/data/moving/graphical_transfer/) on local computer
+### Graphical data transfer tools on local computer
 
 - For example: _FileZilla_ and _WinSCP_
 - For medium amounts of data, < 1 Tb.
 - Very easy, but installation required.
+- [Graphical data transfer tools in Docs](https://docs.csc.fi/data/moving/graphical_transfer/)
 
-## Command line tools
+### Command line tools
 - For any amount of data, practically required if data size > 1 Tb.
 
-### [`scp`](https://docs.csc.fi/data/moving/scp/) 
+#### scp
 
 - The most usual Linux tool for moving file
 - `scp` works even in Windows Powershell
+- [`scp` in Docs](https://docs.csc.fi/data/moving/scp/)
 
 ```
 # One file:
 scp /path/to/a_file [email protected]:/scratch/project_xxxx/data_dir
 # One folder:
-scp -r /path/to/data_directory [email protected]:/scratch/project_#####/data_dir 
+scp -r /path/to/directory [email protected]:/scratch/project_#####/directory 
 ```
 
 
-### rsync
-- [`rsync`](https://docs.csc.fi/data/moving/rsync/)
+#### rsync
+
 - Does not copy what is already there
 - Can resume a copy process which disconnected
 - Efficient for small files 
@@ -38,12 +46,13 @@ scp -r /path/to/data_directory [email protected]:/scratch/project_#####/d
 - Available on Linux and Mac and WSL (windows subsystem linux)
 - Organization firewalls might have forbidden using rsync, especially in Valtori organizations
 - Windows Powershell does not have `rsync`, _MobaXterm_ has `rsync`, but it removes write permissions of copied files
+- [`rsync` in Docs](https://docs.csc.fi/data/moving/rsync/)
 
 ```
 # One file:
 rsync --info=progress2 -a /path/to/a_file [email protected]:/scratch/project_xxxx/data_dir
 # One folder:
-rsync --info=progress2 -a /path/to/data_directory [email protected]:/scratch/project_xxxx/data_dir
+rsync --info=progress2 -a /path/to/directory [email protected]:/scratch/project_xxxx/directory
 ```
 * `progress2` shows time left and percentage
 
@@ -57,8 +66,9 @@ rsync --info=progress2 -a /path/to/data_directory [email protected]:/scra
 	- wget/curl if HTTP-urls avaialable
 	- OGC APIs, STAC ?
 
-### [wget](https://docs.csc.fi/data/moving/wget/)
-- 
+### wget
+
+- [`wget` in Docs](https://docs.csc.fi/data/moving/wget/)
 
 ``` 
 # One file:
@@ -70,4 +80,4 @@ wget -r -nc ftp://ftp.aineistot.metsaan.fi/Metsamaski/Maakunta/ --cut-dirs=2
 
 ## Moving data as part of workflow
 
-> TODO: Example of moving data into local scratch eg from Allas
+> ToDo: Example of moving data into local scratch eg from Allas

materials/supercomputer/jobs.md

@@ -1,6 +1,6 @@
 
 
-# Puhti 
+# Jobs 
 
 ![](../images/puhti_overview.png)
 

materials/supercomputer/software.md

@@ -1,39 +1,56 @@
-## GIS tools available on Puhti
+# GIS tools
+
+## GIS tools available in Puhti
+
+* [Ames Stereo Pipeline](https://docs.csc.fi/apps/ames-stereo.md) for processing stereo images
+* [ArcGIS Python API](https://docs.csc.fi/apps/arcgis.md) 
+* [CloudCompare](https://docs.csc.fi/apps/cloudcompare.md) for visualizing, editing and processing poing clouds
+* [FORCE](https://docs.csc.fi/apps/force.md) for mass-processing of medium-resolution satellite images
+* [GDAL](https://docs.csc.fi/apps/gdal.md) for geospatial data formats
+* [Geoconda](https://docs.csc.fi/apps/geoconda.md) - Python spatial analysis libraries
+* [GRASS GIS](https://docs.csc.fi/apps/grass.md) General purpose GIS software family for viewing, editing and analysing geospatial data
+* [LAStools](https://docs.csc.fi/apps/lastools.md) for LiDAR datasets
+* [OpenDroneMap](https://docs.csc.fi/apps/opendronemap.md) for processing aerial drone imagery
+* [Orfeo ToolBox](https://docs.csc.fi/apps/otb.md) for remote sensing applications
+* [PCL](https://docs.csc.fi/apps/pcl.md) for 2D/3D image and point cloud processing
+* [PDAL](https://docs.csc.fi/apps/pdal.md) for point cloud translations and processing
+* [QGIS](https://docs.csc.fi/apps/qgis.md) General purpose GIS software family for viewing, editing and analysing geospatial data
+* [R for GIS](https://docs.csc.fi/apps/r-env-for-gis.md) R spataial analysis libraries
+* [SAGA GIS](https://docs.csc.fi/apps/saga-gis.md) General purpose GIS software family for viewing, editing and analysing geospatial data
+* [Sen2Cor](https://docs.csc.fi/apps/sen2cor.md) for atmospheric-, terrain and cirrus correction of the Sentinel-2 products
+* [Sen2mosaic](https://docs.csc.fi/apps/sen2mosaic.md) for download, preprocessing and mosaicing of Sentinel-2 products
+* [SNAP](https://docs.csc.fi/apps/snap.md) for remote sensing applications
+* [WhiteboxTools](https://docs.csc.fi/apps/whiteboxtools.md) an advanced geospatial data analysis platform
+* [Zonation](https://docs.csc.fi/apps/zonation.md) Spatial conservation prioritization framework
+* [pytorch](https://docs.csc.fi/apps/pytorch.md) for deep learning
+* [tensorflow](https://docs.csc.fi/apps/tensorflow.md) for deep learning 
+
+
+## GIS tools available in Mahti
 
-* [Ames Stereo Pipeline](ames-stereo.md) for processing stereo images
-* [ArcGIS Python API](arcgis.md) 
-* [CloudCompare](cloudcompare.md) for visualizing, editing and processing poing clouds
-* [FORCE](force.md) for mass-processing of medium-resolution satellite images
-* [GDAL](gdal.md) for geospatial data formats
 * [Geoconda](geoconda.md) - Python spatial analysis libraries
-* [GRASS GIS](grass.md) General purpose GIS software family for viewing, editing and analysing geospatial data
-* [LAStools](lastools.md) for LiDAR datasets
-* [OpenDroneMap](opendronemap.md) for processing aerial drone imagery
-* [Orfeo ToolBox](otb.md) for remote sensing applications
-* [PCL](pcl.md) for 2D/3D image and point cloud processing
-* [PDAL](pdal.md) for point cloud translations and processing
-* [QGIS](qgis.md) General purpose GIS software family for viewing, editing and analysing geospatial data
-* [R for GIS](r-env-for-gis.md) R spataial analysis libraries
-* [SAGA GIS](saga-gis.md) General purpose GIS software family for viewing, editing and analysing geospatial data
-* [Sen2Cor](sen2cor.md) for atmospheric-, terrain and cirrus correction of the Sentinel-2 products
-* [Sen2mosaic](sen2mosaic.md) for download, preprocessing and mosaicing of Sentinel-2 products
-* [SNAP](snap.md) for remote sensing applications
-* [WhiteboxTools](whiteboxtools.md) an advanced geospatial data analysis platform
-* [Zonation](zonation.md) Spatial conservation prioritization framework
-* Deep learning: pytorch, tensorflow
-* [Puhti softwate pages](https://docs.csc.fi/apps/by_discipline/#geosciences)
-
-In Mahti: geoconda, PyTorch, TensorFlow
-
-In LUMI: 
-* GDAL, QGIS, GRASS GIS, PDAL, SagaGIS, PyTorch, TensorFlow
+* [pytorch](https://docs.csc.fi/apps/pytorch.md) for deep learning
+* [tensorflow](https://docs.csc.fi/apps/tensorflow.md) for deep learning 
+
+## GIS tools available in LUMI
+
+* [GDAL](https://docs.csc.fi/apps/gdal.md) for geospatial data formats
+* [GRASS GIS](https://docs.csc.fi/apps/grass.md) General purpose GIS software family for viewing, editing and analysing geospatial data
+* [PDAL](https://docs.csc.fi/apps/pdal.md) for point cloud translations and processing
+* [QGIS](https://docs.csc.fi/apps/qgis.md) General purpose GIS software family for viewing, editing and analysing geospatial data
+* [SAGA GIS](https://docs.csc.fi/apps/saga-gis.md) General purpose GIS software family for viewing, editing and analysing geospatial data
+* [pytorch](https://docs.csc.fi/apps/pytorch.md) for deep learning
+* [tensorflow](https://docs.csc.fi/apps/tensorflow.md) for deep learning 
+
+## Documentation
+* [Puhti and Mahti softwate pages](https://docs.csc.fi/apps/by_discipline/#geosciences)
 * [LUMI software pages](https://docs.lumi-supercomputer.eu/software/)
 
  Something missing?
       Ask us :)
       [email protected]
 
-## Geospatial software NOT available in Puhti
+## Geospatial software NOT available in supercomputers
 
 * Servers -> these can be run in cPouta
 	* Web map servers

materials/supercomputer/supercomputing.md

@@ -1,4 +1,4 @@
-# Supercomputing 
+# Recources and possibilies 
 
 ## Why use a supercomputer?
 
@@ -65,6 +65,7 @@ Puhti - Mahti - LUMI
 ![](../images/lumi2.jpg)
 
 Name | CPUs | GPUs | Pre-installed GIS tools | Finnish spatial data locally | Scope |
+--- | --- | --- | --- | --- | --- |
 Puhti | 28 000 | 240 Nvidia V100 | 20 | Yes | National |
 Mahti | 90 000 | 96 Nvidia A100 | 1 | No | National |
 LUMI | 100 000 | 10 000 AMD MI250X | 5 | No | EU |