first pass docs update

Author: tomkralidis

CONTRIBUTING.rst

@@ -14,7 +14,7 @@ pycsw has the following modes of contribution:
 Code of Conduct
 ---------------
 
-Contributors to this project are expected to act respectfully toward others in accordance with the `OSGeo Code of Conduct <http://www.osgeo.org/code_of_conduct>`_.
+Contributors to this project are expected to act respectfully toward others in accordance with the `OSGeo Code of Conduct <https://www.osgeo.org/code_of_conduct>`_.
 
 Contributions and Licensing
 ---------------------------
@@ -61,7 +61,7 @@ Documentation
 
 - documentation is managed in ``docs/``, in reStructuredText format
 - `Sphinx`_ is used to generate the documentation
-- See the `reStructuredText Primer <http://sphinx-doc.org/rest.html>`_ on rST markup and syntax.
+- See the `reStructuredText Primer <https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html>`_ on rST markup and syntax.
 
 Bugs
 ----
@@ -94,7 +94,7 @@ Coding Guidelines
 
 - pycsw instead of PyCSW, pyCSW, Pycsw
 - always code with `PEP 8`_ conventions
-- always run source code through ``pep8`` and `pylint`_, using all pylint defaults except for ``C0111``.  ``sbin/pycsw-pylint.sh`` is included for convenience
+- always run source code through `flake8`_ and `pylint`_, using all pylint defaults except for ``C0111``.  ``sbin/pycsw-pylint.sh`` is included for convenience
 - for exceptions which make their way to OGC ``ExceptionReport`` XML, always specify the appropriate ``locator`` and ``code`` parameters
 - the pycsw wiki documents `developer tasks`_ for things like releasing documentation, testing, etc.
 
@@ -137,13 +137,10 @@ your own repository to ensure your pycsw repository is up to date with pycsw mas
   git checkout master
   git pull upstream master
 
-.. _`Corporate`: http://www.osgeo.org/sites/osgeo.org/files/Page/corporate_contributor.txt
-.. _`Individual`: http://www.osgeo.org/sites/osgeo.org/files/Page/individual_contributor.txt
 .. _`[email protected]`: mailto:[email protected]
-.. _`OSGeo`: http://www.osgeo.org/content/foundation/legal/licenses.html
-.. _`PEP 8`: http://www.python.org/dev/peps/pep-0008/
-.. _`pep8`: http://pypi.python.org/pypi/pep8/
-.. _`pylint`: http://www.logilab.org/857
-.. _`Sphinx`: http://sphinx-doc.org/
+.. _`PEP 8`: https://www.python.org/dev/peps/pep-0008/
+.. _`flake8`: https://gitlab.com/pycqa/flake8
+.. _`pylint`: https://pylint.org
+.. _`Sphinx`: https://www.sphinx-doc.org
 .. _`developer tasks`: https://github.com/geopython/pycsw/wiki/Developer-Tasks
 .. _`mailing list`: https://pycsw.org/community#mailing-list

LICENSE.txt

@@ -1,9 +1,10 @@
 The MIT License (MIT)
+=====================
 
-Copyright (c) 2010-2015 Tom Kralidis
-Copyright (c) 2011-2015 Angelos Tzotsos
-Copyright (c) 2012-2015 Adam Hinz
-Copyright (c) 2015      Ricardo Garcia Silva
+Copyright © 2010-2021 Tom Kralidis
+Copyright © 2011-2021 Angelos Tzotsos
+Copyright © 2012-2015 Adam Hinz
+Copyright © 2015-2021 Ricardo Garcia Silva
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

MANIFEST.in

@@ -1,3 +1,3 @@
 recursive-include pycsw *.xsd *.yaml *.html *.png *.ico
 include *.txt
-include README.rst
+include README.md

README.md

@@ -0,0 +1,23 @@
+# pycsw
+
+[![Build Status](https://github.com/geopython/pycsw/workflows/build%20%E2%9A%99%EF%B8%8F/badge.svg)](https://github.com/geopython/pycsw/actions)
+[![Join the chat at https://gitter.im/geopython/pycsw](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/geopython/pycsw)
+[![Documentation](https://readthedocs.org/projects/pycsw/badge/)](https://docs.pycsw.org)
+
+[pycsw](https://pycsw.org) is an OGC API - Records and CSW server implementation written in Python.
+
+pycsw fully implements the the [OGC API - Records (OARec) standard](https://ogcapi.ogc.org/records/)
+and the OpenGIS Catalogue Service Implementation Specification (Catalogue Service for
+the Web). Initial development started in 2010 (more formally announced in
+2011). The project is certified OGC Compliant, and is an OGC Reference
+Implementation.  Since 2015, pycsw is an official OSGeo Project.
+
+pycsw allows for the publishing and discovery of geospatial metadata via
+numerous APIs (OGC API - Records, CSW 2/CSW 3, OpenSearch, OAI-PMH, SRU). Existing
+repositories of geospatial metadata can also be exposed, providing a standards-based
+metadata and catalogue component of spatial data infrastructures.
+
+pycsw is Open Source, released under an MIT license, and runs on all major
+platforms (Windows, Linux, Mac OS X).
+
+Please read the docs at [https://pycsw.org/docs](https://pycsw.org/docs) for more information.

README.rst

@@ -53,7 +53,7 @@ allowed_ips=127.0.0.1
 
 [metadata:main]
 identification_title=pycsw Geospatial Catalogue
-identification_abstract=pycsw is an OGC CSW server implementation written in Python
+identification_abstract=pycsw is an OARec and OGC CSW server implementation written in Python
 identification_keywords=catalogue,discovery,metadata
 identification_keywords_type=theme
 identification_fees=None

default-sample.cfg

@@ -3,15 +3,15 @@
 </p>
 
 <p>
-    <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." src="https://portal.opengeospatial.org/public_ogc/compliance/Certified_OGC_Compliant_Logo_Web.gif" height="74"/></a>
+    <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="https://www.ogc.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." src="https://portal.ogc.org/public_ogc/compliance/Certified_OGC_Compliant_Logo_Web.gif" height="74"/></a>
 </p>
 
 <p>
-    <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." src="https://portal.opengeospatial.org/public_ogc/compliance/badge.php?s=CAT%20CSW%203.0.0&r=1" height="38"/></a>
+    <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="https://www.ogc.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 3.0.0. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." src="https://portal.ogc.org/public_ogc/compliance/badge.php?s=CAT%20CSW%203.0.0&r=1" height="38"/></a>
 </p>
 
 <p>
-    <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 2.0.2. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="http://www.opengeospatial.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 2.0.2. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." src="https://portal.opengeospatial.org/public_ogc/compliance/badge.php?s=CAT%20CSW%202.0.2&r=1" height="38"/></a>
+    <a title="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 2.0.2. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." href="https://www.ogc.org/resource/products/details/?pid=1374"><img alt="This product conforms to the OpenGIS Catalogue Service Implementation Specification [Catalogue Service for the Web], Revision 2.0.2. OGC, OGC®, and CERTIFIED OGC COMPLIANT are trademarks or registered trademarks of the Open Geospatial Consortium, Inc. in the United States and other countries." src="https://portal.ogc.org/public_ogc/compliance/badge.php?s=CAT%20CSW%202.0.2&r=1" height="38"/></a>
 </p>
 
 <p>

docs/_templates/indexsidebar.html

@@ -39,11 +39,10 @@
         <a href="https://pycsw.org">Website</a> &mdash;
         <a href="https://pycsw.org/community">Community</a> &mdash;
         <a href="https://pycsw.org/blog">Blog</a> &mdash;
-        <a href="https://plus.google.com/communities/104084873011085696113">Google+</a> &mdash;
         <a href="https://pycsw.org/download">Download</a> &mdash;
         <a href="https://github.com/geopython/pycsw">Source</a> &mdash;
         <a href="https://github.com/geopython/pycsw/issues">Issues</a> &mdash;
-        <a href="http://lists.osgeo.org/mailman/listinfo/pycsw-devel">Mailing List</a>
+        <a href="https://lists.osgeo.org/mailman/listinfo/pycsw-devel">Mailing List</a>
     </p>
 </div>
 {% endblock %}

docs/_templates/layout.html

@@ -16,7 +16,7 @@ Metadata Repository Setup
 pycsw supports the following databases:
 
 - SQLite3
-- PostgreSQL
+- PostgreSQL (without PostGIS)
 - PostgreSQL with PostGIS enabled
 - MySQL
 
@@ -27,7 +27,7 @@ pycsw supports the following databases:
   PostgreSQL support includes support for PostGIS functions if enabled
 
 .. note::
-  If PostGIS (1.x or 2.x) is activated before setting up the pycsw/PostgreSQL database, then native PostGIS geometries will be enabled.
+  If PostGIS is activated before setting up the pycsw/PostgreSQL database, then native PostGIS geometries will be enabled.
 
 To expose your geospatial metadata via pycsw, perform the following actions:
 
@@ -38,7 +38,9 @@ To expose your geospatial metadata via pycsw, perform the following actions:
 Supported Information Models
 ----------------------------
 
-By default, pycsw supports the ``csw:Record`` information model.
+By default, pycsw's API  supports the core OARec and CSW Record information models.  From
+the database perspective, the pycsw metadata model is loosely based on ISO 19115 and is
+able to transform to other formats as part of transformation during OARec/CSW requests.
 
 .. note::
   See :ref:`profiles` for information on enabling profiles
@@ -48,31 +50,31 @@ Setting up the Database
 
 .. code-block:: bash
 
-  $ pycsw-admin.py -c setup_db -f default.cfg
+  pycsw-admin.py -c setup_db -f default.cfg
 
 This will create the necessary tables and values for the repository.
 
-The database created is an `OGC SFSQL`_ compliant database, and can be used with any implementing software.  For example, to use with `OGR`_:
+The database created is an `OGC SFSQL`_ compliant database, and can be used with any implementing software.  For example, to use with `GDAL`_:
 
 .. code-block:: bash
 
-  $ ogrinfo /path/to/records.db
+  ogrinfo /path/to/records.db
   INFO: Open of 'records.db'
   using driver 'SQLite' successful.
   1: records (Polygon)
-  $ ogrinfo -al /path/to/records.db
+  ogrinfo -al /path/to/records.db
   # lots of output
 
 .. note::
-  If PostGIS is detected, the pycsw-admin.py script does not create the SFSQL tables as they are already in the database.
+  If PostGIS is detected, the ``pycsw-admin.py`` script does not create the SFSQL tables as they are already in the database.
 
 
 Loading Records
 ----------------
 
 .. code-block:: bash
 
-  $ pycsw-admin.py -c load_records -f default.cfg -p /path/to/records
+  pycsw-admin.py -c load_records -f default.cfg -p /path/to/records
 
 This will import all ``*.xml`` records from ``/path/to/records`` into the database specified in ``default.cfg`` (``repository.database``).  Passing ``-r`` to the script will process ``/path/to/records`` recursively.  Passing ``-y`` to the script will force overwrite existing metadata with the same identifier.  Note that ``-p`` accepts either a directory path or single file.
 
@@ -84,7 +86,7 @@ Exporting the Repository
 
 .. code-block:: bash
 
-  $ pycsw-admin.py -c export_records -f default.cfg -p /path/to/output_dir
+  pycsw-admin.py -c export_records -f default.cfg -p /path/to/output_dir
 
 This will write each record in the database specified in ``default.cfg`` (``repository.database``) to an XML document on disk, in directory ``/path/to/output_dir``.
 
@@ -93,7 +95,8 @@ Optimizing the Database
 
 .. code-block:: bash
 
-  $ pycsw-admin.py -c optimize_db -f default.cfg
+  pycsw-admin.py -c optimize_db -f default.cfg
+  pycsw-admin.py -c rebuild_db_indexes -f default.cfg
 
 .. note::
   This feature is relevant only for PostgreSQL and MySQL
@@ -103,7 +106,7 @@ Deleting Records from the Repository
 
 .. code-block:: bash
 
-  $ pycsw-admin.py -c delete_records -f default.cfg
+  pycsw-admin.py -c delete_records -f default.cfg
 
 This will empty the repository of all records.
 
@@ -114,14 +117,14 @@ PostgreSQL
 ^^^^^^^^^^
 
 - if PostGIS is not enabled, pycsw makes uses of PL/Python functions.  To enable PostgreSQL support, the database user must be able to create functions within the database. In case of recent PostgreSQL versions (9.x), the PL/Python extension must be enabled prior to pycsw setup
-- `PostgreSQL Full Text Search`_ is supported for ``csw:AnyText`` based queries.  pycsw creates a tsvector column based on the text from anytext column. Then pycsw creates a GIN index against the anytext_tsvector column.  This is created automatically in ``pycsw.admin.setup_db``.  Any query against `csw:AnyText` or `apiso:AnyText` will process using PostgreSQL FTS handling
+- `PostgreSQL Full Text Search`_ is supported for ``csw:AnyText`` based queries.  pycsw creates a tsvector column based on the text from anytext column. Then pycsw creates a GIN index against the anytext_tsvector column.  This is created automatically in ``pycsw.admin.setup_db``.  Any query against OARec's ``q`` parameter or CSW `csw:AnyText` or `apiso:AnyText` will process using PostgreSQL FTS handling
 
 PostGIS
 ^^^^^^^
 
 - pycsw makes use of PostGIS spatial functions and native geometry data type.
 - It is advised to install the PostGIS extension before setting up the pycsw database
-- If PostGIS is detected, the pycsw-admin.py script will create both a native geometry column and a WKT column, as well as a trigger to keep both synchronized. 
+- If PostGIS is detected, the pycsw-admin.py script will create both a native geometry column and a WKT column, as well as a trigger to keep both synchronized
 - In case PostGIS gets disabled, pycsw will continue to work with the `WKT`_ column
 - In case of migration from plain PostgreSQL database to PostGIS, the spatial functions of PostGIS will be used automatically
 - When migrating from plain PostgreSQL database to PostGIS, in order to enable native geometry support, a "GEOMETRY" column named "wkb_geometry" needs to be created manually (along with the update trigger in ``pycsw.admin.setup_db``). Also the native geometries must be filled manually from the `WKT`_ field. Next versions of pycsw will automate this process
@@ -214,8 +217,8 @@ Values of mappings can be derived from the following mechanisms:
 
 Further information is provided in ``pycsw/config.py:MD_CORE_MODEL``.
 
-.. _`OGR`: http://www.gdal.org/ogr
-.. _`OGC SFSQL`: http://www.opengeospatial.org/standards/sfs
-.. _`WKT`: http://en.wikipedia.org/wiki/Well-known_text
-.. _`EWKT`: http://en.wikipedia.org/wiki/Well-known_text#Variations
-.. _`PostgreSQL Full Text Search`: http://www.postgresql.org/docs/9.2/static/textsearch.html
+.. _`GDAL`: https://www.gdal.org
+.. _`OGC SFSQL`: https://www.ogc.org/standards/sfs
+.. _`WKT`: https://en.wikipedia.org/wiki/Well-known_text
+.. _`EWKT`: https://en.wikipedia.org/wiki/Well-known_text#Variations
+.. _`PostgreSQL Full Text Search`: https://www.postgresql.org/docs/current/textsearch.html

docs/administration.rst

@@ -4,7 +4,7 @@ API
 ===
 
 Python applications can integrate pycsw into their custom workflows.  This
-allows for seamless integate within frameworks like Flask and Django
+allows for seamless integate within frameworks such as Flask and Django.
 
 Below are examples of where using the API (as opposed to the default WSGI/CGI
 services could be used:
@@ -14,9 +14,15 @@ services could be used:
 - authentication or authorization logic
 - forcing CSW version 2.0.2 as default
 
+OARec Flask Example
+-------------------
 
-Simple Flask Example
---------------------
+See https://github.com/geopython/pycsw/blob/master/pycsw/wsgi_flask.py for how
+to implement a Flask wrapper atop all pycsw supported APIs.  Note the use of
+Flask blueprints to enable integration with downstream Flask applications.
+
+Simple CSW Flask Example
+------------------------
 
 .. code-block:: python