Syntax error when loading shapefile

[jakerbrown]

I have progressed to the end of this help page: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/PostGIS-on-FASRC

But when running the last command, loading the shapefile, I get the following error:

usvotersdb=# shp2pgsql -I US_place_2010.shp US_place_2010 | psql -d usvotersdb -h localhost -p 8521;
ERROR: syntax error at or near "shp2pgsql"
LINE 1: shp2pgsql -I US_place_2010.shp US_place_2010 | psql -d usvot...
^
Time: 2.448 ms

[dkakkar]

I have progressed to the end of this help page: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/PostGIS-on-FASRC

But when running the last command, loading the shapefile, I get the following error:

usvotersdb=# shp2pgsql -I US_place_2010.shp US_place_2010 | psql -d usvotersdb -h localhost -p 8521;
ERROR: syntax error at or near "shp2pgsql"
LINE 1: shp2pgsql -I US_place_2010.shp US_place_2010 | psql -d usvot...
^
Time: 2.448 ms

You don't need to do this step. It is just for reference in case you want to upload a shapefile. In your case we are using database dumps.

[jakerbrown]

Thank you. As a clarification, each time I want to run some knn analysis in postgis on the FASRC, I need to run through the instructions from this page: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/PostGIS-on-FASRC , and then move to this page https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/KNN-calculation ?

[dkakkar]

Thank you. As a clarification, each time I want to run some knn analysis in postgis on the FASRC, I need to run through the instructions from this page: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/PostGIS-on-FASRC , and then move to this page https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/KNN-calculation ?

Please follow the order:
Launch PostGIS: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/PostGIS-on-FASRC
Import Database:https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/Loading-and-Exporting-US-voters-database
Run KNN: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/KNN-calculation

[jakerbrown]

Okay, thank you. I imported the database (2012 US voterfile) prior to launching PostGIS. Was this incorrect?

[jakerbrown]

Also, importing the database took approximately 10 hours. So when asking for time from FASRC, I need to budget time for this each time? I apologize if my questions are unclear.

[dkakkar]

Okay, thank you. I imported the database (2012 US voterfile) prior to launching PostGIS. Was this incorrect?

Launch Postgis is the first step in Import instructions. Wondering how you imported without launching PostGIS?

[dkakkar]

Also, importing the database took approximately 10 hours. So when asking for time from FASRC, I need to budget time for this each time? I apologize if my questions are unclear.

It Should not take that long. It took me an hour or so for 2018. How big is 2012?

[dkakkar]

Also, importing the database took approximately 10 hours. So when asking for time from FASRC, I need to budget time for this each time? I apologize if my questions are unclear.

It should not take that long. It took me an hour or so for 2018. How big is 2012? How many voters? How many fields?

[jakerbrown]

I might be using the terminology wrong here. I thought the "Importing Database" link you shared referred to creating the pgsql dump?

[dkakkar]

I might be using the terminology wrong here. I thought the "Importing Database" link you shared referred to creating the pgsql dump?

Importing refers to importing to Postgis using CSV file. Once you have imported we export and store as psql dump to use the dump for future imports as it is faster. Since FASRC resource is temporary you might have to import multiple times. Please use 256 GB RAM and 168:00:00 hours (7 days) and 2 CPU device as your POSTGIS parameters when you are ready to do the work, for testing you could use smaller values. These are maximum values of the parameters. However, make sure you use them otherwise your credit on FASRC will go down if you keep them idle for long.

[jakerbrown]

So the instructions in "Loading and Exporting US voters database"
(https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/Loading-and-Exporting-US-voters-database) do not refer to creating the dump? I was under the impression yesterday that I was running those instructions to create the dump, which we discussed in a different issue (#4). But if not, I am confused as to what I was doing.

[dkakkar]

I think we are talking about the same thing. It refers to creating/ exporting the dump from input CSV file. You seem to be on right track.

…

On Fri, Aug 14, 2020, 10:10 AM Jacob Brown ***@***.***> wrote: So the instructions in "Loading and Exporting US voters database" ( https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/Loading-and-Exporting-US-voters-database) do not refer to creating the dump? I was under the impression yesterday that I was running those instructions to create the dump, which we discussed in a different issue (#4 <#4>). But if not, I am confused as to what I was doing. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACWCV2GQAMRL2IRBKER7743SAVAUDANCNFSM4P7QL5YA> .

[jakerbrown]

Thanks for the continued help on this!

So based on the order above:

Launch PostGIS: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/PostGIS-on-FASRC
Import Database:https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/Loading-and-Exporting-US-voters-database
Run KNN: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/KNN-calculation

I was able to get through all of "Launch PostGIS", but if I used "Import Database" to create the dump yesterday, what is it I need to do with those instructions now? Re-run them, but this time it does something different?

[dkakkar]

After you imprt your database, you could start knn processing instructions.

…

On Fri, Aug 14, 2020, 10:31 AM Jacob Brown ***@***.***> wrote: Thanks for the continued help on this! So based on the order above: Launch PostGIS: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/PostGIS-on-FASRC Import Database: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/Loading-and-Exporting-US-voters-database Run KNN: https://github.com/cga-harvard/GIS_Apps_on_HPC/wiki/KNN-calculation I was able to get through all of "Launch PostGIS", but if I used "Import Database" to create the dump yesterday, what is it I need to do with those instructions now? Re-run them, but this time it does something different? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACWCV2CG3G7JQJEVMYQSTJTSAVDEZANCNFSM4P7QL5YA> .

[jakerbrown]

Does importing the database refer to the instructions in "Launch PostGIS" where we are restoring the database from the dump? Or the running of import.sql in "Import Database"?