Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] v.in.wfs fails when special characters exist in the wfs url #4275

Open
karunakar2 opened this issue Sep 5, 2024 · 6 comments
Open
Labels
bug Something isn't working

Comments

@karunakar2
Copy link

Describe the bug

The v.in.wfs throws an error "raise InvalidURL(f"URL can't contain control characters. {url!r} " when the url has special characters as a part of api validation.

To reproduce

the wfs provided by koordinates.co.nz has url with api embedded in the url

  1. Go to QGIS processing toolbox and open v.in.wfs (grass module)
  2. Set the koordinates wfs url (https://data.waikatodistrict.govt.nz/services;key=/wfs?
  3. run the algorithm
  4. See error in the log window
    -->

Expected behavior

WFS should work as usual if the urllib issue is fixed.

Screenshots

System description

Additional context

@karunakar2 karunakar2 added the bug Something isn't working label Sep 5, 2024
@karunakar2
Copy link
Author

owslib circumvents the issue

@tmszi
Copy link
Member

tmszi commented Sep 7, 2024

Hello I tried your WFS URL example https://data.waikatodistrict.govt.nz/services;key=/wfs? with Python OWSLib, with default WFS version 1.0.0 or 1.1.0 but I got HTTP error.

from owslib.wfs import WebFeatureService
wfs = WebFeatureService(url="https://data.waikatodistrict.govt.nz/services;key=/wfs?")

Error message:

HTTPError: 404 Client Error: Not Found for url: https://data.waikatodistrict.govt.nz/services;key=/wfs?service=WFS&request=GetCapabilities&version=1.0.0

Could you provide functional public WFS URL example please?

@karunakar2
Copy link
Author

karunakar2 commented Sep 7, 2024 via email

@tmszi
Copy link
Member

tmszi commented Sep 15, 2024

I registered at page https://data.waikatodistrict.govt.nz/ and I tried download some data via WFS service successfully with GRASS GIS module v.in.wfs.

v.in.wfs command (Change URL <API_KEY> to your relevant service API KEY.):

v.in.wfs "url=https://data.waikatodistrict.govt.nz/services;key=<API_KEY>/wfs/?" output=imported_wfs_data name=data.waikatodistrict.govt.nz:layer-95697

Example:

GRASS epsg_2193/PERMANENT:~ > v.in.wfs "url=https://data.waikatodistrict.govt.nz/services;key=<API_KEY>/wfs/?" output=imported_wfs_data name=data.waikatodistrict.govt.nz:layer-95697
https://data.waikatodistrict.govt.nz/services;key=<API_KEY>/wfs/?REQUEST=GetFeature&SERVICE=WFS&VERSION=1.0.0&TYPENAME=data.waikatodistrict.govt.nz:layer-95697
Retrieving data...
Importing data...
No projection information available for layer <layer-95697>
Over-riding projection check
Check if OGR layer <layer-95697> contains polygons...
 100%
Creating attribute table for layer <layer-95697>...
Importing 1 features (OGR layer <layer-95697>)...
 100%
-----------------------------------------------------
Registering primitives...
-----------------------------------------------------
Cleaning polygons
-----------------------------------------------------
Breaking polygons...
Breaking polygons (pass 1: select break points)...
 100%
Breaking polygons (pass 2: break at selected points)...
 100%
-----------------------------------------------------
Removing duplicates...
 100%
-----------------------------------------------------
Breaking boundaries...
 100%
-----------------------------------------------------
Removing duplicates...
 100%
-----------------------------------------------------
Cleaning boundaries at nodes...
 100%
-----------------------------------------------------
Merging boundaries...
 100%
-----------------------------------------------------
Removing dangles...
 100%
-----------------------------------------------------
Building areas...
 100%
-----------------------------------------------------
Removing bridges...
 100%
-----------------------------------------------------
Registering primitives...
Building areas...
 100%
Attaching islands...
 100%
-----------------------------------------------------
Finding centroids for OGR layer <layer-95697>...
 100%
-----------------------------------------------------
Writing centroids...
 100%
-----------------------------------------------------
1 input polygons
Total area: 472316 (1 areas)
-----------------------------------------------------
Copying features...
 100%
Building topology for vector map <imported_wfs_data@PERMANENT>...
Registering primitives...
Building areas...
 100%
Attaching islands...
 100%
Attaching centroids...
 100%
Vector map <imported_wfs_data> imported from WFS.

Imported WFS vector map imported_wfs_data metadata:

GRASS epsg_2193/PERMANENT:~ > v.info imported_wfs_data
 +----------------------------------------------------------------------------+
 | Name:            imported_wfs_data                                         |
 | Mapset:          PERMANENT                                                 |
 | Project:         epsg_2193                                                 |
 | Database:        /home/tomas/grassdata                                     |
 | Title:                                                                     |
 | Map scale:       1:1                                                       |
 | Name of creator: tomas                                                     |
 | Organization:                                                              |
 | Source date:     Sun Sep 15 04:49:08 2024                                  |
 | Timestamp (first layer): none                                              |
 |----------------------------------------------------------------------------|
 | Map format:      native                                                    |
 |----------------------------------------------------------------------------|
 |   Type of map: vector (level: 2)                                           |
 |                                                                            |
 |   Number of points:       0               Number of centroids:  1          |
 |   Number of lines:        0               Number of boundaries: 1          |
 |   Number of areas:        1               Number of islands:    1          |
 |                                                                            |
 |   Map is 3D:              No                                               |
 |   Number of dblinks:      1                                                |
 |                                                                            |
 |   Projection: NZGD2000 / New Zealand Transverse Mercator 2000              |
 |                                                                            |
 |               N:  5825400.23772001    S:  5824481.42474173                 |
 |               E:  1794852.97836222    W:  1793671.01079498                 |
 |                                                                            |
 |   Digitization threshold: 0                                                |
 |   Comment:                                                                 |
 |                                                                            |
 +----------------------------------------------------------------------------+

Could you try this example (or with another data from this WFS service) with GRASS GIS directly, and confirm that importing data is successful, please?

@karunakar2
Copy link
Author

karunakar2 commented Sep 16, 2024 via email

@tmszi
Copy link
Member

tmszi commented Sep 17, 2024

Any specific version of grass?

I tested with 8.5.0dev version, but you can try last stable 8.4.0 or 8.3 version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants