'nomis_data_info' function times out

[ralphlsmith]

Hi I am getting the following error when trying to use nomisr at work (Birmingham City Council)

datasets <- nomis_data_info()
Error in curl::curl_fetch_memory(url, handle = handle) :
Timeout was reached: Connection timed out after 10000 milliseconds

My code works OK at home on my own MacBook.

Our outsourced ICT help is of little use (they dont know anything about R, and are unwilling to find out) so I was hoping you may have a clue as to how I can get this fixed?

cheers

Ralph

[evanodell]

What's the result of sessionInfo()?

It might be an issue with your internet, is there any throttling or firewalls or things like that? If you're on wifi, try running from a ethernet cable if that's available.

Do other 'nomisr' functions work?

[ralphlsmith]

Hi Sessioninfo() gives the following, although I am not sure

sessionInfo()

R version 3.5.1 (2018-07-02) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 7 x64 (build 7601) Service Pack 1 Matrix products: default locale: [1] LC_COLLATE=English_United Kingdom.1252 [2] LC_CTYPE=English_United Kingdom.1252 [3] LC_MONETARY=English_United Kingdom.1252 [4] LC_NUMERIC=C [5] LC_TIME=English_United Kingdom.1252 attached base packages: [1] stats graphics grDevices utils datasets methods [7] base other attached packages: [1] nomisr_0.4.1 loaded via a namespace (and not attached): [1] Rcpp_1.0.0 XML_3.98-1.16 crayon_1.3.4 [4] dplyr_0.7.8 assertthat_0.2.0 bitops_1.0-6 [7] plyr_1.8.4 R6_2.3.0 jsonlite_1.5 [10] rsdmx_0.5-13 magrittr_1.5 httr_1.3.1 [13] pillar_1.3.0 rlang_0.3.0.1 bindrcpp_0.2.2 [16] tools_3.5.1 readr_1.1.1 glue_1.3.0 [19] RCurl_1.95-4.11 purrr_0.2.5 hms_0.4.2 [22] compiler_3.5.1 pkgconfig_2.0.2 bindr_0.1.1 [25] tidyselect_0.2.5 tibble_1.4.2

There are plenty of firewalls which are locked down pretty tight as we are a local authority. I have had our network help desk on the line but he could not id anything. I am on an ethernet cable. Cheers Ralph Ralph Smith Public Health Knowledge Impact and Outcomes Lead Adult Social Care and Health, Birmingham City Council Office: 0121 303 5702 | 07864 930224 Main email: [email protected]<mailto:[email protected]> From: Evan Odell [mailto:[email protected]] Sent: 14 January 2019 11:45 To: ropensci/nomisr Cc: Ralph Smith; Author Subject: Re: [ropensci/nomisr] 'nomis_data_info' function times out (#13) What's the result of sessionInfo()? It might be an issue with your internet, is there any throttling or firewalls or things like that? If you're on wifi, try running from a ethernet cable if that's available. Do other 'nomisr' functions work? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#13 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AsgyqdSUDUvRWNIrMUdxISmyAxl1UGMWks5vDG2lgaJpZM4Z-D9J>. *********************************************************************** *********************************************************************** The information contained within this e-mail (and any attachment) sent by Birmingham City Council is confidential and may be legally privileged. It is intended only for the named recipient or entity to whom it is addressed. If you are not the intended recipient please accept our apologies and notify the sender immediately. Unauthorised access, use, disclosure, storage or copying is not permitted and may be unlawful. Any e-mail including its content may be monitored and used by Birmingham City Council for reasons of security and for monitoring internal compliance with the office policy on staff use. E-mail blocking software may also be used. Any views or opinions presented are solely those of the originator and do not necessarily represent those of Birmingham City Council. We cannot guarantee that this message or any attachment is virus free or has not been intercepted and amended. ***********************************************************************

[evanodell]

What's the result of httr::GET("https://www.nomisweb.co.uk/api/v01/dataset/def.sdmx.json")?

It should look something like:

httr::GET("https://www.nomisweb.co.uk/api/v01/dataset/def.sdmx.json")
Response [https://www.nomisweb.co.uk/api/v01/dataset/def.sdmx.json]
  Date: 2019-01-14 12:32
  Status: 200
  Content-Type: application/json; charset=utf-8
  Size: 4.49 MB
{
"structure" :{
"header" :{
"id" :"none",
"prepared" :"2019-01-14T12:31:48Z",
"sender" :{
"contact" :{
"email" :"[email protected]",
"name" :"Nomis",
"telephone" :"+441913342680",
...

[ralphlsmith]

I get the same timeout error httr::GET("https://www.nomisweb.co.uk/api/v01/dataset/def.sdmx.json") Error in curl::curl_fetch_memory(url, handle = handle) : Timeout was reached: Connection timed out after 10015 milliseconds Ralph Smith Public Health Knowledge Impact and Outcomes Lead Adult Social Care and Health, Birmingham City Council Office: 0121 303 5702 | 07864 930224 Main email: [email protected]<mailto:[email protected]> From: Evan Odell [mailto:[email protected]] Sent: 14 January 2019 12:33 To: ropensci/nomisr Cc: Ralph Smith; Author Subject: Re: [ropensci/nomisr] 'nomis_data_info' function times out (#13) What's the result of httr::GET("https://www.nomisweb.co.uk/api/v01/dataset/def.sdmx.json")? It should look something like: httr::GET("https://www.nomisweb.co.uk/api/v01/dataset/def.sdmx.json") Response [https://www.nomisweb.co.uk/api/v01/dataset/def.sdmx.json] Date: 2019-01-14 12:32 Status: 200 Content-Type: application/json; charset=utf-8 Size: 4.49 MB { "structure" :{ "header" :{ "id" :"none", "prepared" :"2019-01-14T12:31:48Z", "sender" :{ "contact" :{ "email" :"[email protected]", "name" :"Nomis", "telephone" :"+441913342680", ... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#13 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AsgyqdlVupSV7-6Amgra8Erdwca8pVLDks5vDHjzgaJpZM4Z-D9J>. *********************************************************************** *********************************************************************** The information contained within this e-mail (and any attachment) sent by Birmingham City Council is confidential and may be legally privileged. It is intended only for the named recipient or entity to whom it is addressed. If you are not the intended recipient please accept our apologies and notify the sender immediately. Unauthorised access, use, disclosure, storage or copying is not permitted and may be unlawful. Any e-mail including its content may be monitored and used by Birmingham City Council for reasons of security and for monitoring internal compliance with the office policy on staff use. E-mail blocking software may also be used. Any views or opinions presented are solely those of the originator and do not necessarily represent those of Birmingham City Council. We cannot guarantee that this message or any attachment is virus free or has not been intercepted and amended. ***********************************************************************

[evanodell]

Do other data API packages work in R?

Try:

req <- curl_fetch_memory("https://www.birmingham.gov.uk/info/20018/adult_social_care")
str(req)

If this errors, then the issue is with curl. I am not very familiar with Windows 7, but I don't believe that it comes with curl built in. You may need to install curl from here: https://curl.haxx.se/windows/

[ralphlsmith]

Thanks Evan, I’ll ask my ICT to look into it – if it’s a Windows thing they should be more comfortable in working it out. I am enjoying using nomisr by the way – very useful. We are trying to get all our local public health analysts to use it. Regards Ralph Ralph Smith Public Health Knowledge Impact and Outcomes Lead Adult Social Care and Health, Birmingham City Council Office: 0121 303 5702 | 07864 930224 Main email: [email protected]<mailto:[email protected]> From: Evan Odell [mailto:[email protected]] Sent: 14 January 2019 13:46 To: ropensci/nomisr Cc: Ralph Smith; Author Subject: Re: [ropensci/nomisr] 'nomis_data_info' function times out (#13) Do other data API packages work in R? Try: req <- curl_fetch_memory("https://www.birmingham.gov.uk/info/20018/adult_social_care") str(req) If this errors, then the issue is with curl. I am not very familiar with Windows 7, but I don't believe that it comes with curl built in. You may need to install curl from here: https://curl.haxx.se/windows/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#13 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AsgyqegRzL3It9yhZ7vgCofDhF4UJGfyks5vDIoTgaJpZM4Z-D9J>. *********************************************************************** *********************************************************************** The information contained within this e-mail (and any attachment) sent by Birmingham City Council is confidential and may be legally privileged. It is intended only for the named recipient or entity to whom it is addressed. If you are not the intended recipient please accept our apologies and notify the sender immediately. Unauthorised access, use, disclosure, storage or copying is not permitted and may be unlawful. Any e-mail including its content may be monitored and used by Birmingham City Council for reasons of security and for monitoring internal compliance with the office policy on staff use. E-mail blocking software may also be used. Any views or opinions presented are solely those of the originator and do not necessarily represent those of Birmingham City Council. We cannot guarantee that this message or any attachment is virus free or has not been intercepted and amended. ***********************************************************************

[evanodell]

Ok, let me know if they find a solution, I'm glad to hear that it's otherwise useful for you.

[cderv]

@ralphlsmith As it does not work at inside your company network but work at home, I suspect it comes from proxy you may be behing. Do you know about a proxy ? Can you ask IT for that ?
You can try this to find if there is a proxy

curl::ie_get_proxy_for_url()

If you have one, you need to get the url and credentials to pass through.
You then need to configure it using httr::use_proxy() in a httr::set_config call.
I made a WIP 📦 to help that you may try and find useful. README has some info about proxy;