read of aircraft data strange warning as from 2024

[espinielli]

Data for registered aircraft seems to fail as from 2024.
I am using v1.0.0 of the package.

December 2023 is ok:

> aaa <- read_aircrafts(date = "202312", cache = TRUE)
Download status: 1 done; 0 in progress. Total size: 8.07 Mb (100%)... done!             
  |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=00s  

But 2024 starts to fail...maybe format has changed?

> aaa <- read_aircrafts(date = 202401, cache = TRUE)
Download status: 1 done; 0 in progress. Total size: 10.21 Mb (100%)... done!             
  |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=00s  
Warning message:
In data.table::fread(x, showProgress = showProgress, encoding = "UTF-8",  :
  Found and resolved improper quoting out-of-sample. First healed line 7631: <<"PTPSL";"ARRAIAL D"AJUDA EMPREENDIMENTOS IMOBILIARIOS LTDA";"null";"BA";"07.890.990/0001-20";"ARRAIAL D"AJUDA EMPREENDIMENTOS IMOBILIARIOS LTDA";"null";"null";"null";"null";"048";"TPP";"null";"MONTAER MC01";"MONTAER - MONTAGEM E MANUTENCAO AERONAUTICA LTDA";"null";"600";"null";"null";"null";"null";"null";"null";"null";"null";"null";"R";"null";"null";"null";"null";"RESERVADAS AS MARCAS";"">>. If the fields are not quoted (e.g. field separator does not appear within any field), try quote="" to avoid this warning.
> aaa <- read_aircrafts(date = "202402", cache = TRUE)
Download status: 1 done; 0 in progress. Total size: 10.23 Mb (100%)... done!             
  |++++++++++++++++++++++++++++++++++++++++++++++++++| 100% elapsed=00s  
Warning message:
In data.table::fread(x, showProgress = showProgress, encoding = "UTF-8",  :
  Found and resolved improper quoting out-of-sample. First healed line 7267: <<"PTPSL";"ARRAIAL D"AJUDA EMPREENDIMENTOS IMOBILIARIOS LTDA";"null";"BA";"07.890.990/0001-20";"ARRAIAL D"AJUDA EMPREENDIMENTOS IMOBILIARIOS LTDA";"null";"null";"null";"null";"048";"TPP";"null";"MONTAER MC01";"MONTAER - MONTAGEM E MANUTENCAO AERONAUTICA LTDA";"null";"600";"null";"null";"null";"null";"null";"null";"null";"null";"null";"R";"null";"null";"null";"null";"RESERVADAS AS MARCAS";"">>. If the fields are not quoted (e.g. field separator does not appear within any field), try quote="" to avoid this warning.

[rafapereirabr]

ANAC's data sets come in .csv files and oftentimes there is an issue or two with quotation marks. In this case, the value of the "prorietario" (owner) column is "ARRAIAL D'AJUDA EMPREENDIMENTOS IMOBILIARIOS", but R parses is as D"AJUDA and hence the annoying warning messages.

Nonetheless, please note that this issue is solved, as the message says "Found and resolved improper quoting out-of-sample". The function returns the expected result. Here is the row with the quotation issue.

 a202402 <- flightsbr::read_aircrafts(date = "202402")
  
 a202402[marca=='PTPSL',]

[espinielli]

In fact it is not a problem of R: I downloaded the relevant file and the payload is using double quote instead of a quote.

"PTPSL";"ARRAIAL D"AJUDA EMPREENDIMENTOS IMOBILIARIOS LTDA";"null";"BA";...

[rafapereirabr]

ah, indeed, it's a problem with the data then. In such case, there isn't much we can do from our side. Thanks for the heads up anyway!

[espinielli]

Do you think reporting the issue back to the data owner would help?

[rafapereirabr]

I've contacted a colleague who might be able to help ,but I'm not very optimistic will make a difference, unfortunately