savetxt and loadtxt are not portable

[milancurcic]

While porting stdlib_io tests to the new testing framework (PR in progress in #494), it occurred to me that the loadtxt tests were not testing values, but merely loading the data and printing them to stdout. When I re-wrote the tests to compare the values, though the tests passed for GFortran, they failed for ifort (see awvwgk#87).

We should be able to round-trip the data without loss of information, i.e.

call random_number(a)
call savetxt('data.txt', a)
call loadtxt('data.txt', b)
print *, all(a == b) ! should print T

Currently it seems that doesn't generally work because both loadtxt and savetxt use list-directed input and output, which is not generally portable. We should use specific formats with field widths wide enough so that we can both round-trip the data and so that the files are in the same format between different compilers.

[ivan-pi]

Some guidelines on output in formats with sufficient width for different precision kinds would be very useful (maybe even having a function returning a "standardized" format string?). In the past I made the mistake of not writing enough decimals, leading to false conclusions during benchmarking a CFD code.

[epagone]

Some guidelines on output in formats with sufficient width for different precision kinds would be very useful (maybe even having a function returning a "standardized" format string?)

Does g0 address this or it's not guaranteed to work in this context?

[ivan-pi]

Consulting with N2146, Section 13.7.5, about Generalized editing of the forms Gw , Gw.d , and G w.dEe, the standard says,

When w is zero the processor selects the field width.

which means behavior could differ between compilers. It also does not solve the issue for loadtxt, since the standard says further

On input, w shall not be zero. [...] Note that w cannot be zero for input editing.

[epagone]

Thanks. Another solution might be to keep the number of decimals fixed and deal with different kinds using is_close (and related issues therein specified). However, it might be a bit too risky.

[milancurcic]

Some guidelines on output in formats with sufficient width for different precision kinds would be very useful

Here's what I used:

https://en.wikipedia.org/wiki/IEEE_754#Basic_and_interchange_formats

I took the decimal digits value for each real kind from the table, and rounded up, so we get 8, 16, and 35 for sp, dp, and qp, respectively. Then for full width I added 7 to that (it's possible that's more than what's needed):

! Format strings with edit descriptors for each type and kind
character(*), parameter :: &
    FMT_INT = '(*(i0,1x))', &
    FMT_REAL_SP = '(*(es15.8,1x))', &
    FMT_REAL_DP = '(*(es23.16,1x))', &
    FMT_REAL_QP = '(*(es42.35,1x))', &
    FMT_COMPLEX_SP = '(*(es15.8,1x,es15.8))', &
    FMT_COMPLEX_DP = '(*(es23.16,1x,es23.16))', &
    FMT_COMPLEX_QP = '(*(es42.35,1x,es42.35))'

Even if g0 were to be portable between compilers (I tried that first), the real problem I ran into is what Ivan mentioned, you can't use it on input. For integers I use i0 on output (could be g0 as well), and list-directed (*) on input.

Thanks. Another solution might be to keep the number of decimals fixed and deal with different kinds using is_close (and related issues therein specified).

We can test for exact values and we should be able to reproduce them for sp, dp, or qp. Current tests for loadtxt in #506 do this:

        do n = 1, 100
            call random_number(input)
            call savetxt('test_dp.txt', input)
            call loadtxt('test_dp.txt', expected)
            call check(error, all(input == expected))
            if (allocated(error)) return
        end do

and that works for all real kinds. Granted, I didn't test with very large or very small numbers, and we probably should.

Though I have some basic awareness of how floating-point numbers work, I don't have enough experience to know about all caveats to test for. So any help with that would be very welcome.

[epagone]

I'm not an expert of this area too but I thought that exact comparisons of floating point numbers is always a bad idea 🤷‍♂️

[milancurcic]

Exactly comparing values that you know should be exactly the same is fine. On the other side, comparing a floating point value with the result of a floating point expression can be problematic, thus people use tolerant comparisons there.

What I'm unsure of is the ranges of values to test for, and whether the total width of decimal + 7 is good enough in all cases. Currently I test with a 100 iterations of random_number() for a 10x10 matrix, and that works for all reals. Perhaps I also need a series of tests with very large and very small numbers. I'll try that next.

[epagone]

I'll try that next.

Thank you for doing this, it will definitely give a much more solid answer. @kargl does not seem to be much active on GitHub but he writes excellent messages on Discourse in this domain. I wish he could advise.

[urbanjost]

Is the purpose of savetxt and loadtxt to provide for a round-trip transfer of data? There are many reasons to transfer an array to a data stream -- to inspect it, to format it for use with other applications (plotting, entry into databases and spreadsheets (usually CSV), presentation in a document (converting to HTML, LaTex, *roff, ...), archiving, transfer to another process, ....

There are libraries for self-describing data (NetCFD, HDF5, ...) serialized standards (TOML, JSON, ...) , transfer to other processes or platforms (xdr, ...) ... For simple round-trips hexadecimal format is often used (the GPF M_matrix utility uses that in "save" and "load" commands as a simple example).

xdr is an old standby for a compromise between the efficiency of binary transfers but all the problems that entails (byte size of values, endian, mantissa types, ...) and cross-platform portability.

I have never seen a formal declaration of which of these or other uses savetxt and loadtxt were targetting short and/or long-term. The answer is very different depending on the intended use. Hexadecimal values are far more reliable for a round trip but far less intelligible, something as simple as CSV is better than a plain table for importing into many applications, HTML is often importable and also useful for use with a browser, most people prefer a row-column decimal-justified column format for small tables to be viewed manually, any kind of data archiving is hopefully using a self-describing format , where a large number of digits can be inappropriate and imply often non-existent precision to the values ... and so on. NAMELIST should not be forgotten, as it is self-describing, a part of the Fortran standard, simple to use and supports user-defined types better than anything else I can think of.

[milancurcic]

Is the purpose of savetxt and loadtxt to provide for a round-trip transfer of data?

Thanks @urbanjost, good question. Let's discuss, I think it's up to us to decide. My understanding is that the main purposes are:

1. Easy one-liners for reading and writing arrays
2. Human readable
3. Bit-to-bit reproducible transfer of data? Maybe; I think so.

1 and 2 are very useful for development and prototyping, or to easily send a 2-d array to a friend or colleague and they can read it from any tool or language.

3 is important for use in the early development of simulations (toy models), especially the chaotic kind that are sensitive to initial conditions. Restarting such simulations would need exact round-trip IO. Production code will likely use some more optimal self-describing binary format like NetCDF or HDF5, but the target audience for savetxt and loadtxt is I think different then those of mature production code.

3 is to an extent at odds with 2) for real and complex numbers. To preserve the data, you need to store all the digits. The more digits the less readable.

If 3 is not important, list-directed IO is still not optimal because most compilers will still output all significant digits for that type kind.

We can consider having the user provide the format as an argument to savetxt, like in NumPy.savetxt. Then the user could specify fmt='f5.2' and write pi as 3.14. The problem here is that to load the data, the user would need to specify the format, or we'd need to rely on list-directed IO for input because we can't use "zero-width" descriptors like g0 in read statements.

If 3) is not important, then my marking this as a "bug" is not charitable, of course.

CC also @certik and @jvdp1 who were the original implementors.

Something really confuses me though. I see that there was a PR #89 that made a format fix for ifort in qsavetxt, however the current stdlib_io.fypp doesn't have qsavetxt. Does anybody know what PR removed it? I can't find it.

[Beliavsky]

@milancurcic "The problem here is that to load the data, the user would need to specify the format, or we'd need to rely on list-directed IO for input because we can't use "zero-width" descriptors like g0 in read statements."

Are there or will there be functions in stdlib to read a single real, integer, or logical variable from a string? If there is, one could first decompose a line of text into tokens and then read one of the basic types from the tokens.