Description
Fractal preprocesses the matrices, and the prover must store codewords output by the preprocessing. Currently the library keeps these in memory.
(Recall a codeword is the evaluation of a polynomial over a known domain, so in terms of serialization it is a vector of Field elements)
We should store the prover indexes codewords on disk. Additionally, if we store the raw polynomial the codeword represents, in addition to the codeword, then the prover should be able to get away with never loading these codewords into memory. Then the prover would only require random access to the codewords.
Then when constructing the proof, the codewords entries can be read directly from disk. (Maybe even via mmap if we want lower memory filecoin-project/rust-fil-proofs#986)
This will be crucial for large deployments of Fractal, due to the memory reduction. (Additionally, these codewords can be stored 'by a helper' for the helped setting)