fast-read-byte returns unexpected octet

[MakarovAlexey]

(defun test-fast-io (&optional (file #p"~/test-fast-io.bin"))
  (let* ((byte-size 8)
         (test-value 12345)
         (size (ceiling
                (integer-length test-value) byte-size)))
    (format t "test value: ~a~%" test-value)
    (with-open-file (stream file
                            :element-type 'fast-io:octet
                            :if-exists :supersede
                            :direction :io)
      ;; writing to file
      (dotimes (byte-number size)
        (let* ((position (* byte-size byte-number))
               (bytespec (byte byte-size position))
               (byte (ldb bytespec test-value)))
          (write-byte byte stream)))
      
      (file-position stream 0)

      ;; reading without fast-io
      (let ((value 0))
        (dotimes (byte-number size)
          (let* ((position (* byte-size byte-number))
                 (bytespec (byte byte-size position))
                 (byte (read-byte stream)))
            (setf (ldb bytespec value) byte)))
        (format t "value from file (without fast-io): ~a~%" value))

      (file-position stream 0)

      ;; reading with fast-io
      (let ((value 0)
            (buffer (make-input-buffer :vector (make-octet-vector size)
                                       :stream stream)))
        (dotimes (byte-number size)
          (let* ((position (* byte-size byte-number))
                 (bytespec (byte byte-size position))
                 (byte (fast-read-byte buffer)))
            (setf (ldb bytespec value) byte)))
        (format t "value from file (with fast-io): ~a~%" value)))))

result:
test value: 12345
reading from file (without fast-io): 12345
reading from file (with fast-io): 0

[CloseToZero]

You can't specify both a vector and a stream to be read from.
Change

(make-input-buffer :vector (make-octet-vector size) :stream stream)

to

(make-input-buffer :stream stream)

[samuel-hunter]

I've had this issue before, and I can verify @CloseToZero's solution works.

I assumed that the vector argument in with-fast-input was for buffering. Instead the buffer reads from the vector first, and then the stream:

(with-fast-input (buffer (make-array 4 :element-type '(unsigned-byte 8)) my-stream)
  ;; XXX: The first statement reads and exhausts the vector, and then the buffer reads from the stream.
  (print (read32-le buffer)) ;; => 0
  (print (read32-le buffer))) ;; => 32-bit number from stream...

(with-fast-input (buffer nil my-stream)
  ;; OK
  (print (read32-le buffer)) ;; => first 32-bit number from stream...
  (print (read32-le buffer))) ;; => second 32-bit number, and so on

[CloseToZero]

😅 I overlook something in the source of fast-read-byte, samuel-hunter is right, it will read from the vector first and then the stream.
But from the documentation, it seems that may be just an implementation detail and not part of the interface (If it's part of the interface, the documentation should at least talk about the priorities).

[samuel-hunter]

More than an implementation detail, it's documented in the readme:

• with-fast-input (BUFFER VECTOR &optional STREAM (OFFSET 0)) &body body
  Create an input buffer called BUFFER, optionally reading from VECTOR, followed by reading from STREAM. If OFFSET is specified, start reading from this position in VECTOR.

Same behavior for make-input-buffer since the macro is a direct mapping to this, though not documented. #28 should help clarify this in the future.