Code example for reading raw data?

[fasiha]

I'm working translating the prose in https://github.com/peterschwarz/clj-serial#reading-bytes to code but having any old example in the readme would be helpful.

[fasiha]

Oh, here's what I came up with: the following connects to a serial port and whenever data is available, invokes a function that grabs all the data available and appends it to an atom.

(require '[serial.core :as serial])

(def buffers-atom (atom []))
(def n 1024) ; 1 KiB

(defn exhaust-stream
  ([stream n] (exhaust-stream stream n '()))
  ([stream n buf-so-far]
   (let [new-buf (byte-array n)
         read-len (.read stream new-buf)
         buf-with-new (concat buf-so-far (take read-len new-buf))]
     (if (< read-len n)
       buf-with-new
       (recur stream n buf-with-new)))))

(def port (serial/open "/dev/ttyUSB0" :baud-rate 9600))
(serial/listen! port (fn [stream] (swap! buffers-atom concat (exhaust-stream stream n))))

The only tricky part here is exhaust-stream which .reads all the data from the InputStream object given to the callback by reading chunks of n bytes at a time until there's no more data. I'm not sure if this is the best way to do it—I just wanted something that would completely drain the InputStream each time data was available and then do something with the result.

[peterschwarz]

More like than not, you'll probably want to read until some end character or a given length specified by the beginning of the message coming from the device connected to your serial port. A good example (though a bit more complex to parse, given that it's written in cljx) is clj-firmata which uses clj-serial on the clojure side. Each read on the stream is done a byte at a time until the end of message is hit - messages are pretty short in this protocol, so it's not too bad. Read (wrapped in a protocol that provides read!) is consumed like so.

It's not too far off your solution, really, save that you're buffering the data as you read.

[fasiha]

Very interesting points, thank you!

In my application, the device on the other side of the serial port is streaming data continuously, data which my application wants to get and crunch on as soon as it can, so my code snippet was written with the idea of getting all the data that was available right now out of that stream and into my app (atom, or chan, or whatever).

In that context, would I be correct in thinking that parsing byte-by-byte is inappropriate? Because at the end of a message is just another message (my application parses all that anyway).

Also, in that context, I'm fearful of reading N bytes at a time, in case the stream is somehow filling up faster than N bytes per callback invocation.

But I do see in the docs that InputStream has an available. method which tells me approximately how much data is available to take. I think I can write a callback that takes that many bytes from the stream, instead of the buffering technique in my code snippet? Or alternatively, I could wait till some reasonable number of bytes is available, maybe 100, instead of taking them in twos and threes—this would introduce some latency but that might be made up for in faster processing times of fewer & larger chunks instead of many tiny ones.