Paramiko hanging fix

[briank-git]

I found that the snapshot flow would stop and hang when it tried to receive the exit status from stdout and stderr when collecting the output of zfs list -t snapshot, preventing any new snapshots from being taken.

This happens when the output gets over a certain number of lines, I believe in our case it's because rudaux is now taking four times the number of snapshots it used to because of the section overrides. At the time the hanging started, around 20000 snapshots had been taken.

The issue is described here and is also described in a warning on the documentation here, I increased the window_size and max_packet_size and moved checking exit status until after reading stdout and stderr as recommended in the comments and warning. Also added polling on exit_status_ready to make sure command finished executing before reading stdout and stderr.

rudaux on dsci-100-instructor has been running with these changes for the last three weeks and snapshots are now being taken properly again.

[briank-git]

Polling isn't working properly, looking into it.

[briank-git]

Don't need to poll as for line in stdout: loop will block until there is something to read and will continue reading until EOF is reached (i.e. when command finishes executing).