Clean EthernetProvider error messages

[wcpettus]

Related to #104 and #54

Generic problems:

• EthernetProvider has warning/critical standard established before slack spammed on logger.warning, so some of its warning level loggers should be removed as they are redundant.
• Scheduler should probably not logger.error on a DriplineException, since those have been specifically raised in our code and we can add our own alert messages for this
  • This is a dripline-python comment, but may merit discussion
• We should probably never logger.warning/error/critical a traceback, because that's a mess to hunt through on slack

Specific problem:

• Laphroaig crashed yesterday, probably because it's a TTi power supply where we cannot connect over the appropriate port.
  • In this failure mode the only known fix is to manually power cycle the unit.
• At 02:01:08, laphroaig attempted to query the current output, with the following error sequence:
  • 02:01:13[WARNING ] dragonfly.implementations.ethernet_provider(196) -> socket.timeout condition met; received: ''
  • 02:01:13[CRITICAL] dragonfly.implementations.ethernet_provider(131) -> Unexpected socket.timeout
  • 02:01:18[WARNING ] dragonfly.implementations.ethernet_provider(90) -> connection ('laphroaig.p8', 9221) refused: timed out
  • 02:01:18[CRITICAL] dragonfly.implementations.ethernet_provider(137) -> Ethernet reconnect failed, dead socket
  • 02:01:18[ERROR ] dripline.core.scheduler(77) -> got a: Broken ethernet socket
  • 02:01:18[ERROR ] dripline.core.scheduler(78) -> traceback follows:
• Then in a "dead socket" state, every future query failed like this:
  • 02:01:19[DEBUG ] dragonfly.implementations.ethernet_provider(158) -> sending: 'V1O?'
  • 02:01:19[WARNING ] dragonfly.implementations.ethernet_provider(126) -> socket.error <[Errno 32] Broken pipe> received, attempting reconnect
  • 02:01:18[WARNING ] dragonfly.implementations.ethernet_provider(90) -> connection ('laphroaig.p8', 9221) refused: timed out
  • 02:01:18[ERROR ] dripline.core.scheduler(77) -> got a: Broken ethernet socket
  • 02:01:18[ERROR ] dripline.core.scheduler(78) -> traceback follows:

The complete stdout log is

2018-01-17T02:01:08[INFO    ] dripline.core.scheduler(73) -> beginning scheduled sequence
2018-01-17T02:01:08[DEBUG   ] dragonfly.implementations.ethernet_provider(158) -> sending: 'I1O?'
2018-01-17T02:01:13[WARNING ] dragonfly.implementations.ethernet_provider(196) -> socket.timeout condition met; received:
''
2018-01-17T02:01:13[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:13[DEBUG   ] dripline.core.service(552) -> to status_message.warning.laphroaig sending {
    "timestamp": "2018-01-17T10:01:13Z", 
    "payload": "socket.timeout condition met; received:\n''", 
    "sender_info": {
        "username": "project8", 
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly", 
        "package": "dripline", 
        "service_name": "unknown_service_2e28e31-900", 
        "hostname": "claude", 
        "version": "wp2.1.1", 
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:13[CRITICAL] dragonfly.implementations.ethernet_provider(131) -> Unexpected socket.timeout
2018-01-17T02:01:13[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:13[DEBUG   ] dripline.core.service(552) -> to status_message.critical.laphroaig sending {
    "timestamp": "2018-01-17T10:01:13Z", 
    "payload": "Unexpected socket.timeout", 
    "sender_info": {
        "username": "project8", 
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly", 
        "package": "dripline", 
        "service_name": "unknown_service_2e28e31-900", 
        "hostname": "claude", 
        "version": "wp2.1.1", 
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:18[WARNING ] dragonfly.implementations.ethernet_provider(90) -> connection ('laphroaig.p8', 9221) refused: timed out
2018-01-17T02:01:18[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:18[DEBUG   ] dripline.core.service(552) -> to status_message.warning.laphroaig sending {
    "timestamp": "2018-01-17T10:01:18Z", 
    "payload": "connection ('laphroaig.p8', 9221) refused: timed out", 
    "sender_info": {
        "username": "project8", 
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly", 
        "package": "dripline", 
        "service_name": "unknown_service_2e28e31-900", 
        "hostname": "claude", 
        "version": "wp2.1.1", 
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:18[CRITICAL] dragonfly.implementations.ethernet_provider(137) -> Ethernet reconnect failed, dead socket
2018-01-17T02:01:18[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:18[DEBUG   ] dripline.core.service(552) -> to status_message.critical.laphroaig sending {
    "timestamp": "2018-01-17T10:01:18Z",
    "payload": "Ethernet reconnect failed, dead socket",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:18[ERROR   ] dripline.core.scheduler(77) -> got a: Broken ethernet socket
2018-01-17T02:01:18[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:18[DEBUG   ] dripline.core.service(552) -> to status_message.error.laphroaig sending {
    "timestamp": "2018-01-17T10:01:18Z",
    "payload": "got a: Broken ethernet socket",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:18[ERROR   ] dripline.core.scheduler(78) -> traceback follows:
Traceback (most recent call last):
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/scheduler.py", line 75, in _process_schedule
    result = self.scheduled_action()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/spime.py", line 137, in scheduled_action
    result = self.on_get()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/endpoint.py", line 34, in wrapper
    very_raw = fun(self)
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/spime_endpoints.py", line 145, in on_get
    result = self.provider.send([self._get_str])
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py", line 138, in send
    raise exceptions.DriplineHardwareConnectionError("Broken ethernet socket")
DriplineHardwareConnectionError: Broken ethernet socket

2018-01-17T02:01:18[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:18[DEBUG   ] dripline.core.service(552) -> to status_message.error.laphroaig sending {
    "timestamp": "2018-01-17T10:01:18Z",
    "payload": "traceback follows:\nTraceback (most recent call last):\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/scheduler.py\", line 75, in _process_schedule\n    result = self.scheduled_action()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/spime.py\", line 137, in scheduled_action\n    result = self.on_get()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/endpoint.py\", line 34, in wrapper\n    very_raw = fun(self)\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/spime_endpoints.py\", line 145, in on_get\n    result = self.provider.send([self._get_str])\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py\", line 138, in send\n    raise exceptions.DriplineHardwareConnectionError(\"Broken ethernet socket\")\nDriplineHardwareConnectionError: Broken ethernet socket\n",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:18[DEBUG   ] dripline.core.scheduler(79) -> scheduled sequence complete
2018-01-17T02:01:19[INFO    ] dripline.core.scheduler(73) -> beginning scheduled sequence
2018-01-17T02:01:19[DEBUG   ] dragonfly.implementations.ethernet_provider(158) -> sending: 'V1O?'
2018-01-17T02:01:19[WARNING ] dragonfly.implementations.ethernet_provider(126) -> socket.error <[Errno 32] Broken pipe> received, attempting reconnect
2018-01-17T02:01:19[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:19[DEBUG   ] dripline.core.service(552) -> to status_message.warning.laphroaig sending {
    "timestamp": "2018-01-17T10:01:19Z",
    "payload": "socket.error <[Errno 32] Broken pipe> received, attempting reconnect",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:25[WARNING ] dragonfly.implementations.ethernet_provider(90) -> connection ('laphroaig.p8', 9221) refused: timed out
2018-01-17T02:01:25[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:25[DEBUG   ] dripline.core.service(552) -> to status_message.warning.laphroaig sending {
    "timestamp": "2018-01-17T10:01:25Z",
    "payload": "connection ('laphroaig.p8', 9221) refused: timed out",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:25[ERROR   ] dripline.core.scheduler(77) -> got a: Unable to establish ethernet socket ('laphroaig.p8', 9221)
2018-01-17T02:01:25[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:25[DEBUG   ] dripline.core.service(552) -> to status_message.error.laphroaig sending {
    "timestamp": "2018-01-17T10:01:25Z",
    "payload": "got a: Unable to establish ethernet socket ('laphroaig.p8', 9221)",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:25[ERROR   ] dripline.core.scheduler(78) -> traceback follows:
Traceback (most recent call last):
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/scheduler.py", line 75, in _process_schedule
    result = self.scheduled_action()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/spime.py", line 137, in scheduled_action
    result = self.on_get()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/endpoint.py", line 34, in wrapper
    very_raw = fun(self)
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/spime_endpoints.py", line 145, in on_get
    result = self.provider.send([self._get_str])
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py", line 127, in send
    self._reconnect()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py", line 91, in _reconnect
    raise exceptions.DriplineHardwareConnectionError("Unable to establish ethernet socket {}".format(self.socket_info))
DriplineHardwareConnectionError: Unable to establish ethernet socket ('laphroaig.p8', 9221)

2018-01-17T02:01:25[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:25[DEBUG   ] dripline.core.service(552) -> to status_message.error.laphroaig sending {
    "timestamp": "2018-01-17T10:01:25Z",
    "payload": "traceback follows:\nTraceback (most recent call last):\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/scheduler.py\", line 75, in _process_schedule\n    result = self.scheduled_action()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/spime.py\", line 137, in scheduled_action\n    result = self.on_get()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/endpoint.py\", line 34, in wrapper\n    very_raw = fun(self)\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/spime_endpoints.py\", line 145, in on_get\n    result = self.provider.send([self._get_str])\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py\", line 127, in send\n    self._reconnect()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py\", line 91, in _reconnect\n    raise exceptions.DriplineHardwareConnectionError(\"Unable to establish ethernet socket {}\".format(self.socket_info))\nDriplineHardwareConnectionError: Unable to establish ethernet socket ('laphroaig.p8', 9221)\n",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:25[DEBUG   ] dripline.core.scheduler(79) -> scheduled sequence complete
2018-01-17T02:01:49[INFO    ] dripline.core.scheduler(73) -> beginning scheduled sequence
2018-01-17T02:01:49[DEBUG   ] dragonfly.implementations.ethernet_provider(158) -> sending: 'I1O?'
2018-01-17T02:01:49[WARNING ] dragonfly.implementations.ethernet_provider(126) -> socket.error <[Errno 32] Broken pipe> received, attempting reconnect
2018-01-17T02:01:49[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:49[DEBUG   ] dripline.core.service(552) -> to status_message.warning.laphroaig sending {
    "timestamp": "2018-01-17T10:01:49Z",
    "payload": "socket.error <[Errno 32] Broken pipe> received, attempting reconnect",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:54[WARNING ] dragonfly.implementations.ethernet_provider(90) -> connection ('laphroaig.p8', 9221) refused: timed out
2018-01-17T02:01:54[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:54[DEBUG   ] dripline.core.service(552) -> to status_message.warning.laphroaig sending {
    "timestamp": "2018-01-17T10:01:54Z",
    "payload": "connection ('laphroaig.p8', 9221) refused: timed out",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:54[ERROR   ] dripline.core.scheduler(77) -> got a: Unable to establish ethernet socket ('laphroaig.p8', 9221)
2018-01-17T02:01:54[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:54[DEBUG   ] dripline.core.service(552) -> to status_message.error.laphroaig sending {
    "timestamp": "2018-01-17T10:01:54Z",
    "payload": "got a: Unable to establish ethernet socket ('laphroaig.p8', 9221)",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:54[ERROR   ] dripline.core.scheduler(78) -> traceback follows:
Traceback (most recent call last):
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/scheduler.py", line 75, in _process_schedule
    result = self.scheduled_action()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/spime.py", line 137, in scheduled_action
    result = self.on_get()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/endpoint.py", line 34, in wrapper
    very_raw = fun(self)
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/spime_endpoints.py", line 145, in on_get
    result = self.provider.send([self._get_str])
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py", line 127, in send
    self._reconnect()
  File "/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py", line 91, in _reconnect
    raise exceptions.DriplineHardwareConnectionError("Unable to establish ethernet socket {}".format(self.socket_info))
DriplineHardwareConnectionError: Unable to establish ethernet socket ('laphroaig.p8', 9221)

2018-01-17T02:01:54[INFO    ] dripline.core.service(548) -> sending an alert
2018-01-17T02:01:54[DEBUG   ] dripline.core.service(552) -> to status_message.error.laphroaig sending {
    "timestamp": "2018-01-17T10:01:54Z",
    "payload": "traceback follows:\nTraceback (most recent call last):\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/scheduler.py\", line 75, in _process_schedule\n    result = self.scheduled_action()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/spime.py\", line 137, in scheduled_action\n    result = self.on_get()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dripline/core/endpoint.py\", line 34, in wrapper\n    very_raw = fun(self)\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/spime_endpoints.py\", line 145, in on_get\n    result = self.provider.send([self._get_str])\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py\", line 127, in send\n    self._reconnect()\n  File \"/home/project8/python_environments/drag_v1.9.4_drip_wp2.1.1_python3.5.9_hard_v1.11.4_fire_v1.1.2_swarm_v1.4.9/local/lib/python2.7/site-packages/dragonfly/implementations/ethernet_provider.py\", line 91, in _reconnect\n    raise exceptions.DriplineHardwareConnectionError(\"Unable to establish ethernet socket {}\".format(self.socket_info))\nDriplineHardwareConnectionError: Unable to establish ethernet socket ('laphroaig.p8', 9221)\n",
    "sender_info": {
        "username": "project8",
        "exe": "/home/project8/python_environments/latest_version/bin/dragonfly",
        "package": "dripline",
        "service_name": "unknown_service_2e28e31-900",
        "hostname": "claude",
        "version": "wp2.1.1",
        "commit": "gd93cd40"
    }
}
2018-01-17T02:01:54[DEBUG   ] dripline.core.scheduler(79) -> scheduled sequence complete