Private Address Inconsistent Results #956
Comments
|
What are the different ip ranges to be tested? It looked like all the IP
notifications were from 10.99.246.73, or 10.99.246.72
…On Thu, May 12, 2022 at 9:29 AM jhughesbiot ***@***.***> wrote:
I can't pinpoint to a specific release or commit but I have noticed some
inconsistency with the private address test in the ipaddr module.
What I have found is that I can run DAQ and get an output that will return
a failure for private address test and then I can run it again a few more
times and maybe on the 2nd, 3rd, 4th, etc. I get a pass, which is weird. If
I look even closer at these individual results, I will see that maybe in
the first run it failed on the second IP range but then on the 2nd it
passes that range but fails on the 3rd range. This tells me the device is
completely capable of accepting these ranges but there is some issue with
how the test is either providing them or an issue with timing on how it is
validating. Additionally, I have found in the logs what appears to be a
passing result in the individual tests but still fails in the final result.
In this output, you see the IP change to 10.99.246.73 and then fail a ping
but then after the timeout, a ping again and it passes. The device fails
the private address but is receiving all the addresses.
Apr 28 21:22:13 ipaddr.ipaddr01 INFO Device f833312cd6a4 running dhcp
change test
Apr 28 21:22:13 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp
timeout at 2022-04-28 21:30:33.717266
Apr 28 21:22:21 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification
10.99.246.73 (done)
Apr 28 21:22:32 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result False
Apr 28 21:22:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 running analyze
results
Apr 28 21:22:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp
timeout at 2022-04-28 21:30:52.531365
Apr 28 21:22:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification
10.99.246.73 (done)
Apr 28 21:22:35 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True
On a subsequent test, with no change to any configurations on the DAQ
system or the device, this passes.
May 11 23:17:03 ipaddr.ipaddr01 INFO Device f833312cd6a4 running ip change
test
May 11 23:17:03 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp
timeout at 2022-05-11 23:25:23.189214
May 11 23:21:18 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification
10.99.246.72 (long)
May 11 23:21:19 ipaddr.ipaddr01 INFO Ping to 10.99.246.72, result True
May 11 23:21:19 ipaddr.ipaddr01 INFO Device f833312cd6a4 running dhcp
change test
May 11 23:21:19 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp
timeout at 2022-05-11 23:29:39.448075
May 11 23:21:32 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification
10.99.246.73 (done)
May 11 23:21:34 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True
May 11 23:21:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 running analyze
results
May 11 23:21:34 ipaddr.ipaddr01 INFO Device f833312cd6a4 setting dhcp
timeout at 2022-05-11 23:29:54.526310
May 11 23:21:38 ipaddr.ipaddr01 INFO Device f833312cd6a4 ip notification
10.99.246.73 (done)
May 11 23:21:49 ipaddr.ipaddr01 INFO Ping to 10.99.246.73, result True
I am currently having to run tests against the ipaddr module multiple
times and inspect the logs to get the result due to this inconsistency
since the test result isn't necessarily reliable.
—
Reply to this email directly, view it on GitHub
<#956>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABQ5NYKNUZKXYZTCNRL7DMDVJUWW3ANCNFSM5VY2FUVQ>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
The ranges used are all the default ranges defined. https://github.com/faucetsdn/daq/blob/master/resources/setups/common/base_config.json#L7 This was just an example from the most recent device being qualified to show the inconsistencies over the same range. Other devices and other runs of DAQ on this device show failures and success over different ranges so there's not just one range where it happens, it's the whole test that doesn't act consistently. |
|
I think I found the cause of this issue: daq/subset/ipaddr/dhcp_tests.py Line 66 in 8d279c4
and the private address test is now passing for me. If this looks good, I can make a pull request for the change. On a related note, the same error is present in the function _get_dhcp_type() in the same file. However, this function is never used. From what I can tell in the Git history, _get_dhcp_type() has never been used. Should _get_dhcp_type() be removed? If so, would be okay to include it in the same pull request as the other change? |
I can't pinpoint to a specific release or commit but I have noticed some inconsistency with the private address test in the ipaddr module.
What I have found is that I can run DAQ and get an output that will return a failure for private address test and then I can run it again a few more times and maybe on the 2nd, 3rd, 4th, etc. I get a pass, which is weird. If I look even closer at these individual results, I will see that maybe in the first run it failed on the second IP range but then on the 2nd it passes that range but fails on the 3rd range. This tells me the device is completely capable of accepting these ranges but there is some issue with how the test is either providing them or an issue with timing on how it is validating. Additionally, I have found in the logs what appears to be a passing result in the individual tests but still fails in the final result.
In this output, you see the IP change to 10.99.246.73 and then fail a ping but then after the timeout, a ping again and it passes. The device fails the private address but is receiving all the addresses.
On a subsequent test, with no change to any configurations on the DAQ system or the device, this passes.
I am currently having to run tests against the ipaddr module multiple times and inspect the logs to get the result due to this inconsistency since the test result isn't necessarily reliable.
The text was updated successfully, but these errors were encountered: