From 462fdec217977ab788dc08723dd14fa4719ad622 Mon Sep 17 00:00:00 2001 From: tom-votes Date: Mon, 3 Dec 2018 05:54:40 +0000 Subject: [PATCH 1/3] Website: move static form HTML out into its own file --- code/index.html | 363 ++++++++++++++++++++++++++++++++++++++++++++++ code/website.py | 377 ++---------------------------------------------- 2 files changed, 374 insertions(+), 366 deletions(-) create mode 100644 code/index.html diff --git a/code/index.html b/code/index.html new file mode 100644 index 0000000..8b4e92a --- /dev/null +++ b/code/index.html @@ -0,0 +1,363 @@ + +

Bayesian Ballot Comparison Tool

+

This module provides support for auditing of a single plurality contest + over multiple jurisdictions using a Bayesian ballot-level + comparison audit.

+ +

This module provides routines for computing the winning probabilities + for various choices, given audit sample data. Thus, this program + provides a risk-measuring functionality.

+ +

More precisely, the code builds a Bayesian model of the unsampled + ballots, given the sampled ballots. This model is probabilistic, + since there is uncertainty about what the unsampled ballots are. + However, the model is generative: we generate many possible + sets of likely unsampled ballots, and report the probability that + various choices win the contest. (See References below for + more details.)

+ +

The main output of the program is a report on the probability + that each choice wins, in these simulations.

+ +

The contest may be single-jurisdiction or multi-jurisdiction. + More precisely, we assume that there are a number of "collections" + of ballots that may be sampled. Each relevant jurisdiction may + have one or more such collections. For example, a jurisdiction + may be a county, with one collection for ballots submitted by + mail, and one collection for ballots cast in-person.

+ +

This module may be used for ballot-polling audits (where there + no reported choices for ballots) or "hybrid" audits (where some + collections have reported choices for ballots, and some have not).

+ +

References and Code

+

Descriptions of Bayesian auditing methods can be found in:

+ + +

Implementation Note

+

The code for this tool is available on github at + www.github.com/ron-rivest/2018-bctool. + This web form provides exactly the same functionality as the stand-alone + Python tool + www.github.com/ron-rivest/2018-bctool/BCTool.py. + The Python tool + requires an environment set up with Python 3 and Numpy. + This web form was implemented using + + CherryPy + . +

(See www.github.com/ron-rivest/2018-bptool + for similar code for a Bayesian ballot-level polling audit. The code here was based + in part on that code.)

+

+ + +
+ +

Step 1: Upload Collections File

+ +

In the box below, upload a CSV file with the data about collections + in the contest being audited.

+ +

+ COLLECTIONS.CSV format:

+ +

Example collections.csv file:

+ + + + + + + + + + + + + + + + + + + + + + +
CollectionVotesComment
Bronx11000
Queens120000
Mail-In56000
+ +

with one header line, as shown, then one data line for + each collection of paper ballots that may be sampled. + Each data line gives the name of the collection and + then the number of cast votes in that collection + (that is, the number of cast paper ballots in the collection). + An additional column for optional comments is provided.

+ + + Collections File: + +

Step 2: Upload Reported Votes File

+ +

In the box below, upload a CSV file with the data about the reported votes + in the contest being audited.

+ +

+ REPORTED.CSV format:

+ +

Example reported.csv file:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CollectionReportedVotesComment
BronxYes8500
BronxNo2500
QueensYes100000
QueensNo20000
Mail-InYes5990
Mail-InNo50000
Mail-InOvervote10
+ +

with one header line, as shown, then one data line for each + collection and each reported choice in that collection, + giving the number of times such reported choice is reported + to have appeared. An additional column for optional comments is + provided.

+ +

A reported choice need not be listed, if it occurred zero times, + although every possible choice (except write-ins) should be listed + at least once for some contest, so that this program knows what the + possible votes are for this contest.

+ +

For each collection, the sum, over reported choices, of the Votes given + should equal the Votes value given in the corresponding line in + the collections.csv file.

+ +

Write-ins may be specified as desired, perhaps with a string + such as "Write-in:Alice Jones".

+ +

For ballot-polling audits, use a reported choice of "-MISSING" + or "-noCVR" or any identifier starting with a "-". (Tagging + the identifier with an initial "-" prevents it from becoming + elegible for winning the contest.)

+ + Reported Votes File: + +

Step 3: Upload Sampled Votes File

+ +

In the box below, upload a CSV file with the data about the samples + from the contest being audited.

+ +

+ SAMPLE.CSV format:

+ +

Example sample.csv file:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
CollectionReportedActualVotesComment
BronxYesYes62
BronxNoNo21
BronxNoOvervote1
QueensYesYes504
QueensNoNo99
Mail-InYesYes17
Mail-InNoNo73
Mail-InNoLizard People2
Mail-InNoLizard People1
+ +

with one header line, as shown then at least one data line for + each collection for each reported/actual choice pair that was seen + in the audit sample at least once, giving the number of times it + was seen in the sample. An additional column for comments is + provided.

+ +

There is no need to give a data line for a reported/actual pair + that wasn't seen in the sample.

+ +

If you give more than one data line for a given collection, + reported choice, and actual choice combination, then the votes + are summed. (So the mail-in collection has three ballots the scanner said + showed "No", but that the auditors said actually showed "Lizard + People".) For example, you may give one line per audited ballot, + if you wish.

+ +

The lines of this file do not need to be in any particular order. + You can just add more lines to the end of this file as the audit + progresses, if you like.

+ + Sample Votes File: + +

(Optional) Specify random number seed

+

+ The computation uses a random number seed, which defaults to 1. + You may if you wish enter a different seed here. + (Using the same seed with the same data always returns the same results.) + This is an optional parameter; there should be no reason to change it. +

+ + Seed: + +

(Optional) Specify number of trials

+

Bayesian audits work by simulating the data which hasn't been sampled to + estimate the chance that each candidate would win a full hand recount. + You may specify in the box below the number of + trials used to compute these estimates. + This is an optional parameter, defaulting to 10000. Making it smaller + will decrease accuracy and improve running time; making it larger will + improve accuracy and increase running time. +

+ + Number of trials: + +

Compute results

+ Click on the "Submit" button below to compute the desired answers, + which will be shown on a separate page. + + + Note: The Bayesian prior is represented by a pseudocount of one vote for + each choice. This may become an optional input parameter later. +
+ diff --git a/code/website.py b/code/website.py index c34e50b..2f337e7 100644 --- a/code/website.py +++ b/code/website.py @@ -11,378 +11,20 @@ import bctool +CODE_DIR = os.path.dirname(os.path.realpath(__file__)) -class BCToolPage: +# Serving static files with CherryPy is a bit kludgy no matter how you do it, +# so let's just do it directly: +with open(os.path.join(CODE_DIR, 'index.html'), 'r') as content_file: + index_string = content_file.read() +class BCToolPage: @cherrypy.expose def index(self): # Ask for the parameters required for the Bayesian Audit. # Style parameters from # https://www.w3schools.com/css/tryit.asp?filename=trycss_forms - return ''' - -

Bayesian Ballot Comparison Tool

-

This module provides support for auditing of a single plurality contest - over multiple jurisdictions using a Bayesian ballot-level - comparison audit.

- -

This module provides routines for computing the winning probabilities - for various choices, given audit sample data. Thus, this program - provides a risk-measuring functionality.

- -

More precisely, the code builds a Bayesian model of the unsampled - ballots, given the sampled ballots. This model is probabilistic, - since there is uncertainty about what the unsampled ballots are. - However, the model is generative: we generate many possible - sets of likely unsampled ballots, and report the probability that - various choices win the contest. (See References below for - more details.)

- -

The main output of the program is a report on the probability - that each choice wins, in these simulations.

- -

The contest may be single-jurisdiction or multi-jurisdiction. - More precisely, we assume that there are a number of "collections" - of ballots that may be sampled. Each relevant jurisdiction may - have one or more such collections. For example, a jurisdiction - may be a county, with one collection for ballots submitted by - mail, and one collection for ballots cast in-person.

- -

This module may be used for ballot-polling audits (where there - no reported choices for ballots) or "hybrid" audits (where some - collections have reported choices for ballots, and some have not).

- -

References and Code

-

Descriptions of Bayesian auditing methods can be found in:

- - -

Implementation Note

-

The code for this tool is available on github at - www.github.com/ron-rivest/2018-bctool. - This web form provides exactly the same functionality as the stand-alone - Python tool - www.github.com/ron-rivest/2018-bctool/BCTool.py. - The Python tool - requires an environment set up with Python 3 and Numpy. - This web form was implemented using - - CherryPy - . -

(See www.github.com/ron-rivest/2018-bptool - for similar code for a Bayesian ballot-level polling audit. The code here was based - in part on that code.)

-

- - -
- -

Step 1: Upload Collections File

- -

In the box below, upload a CSV file with the data about collections - in the contest being audited.

- -

- COLLECTIONS.CSV format:

- -

Example collections.csv file:

- - - - - - - - - - - - - - - - - - - - - - -
CollectionVotesComment
Bronx11000
Queens120000
Mail-In56000
- -

with one header line, as shown, then one data line for - each collection of paper ballots that may be sampled. - Each data line gives the name of the collection and - then the number of cast votes in that collection - (that is, the number of cast paper ballots in the collection). - An additional column for optional comments is provided.

- - - Collections File: - -

Step 2: Upload Reported Votes File

- -

In the box below, upload a CSV file with the data about the reported votes - in the contest being audited.

- -

- REPORTED.CSV format:

- -

Example reported.csv file:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
CollectionReportedVotesComment
BronxYes8500
BronxNo2500
QueensYes100000
QueensNo20000
Mail-InYes5990
Mail-InNo50000
Mail-InOvervote10
- -

with one header line, as shown, then one data line for each - collection and each reported choice in that collection, - giving the number of times such reported choice is reported - to have appeared. An additional column for optional comments is - provided.

- -

A reported choice need not be listed, if it occurred zero times, - although every possible choice (except write-ins) should be listed - at least once for some contest, so that this program knows what the - possible votes are for this contest.

- -

For each collection, the sum, over reported choices, of the Votes given - should equal the Votes value given in the corresponding line in - the collections.csv file.

- -

Write-ins may be specified as desired, perhaps with a string - such as "Write-in:Alice Jones".

- -

For ballot-polling audits, use a reported choice of "-MISSING" - or "-noCVR" or any identifier starting with a "-". (Tagging - the identifier with an initial "-" prevents it from becoming - elegible for winning the contest.)

- - Reported Votes File: - -

Step 3: Upload Sampled Votes File

- -

In the box below, upload a CSV file with the data about the samples - from the contest being audited.

- -

- SAMPLE.CSV format:

- -

Example sample.csv file:

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
CollectionReportedActualVotesComment
BronxYesYes62
BronxNoNo21
BronxNoOvervote1
QueensYesYes504
QueensNoNo99
Mail-InYesYes17
Mail-InNoNo73
Mail-InNoLizard People2
Mail-InNoLizard People1
- -

with one header line, as shown then at least one data line for - each collection for each reported/actual choice pair that was seen - in the audit sample at least once, giving the number of times it - was seen in the sample. An additional column for comments is - provided.

- -

There is no need to give a data line for a reported/actual pair - that wasn't seen in the sample.

- -

If you give more than one data line for a given collection, - reported choice, and actual choice combination, then the votes - are summed. (So the mail-in collection has three ballots the scanner said - showed "No", but that the auditors said actually showed "Lizard - People".) For example, you may give one line per audited ballot, - if you wish.

- -

The lines of this file do not need to be in any particular order. - You can just add more lines to the end of this file as the audit - progresses, if you like.

- - Sample Votes File: - -

(Optional) Specify random number seed

-

- The computation uses a random number seed, which defaults to 1. - You may if you wish enter a different seed here. - (Using the same seed with the same data always returns the same results.) - This is an optional parameter; there should be no reason to change it. -

- - Seed: - -

(Optional) Specify number of trials

-

Bayesian audits work by simulating the data which hasn't been sampled to - estimate the chance that each candidate would win a full hand recount. - You may specify in the box below the number of - trials used to compute these estimates. - This is an optional parameter, defaulting to 10000. Making it smaller - will decrease accuracy and improve running time; making it larger will - improve accuracy and increase running time. -

- - Number of trials: - -

Compute results

- Click on the "Submit" button below to compute the desired answers, - which will be shown on a separate page. - - - Note: The Bayesian prior is represented by a pseudocount of one vote for - each choice. This may become an optional input parameter later. -
- ''' + return index_string @cherrypy.expose def ComparisonAudit( @@ -531,9 +173,12 @@ def get_html_results(self, actual_choices, win_probs, n_winners): -server_conf = os.path.join(os.path.dirname(__file__), 'server_conf.conf') +server_conf = os.path.join(CODE_DIR, 'server_conf.conf') if __name__ == '__main__': + # cherrypy.tree.mount(BCToolPage(), config=server_conf) + # cherrypy.engine.start() + # cherrypy.engine.block() # cherrypy.config.update({'tools.sessions.on': True, # 'tools.sessions.storage_type': "File", From 2044b074c312c926b5080a2c81d9e6a343ecdbbd Mon Sep 17 00:00:00 2001 From: tom-votes Date: Mon, 3 Dec 2018 06:05:41 +0000 Subject: [PATCH 2/3] Standards-compliant index.html --- code/index.html | 135 ++++++++++++++++++++++++++---------------------- 1 file changed, 72 insertions(+), 63 deletions(-) diff --git a/code/index.html b/code/index.html index 8b4e92a..ccd82cc 100644 --- a/code/index.html +++ b/code/index.html @@ -1,3 +1,7 @@ + + + + + Bayesian Ballot Comparison Tool + +

Bayesian Ballot Comparison Tool

This module provides support for auditing of a single plurality contest over multiple jurisdictions using a Bayesian ballot-level @@ -64,9 +71,9 @@

References and Code

  • Bayesian Tabulation Audits: Explained and Extended - by Ronald L. Rivest + by Ronald L. Rivest 2018 -
    • +
  • Bayesian Election Audits in One Page @@ -77,20 +84,20 @@

    References and Code

    Implementation Note

    -

    The code for this tool is available on github at +

    The code for this tool is available on github at www.github.com/ron-rivest/2018-bctool. - This web form provides exactly the same functionality as the stand-alone + This web form provides exactly the same functionality as the stand-alone Python tool - www.github.com/ron-rivest/2018-bctool/BCTool.py. + www.github.com/ron-rivest/2018-bctool/BCTool.py. The Python tool requires an environment set up with Python 3 and Numpy. - This web form was implemented using + This web form was implemented using CherryPy .

    (See www.github.com/ron-rivest/2018-bptool for similar code for a Bayesian ballot-level polling audit. The code here was based - in part on that code.)

    + in part on that code.)

    @@ -109,22 +116,22 @@

    - + - + - + - +
    CollectionVotesVotes Comment
    Bronx1100011000
    Queens120000120000
    Mail-In5600056000
    @@ -136,9 +143,9 @@

    (that is, the number of cast paper ballots in the collection). An additional column for optional comments is provided.

    - + Collections File: - +

    Step 2: Upload Reported Votes File

    In the box below, upload a CSV file with the data about the reported votes @@ -152,50 +159,50 @@

    - - + + - - + + - - + + - - + + - - + + - - + + - - + + - - + +
    CollectionReportedVotesReportedVotes Comment
    BronxYes8500Yes8500
    BronxNo2500No2500
    QueensYes100000Yes100000
    QueensNo20000No20000
    Mail-InYes5990Yes5990
    Mail-InNo50000No50000
    Mail-InOvervote10Overvote10
    @@ -222,7 +229,7 @@

    or "-noCVR" or any identifier starting with a "-". (Tagging the identifier with an initial "-" prevents it from becoming elegible for winning the contest.)

    - + Reported Votes File:

    Step 3: Upload Sampled Votes File

    @@ -238,72 +245,72 @@

    - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + +
    CollectionReportedActualVotesReportedActualVotes Comment
    BronxYesYes62YesYes62
    BronxNoNo21NoNo21
    BronxNoOvervote1NoOvervote1
    QueensYesYes504YesYes504
    QueensNoNo99NoNo99
    Mail-InYesYes17YesYes17
    Mail-InNoNo73NoNo73
    Mail-InNoLizard People2NoLizard People2
    Mail-InNoLizard People1NoLizard People1
    @@ -343,13 +350,13 @@

    (Optional) Specify random number seed

    (Optional) Specify number of trials

    Bayesian audits work by simulating the data which hasn't been sampled to estimate the chance that each candidate would win a full hand recount. - You may specify in the box below the number of + You may specify in the box below the number of trials used to compute these estimates. This is an optional parameter, defaulting to 10000. Making it smaller will decrease accuracy and improve running time; making it larger will - improve accuracy and increase running time. + improve accuracy and increase running time.

    - + Number of trials:

    Compute results

    @@ -360,4 +367,6 @@

    Compute results

    Note: The Bayesian prior is represented by a pseudocount of one vote for each choice. This may become an optional input parameter later. + + From 0ba36a0bab588073959e6db2193be99dc8266b12 Mon Sep 17 00:00:00 2001 From: tom-votes Date: Mon, 3 Dec 2018 06:37:16 +0000 Subject: [PATCH 3/3] Add note about working with HTML and 'autoreloader' --- code/website.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/code/website.py b/code/website.py index 2f337e7..2226cf0 100644 --- a/code/website.py +++ b/code/website.py @@ -15,6 +15,8 @@ # Serving static files with CherryPy is a bit kludgy no matter how you do it, # so let's just do it directly: +# (If you'd like to be able to change the HTML with 'autoreloader', move this +# block into the definition of 'index'): with open(os.path.join(CODE_DIR, 'index.html'), 'r') as content_file: index_string = content_file.read()