Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adult notebook #120

Open
wants to merge 24 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
6ed3e04
add adult notebook
Hilly12 Aug 26, 2021
c657c9d
random forest
Hilly12 Aug 26, 2021
531d4f8
update adult
bogdansurdu Sep 3, 2021
7a86d21
fix adult outputs
bogdansurdu Sep 3, 2021
3729c9d
fix adult tests with correct and all sensitive column names
bogdansurdu Sep 3, 2021
b3b2c10
fix adult detection tests
bogdansurdu Sep 3, 2021
3ea3d56
update version to use right tests
bogdansurdu Sep 3, 2021
b79d411
add heatmap example in Adult notebook
bogdansurdu Sep 6, 2021
d6cd3fd
Merge branch 'main' into adult-notebook
simonhkswan Sep 8, 2021
1ae98af
Update ci.yml
simonhkswan Sep 8, 2021
98182c2
change column names in tests to reflect new Adult dataset
bogdansurdu Sep 8, 2021
7aeb5a7
Merge branch 'adult-notebook' of https://github.com/synthesized-io/fa…
bogdansurdu Sep 8, 2021
dd83de2
change column names in tests/test_plot.py
bogdansurdu Sep 8, 2021
1108377
Merge branch 'main' into adult-notebook
Hilly12 Sep 13, 2021
d661fab
add adult notebook to docs
Hilly12 Sep 13, 2021
b2c3138
Merge branch 'adult-notebook' of https://github.com/synthesized-io/fa…
bogdansurdu Sep 16, 2021
594d656
remove _version.py and update .gitignore
bogdansurdu Sep 16, 2021
f6d5ef0
Merge branch 'main' into adult-notebook
bogdansurdu Sep 16, 2021
153b7fc
Merge branch 'main' into adult-notebook
Hilly12 Sep 17, 2021
d12661a
Merge remote-tracking branch 'origin/main' into adult-notebook
simonhkswan Apr 15, 2022
6f6d6fd
Remove references to source data files
simonhkswan Apr 15, 2022
83ca775
Merge branch 'main' into adult-notebook
simonhkswan Apr 15, 2022
edea9f0
Amend df URLs
simonhkswan Apr 15, 2022
338d281
Update broken df URLs
simonhkswan Apr 15, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,6 @@ jobs:
name: Check Python ${{ matrix.python-version }}
steps:
- uses: actions/[email protected]
if: ${{ github.event_name == 'pull_request' }}
with:
ref: ${{ github.head_ref }}

- uses: actions/[email protected]
if: ${{ github.event_name == 'push' }}

- name: Setup Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ build/
dist/
*.swp
src/fairlens/version.py
src/_version.py

docs/_build/*
docs/savefig/*
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ import pandas as pd
import fairlens as fl

# Load in the data
df = pd.read_csv("datasets/compas.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")

# Automatically generate a report
fscorer = fl.FairnessScorer(
Expand Down
48,843 changes: 0 additions & 48,843 deletions datasets/adult.csv

This file was deleted.

20,282 changes: 0 additions & 20,282 deletions datasets/compas.csv

This file was deleted.

1,001 changes: 0 additions & 1,001 deletions datasets/german_credit_data.csv

This file was deleted.

892 changes: 0 additions & 892 deletions datasets/titanic.csv

This file was deleted.

3 changes: 3 additions & 0 deletions docs/user_guide/adult.nblink
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"path": "../../notebooks/adult.ipynb"
}
2 changes: 1 addition & 1 deletion docs/user_guide/correlations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ the data and check what columns in contains.

.. ipython:: python

df = pd.read_csv("../datasets/german_credit_data.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/templates/german_credit_data.csv")
df

We can generate a correlation heatmap to get a rough idea of any potentially hidden correlations.
Expand Down
1 change: 1 addition & 0 deletions docs/user_guide/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ User Guide
:glob:
:caption: Tutorials

adult
compas

.. toctree::
Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/metrics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Let's import this method and load in the compas dataset.
import pandas as pd
import fairlens as fl

df = pd.read_csv("../datasets/compas.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")
df.info()


Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Below we show an assessment of fairness on a subset of Propublica's COMPAS datas
import pandas as pd
import fairlens as fl

df = pd.read_csv("../datasets/compas.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")
df.info()

fscorer = fl.FairnessScorer(df, "RawScore")
Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/scorer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ and identifying any inherent biases present in a dataset.
import pandas as pd
import fairlens as fl

df = pd.read_csv("../datasets/compas.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")
df.info()

fscorer = fl.FairnessScorer(df, "RawScore", ["Ethnicity", "Sex"])
Expand Down
4 changes: 2 additions & 2 deletions docs/user_guide/sensitive.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ dataset:

.. ipython:: python

df = pd.read_csv("../datasets/compas.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")
df.head()

# Apply shallow detection algorithm.
Expand All @@ -74,7 +74,7 @@ the sensitive columns to have random names.

.. ipython:: python

df_deep = pd.read_csv("../datasets/compas.csv")
df_deep = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")
df_deep = df_deep.rename(columns={"Ethnicity": "A", "Language": "Random", "MaritalStatus": "B", "Sex": "C"})

# Apply deep detection algorithm.
Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/significance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ we can do the following.
import pandas as pd
import fairlens as fl

df = pd.read_csv("../datasets/compas.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")

group1 = df[df["Sex"] == "Male"]["RawScore"]
group2 = df[df["Sex"] == "Female"]["RawScore"]
Expand Down
2 changes: 1 addition & 1 deletion docs/user_guide/viz.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ First we will import the required packages and load the compas dataset.

fl.plot.use_style()

df = pd.read_csv("../datasets/compas.csv")
df = pd.read_csv("https://raw.githubusercontent.com/synthesized-io/datasets/master/tabular/biased/compas.csv")
df.info()


Expand Down
Loading