Skip to content

Latest commit

 

History

History
201 lines (145 loc) · 9.05 KB

README.md

File metadata and controls

201 lines (145 loc) · 9.05 KB
Raw

Quantipy3

Python for people data

Quantipy is an open-source data processing, analysis and reporting software project that builds on the excellent pandas and numpy libraries. Aimed at people data, Quantipy offers support for native handling of special data types like multiple choice variables, statistical analysis using case or observation weights, DataFrame metadata and pretty data exports.

Quantipy for Python 3

This repository is a port of Quantipy from Python 2.x to Python 3.

Key features

  • Reads plain .csv, converts from Dimensions, SPSS, Decipher, or Ascribe
  • Open metadata format to describe and manage datasets
  • Powerful, metadata-driven cleaning, editing, recoding and transformation of datasets
  • Computation and assessment of data weights
  • Easy-to-use analysis interface

Features not yet supported in Python 3 version

  • Structured analysis and reporting via Chain and Cluster containers
  • Exports to SPSS, Dimensions ddf/mdd, MS Excel and Powerpoint with flexible layouts and various options
  • Python 3.8 is not yet fully supported, but 3.5, 3.6, and 3.7 are.

Origins

Contributors

Installation

pip install quantipy3

or

python3 -m pip install quantipy3

Note that the package is called quantipy3 on pip.

Create a virtual envirionment

If you want to create a virtual environment when using Quantipy:

conda

conda create -n envqp python=3

with venv

python -m venv [your_env_name]

5-minutes to Quantipy

Get started

If you are working with SPSS, import your sav file.

import quantipy as qp
dataset = qp.DataSet("My dataset, wave 1")
dataset.read_spss('my_file.sav')

You can start straight away by exploring what variables are in your file.

dataset.variables()

['gender',
 'agecat',
 'price_satisfaction',
 'numitems_satisfaction',
 'org_satisfaction',
 'service_satisfaction',
 'quality_satisfaction',
 'overall_satisfaction',
 'weight']

If you want more details on a variable, explore it's meta data.

dataset.meta('agecat')
single codes texts missing
agecat: Age category
1 1 18-24 None
2 2 25-34 None
3 3 35-49 None
4 4 50-64 None
5 5 64+ None

Quantipy knows out-of-the-box what SPSS's meta data means and uses it correctly. All codes and labels are the same as in the sav file.

Calculate some results, counts or percentages

dataset.crosstab('price_satisfaction', 'gender')
Question agecat. Age category
Values All 18-24 25-34 35-49 50-64 64+
Question Values
price_satisfaction. Price satisfaction All 582.0 46.0 127.0 230.0 147.0 32.0
Strongly Negative 72.0 8.0 20.0 22.0 17.0 5.0
Somewhat Negative 135.0 10.0 30.0 52.0 38.0 5.0
Neutral 140.0 9.0 32.0 59.0 36.0 4.0
Somewhat Positive 145.0 12.0 25.0 63.0 33.0 12.0
Strongly Positive 90.0 7.0 20.0 34.0 23.0 6.0

You can also filter

dataset.crosstab('price_satisfaction', 'agecat', f={'gender':1})

and use a weight column

dataset.crosstab('price_satisfaction', 'agecat', f={'gender':1}, w="weight")

Variables can be created, recoded or edited with DataSet methods, e.g. derive():

mapper = [(1,  '18-35 year old', {'agecat': [1,2]}),
          (2, '36 and older', {'agecat': [3,4,5]})]

dataset.derive('two_age_groups', 'single', dataset.text("Older or younger than 35"), mapper)
dataset.meta('two_age_groups')
single                                              codes     texts              missing
two_age_groups: "Older or youngar than 35"
1                                                       1     18-35 years old    None
2                                                       2     36 and older       None

The DataSet case data component can be inspected with the []-indexer, as known from a pd.DataFrame:

dataset[['gender', 'age']].head(5)
        gender  age
0       1.0    1.0
1       2.0    1.0
2       2.0    2.0
3       1.0    NaN
4       NaN    1.0

Weighting

If your data hasn't been weighted yet, you can use Quantipy's RIM weighting algorithm.

Assuming we have the same variables as before, gender and agecat we can weight the dataset with these two variables:

from quantipy.core.weights.rim import Rim

age_targets = {'agecat':{1:5.0, 2:30.0, 3:26.0, 4:19.0, 5:20.0}}
gender_targets = {'gender':{0:49, 1:51}}
scheme = Rim('gender_and_age')
scheme.set_targets(targets=[age_targets, gender_targets])
dataset.weight(scheme,unique_key='respondentId',
               weight_name="my_weight",
               inplace=True)

Quantipy will show you a weighting report:

Weight variable       weights_gender_and_age
Weight group                  _default_name_
Weight filter                           None
Total: unweighted                 582.000000
Total: weighted                   582.000000
Weighting efficiency               60.009826
Iterations required                14.000000
Mean weight factor                  1.000000
Minimum weight factor               0.465818
Maximum weight factor               6.187700
Weight factor ratio                13.283522

And you can test whether the weighting has worked by running crosstabs:

dataset.crosstab('agecat', ci=['c%'], w='my_new_weight')
Question agecat. Age category
Question Values
agecat. Age category All 100.0
18-24 5.0
25-34 30.0
35-49 26.0
50-64 19.0
64+ 20.0 
dataset.crosstab('gender', ci=['c%'], w='my_new_weight')
Question gender. Gender
Question Values
gender. Gender All 100.0
Male 49.0
Female 51.0

Contributing

The test suite for Quantipy can be run with the command

python3 -m pytest tests

But when developing a specific aspect of Quantipy, it might be quicker to run (e.g. for the DataSet)

python3 -m unittest tests.test_dataset

Tests for unsupported features are skipped, see here for what tests are supported.

We welcome volunteers and supporters. Please include a test case with any pull request, especially those that run calculations.