Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add causal product incrementality analysis with multivariate interrupted time series models #803

Open
wants to merge 259 commits into
base: main
Choose a base branch
from

Conversation

drbenvincent
Copy link
Contributor

@drbenvincent drbenvincent commented Jul 3, 2024

This PR will add new functionality to pymc-marketing. Namely it will allow causal analysis of product incrementality/cannibalization.

In this initial PR we will focus on:
a) adding docs to introduce the idea behind incrementality/cannibalization analysis
b) in terms of modeling we will start with a multivariate interrupted time series model. This will include a notebook to illustrate its use, as well as docs in the guide section to convey the basic workings.

Later PR's will add additional modeling options, such as nested discrete choice models.

TODO

  • Finish "Introduction to product incrementality" docs page
  • Finish MV-ITS model explanation
  • Add tests
  • Finish how-to notebook demonstrating use
    • Add module code
    • Add Scenario 3 where we operate on product level data
    • Add scenario where market is not saturated + update relevant docs + tests
  • Add a QuickStart on the Getting Started docs page
    • add python code block for QuickStart
  • Rename module to consumer-choice. Update docs and imports and re-run notebooks etc.
  • Fix typo "isoluate" -> "isolate"
  • Update docs homepage and README.md

Known limitations

At the moment the MVITS model is very simple. It only has an intercept term, embodying the assumption that sales are relatively static over time. Clearly there are many ways to expand this model, so the proposal is that we get this in as a first attempt and stepwise add features over time. We can be lead by feature requests (if there are any), but this establishes an MVP which can be built upon.

Checklist

Modules affected

This PR will create a new module and will not affect the MMM or CLV modules.

Type of change

  • New feature / enhancement
  • Bug fix
  • Documentation
  • Maintenance
  • Other (please specify):

📚 Documentation preview 📚: https://pymc-marketing--803.org.readthedocs.build/en/803/

juanitorduz and others added 30 commits April 11, 2024 13:11
remove ruff E501 ignore
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.3.5 → v0.3.7](astral-sh/ruff-pre-commit@v0.3.5...v0.3.7)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Juan Orduz <[email protected]>
* fix potential bugs

* minor improvements

* remove rule for nb

* fix test

* improve tests syntax

* use stacklevel 2 for warnings

* use stacklevel 1 for warnings as they are used in public methods

* ignore B904

* undo change

* ricardos feedback

* use fit_posterior

* Update pymc_marketing/clv/models/gamma_gamma.py

Co-authored-by: Ricardo Vieira <[email protected]>

* last fix XD

---------

Co-authored-by: Ricardo Vieira <[email protected]>
* notebook opening and imports

* model definition markdown

* Data Load Notebook section

* WIP model fitting section

* added notebook to docs directory

* notebook edits and graph code

* ppc section and nb cleanup

* demz sampling and WIP plotting

* WIP predictive plots

* WIP heatmap plots

* predictive plots

* WIP covariates and nbqa-ruff edits

* covariate section

* plot additions

* fig sizes

* remove model file
…book (#651)

* add spaces, increase indentation, and fix number order

* explicit with 6
* Creating plot waterfall

Co-Authored-By: Carlos Trujillo <[email protected]>

* requested changes

* pre-commit

---------

Co-authored-by: Carlos Trujillo <[email protected]>
Databricks should have a lower-case b.
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.4.1 → v0.4.2](astral-sh/ruff-pre-commit@v0.4.1...v0.4.2)
- [github.com/pre-commit/mirrors-mypy: v1.9.0 → v1.10.0](pre-commit/mirrors-mypy@v1.9.0...v1.10.0)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* support for negative values and dates (if used)

* fix terrible spelling

* test dates in coords

* cover numpy objects

* consolidate the tests
* add quickstartt to readme

* add pymc labs logo

* typos

* add community links

* fixes
* Add contributors to README

* small code style improvements
* Add time-varying prior functionality to DelayedSaturatedMMM

* resolve wd's comments

* resolve failing pre-commits

* add tvp_kwargs to model_config

* fix typo

* replace softplus

* resolve minor review comments

* Add option to supply `ax` to `plot_posterior_predictive`

* bugfix: time_index was not set correctly for OOS

If provided `X` to `_data_setter` was *not* the immediate sequence following the training set, the time_index would we wrong. With this fix, the `date["time_index"]` which gets set upon providing a new `X`, is inferred from the `self.date_column` column of provided `X` (by comparing it to same date column in the training data `self.X`).

* Clean up example notebook

* Make utility function `transform_1d_array`

* 'tvp_kwargs' -> 'intercept_tvp_kwargs'

* move `infer_time_index` into utils

* add tests for new utils

* small fixes (found in tests)

* add tests to cover all added cases

* fix ruff check

* update typehints

* resolve review comments

* refactor model logic for tv intercept

* address review comment for util test

* .

* fix documentation link

* change variable name

* fix hsgp_dims

* update time_varying_prior to be centered on 1

* review fixes

* fix broken test

* add final tests

* fix coverage issues

* Update tests/mmm/test_tvp.py

Co-authored-by: Will Dean <[email protected]>

* Update pymc_marketing/mmm/tvp.py

Co-authored-by: Will Dean <[email protected]>

* Update tests/mmm/test_tvp.py

Co-authored-by: Will Dean <[email protected]>

* Update tests/mmm/test_tvp.py

Co-authored-by: Will Dean <[email protected]>

* significant improvements to notebook

* fix heading

* update notebook to make it EVEN better

* update legend, add watermark

* fix intro

* fix broken test

* copy sweep with grammarly

---------

Co-authored-by: Will Dean <[email protected]>
* add tv intrecept to readme

* add to comparison table
* ignore non-implemented

* remove not implemented error from abstract classes

* simplify docstrings
@drbenvincent
Copy link
Contributor Author

Everything is passing now, except codecov

@wd60622
Copy link
Contributor

wd60622 commented Oct 29, 2024

All looks really clean @drbenvincent Nice work!
I will take a look later and can do those edits to docstrings, tests, etc when I get the chance

@drbenvincent
Copy link
Contributor Author

@wd60622 All tests pass again now - aside from code coverage. Am I right in understanding that you might do some work on this PR? I probably need to put this aside until late next week until things free up a bit.

@wd60622
Copy link
Contributor

wd60622 commented Oct 31, 2024

@wd60622 All tests pass again now - aside from code coverage. Am I right in understanding that you might do some work on this PR? I probably need to put this aside until late next week until things free up a bit.

Nice. Yes, I take a look today

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
causal inference customer choice Related to customer choice module docs Improvements or additions to documentation enhancement New feature or request streamlit tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.