Description:

This project is an automated testing framework designed to improve researchers work and data analysts in testing the statistical significance of events and hypotheses. The framework is designed to be user-friendly and customizable, so users can calculate and configure different statistical tests based on their specific needs.

The framework includes several versions, starting with an automated testing of statistical significance of events using the Bernoulli distribution. The next version implements multiple hypothesis testing using the Holm-Bonferroni method. The following versions include combining the code for testing multiple hypotheses with the multiple hypothesis testing, adding Bayesian testing, and automating the testing of statistical significance of revenue using bootstrap, chi-square distribution, and Bernoulli.

The project is open-source and available on Github, allowing users to contribute to the framework and customize it based on their specific needs.

Development Testing Plan:

Done: Version 1. Automated testing of statistical significance of events: Bernoulli distribution.

Done: Version 2. Implemented multiple hypothesis testing, starting with the Holm-Bonferroni method.

Version 3. Add Bayesian testing.

Version 4. Automated testing of statistical significance of revenue: bootstrap, chi-square distribution, Bernoulli.

Version 5. All of the above is wrapped into a microservice, deployed in Docker, and deployed in Kubernetes.

Version 6. All of the above is tested on large numbers for the stability of results.

Version 7. Monitoring of metrics.