Merge pull request #11 from lgabs/evolver-janus-multivariant

Refactor Janus to support Multivariant

Author: lgabs

.github/workflows/ci.yml

@@ -7,7 +7,7 @@ on:
     branches: [ master ]
 
 jobs:
-  build:
+  test:
     runs-on: ubuntu-latest
     strategy:
       matrix:
@@ -29,4 +29,46 @@ jobs:
 
     - name: Run tests
       run: |
-        python -m unittest discover tests
+        pytest
+
+  build:
+    needs: test
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v2
+      
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+      
+      - name: Log in to GitHub Container Registry
+        uses: docker/login-action@v1
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      
+      - name: Extract metadata for Docker
+        id: meta
+        uses: docker/metadata-action@v3
+        with:
+          images: ghcr.io/${{ github.repository }}
+          tags: |
+            type=sha,format=long
+            type=ref,event=branch
+            type=ref,event=tag
+            latest
+      
+      - name: Build and push Docker image
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          push: true
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          cache-from: type=gha
+          cache-to: type=gha,mode=max

Dockerfile

@@ -8,8 +8,6 @@ RUN pip install --no-cache-dir -r requirements.txt
 
 COPY . .
 
-EXPOSE 8501
+EXPOSE 8000
 
-ENV PYTHONPATH=/app
-
-CMD ["streamlit", "run", "Homepage.py"]
+CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Makefile

@@ -4,53 +4,103 @@
 ![Python Version](https://img.shields.io/badge/python-3.7%20%7C%203.8-brightgreen.svg)
 [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
 
-> **Warning**
-> We're making a major breaking change in the project to use the [bayesian-testing](https://github.com/Matt52/bayesian-testing) library for better experiment management, and a full stack application will be developed to build a website for Janus.
-> Please, consider using the distributed [package in pypi](https://pypi.org/project/janus-web-ab-testing/), which comes from the `evolve-janus-backend` branch.
+Janus is a Bayesian A/B Testing application that supports multivariant experiments. It's designed to help you make data-driven decisions by analyzing conversion rates, revenue, and ARPU (Average Revenue Per User) across multiple variants.
 
+## Features
 
-Janus is an A/B Test Engine to be used in a variety use cases, especially to measure conversion, ticket and ARPU difference between variants, i.e, typical metrics for tests in marketplaces. The engine name is an analogy to _Janus_, the god of changes and transitions.
-
-This library came as an ideia of separate the statistical calculations in A/B Tests from other code that is typically used to manage tests and execute queries over the company's database, and hence usually carry proprietary code and even business logic, which should not be open sourced. There was the bud to build this library and get it open sourced.
-
-Checkout the [streamlit app](https://lgabs-janus-homepage-31diny.streamlit.app/) from this repo.
+- **Multivariant Testing**: Compare multiple variants simultaneously (not just A vs B)
+- **Bayesian Statistics**: Get more insightful results faster than traditional frequentist methods
+- **Key Metrics Analysis**:
+  - Conversion rate
+  - Revenue for conversions
+  - Average revenue per impression (ARPU)
+- **Modern Web Interface**: Clean, responsive UI built with Bootstrap
+- **FastAPI Backend**: High-performance API for experiment analysis
 
 ## Installation
 
-Open a terminal, clone this repository into your machine and stay into the project directory.
-
-Using a virtual environment is a good practice, but it is optional. If you enjoy it, go ahead and create a virtual environment by typing:
-```
-python3 -m venv venv -r requirements.txt
-```
-Once it is created, you must now activate the environment by using:
-```
-source venv/bin/activate
+1. Clone this repository:
+```bash
+git clone https://github.com/lgabs/janus.git
+cd janus
 ```
-Now, you can install our lib (if you are not using virtual env, go straight to this command):
+
+2. Create a virtual environment (optional but recommended):
+```bash
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
 ```
-make install
+
+3. Install dependencies:
+```bash
+pip install -r requirements.txt
 ```
 
-And that's it! Now, inside our environment, we can import the `janus` lib inside our scripts with plain `import janus` etc. Try to test using the same code on `experiment_example.ipynb` notebook here or in a plain terminal. 
+## Running the Application
+
+### Using Python directly:
+
+```bash
+uvicorn main:app --reload
+```
 
-## Using as an Application
+Then open your browser and navigate to `http://localhost:8000`
 
-You can use _janus_ as a streamlit product in two ways:
+### Using Docker:
 
-### 1. Using Docker (Recommended)
-Build and run the application locally using Docker:
 ```bash
 # Build the Docker image
 docker build -t janus .
 
 # Run the container
-docker run -p 8501:8501 janus
+docker run -p 8000:8000 janus
 ```
-Or use `make run`. Then open your browser and navigate to `http://localhost:8501`
 
+### Using Docker Compose (Recommended):
+
+```bash
+# Start the application
+docker-compose up -d
+
+# View logs
+docker-compose logs -f
+
+# Stop the application
+docker-compose down
+```
+
+Then open your browser and navigate to `http://localhost:8000`
+
+## How to Use
+
+1. Enter your baseline variant name (e.g., "A" or "Control")
+2. Add your variants with their respective data:
+   - Name: A unique identifier for the variant
+   - Impressions: Total number of users/sessions exposed to this variant
+   - Conversions: Number of successful conversions
+   - Revenue: Total revenue generated by this variant
+
+3. Click "Run Analysis" to see the results:
+   - Summary statistics for each variant
+   - Conversion statistics with probability of being the best variant
+   - ARPU statistics with probability of being the best variant
+
+4. Export your results as CSV if needed
+
+## API Documentation
+
+The API documentation is available at `/docs` when the application is running.
+
+## Technical Details
+
+This application uses:
+- [FastAPI](https://fastapi.tiangolo.com/) for the backend
+- [Bayesian-Testing](https://github.com/Matt52/bayesian-testing) for statistical calculations
+- Bootstrap 5 for the frontend UI
+- Jinja2 for HTML templating
 
 ## References
+
 * [What is A/B Testing](https://en.wikipedia.org/wiki/A/B_testing)
 * The bayesian calculations were implemented based on [this VWO white paper](https://vwo.com/downloads/VWO_SmartStats_technical_whitepaper.pdf)
 * [Agile A/B testing with Bayesian Statistics and Python](https://web.archive.org/web/20150419163005/http://www.bayesianwitch.com/blog/2014/bayesian_ab_test.html)

README.md

@@ -0,0 +1,20 @@
+version: '3.8'
+
+services:
+  janus:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    ports:
+      - "8000:8000"
+    volumes:
+      - .:/app
+    environment:
+      - PYTHONPATH=/app
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 10s