convez.ai
MCP ServerIntegração & APIsAvançado
pgundlupetvenkatesh

MDB API Layer - Framework de Testes para TMDB

pgundlupetvenkatesh/mdb_api_layer

Framework SDET para TMDB com diagnóstico de falhas powered by IA

Ver no GitHub
InicialPluginScore · de 0 a 100

Framework de testes Python para a API REST do TMDB com validação de contrato Pact, testes E2E, execução paralela em Docker/Kubernetes e diagnóstico de falhas com IA via Groq. Expõe funcionalidades como MCP server para agentes Claude analisarem e corrigirem falhas de teste automaticamente.

Instalação

$npx convez-ai pgundlupetvenkatesh/mdb_api_layer

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
28%
Último commit
há 2 dias
Versão
—
Issues abertas
1
Contribuidores
1

Casos de uso

  • Testes automatizados e CI/CD para APIs REST
  • Validação de contratos entre consumidor e provedor de API
  • Diagnóstico inteligente de falhas em testes via LLM
  • Integração com agentes IA para análise de quebras de teste

Veredito

Solução robusta e profissional para testar APIs com práticas SDET reais, contract testing e orquestração em containers. O diagnóstico de falhas com IA via MCP é diferencial, mas exige Python 3.10+ e conhecimento sólido de pytest, Docker e K8s.

README

A tradução para o português deste README ainda está na fila.

mdb_api_layer

Tests Docs Python pytest Allure Loguru Pact pydantic Docker Sphinx Kubernetes Groq CI/CD

API Testing Framework for The Movie Database (TMDB) with Python

Overview

A Python-based API testing framework for TMDB API, featuring structured test organization, JSON schema validation, Contract tests, and comprehensive response assertions.

View live code documentation & latest test report here

Quick Start

New here? Get from clone to a passing test in four steps:

# 1. Clone & install
git clone https://github.com/pgundlupetvenkatesh/mdb_api_layer.git
cd mdb_api_layer
poetry install

# 2. Add your TMDB credentials to a .env file (see Configuration below)
echo "TMDB_API_KEY=your_api_key_here"        >> .env
echo "TMDB_AUTH_TOKEN=your_read_access_token" >> .env

# 3. Run a single test to verify the setup
poetry run pytest tests/movie_lists/test_popular.py -v -s

# 4. Run the full suite
poetry run pytest tests/ -v -m "not contract"

Each step is expanded in Prerequisites, Installation, Configuration, and Running Tests below.

Features

  • RESTful API client with session management (GET, POST, PUT, DELETE)
  • Multiple API clients: Movies, People, Lists, Account
  • Strict Pydantic model validation for response structure (single source of truth)
  • Consumer-driven contract testing with Pact
  • Data-driven testing with per-section YAML test data and cached dynamic generators
  • Automatic retries for flaky live-API latency via pytest-rerunfailures
  • Reusable HTTP response assertion helpers (status, method, content-type, response time)
  • Dedicated per-client pytest fixtures (movies_api, people_api, lists_api, search_api)
  • Structured logging with Loguru (configurable level & file output)
  • Configurable environment-based settings via .env
  • AI-powered test failure analysis using open-source LLMs via Groq (opt-in)
  • Pytest integration with HTML and Allure reporting
  • Sphinx documentation with GitHub Pages deployment

Architecture

Architecture Diagram

How the Layers Connect

LayerResponsibility
ConfigLoads .env, exposes Config class, sets up Loguru logging
APIBaseAPI handles HTTP + auth; endpoint classes (MoviesAPI, etc.) add domain methods
Testsconftest.py wires fixtures/hooks; suites use helpers, schemas, and data-driven YAML
ContractsPact CDC tests generate .json contract files in tests/pacts/
DockerDockerfile builds image; docker-compose.yml runs tests locally with .env and volume
CI/CDTwo GH Actions workflows: one for Docker test runs + report deploy, one for Sphinx docs
DocsSphinx auto-generates API docs from docstrings, deployed to GitHub Pages
MCPfailure_mcp/ exposes FailureAnalyzer as 3 callable tools over stdio transport

Prerequisites

  • Python 3.10+
  • Poetry for dependency management
  • TMDB API account and API key
  • Docker for containerized test runs (optional)

Installation

# Clone the repository
git clone https://github.com/pgundlupetvenkatesh/mdb_api_layer.git
cd mdb_api_layer

# Install dependencies
poetry install

# (optional) add a new Python package dependency later
poetry add <package_name>

Sanity Checks

Verify the virtual environment was created and dependencies resolved:

poetry env info
poetry show

Configuration

Getting TMDB API Credentials

  1. Create a TMDB account at themoviedb.org
  2. Request an API key:
    • Go to Settings → API
    • Click "Create" or "Request an API Key"
    • Choose "Developer" for personal/testing use
    • Fill in the application details (use your project URL or localhost for testing)
    • Once approved, you'll receive your API Key (v3 auth) and API Read Access Token
  3. Locate your credentials:
    • API Key: Found under "API Key (v3 auth)" on the API settings page
    • Auth Token: Found under "API Read Access Token (v4 auth)" — this is your Bearer token

Setting Up Environment Variables

Create a .env file in the project root (this file is ignored):

# Required
TMDB_API_KEY=your_api_key_here
TMDB_AUTH_TOKEN=your_read_access_token_here

# Optional (defaults shown)
TMDB_BASE_URL=https://api.themoviedb.org
TMDB_API_VERSION=3
TMDB_TIMEOUT=30
TMDB_USER_ACCESS_TOKEN=your_user_access_token_here

Alternative: Export variables directly in your terminal:

export TMDB_API_KEY="your_api_key_here"
export TMDB_AUTH_TOKEN="your_read_access_token_here"

Environment Variables Reference

VariableDescriptionRequiredDefault
TMDB_API_KEYYour TMDB API key (v3 auth)Yes-
TMDB_AUTH_TOKENYour TMDB API read access token (v4 auth)Yes-
TMDB_BASE_URLBase URL for TMDB APINohttps://api.themoviedb.org
TMDB_API_VERSIONAPI version to use (3 or 4)No3
TMDB_TIMEOUTRequest timeout in secondsNo30
TMDB_USER_ACCESS_TOKENUser Access Token for Lists API (v4 auth)No-
TMDB_ACCOUNT_IDTMDB account IDNo12016691
TMDB_SESSION_IDSession ID for authenticated requestsNo-
TMDB_REQ_TOKENRequest token for authenticationNo-
TMDB_MOVIE_IDDefault movie ID for testsNo346698

Note: Keep your API credentials secure. Never commit .env files or expose tokens in public repositories.

Running Tests

First, confirm your setup works by running a single test:

# Verify your setup
poetry install

# Run one test
poetry run pytest tests/movie_lists/test_popular.py -v -s

Once that passes, use the commands below for everyday runs:

# Run all tests (integration + contract)
poetry run pytest tests/ -v

# Run all tests except contract tests
poetry run pytest tests/ -v -m "not contract"

# Run tests by module
poetry run pytest tests/movies/ -v -s
poetry run pytest tests/people/ -v -s
poetry run pytest tests/lists/ -v -s
poetry run pytest tests/movie_lists/ -v -s
poetry run pytest tests/search/ -v -s

# Run a specific test function
poetry run pytest tests/movie_lists/test_popular.py::TestClassName::test_func_name -v -s

# Run with debug logging
poetry run pytest tests/ -v -s --loguru-log-level=DEBUG

# Run with log file output
poetry run pytest tests/ -v -s --log-to-file

# Run with HTML report
poetry run pytest tests/ --html=report/tmdb_report.html --self-contained-html -v -s

# Clean run with Allure reporting
rm -rf allure-results && poetry run pytest tests/ --alluredir=allure-results -v && allure serve allure-results
# View report (requires Allure CLI installed locally: brew install allure)
allure serve allure-results

# Generate static HTML from results. allure-report/ contains a full static site
allure generate allure-results -o allure-report --clean

# Keep history between local runs
cp -r allure-report/history allure-results/ 2>/dev/null || true

Project Structure

mdb_api_layer/
├── api/
│   ├── base_api.py             # Base API client with HTTP methods (GET, POST, PUT, DELETE)
│   ├── movies_api.py           # Movies endpoint implementation
│   ├── people_api.py           # People endpoint implementation
│   ├── lists_api.py            # Lists endpoint implementation (v4 API)
│   ├── account_api.py          # Account endpoint implementation
│   └── search_api.py           # Search endpoint implementation
├── config/
│   └── config.py               # Environment configuration & Loguru logging setup
├── tests/
│   ├── allure/
│   │   └── categories.json
│   ├── ai_analysis/                 # Generated analysis reports (gitignored)
│   │   └── failure_analysis.json
│   ├── conftest.py             # Pytest fixtures, hooks & logging CLI options
│   ├── contracts/
│   │   ├── conftest.py             # Shared Pact fixtures (pact, pact_movies_api, pact_address)
│   │   ├── test_movie_details.py   # Movie details contract tests (Pact CDC)
│   │   └── test_popular_movies.py  # Popular movies contract tests (Pact CDC)
│   ├── data/
│   │   ├── data_loader.py      # Per-section YAML test data loader + dynamic generators
│   │   ├── test_data.yaml      # Parametrized test data for data-driven tests
│   │   └── movie_ids.txt       # Movie IDs for random test data generation (cached on first read)
│   ├── helpers/
│   │   ├── failure_analyzer.py     # LLM failure analysis client
│   │   ├── field_assertions.py     # Legacy field-check mixin (now unused; superseded by Pydantic models)
│   │   ├── response_assertions.py  # HTTP response assertion helpers
│   │   └── test_data_generators.py # Dynamic test data generators
│   ├── movies/
│   │   ├── test_details.py     # Movie details integration tests
│   │   ├── test_add_rating.py  # Add rating tests
│   │   └── test_delete_rating.py   # Delete rating tests
│   ├── movie_lists/
│   │   └── test_popular.py     # Popular movies list tests
│   ├── people/
│   │   └── test_details.py     # People details integration tests
│   ├── lists/
│   │   └── test_update.py      # Lists update tests (v4 API)
│   ├── search/
│   │   └── test_movie.py       # Movie search integration tests
│   ├── pacts/
│   │   └── *.json              # Generated Pact contract files
│   └── schemas/
│       └── models.py             # Pydantic response models (source of truth for validation)
├── docs/
│   ├── conf.py                 # Sphinx configuration
│   ├── index.rst               # Documentation index
│   ├── api.rst                 # API module documentation
│   ├── config.rst              # Config module documentation
│   ├── tests.rst               # Tests module documentation
│   ├── make.bat                # Windows build script
│   └── Makefile                # Unix build script
├── logs/
│   └── test_run.log            # Log file (when --log-to-file is used)
├── failure_mcp/
│   ├── __init__.py
│   ├── server.py               # entry point, registers tools, runs stdio server
│   └── tools/
│       ├── __init__.py
│       ├── analyze_test_failure.py     # TOOLS list + handle_call() dispatcher
├── report/
│   └── tmdb_report.html        # Generated HTML test reports
├── .env                        # Environment variables (not committed)
├── .gitignore
├── Dockerfile                  # Docker image definition for containerized test runs
├── docker-compose.yml          # Docker Compose config for local containerized runs
├── pyproject.toml              # Poetry configuration & pytest settings
├── k8s/                        # Kubernetes Job manifests for test orchestration
│   ├── integration-test-job.yaml
│   └── contract-test-job.yaml
├── poetry.lock
└── README.md

Logging

The framework uses Loguru for structured logging with colored output and option to save logs to files. You can control the log level to filter messages based on severity.

Log Levels

LevelDescription
DEBUGDetailed diagnostic information
INFOGeneral operational messages
WARNINGPotential issues
ERRORError events
CRITICALSerious failures

Log level hierarchy

Level SetLogs Shown
DEBUGDEBUG, INFO, WARNING, ERROR, CRITICAL
INFOINFO, WARNING, ERROR, CRITICAL
WARNINGWARNING, ERROR, CRITICAL
ERRORERROR, CRITICAL
CRITICALCRITICAL only

Controlling Log Level

  • Default (INFO) - poetry run pytest tests/*
  • Debug logging - poetry run pytest tests/* --loguru-log-level=DEBUG
  • Only errors - poetry run pytest tests/* --loguru-log-level=ERROR

Log Output

By default, logs are printed to the console with color coding. You can also add --log-to-file option to write logs to a logs/test_run.log for persistent storage and later analysis.

poetry run pytest tests/* --log-to-file

Running Tests in Docker

Tests run inside Docker containers for a consistent, isolated environment — no local Python or dependency setup needed. Integration and contract tests run in parallel as separate containers from the same image.

Prerequisites

  • Docker installed and running

Using Docker Compose (Recommended for Local)

# Build and run both test suites in parallel (reads secrets from .env file)
docker compose up --build

# Flow
build image (mdb-api-tests)
  ↓
start 2 containers in parallel
  ├── tmdb-api-integration-tests  → pytest tests/ -m "not contract"
  └── tmdb-api-contract-tests     → pytest tests/contracts/ -m contract
  ↓
reports saved to ./report/

# Run in detached mode - Run in background
docker compose up --build -d

# View logs for a specific service
docker compose logs -f integration_tests
docker compose logs -f contract_tests

# Tear down
docker compose down

Both containers share the same ./report/ volume mount but write to separate report files:

  • tmdb_non_contract_report.html — integration tests
  • tmdb_contract_report.html — contract tests

Using Docker Directly (Single Run)

# Build the image
docker build -t mdb-api-tests .

# Run all tests in one container (pass env vars explicitly)
docker run --rm \
  -e TMDB_API_KEY=your_key \
  -e TMDB_AUTH_TOKEN=your_token \
  -v $(pwd)/report:/app/report \
  mdb-api-tests

How It Works

FileRole
DockerfileDefines the image: Python 3.12-slim, installs Poetry & dependencies
docker-compose.ymlRuns integration & contract tests in parallel via two services
GH CI workflowBuilds with Buildx caching, runs parallel compose, merges reports

Parallel Execution Architecture

docker-compose.yml
  │
  ├── x-common-config (shared anchor)
  │     ├── build: .
  │     ├── image: mdb-api-tests
  │     ├── env_file: .env
  │     └── volumes: ./report:/app/report
  │
  ├── integration_tests (container: tmdb-api-integration-tests)
  │     └── pytest tests/ -m "not contract" → tmdb_non_contract_report.html
  │
  └── contract_tests (container: tmdb-api-contract-tests)
        └── pytest tests/contracts/ -m contract → tmdb_contract_report.html

CI/CD (GitHub Actions)

In CI, tests run inside the same Docker image with additional optimizations:

  • Docker Buildx for layer caching between workflow runs
  • Secrets are written to .env file (never baked into the image)
  • Parallel execution via docker compose up --no-build (image pre-built by Buildx)
  • Individual HTML reports are merged into merged_tmdb_full_report.html using pytest-html-report-merger
  • Merged report is deployed to GitHub Pages
  • If tests fail, the workflow still merges and deploys the report before marking the job as failed

Contract Testing

This project includes consumer-driven contract testing(CDC) using Pact to verify API structure expectations without making real network calls.

Running Contract Tests

# Run a specific contract test. -m is a marker defined in .toml file
poetry run pytest tests/contracts/test_movie_details.py -v -m contract

# Run all contract tests
poetry run pytest tests/contracts/ -v -m contract

# With report generation
poetry run pytest tests/contracts/ --html=report/tmdb_contract_report.html --self-contained-html -v -s -m contract

# Run all tests except contract tests
poetry run pytest tests/ -v -m "not contract"

# Run all tests (integration + contract)
poetry run pytest tests/ -v

# With report generation for all tests
poetry run pytest tests/ --html=report/tmdb_full_report.html --self-contained-html -v -s

Generated Pact Files

Contract tests generate a single merged JSON pact file in tests/pacts/ (all interactions share one consumer→provider pair):

tests/pacts/
└── mdb_api_layer-api_pvd.json   # Consumer (mdb_api_layer) → provider (api_pvd) contract

Pact files document the expected request/response structure and can be:

  • Versioned for tracking in git
  • Shared with API providers to verify
  • Used by CI/CD to detect breaking changes

AI-Powered Failure Analysis

When a test fails, the framework can automatically send the failure context to an open-source LLM for instant root-cause diagnosis. This is disabled by default — no API calls are made unless the user explicitly opt in.

How It Works

Test Fails
  ↓
pytest_runtest_makereport hook captures:
  • test name & file
  • error message & traceback
  • API URL, status code, response body (if available)
  ↓
FailureAnalyzer sends context to Groq API (Llama 4 Scout / Qwen 3)
  ↓
LLM returns structured JSON diagnosis:
  { root_cause, category, suggested_fix, confidence, explanation, evidence }
  ↓
Diagnosis is:
  • Logged to console (🤖 emoji prefix)
  • Attached to Allure report as JSON
  • Saved to tests/ai_analysis/failure_analysis.json at session end

Failure Categories

LLM classifies each failure into one of these categories:

CategoryMeaning
api_bugAPI returned unexpected response (status code, missing field)
test_bugTest assertion or logic is incorrect
data_issueTest data is stale, invalid, or resource was deleted
timeoutResponse time exceeded threshold
auth_errorAuthentication/authorization failure (expired token, key)
schema_mismatchResponse doesn't match Pydantic model or Pact contract
environmentConfig, connectivity, or environment setup issue

Setup

  1. Get a free Groq API key at console.groq.com
  2. Add to .env file:
    GROQ_API_KEY=your_groq_api_key_here
    AI_ANALYSIS_ENABLED=true          # or use --failure-analysis CLI flag instead
    
  3. Install dependencies (should be in pyproject.toml):
    poetry install  # groq package is included
    

Usage

# Enable via CLI flag to keep .env clean
poetry run pytest tests/ --failure-analysis -v

# Or enable via environment variable
AI_ANALYSIS_ENABLED=true poetry run pytest tests/ -v

# Combine with other options
poetry run pytest tests/ --failure-analysis --loguru-log-level=DEBUG --alluredir=allure-results -v

When enabled, each failed test produces a console log like:

🤖 AI Analysis [auth_error]: The Bearer token has expired, causing a 401 Unauthorized response.

Output

DestinationFormatWhen
Console logTextImmediately on failure
Allure report (🤖 AI Failure Analysis)JSONAttached to failed test
tests/ai_analysis/failure_analysis.jsonJSONEnd of test session

Sample diagnosis:

{
  "root_cause": "The movie ID 999999 does not exist, causing a 404 response",
  "category": "data_issue",
  "suggested_fix": "Update test data to use a valid movie ID from TMDB",
  "confidence": 90,
  "explanation": "The test expects a 200 OK but the API returned 404 Not Found because the movie resource was deleted or never existed. Refresh test data with current valid IDs.", 
  "evidence": ["HTTP status code: 404", "Response body: {'status_code':34, 'status_message':'The resource you requested could not be found.'}"],
  "test_name": "test_get_movie_details[invalid_id]",
  "model": "meta-llama/llama-4-scout-17b-16e-instruct"
}

allure_failure_ss

Supported Models

The default model is meta-llama/llama-4-scout-17b-16e-instruct on Groq's free tier. Override via environment variable:

# In .env
AI_MODEL=qwen/qwen3-32b

CI Integration

In GitHub Actions, the feature works automatically when GROQ_API_KEY secret and AI_ANALYSIS_ENABLED variable is set in repository settings. The workflow writes them to the .env file so both Docker containers have access. Analysis results are attached to the Allure report deployed to GitHub Pages.

AI Environment Variables

VariableDescriptionRequiredDefault
AI_ANALYSIS_ENABLEDEnable AI failure analysisNofalse
GROQ_API_KEYGroq API key for LLM accessIf enabled-
AI_MODELLLM model identifier on GroqNometa-llama/llama-4-scout-17b-16e-instruct

Note: Groq free tier has rate limits. For large test suites with many failures, analysis may be throttled. The analyzer gracefully handles errors — if an LLM call fails, the test result is unaffected.

Reports

Pytest HTML Report

poetry run pytest tests/ --html=report/tmdb_report.html --self-contained-html -v -s

Generate a simple pytest HTML test report: Open the generated tmdb_report.html in your web browser to view the test results. sample_report

Allure Report

# Generate Allure results
poetry run pytest tests/ --alluredir=allure-results -v

--alluredir option tells pytest to save the test results in a format that Allure can process. See Running Tests section for more Allure details. sample_allure_report

Docker Parallel Reports

When running via docker compose, two separate reports are generated and then merged in CI:

ReportSource
tmdb_non_contract_report.htmlIntegration tests
tmdb_contract_report.htmlContract tests
merged_tmdb_full_report.htmlCombined (CI only)

Click here to see the latest merged report

Documentation

cd docs
make html

To clean previous builds and rebuild run - make clean && make html

View documentation

Open docs/_build/html/index.html in your web browser. doc_sample

Click here to See the current live documentation

Dependencies

Runtime

  • requests — HTTP client
  • python-dotenv — .env file loading
  • pydantic — Model validation
  • pyyaml — YAML test data parsing
  • loguru — Structured logging
  • groq — LLM client for AI failure analysis
  • mcp — Model Context Protocol server SDK (MCP integration)

Development

  • pytest — Test framework
  • pytest-html — HTML report generation
  • pytest-html-report-merger — Merges multiple HTML reports into one
  • allure-pytest — Allure reporting
  • sphinx — Documentation generation
  • pytest-order — Test execution ordering
  • pytest-rerunfailures — Retries flaky tests (absorbs live-API latency spikes)
  • pact-python — Consumer-driven contract testing
# Update dependencies
poetry update

Running Tests in Kubernetes(Optional)

Tests can also run in Kubernetes for container orchestration. Both test suites run as parallel Jobs on your local Docker Desktop K8s cluster.

Prerequisites

  • Docker Desktop with Kubernetes enabled
  • kubectl CLI configured (kubectl cluster-info should respond)

Step-by-step

Run the run_k8s_tests.sh file to execute k8 commands in sequence. Feel free to see the step-by-step commands in the script.

How It Works

k8s/
├── integration-test-job.yaml   → Job: pytest tests/ -m "not contract"
└── contract-test-job.yaml      → Job: pytest tests/contracts/ -m contract

.env → kubectl create secret → tmdb-secrets (K8s Secret)
                                    ↓
                          envFrom: secretRef
                                    ↓
                    ┌───────────────────────────────────┐
                    │         mdb-api-tests image       │
                    ├──────────────┬────────────────────┤
                    │ integration  │    contract        │
                    │ tests Job    │    tests Job       │
                    └──────────────┴────────────────────┘

MCP Integration

The framework ships a Model Context Protocol (MCP) server that exposes FailureAnalyzer capabilities as callable tools for any MCP-compatible AI client (Claude Desktop, Cursor, VS Code Copilot Chat, custom agents, etc.).

Project Structure

failure_mcp/
├── __init__.py
├── server.py                   # Entry point — registers tools, runs stdio transport
└── tools/
    ├── __init__.py
    └── analyze_test_failure.py # TOOLS list + handle_call() dispatcher

failure_analyzer.py is not modified — the MCP server wraps the existing singleton.

Exposed Tools

ToolRequired ArgsOptional ArgsDescription
analyze_failuretest_name, error_messagetest_file, traceback, api_url, status_code, response_bodySend failure context to LLM and get structured diagnosis
get_results—min_confidence (0–100, default 0)Return accumulated diagnosis results, optionally filtered by confidence threshold
save_results—output_dir (default "ai_analysis")Flush all results to <output_dir>/failure_analysis.json

Sample analyze_failure Response

{
  "root_cause": "The Bearer token has expired, causing a 401 Unauthorized response.",
  "category": "auth_error",
  "suggested_fix": "Refresh the TMDB_AUTH_TOKEN in your .env file and re-run the tests.",
  "confidence": 92,
  "explanation": "The API rejected the request with HTTP 401...",
  "evidence": ["HTTP status code 401", "Response body contains 'Invalid API key'"],
  "test_name": "test_get_movie_details",
  "model": "meta-llama/llama-4-scout-17b-16e-instruct",
  "confidence_tier": "high"
}

confidence_tier is derived from the raw confidence score:

ScoreTier
>= 80high — act on it
50–79medium — review it
< 50low — treat with caution

Testing MCP Server

MCP Inspector is a web-based UI to interactively call MCP tools and inspect responses — no AI client needed.

Prerequisites

Node.js must be installed (brew install node on macOS).

Launch

npx @modelcontextprotocol/inspector

Opens at http://localhost:6274 in your browser.

Step 1 — Connect

Important: MCP Inspector does not have a cwd field. Use the full path to the poetry binary so it resolves the correct virtual environment regardless of where Inspector is launched from.

FieldValue
Transport TypeSTDIO
Command<path_to>/mdb_api_layer/.venv/bin/poetry
Argumentsrun python -m failure_mcp.server

Expand Environment Variables and add:

KeyValue
AI_ANALYSIS_ENABLEDtrue
GROQ_API_KEYyour actual key from .env
PYTHONPATH<path_to>/mdb_api_layer

Click Connect. On success the right panel shows all 3 tools: analyze_failure, get_results, save_results.

mcp_inspector

Step 2 — Call analyze_failure

Select the tool, paste the input JSON and click Run:

  • Feed this sample input JSON
{
  "test_name": "test_get_movie_details",
  "error_message": "HTTP 401 Unauthorized: Invalid API key",
  "test_file": "tests/movies/test_details.py",
  "traceback": "Traceback (most recent call last): ...",
  "api_url": "https://api.themoviedb.org/3/movie/12345",
  "status_code": 401,
  "response_body": "{\"status_code\": 7, \"status_message\": \"Invalid API key: You must be granted a valid key.\"}"
}

Note: response_body must be a string, not a JSON object. Stringify it before passing.

Expected response includes root_cause, category, confidence (0–100), confidence_tier, evidence, and suggested_fix.

mcp_inspector_op

Step 3 — Call get_results

Returns all diagnoses accumulated in the current session. Optionally filter by confidence:

{ "min_confidence": 80 }

Pass {} (empty object) to return everything.

Step 4 — Call save_results

Flushes all accumulated results to ai_analysis/failure_analysis.json:

{ "output_dir": "ai_analysis" }

Returns { "saved": <count>, "path": "ai_analysis/failure_analysis.json" }.

Troubleshooting Connection Errors

SymptomFix
Connection Error — Check if your MCP server is runningVerify the Command path is correct with which poetry
ModuleNotFoundError: No module named 'tests'Make sure PYTHONPATH is set to the project root
{"error": "Analysis disabled or failed.", "hint": "Set AI_ANALYSIS_ENABLED=true..."}Confirm AI_ANALYSIS_ENABLED=true and GROQ_API_KEY are set in Environment Variables
Server starts but immediately exitsRun PYTHONPATH=. poetry run python -m failure_mcp.server in terminal — a hanging prompt (no output) means it's working correctly

Running the MCP Server

# Requires AI_ANALYSIS_ENABLED=true and GROQ_API_KEY in .env
poetry run python -m failure_mcp.server

MCP Client Config

Add to ~/.cursor/mcp.json (Cursor) or Claude Desktop's settings:

{
  "mcpServers": {
    "failure-analyzer": {
      "command": "poetry",
      "args": ["run", "python", "-m", "failure_mcp.server"],
      "cwd": "/path/to/mdb_api_layer",
      "env": {
        "AI_ANALYSIS_ENABLED": "true",
        "GROQ_API_KEY": "<your-groq-key>"
      }
    }
  }
}

Once connected, your AI client can call analyze_failure directly by passing a failure context dict — no changes to any test file needed.

Claude Code Integration

This repo is set up for Claude Code. It ships project-scoped skills (reusable instructions Claude loads on demand) and hooks (scripts the harness runs automatically around tool calls) under .claude/. None of this is required to run the tests — it only shapes how Claude Code behaves when working in the repo.

.claude/
├── settings.json               # wires the hooks (PreToolUse + Stop)
├── skills/
│   ├── commit-rules/           # how to scope, stage & word a commit
│   ├── update-claude-md/       # keep CLAUDE.md in sync after a change
│   └── review-tests/           # review a diff against this repo's conventions
└── hooks/
    ├── guard-commit.sh         # block commits that stage secrets
    └── claude-md-reminder.sh   # nudge to reconcile CLAUDE.md after code changes

Skills

Invoke explicitly with /<name>, or let Claude trigger one automatically. The table is a summary; each SKILL.md is authoritative.

SkillWhat it does
commit-rulesEnforces atomic commits, message style, secret/artifact exclusion, required trailer
update-claude-mdReconciles CLAUDE.md against the code after a structural change
review-testsReviews a diff against this repo's test conventions (complements /code-review)

Hooks

Configured in .claude/settings.json, run automatically, and fail-open (never wedge legitimate work).

HookEventBehavior
guard-commit.shPreToolUse(Bash)Blocks a git commit that stages a .env or a credential-like token
claude-md-reminder.shStopNudges to run update-claude-md when api//config/ changed but CLAUDE.md didn't

Skills and hooks are version-controlled, so every contributor using Claude Code in this repo gets the same commit hygiene, doc-sync reminders, and convention checks automatically.

GitHub Pages Deployment (Maintainers)

This section is for maintainers deploying reports/docs to GitHub Pages — it is not required to run the tests locally.

The project uses multi-deployment strategy to host both Sphinx doc and Pytest HTML reports on the same GitHub Pages site. I am using peaceiris/actions-gh-pages action across two separate workflows:

  • API Tests: Generate and deploys test reports to /report.
  • Docs: Build and deploys docs to /docs.

Configs

  1. Settings > Pages: - To support both docs and reports, the repository is configured to deploy from a specific branch rather the default "GitHub Actions" method. Source: Deploy from a branch Branch: gh-pages Folder: / (root)
  2. Environment & Protection Rules - Modified the default github-pages environment to allow the automated workflows to push updates. Settings > Environments > github-pages: Deployment branches: Change from Selected branches to All branches or No Restriction. This allows the peaceiris/actions-gh-pages action to push the gh-pages branch without being blocked by "Main-only" protection rules GitHub Docs: Deployment Branches.
  3. Settings > Actions > General: Workflow permissions: Set to Read and write permissions. This grants the GITHUB_TOKEN the authority to create and update the gh-pages branch GitHub Docs: Token Permissions.

Future Improvements

  • AI-based test generation
  • Load perf testing with Locust
  • Send test results to Grafana
Ver o original no GitHub

Bom para

Relacionados