convez.ai
SkillAutomaçãoAvançado
rqzhu-aide

Interactive Test para Claude Code

rqzhu-aide/interactive-test-cc

Teste skills de análise causal via conversas multi-turno simuladas

Ver no GitHub
InicialPluginScore · de 0 a 100

Skill para testar competências de análise causal em Claude Code através de conversas multi-turno adaptativas. Simula um usuário que gradualmente divulga informações de estudo e um consultor que responde com questões e escolhas. Um coordenador registra problemas, incertezas e evidências para investigação posterior.

Instalação

$npx convez-ai rqzhu-aide/interactive-test-cc

1 comando no terminal: detecta o formato do repositório (marketplace ou plugin avulso) e instala via Claude Code. Requer Node 18+ e o Claude Code instalado.

Métricas

Estrelas
0
Forks
0
Acompanhando
0
Saúde
28%
Último commit
há 2 semanas
Versão
—
Issues abertas
0
Contribuidores
2

Casos de uso

  • Validar comportamento de consultores de causalidade em cenários complexos
  • Testar raciocínio estatístico e design de estudos observacionais
  • Avaliar qualidade de dados e suporte a decisões analíticas

Veredito

Ferramenta altamente especializada para pesquisadores e desenvolvedores que trabalham com análise causal e consultoria estatística. Bem estruturada com casos de teste robustos, mas requer expertise significativa em econometria e desenho de estudos para aproveitar plenamente.

README

A tradução para o português deste README ainda está na fila.

Interactive Test

Version 8.5.6

A companion skill for testing Causal Consultant through adaptive, multi-turn conversations. A simulated user discloses study information gradually and responds to the consultant's questions and choices. A coordinator runs the exchange and records observable issues, uncertainty and supporting evidence so another programmer can investigate the problems. The simulated user and consultant each run as a separate agent.

Supports the latest version of Causal Consultant, currently 8.1.4.

Available cases

There are 17 configurations: four reusable synthetic problems paired with three personas, plus five standalone cases with fixed user roles.

Reusable problems

SelectorWhat the case tests
study-designPlan a prospective tutoring evaluation under allocation, measurement and resource constraints.
observational-didAnalyze a tutoring-policy panel using difference-in-differences, distinguishing effects for adopting sites from effects across all sites.
cate-policyLearn whom to offer tutoring using randomized data, costs and capacity, then evaluate the saved policy on separate data.
data-quality-edgeAudit duplicated and incomplete tutoring records and determine which descriptive or causal conclusions they support.

Choose one of three personas for any reusable problem:

  • standard (default): a typical user who shares context progressively and discusses analysis choices.
  • novice-adversarial: a user with limited statistical fluency who presses for a preferred conclusion but can learn from explanations.
  • expert: a knowledgeable user who proposes distinct analysis ideas and accepts justified rejection or refinement.

These 12 problem/persona pairs require a saved report appropriate to the study. The study-design case produces a prospective design report, without inventing trial outcomes. Detailed disclosure and completion rules are in the catalog guide.

Standalone cases

rggvy-electrification: Burlig and Preonas (2024), Out of the Darkness and into the Light? Development Effects of Rural Electrification. Journal of Political Economy, 132(9), 2937-2971. Includes actual Chhattisgarh village and household data to analyze from the replication archive, for a household-welfare briefing with a fixed Research Coordinator. It requires a saved report and continuity participation.

The four planning cases are fictional prospective adaptations of published studies:

The papers inspire the design challenges; their findings do not supply expected answers. These four cases seek useful collection and conditional analysis plans, with no required report or numerical effect estimate. Each planning reply offers one small answer, choice or clarification, selected for its usefulness to the researcher's goal. Actors may return to relevant earlier questions, retain unanswered requests, and preserve uncertainty about unknown study facts. All five standalone cases have fixed user roles and do not accept --persona; see the standalone guide.

Setup and running

Requires shared Python 3.10+, Node.js 18.18+, a compatible consultant folder, and a configured agent CLI: Claude Code, or Kimi/Pi through the supplied adapter. See runner setup for configuration and permissions. Keep tester materials outside the consultant's permitted access under the host contract.

From the tester directory, list cases or compose one into a fresh private directory:

python scripts/compose_case.py --list
python scripts/compose_case.py --problem observational-did --persona standard --output /private/cases/run001
python scripts/compose_case.py --standalone masks-planning --output /private/cases/run002

Paths are examples; choose one case. Composition does not call a model. Then:

  1. Configure and prepare with preflight and start. Set the actual reply cap to 12 for standalone cases. start prepares the attempt without sending a user message.
  2. Run the conversation using the frozen opening, then a separate simulated user and repeated inspect/step calls. Each reply responds to the actual conversation. If the next step requires obtaining information outside the conversation, pause after the consultant responds to that deferral and keep the attempt resumable.
  3. Record issues and export the diagnostic snapshot, private issues.md and raw evidence with export and check-package. The tester supplies no final grade; a complete evidence package does not itself establish scientific validity.

Further details

Historical attempts retain their original cases, versions and rules. Local fixture checks do not establish live host or model performance.

Ver o original no GitHub

Bom para

Relacionados