日本語版はこちら

🔎 Latest CI reports: JUnit summary / Flaky ranking / Coverage HTML

Demos

01

Spec to Cases

A pipeline that turns specification Markdown into test case JSON with LLM drafting and rule-based post-processing.

• Schema validation and type-preserving transformation logic.
• CLI helpers and JSON samples for quick onboarding.

Evidence →

02

LLM to Playwright

A proof of concept where an LLM expands acceptance criteria and generates Playwright tests automatically.

• Robust selector strategy based on data-testid with a11y scanning baked in.
• Data-driven tests via JSON / CSV drivers for a minimal setup.

Evidence →

03

CI Flaky Analyzer

A CLI that detects flaky tests from CI logs and automatically produces HTML reports and ticket templates.

• Streaming analysis for large JUnit XML files with scoring stored as JSONL / HTML.
• One-command generation of HTML reports, JSONL history, and GitHub Issue templates.

Evidence →

04

LLM Adapter — Shadow Execution

An adapter that runs a shadow provider alongside the primary LLM to capture anomalies while keeping the main response.

• Collects shadow diff metrics as JSONL for vendor comparison.
• Mocks timeouts, rate limits, and malformed responses to validate fallbacks.

Evidence →

Weekly Summary

Weekly QA Snapshot — 2025-09-23

• TotalTests: 126
• PassRate: 100.00%
• NewDefects: 0
• TopFailureKinds: -

週次サマリを詳しく読む →

01. Spec to Cases

• Minimal pipeline that generates test cases from specification Markdown.
• Deliverable: cases.sample.json
• Extra materials: spec.sample.md

02. LLM to Playwright

• PoC that enriches acceptance criteria with an LLM and auto-generates Playwright tests.
• Deliverables: tests/generated/
• Samples: blueprint.sample.json / demo/
• References: tests/README.md

03. CI Flaky Analyzer

• Detects flaky tests from CI logs and enables re-runs and automated ticketing end-to-end.
• Deliverable: Running npx flaky analyze generates projects/03-ci-flaky/out/index.html (HTML/CSV/JSON) retrievable from CI artifacts.
• Sample workflow: Ingest any JUnit XML via npx flaky parse --input <path-to-xml> and store the history for later analysis.

04. LLM Adapter — Shadow Execution

• Runs a shadow provider in parallel, records response diffs as JSONL, and visualizes the findings.
• Recreates timeout, rate limit, and malformed responses with mocks to verify fallback design.
• Reference: evidence/llm-adapter

View all weekly summaries →

Evidence Library

• QA Evidence Catalog
• Test Plan
• Defect Report Sample

Operations Notes

• The weekly-qa-summary.yml workflow automatically updates docs/weekly-summary.md.
• tools/generate_gallery_snippets.py generates highlight cards from the weekly summary.
• .github/workflows/pages.yml deploys everything under docs/ to GitHub Pages.