companion-inc/feynman — Repo Appraisal

Overview

What it is — Feynman is an open-source, terminal-based AI research agent that searches papers, conducts literature reviews, audits paper-vs-codebase consistency, replicates ML experiments, and generates structured research briefs from a single CLI.
Problem — ML research workflows are fragmented across disconnected tools for paper search, experiment orchestration, and cloud compute; Feynman unifies them under one multi-agent CLI with a shared session context.
Who it's for — ML researchers and practitioners who want an agent-driven research assistant with deep alphaXiv and Hugging Face integration and the ability to run or replicate experiments on local or cloud GPUs.
Notable — Built on the Pi agent runtime with four specialized sub-agents (Researcher, Reviewer, Writer, Verifier) dispatched automatically, supports local models via LM Studio/Ollama/vLLM, and ships a skills-only install path that drops directly into Claude/Codex agent skill directories.

Verdict

	Rating	Summary
Quality	excellent (20/24)	Polished, actively maintained CLI with strong adoption (7K+ stars, 898 forks), comprehensive documentation, and clean TypeScript engineering — held back only by a missing CI badge and an empty GitHub description.
PAI Relevance	integrate (0.63)	Fills a real gap in PAI's research toolchain (experiment replication, paper auditing, GPU compute) and its TypeScript CLI with a skills-only install path is structurally drop-in compatible with PAI's skill model.

Quality Assessment

20/24 — actively-maintained / adequately-documented / high-discipline

Health: 7/8 (actively-maintained)

Failed:

H8: FAIL — No CI badge in README and no explicit reference to .github/workflows/; contributing section shows npm test and npm run typecheck commands but no automated CI pipeline is documented.

Passed:

H1: PASS — Tagged release v0.2.58 exists.
H2: PASS — Latest release dated 2026-05-17, well within 12 months.
H3: PASS — Last commit 2026-05-17, within 6 months.
H4: PASS — Last commit 2026-05-17, 10 days before appraisal date.
H5: PASS — archived: false.
H6: PASS — 4 open issues; healthy triage signal, well under 100.
H7: PASS — MIT license explicitly declared in repo and package.json.

Documentation: 6/8 (adequately-documented)

Failed:

D5: FAIL — No heading matching API, Configuration, Options, Reference, Commands, or Parameters; "Workflows" and "Skills & Tools" are the closest sections but neither uses the probe keywords verbatim.
D8: FAIL — No Limitations, Caveats, Known Issues, or Trade-offs section anywhere in the README.

Passed:

D1: PASS — README is present and non-empty with substantial content.
D2: PASS — README is well over 1000 bytes with install instructions, command tables, agent descriptions, and how-it-works prose.
D3: PASS — Detailed install instructions for macOS, Linux, and Windows via curl and PowerShell one-liners, including version pinning and uninstall guidance.
D4: PASS — Multiple annotated code blocks under usage heading (e.g., feynman "what do we know about scaling laws" with inline explanations of what happens).
D6: PASS — "The open source AI research agent." appears in the first paragraph, immediately below the hero image.
D7: PASS — Links to https://feynman.is/docs externally; also links to RELEASES.md and Docs badge.

Engineering Signals: 7/8 (high-discipline)

Failed:

E8: FAIL — GitHub description is "No description" — empty and meaningless; the package.json description ("Research-first CLI agent built on Pi and alphaXiv") exists but doesn't propagate to the GitHub field.

Passed:

E1: PASS — Primary language is TypeScript.
E2: PASS — Full package.json present with version, bin, scripts, engines, files, and dependency declarations.
E3: PASS — Only 6 direct runtime dependencies (@clack/prompts, @companion-ai/alpha-hub, @mariozechner/pi-ai, @mariozechner/pi-coding-agent, @sinclair/typebox, dotenv), well under 15 for a CLI tool.
E4: PASS — Test script defined: node --import tsx --test --test-concurrency=1 tests/*.test.ts; typecheck script also present confirming active type discipline.
E5: PASS — 7263 stars, far above the 50-star threshold.
E6: PASS — ~69 days from creation to appraisal date yields ~3157 stars/month, far above the 2/month threshold.
E7: PASS — 898 forks, far above the 5-fork threshold.

PAI Relevance

Dimension	Score	Assessment
Harvest Value	1	The role-segmented multi-agent dispatch (Researcher/Reviewer/Writer/Verifier with automatic routing) and the skills-only install pattern (`--repo` flag writing into `.agents/skills/feynman`) are design patterns worth studying for PAI's Agents and Delegation subsystems; PAI's Research skill covers web investigation but lacks this structured role decomposition and per-agent verification pass.
Integration Readiness	2	Pure TypeScript MIT CLI, Bun-compatible; the skills-only installer drops directly into `.agents/skills/feynman` with Feynman's skill/prompt/extension tree — PAI's skill model is structurally identical to what Feynman targets with its Pi-based skill packaging, making selective skill adoption feasible without the full terminal bundle.
Overlap Risk	1	Partial overlap with PAI's Research skill (web investigation + vault) and ArXiv skill (academic paper search); Feynman's experiment replication pipeline, paper-vs-codebase auditing, peer review simulation, and GPU compute integrations (Modal, RunPod) are absent from PAI's Capability Manifest.
Gap Fill	1	Addresses limited-coverage areas in PAI — ML experiment replication, paper auditing against public codebases, simulated peer review with severity grading — though PAI's Research and ArXiv skills already cover the core research-and-retrieval workflow.

Composite: 0.63

What Next

Capture-to-Knowledge Pipeline, manual validation stage: Run feynman review <arxiv-url> as the structured-extraction step on each paper entering the pipeline, replacing the current manual Haiku validation pass — Feynman's Reviewer sub-agent outputs a brief with explicit claims, methods, limitations, and reproducibility notes, turning a bottleneck that requires human review time into a single CLI invocation with consistent output schema.
PAI agent setup, Claude skill directory: Run Feynman's skills-only install to drop feynman-search, feynman-review, and feynman-brief directly into the Claude skill directory — Claude agents operating inside existing task loops can invoke paper search and literature review as native skills without spawning a separate session or losing task context, making research lookups a first-class capability rather than a context switch.
fab recommender, research-domain content: Pipe feynman brief <topic> output into fab as the content argument when the intent is pattern selection for research-adjacent material — the structured brief format (problem statement, methods, findings, limitations) gives fab a richer, more semantically consistent input signal than raw paper text or unstructured notes, improving pattern match quality for the research and analysis end of the content spectrum.

Landscape Position

Category: AI Research & Papers

In this category: karpathy--autoresearch (decent, 14/24, watch) — the only prior occupant; feynman is the first repo formally assigned this category_id in this appraisal run per crowding metadata, but karpathy/autoresearch appears in the rolling summary under the same category.

Standing: Feynman significantly outclasses karpathy/autoresearch on every dimension — quality (20 vs 14), maintenance cadence (58 versioned releases vs no releases), feature breadth (full multi-agent pipeline vs single training-loop prototype), and adoption (7K+ stars vs early-stage); the two repos share the autonomous-research-agent idea but are not comparable in scope or maturity.

Evidence Base

Density: 8/10 — Available: full README (8KB), complete package.json with dependency manifest, GitHub metadata (stars, forks, issues, dates, license, release history). Missing: source code structure, CI configuration files (.github/workflows/), test suite content and coverage, CONTRIBUTING.md body.

Notes

Despite the empty GitHub description field (penalizing E8 only), this is the highest-quality research-agent CLI appraised in this landscape run; the cosmetic oversight does not reflect the project's actual maturity.
The skills-only install path (curl | bash -s -- --repo → .agents/skills/feynman) mirrors PAI's skill directory structure precisely, making selective skill adoption a low-friction integration option that bypasses the full terminal runtime.
Star velocity (~3157/month over ~69 days) is exceptional; the 898 forks suggest active downstream usage and derivative development rather than passive star accumulation.
The v0.2.x series with 58 patch increments in ~2 months indicates fast iteration with disciplined release tagging — a notably healthy cadence for a project this young.
The overrides block in package.json (patching hono, express, path-to-regexp, protobufjs, etc.) signals proactive supply-chain hygiene, which is a positive engineering signal beyond what the probe rubric captures.