mcp-hr-knowledge-evals

An enterprise-style Model Context Protocol server in TypeScript that exposes structured HR policy tools, RAG-style retrieval with citations, and an automated evaluation harness benchmarked against 20 synthetic HR scenarios.

GitHub ↗ Live demo ↗ npm ↗ TypeScript · strict stdio transport 20 eval cases

What it is

MCP servers give LLM clients (Claude Desktop, IDEs, agents) a typed tool surface. This one models a realistic internal tool: an HR knowledge base that answers policy questions with citations, exposes onboarding milestones, and generates manager coaching rubrics — plus the eval harness you'd build to prove it doesn't hallucinate.

MCP client (Claude / IDE) │ tools/call · prompts/get · resources/read (JSON-RPC over stdio) ▼ hr-knowledge-mcp-server ├── tools query_hr_policy · get_onboarding_milestones · fetch_manager_coaching_rubric ├── prompts conduct_manager_1on1 ├── resources hr://policies/<id> (markdown, citation ids) └── evals 20 scenarios → tool-selection accuracy · retrieval precision · hallucination rate

Tools

query_hr_policy

Keyword/semantic retrieval over the HR handbook. Returns top matching policy documents with full section text and citation ids.

param	type	req	notes
`query`	string	yes	natural-language HR question
`department`	`"HR" \| "Engineering" \| "Sales"`	no	scope filter

get_onboarding_milestones

Role-family-specific (Engineering / Sales / HR / General) 30-60-90 day expectations and completion criteria.

param	type	req	notes
`role`	string	yes	e.g. `"backend engineer"`
`tenure_days`	`30 \| 60 \| 90`	yes	onboarding checkpoint

fetch_manager_coaching_rubric

Synthesizes a 1-on-1 agenda, coaching prompts, and SBI feedback guidance keyed to scenario topic and rating.

param	type	req	notes
`scenario`	string	yes	e.g. `"missed two sprint deadlines"`
`performance_rating`	`"Exceeds" \| "Meets" \| "Needs_Improvement"`	yes	rating tier

Prompts & resources

Prompt: conduct_manager_1on1 formats employee name, rating, and scenario into a policy-grounded system prompt. Resources: every policy is readable at hr://policies/<id> (e.g. hr://policies/parental-leave) as markdown with its citation id.

Eval harness

npm run eval runs 20 synthetic scenarios through the same dispatch path as tools/call and scores each output against ground-truth facts, expected citations, and forbidden-content rules. With ANTHROPIC_API_KEY set, a Claude-as-judge faithfulness verdict is added per case.

----------------------------------------------------
  HR KNOWLEDGE EVAL BENCHMARK          (verified)
----------------------------------------------------
  Cases evaluated .......... 20
  Judge .................... deterministic
  Tool selection accuracy .. 100.0%
  Policy retrieval precision 100.0%
  Hallucination rate ....... 0.0%  (goal 0.0%)
----------------------------------------------------

metric	what it measures
Tool selection accuracy	heuristic router picks the expected tool for each scenario
Policy retrieval precision	fraction of ground-truth facts present in tool output
Hallucination rate	forbidden content, fabricated citation ids, or missing required citations

Use it

From npm

npm install -g mcp-hr-knowledge-evals

Then register it with an MCP client, e.g. Claude Desktop claude_desktop_config.json:

{
  "mcpServers": {
    "hr-knowledge": {
      "command": "mcp-hr-knowledge-evals"
    }
  }
}

From source

git clone https://github.com/mellowMike92/mcp-hr-knowledge-evals
cd mcp-hr-knowledge-evals
npm install
npm run build
npm run eval       # 20-scenario benchmark
npm run inspector  # interactive tool explorer
npm run web        # local live demo on :8000

Layout

src/
  index.ts            entry — StdioServerTransport (bin entry for npm)
  server.ts           MCP server: tools, prompts, resources, dispatchTool
  tools/hrTools.ts    tool implementations (pure functions)
  data/hrPolicies.ts  mock HR handbook with citation ids
  types/index.ts      interfaces + zod schemas
evals/                20 scenarios + deterministic scorer + Claude judge
web/                  zero-dep HTTP demo (server.mjs + public/)
docs/                 this site