Why Adding Open‑Source Agent Skills Makes HCLS AI More Reliable
Lead
AI agents built on large language models often appear to follow healthcare and life‑sciences (HCLS) guidelines, yet they miss key decision steps. A new collection of 38 open‑source agent skills claims to close that gap, delivering measurable gains in variant interpretation, drug discovery, and imaging workflows.
How the skills work
Agent skills are markdown files ( SKILL.md ) that encode a domain’s decision procedure in a structured way. Each file starts with a YAML front‑matter block that lists triggers (the phrases that should activate the skill), dependencies, and metadata. The body then contains the actual framework – for example, the full ACMG/AMP criteria for classifying a TP53 missense variant, complete with evidence categories, population‑frequency cut‑offs, and predictor score thresholds.
When an agent receives a query, it scans the prompt for trigger patterns. If a match is found, the skill is injected into the model’s context, effectively “teaching” the model the exact steps it should follow before it starts generating text. This is different from Retrieval‑Augmented Generation (RAG), which merely adds raw document excerpts, and from fine‑tuning, which changes the model weights. Skills are pure, human‑readable prompts that can be swapped in or out without retraining.
Reasoning vs. pipeline skills
The catalog splits the 38 items into two families:
| Skill family | What it provides | Typical token load |
|---|---|---|
| Reasoning skills | Decision frameworks, evidence hierarchies, validation criteria | ~15 K tokens per specialist when grouped |
| Pipeline skills | Command‑line snippets, parameter tables, code templates (e.g., GATK4 HaplotypeCaller) | ~5 K tokens per skill |
Reasoning skills give the agent judgment – the “why” behind a choice – while pipeline skills give the how – concrete commands that can be run immediately.
Real‑world impact measured in tests
In head‑to‑head evaluations, agents equipped with the skills beat the same agents without them 70 %–86 % of the time, depending on the harness used. The strongest improvement appears in “critical thinking” tasks, where the win rate climbs to 78 %–85 % with a Cohen’s d (effect size) of 0.65–1.03. Those numbers come from a benchmark set that includes variant classification, drug‑repurposing, and imaging analysis.
The win rates translate to fewer silent failures. For example, an agent that previously mis‑applied ACMG/AMP evidence now follows the exact threshold tables, eliminating the hallucinated predictor scores that could have led to a wrong clinical interpretation.
What the win rates really mean for everyday users
A 70 %–86 % advantage sounds impressive, but the trade‑off is token consumption and orchestration complexity. Loading all 38 skills into a single agent uses roughly 80 K tokens – manageable for large‑context models but costly in latency and API spend. The multi‑agent pattern solves this by splitting the skill set across eight specialist agents, each holding about 15 K tokens. A lightweight coordinator routes the user’s query to the appropriate specialist based on intent classification.
For a typical lab or biotech team, the decision comes down to three factors:
- Cost vs. accuracy – If you already pay per‑token, the multi‑agent setup saves money while preserving most of the accuracy gain.
- Operational overhead – Running a coordinator and several specialists adds deployment steps, but the AWS Strands SDK and Bedrock AgentCore automate much of the wiring.
- Maintainability – Because skills are plain markdown, updating a guideline (e.g., a new ACMG frequency threshold) is a matter of editing a text file, not retraining a model.
In practice, most users will start with a single‑agent Quick Desktop trial to gauge the improvement, then migrate to a multi‑agent architecture if token cost becomes a blocker.
Try it today: a step‑by‑step starter
- Clone the repository
git clone https://github.com/awslabs/hcls-agent-skills.git cd hcls-agent-skills - Install a single skill (e.g., the genomics variant‑interpretation skill) using the universal CLI:
npx skills add awslabs/hcls-agent-skills --skill genomics/variant-interpretation - Run a quick test in your preferred agent harness. With Bedrock’s Python SDK it looks like:
from strands import Agent from strands.skills import AgentSkills agent = Agent(model="anthropic.claude-v2", skills=AgentSkills(skills="./skills/genomics/")) response = agent("Classify NM_000546.6:c.743G>A in TP53 using ACMG criteria") print(response) - Compare the output to a baseline call without the skill (just the model). Note whether the response includes the correct population‑frequency cut‑offs and evidence categories.
- Iterate – if the skill improves the answer, consider adding more skills or moving to the multi‑agent Kiro CLI setup for larger workloads.
By the end of the day you’ll have a concrete sense of how much the structured reasoning changes the answer and whether the token overhead fits your budget.
