Skill

llm-guidelines is an Agent Skill bundle that packages these guidelines for authors to use directly inside an AI coding assistant. It bundles the same content this website renders (scope, study types, guidelines, checklist) and exposes two slash commands: one for exploring the guidelines (asking questions, discussing study designs, deciding which study type fits) and one for reviewing a paper draft (and any supplementary material) against the eight guidelines, producing an llm-guidelines-report.md for the paper at hand.

Agent Skills is an open standard originally developed by Anthropic. The format is now read by Cursor, GitHub Copilot, OpenAI Codex, Gemini CLI, Claude Code, and JetBrains Junie, among others; see the client list for the current set. The instructions below cover Claude Code; for any other client, copy the plugins/llm-guidelines/ subtree from the repository into the location that client expects, preserving the relative paths between skills/<mode>/SKILL.md and shared/ (each client’s docs are linked from the client list).

The skill is published in a separate repository at se-uhd/llm-guidelines-skill. Its version reuses the guideline YYYY.MM tag shown in the header of this site for releases that match a paper version, and adds a -revN suffix for skill-only updates against the same guideline version (see the skill repo README.md for details).

Install

/plugin marketplace add se-uhd/llm-guidelines-skill
/plugin install llm-guidelines

Invoke

The bundle exposes two slash commands.

Explore the guidelines

Use this when you want to ask a question, plan a study, walk through the checklist, or decide which study type your project falls under, before any draft exists.

/llm-guidelines:explore

Explore mode is a conversation; it does not write any files unless you ask.

Review a draft

Use this when you have a draft paper (and optionally supplementary material) and want a structured pass against each of the eight guidelines.

With a LaTeX source:

/llm-guidelines:review path/to/paper.tex

Or with a PDF:

/llm-guidelines:review path/to/paper.pdf

The supplementary-material path is optional, and you can pass more than one when artifacts live in separate locations (e.g., code on GitHub, data on Zenodo):

/llm-guidelines:review path/to/paper.tex path/to/code-repo path/to/dataset

If invoked without arguments, review mode asks for paths to your paper (LaTeX source or PDF) and any supplementary directories. It will not modify your files; it writes its findings to llm-guidelines-report.md in your working directory and prints the same content to the console.

Auto-invocation

Each skill can also load itself without an explicit slash command if the agent decides its description matches the task. Mention a paper path or ask to “check” or “review” a draft and review mode loads; ask a question about the guidelines, the study-type taxonomy, or your study design and explore mode loads.

Bundled Files

The bundle ships a structured set of Markdown files. The first column links to the rich rendering on this website (where one exists); the second column links to the raw Markdown file in the skill repository.

File	Markdown
Explore-mode skill instructions (not rendered here)	skills/explore/SKILL.md
Review-mode skill instructions (not rendered here)	skills/review/SKILL.md
Scope	shared/scope.md
Reporting Checklist	shared/checklist.md

Guidelines

Guideline	Markdown
Declare LLM Usage and Role	declare-llm-usage-and-role.md
Report Model Version, Configuration, and Customizations	report-model-version-configuration-and-customizations.md
Report System and Prompt Design	report-system-and-prompt-design.md
Report Session Traces	report-session-traces.md
Use Suitable Baselines, Benchmarks, and Metrics	use-suitable-baselines-benchmarks-and-metrics.md
Use an Open LLM as a Baseline	use-an-open-llm-as-a-baseline.md
Use Human Validation for LLM Outputs	use-human-validation-for-llm-outputs.md
Report Limitations and Mitigations	report-limitations-and-mitigations.md

Study Types

Study Type	Markdown
LLMs as Tools for Software Engineering Researchers	llms-as-tools-for-software-engineering-researchers.md
LLMs as Annotators	llms-as-annotators.md
LLMs as Judges	llms-as-judges.md
LLMs for Synthesis	llms-for-synthesis.md
LLMs as Subjects	llms-as-subjects.md
Advantages and Challenges	advantages-and-challenges.md
LLMs as Tools for Software Engineers	llms-as-tools-for-software-engineers.md
Studying LLM Usage in Software Engineering	studying-llm-usage-in-software-engineering.md
LLMs for New Software Engineering Tools	llms-for-new-software-engineering-tools.md
Benchmarking LLMs for Software Engineering Tasks	benchmarking-llms-for-software-engineering-tasks.md