Public LLM Datasets
LLM interpretability · Activation steering
Contrast datasets — minimal pairs that isolate one behavioral axis while holding meaning fixed — for finding LLM steering directions and evaluating control.
A prompt, two answers — verbose and concise; each answer's last-token embedding flies into space.
Concise and verbose, in a 768-dimensional embedding space.
mean(verbose) − mean(concise).
Run GPT-2 normally, then open two blocks and add the vector — the output turns concise, weights frozen.
from datasets import load_dataset pairs = load_dataset("PhillipsLab/conciseness_verbosity_contrast")
Public LLM Datasets
Contrast datasets — minimal pairs that isolate one behavioral axis while holding meaning fixed — for finding LLM steering directions and evaluating control.
Contrast pairs become a steering direction — and the model follows it.
Conciseness ↔ Verbosity Positivity ↔ Negativity Formal ↔ Informal Read our paper
I create and release contrast datasets — minimal pairs that isolate one behavioral axis while holding meaning fixed — for finding LLM steering directions and evaluating control. The conciseness and positivity sets were built with Martian AI. Three are currently public on Hugging Face under PhillipsLab (all CC BY 4.0) — the film above walks through how they work, and links each set at the end.
These pairs drive activation steering — residual-stream injections (layer/α sweeps) that steer behaviors like verbosity and tone in models such as Llama 3.1 and Qwen-2.5, with guardrailed LLM-judge + JSON-schema evaluation. For how corruption in steering datasets degrades that control — and how robust estimation mitigates it — see Understanding and Mitigating Dataset Corruption in LLM Steering (2026).