aiticker

Pack · #115 trending skills repo · ↗ +194 stars this week

AI-Research-SKILLs

Orchestra-Research/AI-Research-SKILLs

Comprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.

Research & science
Install ↓GitHub ↗Website ↗
Stars
14k
This week
+194
Forks
995
Open issues
24
Skills inside
98
Star trend · last 17 days+542 ★
Oct 21, 2026: 13,795 starsOct 22, 2026: 13,837 starsOct 23, 2026: 13,882 starsOct 24, 2026: 13,907 starsOct 25, 2026: 13,936 starsOct 26, 2026: 13,977 starsOct 27, 2026: 14,009 starsOct 28, 2026: 14,053 starsOct 29, 2026: 14,103 starsOct 30, 2026: 14,124 starsOct 31, 2026: 14,143 starsNov 1, 2026: 14,172 starsNov 2, 2026: 14,203 starsNov 3, 2026: 14,233 starsNov 4, 2026: 14,269 starsNov 5, 2026: 14,297 starsNov 6, 2026: 14,317 starsNov 7, 2026: 14,337 stars

Get started

Install the skills

npx skills add Orchestra-Research/AI-Research-SKILLs

Installs into Claude Code, Codex, Cursor, OpenCode and the other agents it finds on your machine.

Inside this repo

48 of 98 skills

autoresearchOrchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for executi0-autoresearch-skillimplementing-llms-litgptImplements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction l01-model-architecture/litgptmamba-architectureState-space model with O(n) complexity vs Transformers' O(n²). 5× faster inference, million-token sequences, no KV cache. Selective SSM with hardware-aware design. Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head). Models 130M-2.8B on HuggingFace.01-model-architecture/mambananogptEducational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).01-model-architecture/nanogptrwkv-architectureRNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.01-model-architecture/rwkvdistributed-llm-pretraining-torchtitanProvides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.01-model-architecture/torchtitanhuggingface-tokenizersFast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-perf02-tokenization/huggingface-tokenizerssentencepieceLanguage-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK la02-tokenization/sentencepieceaxolotlExpert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support03-fine-tuning/axolotlllama-factoryExpert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support03-fine-tuning/llama-factorypeft-fine-tuningParameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transform03-fine-tuning/peftunslothExpert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization03-fine-tuning/unslothnnsight-remote-interpretabilityProvides guidance for interpreting and manipulating neural network internals using nnsight with optional NDIF remote execution. Use when needing to run interpretability experiments on massive models (70B+) without local GPU resources, or when working with any PyTorch architecture.04-mechanistic-interpretability/nnsightpyvene-interventionsProvides guidance for performing causal interventions on PyTorch models using pyvene's declarative intervention framework. Use when conducting causal tracing, activation patching, interchange intervention training, or testing causal hypotheses about model behavior.04-mechanistic-interpretability/pyvenesparse-autoencoder-trainingProvides guidance for training and analyzing Sparse Autoencoders (SAEs) using SAELens to decompose neural network activations into interpretable features. Use when discovering interpretable features, analyzing superposition, or studying monosemantic representations in language models.04-mechanistic-interpretability/saelenstransformer-lens-interpretabilityProvides guidance for mechanistic interpretability research using TransformerLens to inspect and manipulate transformer internals via HookPoints and activation caching. Use when reverse-engineering model algorithms, studying attention patterns, or performing activation patching experiments.04-mechanistic-interpretability/transformer-lensnemo-curatorGPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cle05-data-processing/nemo-curatorray-dataScalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipe05-data-processing/ray-datagrpo-rl-trainingExpert guidance for GRPO/RL fine-tuning with TRL for reasoning and task-specific model training06-post-training/grpo-rl-trainingmiles-rl-trainingProvides guidance for enterprise-grade RL training using miles, a production-ready fork of slime. Use when training large MoE models with FP8/INT4, needing train-inference alignment, or requiring speculative RL for maximum throughput.06-post-training/milesopenrlhf-trainingHigh-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.06-post-training/openrlhfsimpo-trainingSimple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.06-post-training/simposlime-rl-trainingProvides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.06-post-training/slimetorchforge-rl-trainingProvides guidance for PyTorch-native agentic RL using torchforge, Meta's library separating infra from algorithms. Use when you want clean RL abstractions, easy algorithm experimentation, or scalable training with Monarch and TorchTitan.06-post-training/torchforgefine-tuning-with-trlFine-tune LLMs using reinforcement learning with TRL - SFT for instruction tuning, DPO for preference alignment, PPO/GRPO for reward optimization, and reward model training. Use when need RLHF, align model with preferences, or train from human feedback. Works with HuggingFace Transformers.06-post-training/trl-fine-tuningverl-rl-trainingProvides guidance for training LLMs with reinforcement learning using verl (Volcano Engine RL). Use when implementing RLHF, GRPO, PPO, or other RL algorithms for LLM post-training at scale with flexible infrastructure backends.06-post-training/verlconstitutional-aiAnthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.07-safety-alignment/constitutional-aillamaguardMeta's 7-8B specialized moderation model for LLM input/output filtering. 6 safety categories - violence/hate, sexual content, weapons, substances, self-harm, criminal planning. 94-95% accuracy. Deploy with vLLM, HuggingFace, Sagemaker. Integrates with NeMo Guardrails.07-safety-alignment/llamaguardnemo-guardrailsNVIDIA's runtime safety framework for LLM applications. Features jailbreak detection, input/output validation, fact-checking, hallucination detection, PII filtering, toxicity detection. Uses Colang 2.0 DSL for programmable rails. Production-ready, runs on T4 GPU.07-safety-alignment/nemo-guardrailsprompt-guardMeta's 86M prompt injection and jailbreak detector. Filters malicious prompts and third-party data for LLM apps. 99%+ TPR, <1% FPR. Fast (<2ms GPU). Multilingual (8 languages). Deploy with HuggingFace or batch processing for RAG security.07-safety-alignment/prompt-guardhuggingface-accelerateSimplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.08-distributed-training/acceleratedeepspeedExpert guidance for distributed training with DeepSpeed - ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, sparse attention08-distributed-training/deepspeedtraining-llms-megatronTrains large language models (2B-462B parameters) using NVIDIA Megatron-Core with advanced parallelism strategies. Use when training models >1B parameters, need maximum GPU efficiency (47% MFU on H100), or require tensor/pipeline/sequence/context/expert parallelism. Production-ready framework used f08-distributed-training/megatron-corepytorch-fsdp2Adds PyTorch FSDP2 (fully_shard) to training scripts with correct init, sharding, mixed precision/offload config, and distributed checkpointing. Use when models exceed single-GPU memory or when you need DTensor-based sharding with DeviceMesh.08-distributed-training/pytorch-fsdp2pytorch-lightningHigh-level PyTorch framework with Trainer class, automatic distributed training (DDP/FSDP/DeepSpeed), callbacks system, and minimal boilerplate. Scales from laptop to supercomputer with same code. Use when you want clean training loops with built-in best practices.08-distributed-training/pytorch-lightningray-trainDistributed training orchestration across clusters. Scales PyTorch/TensorFlow/HuggingFace from laptop to 1000s of nodes. Built-in hyperparameter tuning with Ray Tune, fault tolerance, elastic scaling. Use when training massive models across multiple machines or running distributed hyperparameter swe08-distributed-training/ray-trainlambda-labs-gpu-cloudReserved and on-demand GPU cloud instances for ML training and inference. Use when you need dedicated GPU instances with simple SSH access, persistent filesystems, or high-performance multi-node clusters for large-scale training.09-infrastructure/lambda-labsmodal-serverless-gpuServerless GPU cloud platform for running ML workloads. Use when you need on-demand GPU access without infrastructure management, deploying ML models as APIs, or running batch jobs with automatic scaling.09-infrastructure/modalskypilot-multi-cloud-orchestrationMulti-cloud orchestration for ML workloads with automatic cost optimization. Use when you need to run training or batch jobs across multiple clouds, leverage spot instances with auto-recovery, or optimize GPU costs across providers.09-infrastructure/skypilotawq-quantizationActivation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 10-optimization/awqquantizing-models-bitsandbytesQuantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.10-optimization/bitsandbytesoptimizing-attention-flashOptimizes transformer attention with Flash Attention for 2-4x speedup and 10-20x memory reduction. Use when training/running transformers with long sequences (>512 tokens), encountering GPU memory issues with attention, or need faster inference. Supports PyTorch native SDPA, flash-attn library, H10010-optimization/flash-attentiongguf-quantizationGGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.10-optimization/ggufgptqPost-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-t10-optimization/gptqhqq-quantizationHalf-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.10-optimization/hqqml-training-recipesBattle-tested PyTorch training recipes for all domains — LLMs, vision, diffusion, medical imaging, protein/drug discovery, spatial omics, genomics. Covers training loops, optimizer selection (AdamW, Muon), LR scheduling, mixed precision, debugging, and systematic experimentation. Use when training o10-optimization/ml-training-recipesevaluating-code-modelsEvaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace 11-evaluation/bigcode-evaluation-harnessevaluating-llms-harnessEvaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vL11-evaluation/lm-evaluation-harness

Documentation

README

View on GitHub ↗

AI Research Skills Library

The most comprehensive open-source skills library enabling AI agents to autonomously conduct AI research — from idea to paper

AI Research Skills Demo

License: MIT npm version Blog Post Slack Twitter LinkedIn

98 Skills Powering AI Research in 2026

View All 23 Categories
Autoresearch (1) Ideation (2) ML Paper Writing (2)
Model Architecture (5) Fine-Tuning (4) Post-Training (8)
Distributed Training (6) Optimization (6) Inference (4)
Tokenization (2) Data Processing (2) Evaluation (3)
Safety & Alignment (4) Agents (4) RAG (5)
Multimodal (7) Prompt Engineering (4) MLOps (3)
Observability (2) Infrastructure (3) Mech Interp (4)
Emerging Techniques (6) Agent-Native Research Artifact (3)

Table of Contents

Our Mission

We enable AI agents to autonomously conduct AI research — from literature survey and idea generation through experiment execution to paper writing. The library provides both the research orchestration layer (autoresearch, ideation, paper writing) and the engineering skills (training, evaluation, deployment) needed at each stage.

AI Research Agent System
System diagram of an AI research agent

Path Towards AI Research Agent

Modern AI research requires mastering dozens of specialized tools and frameworks. AI Researchers spend more time debugging infrastructure than testing hypotheses — slowing the pace of scientific discovery. We provide a comprehensive skills library that enables AI agents to autonomously conduct the full research lifecycle — from brainstorming ideas to writing the paper.

  • Autonomous Research - The autoresearch skill orchestrates the entire research workflow using a two-loop architecture, routing to domain skills as needed
  • Specialized Expertise - Each domain skill provides deep, production-ready knowledge of a specific framework (Megatron-LM, vLLM, TRL, etc.)
  • End-to-End Coverage - 98 skills spanning the full AI research lifecycle, from ideation and literature survey to experiments and paper writing
  • Research-Grade Quality - Documentation sourced from official repos, real GitHub issues, and battle-tested production workflows

Available AI Research Engineering Skills

Quality over quantity: Each skill provides comprehensive, expert-level guidance with real code examples, troubleshooting guides, and production-ready workflows.

For humans — interactive installer with one command:

npx @orchestra-research/ai-research-skills

For AI agents — point your agent to the welcome doc and it handles the rest:

Read https://www.orchestra-research.com/ai-research-skills/welcome.md and follow the instructions to install and use AI Research Skills.

This installs all 98 skills, loads the autoresearch orchestration layer, and starts autonomous research.

What the installer does
  • Auto-detects your installed coding agents (Claude Code, Hermes Agent, OpenCode, Qoder, Cursor, Gemini CLI, etc.)
  • Installs skills to ~/.orchestra/skills/ with symlinks to each agent (falls back to copy on Windows)
  • Offers everything, quickstart bundle, by category, or individual skills
  • Updates installed skills with latest versions
  • Uninstalls all or selected skills
CLI Commands
# Interactive installer (recommended)
npx @orchestra-research/ai-research-skills

# Direct commands
npx @orchestra-research/ai-research-skills list      # View installed skills
npx @orchestra-research/ai-research-skills update    # Update installed skills
Claude Code Marketplace (Alternative)

Install skill categories directly using the Claude Code CLI:

# Add the marketplace
/plugin marketplace add orchestra-research/AI-research-SKILLs

# Install by category (23 categories available)
/plugin install fine-tuning@ai-research-skills        # Axolotl, LLaMA-Factory, PEFT, Unsloth
/plugin install post-training@ai-research-skills      # TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge
/plugin install inference-serving@ai-research-skills  # vLLM, TensorRT-LLM, llama.cpp, SGLang
/plugin install distributed-training@ai-research-skills
/plugin install optimization@ai-research-skills

All 23 Categories (98 Skills)

Category Skills Included
Autoresearch 1 Autonomous research orchestration — central layer that manages the full lifecycle and routes to all other skills
Ideation 2 Research Brainstorming, Creative Thinking
ML Paper Writing 2 ML Paper Writing (LaTeX templates, citation verification), Academic Plotting
Model Architecture 5 LitGPT, Mamba, NanoGPT, RWKV, TorchTitan
Tokenization 2 HuggingFace Tokenizers, SentencePiece
Fine-Tuning 4 Axolotl, LLaMA-Factory, PEFT, Unsloth
Mech Interp 4 TransformerLens, SAELens, pyvene, nnsight
Data Processing 2 NeMo Curator, Ray Data
Post-Training 8 TRL, GRPO, OpenRLHF, SimPO, verl, slime, miles, torchforge
Safety 4 Constitutional AI, LlamaGuard, NeMo Guardrails, Prompt Guard
Distributed 6 DeepSpeed, FSDP, Accelerate, Megatron-Core, Lightning, Ray Train
Infrastructure 3 Modal, Lambda Labs, SkyPilot
Optimization 6 Flash Attention, bitsandbytes, GPTQ, AWQ, HQQ, GGUF
Evaluation 3 lm-eval-harness, BigCode, NeMo Evaluator
Inference 4 vLLM, TensorRT-LLM, llama.cpp, SGLang
MLOps 3 W&B, MLflow, TensorBoard
Agents 4 LangChain, LlamaIndex, CrewAI, AutoGPT
RAG 5 Chroma, FAISS, Pinecone, Qdrant, Sentence Transformers
Prompt Eng 4 DSPy, Instructor, Guidance, Outlines
Observability 2 LangSmith, Phoenix
Multimodal 7 CLIP, Whisper, LLaVA, BLIP-2, SAM, Stable Diffusion, AudioCraft
Emerging 6 MoE, Model Merging, Long Context, Speculative Decoding, Distillation, Pruning
Agent-Native Research Artifact 3 ARA Compiler, Research Manager, Rigor Reviewer
View All 98 Skills in Details

🔬 Autoresearch (1 skill) — Central Orchestration Layer

  • Autoresearch - Autonomous research orchestration using a two-loop architecture (inner optimization + outer synthesis). Manages the full lifecycle from literature survey to paper writing, routing to all domain-specific skills. Supports Claude Code /loop and OpenClaw heartbeat for continuous operation (390 lines + 3 refs)

🏗️ Model Architecture (5 skills)

  • LitGPT - Lightning AI's 20+ clean LLM implementations with production training recipes (462 lines + 4 refs)
  • Mamba - State-space models with O(n) complexity, 5× faster than Transformers (253 lines + 3 refs)
  • RWKV - RNN+Transformer hybrid, infinite context, Linux Foundation project (253 lines + 3 refs)
  • NanoGPT - Educational GPT in ~300 lines by Karpathy (283 lines + 3 refs)
  • TorchTitan - PyTorch-native distributed training for Llama 3.1 with 4D parallelism

🔤 Tokenization (2 skills)

  • HuggingFace Tokenizers - Rust-based, <20s/GB, BPE/WordPiece/Unigram algorithms (486 lines + 4 refs)
  • SentencePiece - Language-independent, 50k sentences/sec, used by T5/ALBERT (228 lines + 2 refs)

🎯 Fine-Tuning (4 skills)

  • Axolotl - YAML-based fine-tuning with 100+ models (156 lines + 4 refs)
  • LLaMA-Factory - WebUI no-code fine-tuning (78 lines + 5 refs)
  • Unsloth - 2x faster QLoRA fine-tuning (75 lines + 4 refs)
  • PEFT - Parameter-efficient fine-tuning with LoRA, QLoRA, DoRA, 25+ methods (431 lines + 2 refs)

🔬 Mechanistic Interpretability (4 skills)

  • TransformerLens - Neel Nanda's library for mech interp with HookPoints, activation caching (346 lines + 3 refs)
  • SAELens - Sparse Autoencoder training and analysis for feature discovery (386 lines + 3 refs)
  • pyvene - Stanford's causal intervention library with declarative configs (473 lines + 3 refs)
  • nnsight - Remote interpretability via NDIF, run experiments on 70B+ models (436 lines + 3 refs)

📊 Data Processing (2 skills)

  • Ray Data - Distributed ML data processing, streaming execution, GPU support (318 lines + 2 refs)
  • NeMo Curator - GPU-accelerated data curation, 16× faster deduplication (375 lines + 2 refs)

🎓 Post-Training (8 skills)

  • TRL Fine-Tuning - Transformer Reinforcement Learning (447 lines + 4 refs)
  • GRPO-RL-Training (TRL) - Group Relative Policy Optimization with TRL (569 lines, gold standard)
  • OpenRLHF - Full RLHF pipeline with Ray + vLLM (241 lines + 4 refs)
  • SimPO - Simple Preference Optimization, no reference model needed (211 lines + 3 refs)
  • verl - ByteDance's HybridFlow RL framework, FSDP/Megatron + vLLM/SGLang backends (389 lines + 2 refs)
  • slime - THUDM's Megatron+SGLang framework powering GLM-4.x models (464 lines + 2 refs)
  • miles - Enterprise fork of slime with FP8, INT4, speculative RL for MoE training (315 lines + 2 refs)
  • torchforge - Meta's PyTorch-native RL with Monarch+TorchTitan+vLLM (380 lines + 2 refs)

🛡️ Safety & Alignment (4 skills)

  • Constitutional AI - AI-driven self-improvement via principles (282 lines)
  • LlamaGuard - Safety classifier for LLM inputs/outputs (329 lines)
  • NeMo Guardrails - Programmable guardrails with Colang (289 lines)
  • Prompt Guard - Meta's 86M prompt injection & jailbreak detector, 99%+ TPR, <2ms GPU (313 lines)

⚡ Distributed Training (6 skills)

  • Megatron-Core - NVIDIA's framework for training 2B-462B param models with 47% MFU on H100 (359 lines + 4 refs)
  • DeepSpeed - Microsoft's ZeRO optimization (137 lines + 9 refs)
  • PyTorch FSDP2 - Fully Sharded Data Parallel v2 with fully_shard and DTensor (231 lines + 12 refs)
  • Accelerate - HuggingFace's 4-line distributed training API (324 lines + 3 refs)
  • PyTorch Lightning - High-level training framework with Trainer class (339 lines + 3 refs)
  • Ray Train - Multi-node orchestration and hyperparameter tuning (399 lines + 1 ref)

🚀 Optimization (6 skills)

  • Flash Attention - 2-4x faster attention with memory efficiency (359 lines + 2 refs)
  • bitsandbytes - 8-bit/4-bit quantization for 50-75% memory reduction (403 lines + 3 refs)
  • GPTQ - 4-bit post-training quantization, 4× memory reduction, <2% accuracy loss (443 lines + 3 refs)
  • AWQ - Activation-aware weight quantization, 4-bit with minimal accuracy loss (310 lines + 2 refs)
  • HQQ - Half-Quadratic Quantization, no calibration data needed, multi-backend (370 lines + 2 refs)
  • GGUF - llama.cpp quantization format, K-quant methods, CPU/Metal inference (380 lines + 2 refs)

📊 Evaluation (3 skills)

  • lm-evaluation-harness - EleutherAI's standard for benchmarking LLMs across 60+ tasks (482 lines + 4 refs)
  • BigCode Evaluation Harness - Code model benchmarking with HumanEval, MBPP, MultiPL-E, pass@k metrics (406 lines + 3 refs)
  • NeMo Evaluator - NVIDIA's enterprise platform for 100+ benchmarks across 18+ harnesses with multi-backend execution (454 lines + 4 refs)

☁️ Infrastructure (3 skills)

  • Modal - Serverless GPU cloud with Python-native API, T4-H200 on-demand (342 lines + 2 refs)
  • SkyPilot - Multi-cloud orchestration across 20+ providers with spot recovery (390 lines + 2 refs)
  • Lambda Labs - Reserved/on-demand GPU cloud with H100/A100, persistent filesystems (390 lines + 2 refs)

🔥 Inference & Serving (4 skills)

  • vLLM - High-throughput LLM serving with PagedAttention (356 lines + 4 refs, production-ready)
  • TensorRT-LLM - NVIDIA's fastest inference, 24k tok/s, FP8/INT4 quantization (180 lines + 3 refs)
  • llama.cpp - CPU/Apple Silicon inference, GGUF quantization (251 lines + 3 refs)
  • SGLang - Structured generation with RadixAttention, 5-10× faster for agents (435 lines + 3 refs)

🤖 Agents (4 skills)

  • LangChain - Most popular agent framework, 500+ integrations, ReAct pattern (658 lines + 3 refs, production-ready)
  • LlamaIndex - Data framework for LLM apps, 300+ connectors, RAG-focused (535 lines + 3 refs)
  • CrewAI - Multi-agent orchestration, role-based collaboration, autonomous workflows (498 lines + 3 refs)
  • AutoGPT - Autonomous AI agent platform, visual workflow builder, continuous execution (400 lines + 2 refs)

🔍 RAG (5 skills)

  • Chroma - Open-source embedding database, local/cloud, 24k stars (385 lines + 1 ref)
  • FAISS - Facebook's similarity search, billion-scale, GPU acceleration (295 lines)
  • Sentence Transformers - 5000+ embedding models, multilingual, 15k stars (370 lines)
  • Pinecone - Managed vector database, auto-scaling, <100ms latency (410 lines)
  • Qdrant - High-performance vector search, Rust-powered, hybrid search with filtering (493 lines + 2 refs)

🎨 Multimodal (7 skills)

  • CLIP - OpenAI's vision-language model, zero-shot classification, 25k stars (320 lines)
  • Whisper - Robust speech recognition, 99 languages, 73k stars (395 lines)
  • LLaVA - Vision-language assistant, image chat, GPT-4V level (360 lines)
  • Stable Diffusion - Text-to-image generation via HuggingFace Diffusers, SDXL, ControlNet (380 lines + 2 refs)
  • Segment Anything - Meta's SAM for zero-shot image segmentation with points/boxes (500 lines + 2 refs)
  • BLIP-2 - Vision-language pretraining with Q-Former, image captioning, VQA (500 lines + 2 refs)
  • AudioCraft - Meta's MusicGen/AudioGen for text-to-music and text-to-sound (470 lines + 2 refs)

🎯 Prompt Engineering (4 skills)

  • DSPy - Declarative prompt programming with optimizers, Stanford NLP, 22k stars (438 lines + 3 refs)
  • Instructor - Structured LLM outputs with Pydantic validation, 15k stars (726 lines + 3 refs)
  • Guidance - Constrained generation with regex/grammars, Microsoft Research, 18k stars (485 lines + 3 refs)
  • Outlines - Structured text with FSM, zero-overhead, 8k stars (601 lines + 3 refs)

📊 MLOps (3 skills)

  • Weights & Biases - Experiment tracking, sweeps, artifacts, model registry (427 lines + 3 refs)
  • MLflow - Model registry, tracking, deployment, autologging (514 lines + 3 refs)
  • TensorBoard - Visualization, profiling, embeddings, scalars/images (538 lines + 3 refs)

👁️ Observability (2 skills)

  • LangSmith - LLM observability, tracing, evaluation, monitoring for AI apps (422 lines + 2 refs)
  • Phoenix - Open-source AI observability with OpenTelemetry tracing and LLM evaluation (380 lines + 2 refs)

🔬 Emerging Techniques (6 skills)

  • MoE Training - Mixture of Experts training with DeepSpeed, Mixtral 8x7B, 5× cost reduction (515 lines + 3 refs)
  • Model Merging - Combine models with TIES, DARE, SLERP using mergekit (528 lines + 3 refs)
  • Long Context - Extend context windows with RoPE, YaRN, ALiBi, 32k-128k tokens (624 lines + 3 refs)
  • Speculative Decoding - 1.5-3.6× faster inference with Medusa, Lookahead (379 lines)
  • Knowledge Distillation - Compress models 70B→7B with MiniLLM, temperature scaling (424 lines)
  • Model Pruning - 50% sparsity with Wanda, SparseGPT, <1% accuracy loss (417 lines)

📝 ML Paper Writing (2 skills)

  • ML Paper Writing - Write publication-ready papers for NeurIPS, ICML, ICLR, ACL, AAAI, COLM with LaTeX templates, citation verification, and writing best practices (532 lines + 5 refs)
  • Academic Plotting - Generate publication-quality figures for ML papers: architecture diagrams via Gemini AI and data-driven charts via matplotlib/seaborn with venue-specific styling (479 lines + 3 refs)

💡 Ideation (2 skills)

  • Research Brainstorming - Structured ideation frameworks for discovering high-impact research directions with 10 complementary lenses (384 lines)
  • Creative Thinking - Cognitive science frameworks (bisociation, structure-mapping, constraint manipulation) for genuinely novel research ideas (366 lines)

🧬 Agent-Native Research Artifact (3 skills)

  • ARA Compiler - Compiles any research input (PDF papers, repos, experiment logs, raw notes) into a complete Agent-Native Research Artifact with claims, exploration graph, evidence, and code stubs (245 lines + 3 refs)
  • ARA Research Manager - Post-task research recorder that runs at session end to extract decisions, experiments, dead ends, and pivots from conversation history into the ara/ directory with user-vs-AI provenance tags (324 lines + 3 refs)
  • ARA Rigor Reviewer - ARA Seal Level 2 semantic epistemic review scoring six dimensions of research rigor (evidence relevance, falsifiability, scope, coherence, exploration integrity, methodology) with severity-ranked findings (322 lines + 1 ref)

Demos

All 98 skills in this repo are automatically synced to Orchestra Research, where you can add them to your projects with one click and use them with AI research agents.

See skills in action → demos/

We maintain a curated collection of demo repositories showing how to use skills for real AI research tasks:

Demo Skills Used What It Does
Norm Heterogeneity → LoRA Brittleness Autoresearch, ML Paper Writing, Ideation Agent autonomously discovered norm heterogeneity predicts fine-tuning difficulty (r=-0.99), pivoting from a null result on ETF overlaps
RL Algorithm Brain Scan Autoresearch, GRPO, TRL, SAELens, TransformerLens, ML Paper Writing Agent found DPO is a rank-1 perturbation (95.6% recovery from one SVD direction) while online RL is distributed and structure-preserving
NeMo Eval: GPQA Benchmark NeMo Evaluator Compare Llama 8B/70B/405B on graduate-level science questions
LoRA Without Regret Reproduction GRPO, TRL Reproduce SFT + GRPO RL experiments via prompting
Layer-Wise Quantization Experiment llama.cpp, GGUF Investigate optimal layer precision allocation—early layers at Q8 achieve 1.9× compression with 1.3% perplexity loss
Cross-Lingual Alignment Analysis FAISS Quantify how well multilingual embeddings align semantic concepts across 8 languages using FAISS similarity search
Scientific Plotting Demo Academic Plotting Generate publication-quality figures for the Andes QoE-aware LLM serving paper — Gemini AI architecture diagrams + matplotlib data charts (CDF, multi-panel grids, bar charts)

Featured Demos: Two papers produced entirely by AI agents using the autoresearch skill. The Norm Heterogeneity paper demonstrates autonomous research pivoting — the agent refuted its own hypothesis and discovered a stronger finding. The RL Brain Scan paper demonstrates multi-skill orchestration — the agent trained RL models, analyzed internals with interpretability tools, and synthesized the insight that "DPO is rank-1 alignment." Both papers written end-to-end by the agent.

Skill Structure

Each skill follows a battle-tested format for maximum usefulness:

skill-name/
├── SKILL.md                    # Quick reference (50-150 lines)
│   ├── Metadata (name, description, version)
│   ├── When to use this skill
│   ├── Quick patterns & examples
│   └── Links to references
│
├── references/                 # Deep documentation (300KB+)
│   ├── README.md              # From GitHub/official docs
│   ├── api.md                 # API reference
│   ├── tutorials.md           # Step-by-step guides
│   ├── issues.md              # Real GitHub issues & solutions
│   ├── releases.md            # Version history & breaking changes
│   └── file_structure.md      # Codebase navigation
│
├── scripts/                    # Helper scripts (optional)
└── assets/                     # Templates & examples (optional)
Quality Standards
  • 300KB+ documentation from official sources
  • Real GitHub issues & solutions (when available)
  • Code examples with language detection
  • Version history & breaking changes
  • Links to official docs

Roadmap

The library spans 98 comprehensive skills across the full AI research lifecycle. See our detailed roadmap for the complete development plan.

View Full Roadmap →

View Detailed Statistics
Metric Current Target
Skills 87 (high-quality, standardized YAML) 80 ✅
Avg Lines/Skill 420 lines (focused + progressive disclosure) 200-600 lines
Documentation ~130,000 lines total (SKILL.md + references) 100,000+ lines
Gold Standard Skills 65 with comprehensive references 50+
Contributors 1 100+
Coverage Architecture, Tokenization, Fine-Tuning, Mechanistic Interpretability, Data Processing, Post-Training, Safety, Distributed, Optimization, Evaluation, Infrastructure, Inference, Agents, RAG, Multimodal, Prompt Engineering, MLOps, Observability, ML Paper Writing, Ideation, Autoresearch Full Lifecycle ✅

Recent Progress: npm package @orchestra-research/ai-research-skills for one-command installation across all coding agents

…

Releases

Recent versions

All releases ↗
  1. v1.7.2v1.7.2 — Qoder Agent Support
  2. v1.7.1v1.7.1 — Inventory Consistency, Drift Guard & Security Hardening
  3. v1.4.0v1.4.0 - Autoresearch: Autonomous AI Research from Idea to Paper
  4. v1.3.6v1.3.6 - Local Project Installation & OIDC Publishing
  5. v1.2.0v1.2.0 - OpenClaw & Shared Agents Support
  6. v1.1.0v1.1.0 - OpenCode Support