ComfyUI-omni-llm

v3.4.0 (2026-09-12)

• Node Optimization and Adjustments: Added API loading nodes (expanding model support types to meet different user scenario needs), added multi-turn conversation nodes (for skill usage, making it easier to generate more professional prompt content), added skill loading nodes (expanding text generation capabilities, providing more options for prompt generation), optimized and adjusted parameter node options, adapted for multi-turn conversation node usage.
• Preset Template Optimization: Optimized and adjusted preset template content, added Skill-related files.

v3.3.4 (2026-08-22)

• Node Optimization Adjustments: Added support for VoxCPM2 model, optimized flashattention processing.
• Preset Template Optimization: Optimized anime preset templates negative prompt content.

v3.3.3 (2026-08-16)

• Node Optimization Adjustments: Added support for more raw image models to meet a wider range of user needs, and fixed the issue where model labels wouldn’t show when forward constraint options were not enabled.
• Preset Template Optimization: Optimized raw image preset templates. Natural Paragraph: doesn’t output camera parameter info. Structured: outputs camera parameter info. You can choose based on your own needs (it’s recommended to add a facial optimization node when generating real natural portraits to enhance the effect).

v3.3.2 （2026-08-15）

• Node Optimization and Adjustments: 1. Optimized internal code, adapted parameters for llama-cpp-python 0.3.46, added support for the Qwen3.8 model (untested), and improved MTP processing.

1. Added audio model options, supporting IndexTTS-2.5, Ace-Step1.5, and MiniMax-Music3 models.
2. Added output model labels to help users confirm results.

• Preset Template Optimization: The multi-person chat template now aligns with the output of the IndexTTS-2.5 model, the music template aligns with Ace-Step1.5 and MiniMax-Music3 outputs, the video generation template aligns with LTX-2.5 and MiniMax-H3 outputs, and the image creation template hasn't been fully aligned yet due to the volume of content (feedback or co-creation is welcome).

v3.3.1 （2026-08-12）

• Node Optimization Adjustments: The multi-image input node interface has been adjusted to 9 inputs, LTX2.3 updated to LTX2.5
• Preset Template Optimization: Optimized video preset templates to fit LTX-2.5 model prompt generation

v3.3.0 (2026-08-08)

• Node Optimization & Adjustment:
  1. Unified node name prefixes to avoid node name conflicts
  2. To support transformers-5.0 and above, removed TTS-related nodes. Text-to-audio is no longer supported; for TTS-related features please use a dedicated plugin. Only the ASR node is retained
  3. To improve image generation controllability, added a model selection feature to adapt to the prompt requirements of different models
  4. Refactored inference logic to fit the new preset template requirements
• Preset Template Optimization: Reconstructed the preset template format to adapt to the prompt requirements of different models, integrated Chinese and English templates (functionality unchanged), optimized thinking logic, improved generation speed and quality, and enhanced the controllability of generated content Specific adjustments: illustrious, anime, audio templates, and prompt_expander do not support model selection output; other text generation preset templates support model selection output; reverse engineering templates support inputting text content as a reference to reduce incorrect style judgment in reverse engineering (generally no need to add)
• Dependency Installation Optimization: Simplified dependency installation steps, lowering plugin installation difficulty

v3.2.3 （2026-07-25）

• Preset Template Optimization: Optimize and adjust the image reverse engineering preset templates, add more reverse reasoning dimensions, introduce prompt weighting to enhance reverse engineering results, include a figure art category, reduce forced clothing generation that conflicts with user requests (non-NSFW), and refine the logical guidance for inference thinking
• Dependency Detection Optimization: Optimized the dependency check logic for llama-cpp-python, mainly for the first time setting up this plugin environment

v3.2.2 （2026-07-13）

• Node Optimization & Adjustment: Add pre-constraint options and negative prompt options (built-in fixed content that does not participate in model inference and only applies to text generation preset templates) to enhance controllability of generated content. Example outputs are adjusted to display randomly, and example information for corresponding categories will pop up when key categories are triggered. (If unified inference nodes report errors, re-add them to the interface for execution)
• Model Optimization & Adjustment: Add support for the variant model Qwen3.5-DeepSeek-V4-Flash, which delivers stronger context comprehension; adjust parameters according to actual usage scenarios.
• temperature (temperature parameter): 0.7-1.0 (lower values for coding tasks, higher values for creative reasoning)
• top_p (core sampling threshold): 0.95
• For complex reasoning tasks, it is recommended to increase the value of the max_tokens parameter to obtain more detailed generated content
• Preset Template Optimization: Rename the all-round portrait template to ultra-realistic portrait template, and add thick painting and CG character templates.

v3.2.1 (2026-07-05)

• Node Optimization Adjustments：Integrate cleanup, uninstall nodes, and simplify some parameter options by setting them to auto in the background
• Model Optimization Adjustments：Add MiMo-VL series model support (due to preset templates adding thinking prompts, MiMo-VL models require custom templates, as built-in templates generate undesirable content)

v3.2.0 (2026-07-04)

• Code Optimization: Removed redundant processing code and streamlined the inference logic (please re-add any nodes that throw errors and run them in the interface, otherwise they won't work properly).
• Node Optimization Adjustments:
  1. ASR: Removed unnecessary code, keeping only Qwen ASR support.
  2. Model Unload Node: Added forced alignment, ASR, and TTS model unload functionality.
• Node Grouping Adjustments: All node groups have been changed from llama-cpp-llm to omni-llm, aligning with plugin names.
• Preset Template Optimization:
  1. Added templates for building exterior/garden rendering.
  2. Optimized natural paragraphs to 2-3 sections.
  3. Improved portrait composition and poses, emphasizing a realistic natural feel, reducing AI-like stiffness or awkwardness.
  4. Enhanced storytelling feel for videos.
  5. Strengthened breathing sense in audio.
  6. Optimized prompt professional descriptions to better match open-source model understanding, allowing for better content control {generated content may include some irrelevant information {uncontrollable}, please selectively refine it yourself}.
  7. To improve content quality, generated texts will be lengthy, increasing inference time and memory usage. Please adjust based on actual needs to avoid program crashes from excessively long text. Larger models also provide higher-quality text generation.

v3.1.6 (2026-06-27)

• Preset Template Optimization: Optimized preset templates to improve out-image quality and user experience. Added detailed preset templates examples for different categories (portrait, design, video, etc.).
• Rewrite Prompt Style preset information

v3.1.5 (2026-06-18)

• Optimized Node Parameters: Improved Qwen3 series inference logic. For Qwen3.5, Qwen3.6 and other thinking models using text generation mode, mmproj model must be enabled. Some preset templates may not work, please test and adjust or use non-thinking models
• Fixed Inference Issues: Fixed inference failures or content mismatch issues caused by preset template function modifications. Added example guidance to reduce the chance of generated content deviating from expected results
• Preset Template Optimization: Improved preset template examples (different preset templates have different optimization directions, e.g., use zimage preset template for portraits, use other models for generation, please adjust according to actual needs)

v3.1.4 (2026-06-13)

I. Node Optimization

• Optimized Node Parameters: Removed redundant optional parameters from ASR and TTS nodes, streamlined code to retain only qwen-tts model parameters to avoid parameter errors during use
• Reasoning Budget Parameter: Added reasoning_budget parameter to support thinking mode control for Qwen3.5-Thinking and similar models (-1=unlimited, 0=disable thinking, N=limit to N tokens), thinking models can now use text generation mode normally

II. Dependency Management Optimization

• Multi-Platform Support: Optimized requirements.txt to automatically select precompiled wheels for Windows/Linux/macOS (manual installation recommended to avoid automatic installation failures)

III. Code Quality Improvement

• Exception Handling Optimization: Replaced exit(1) forced exit with custom exception classes LlamaCppDependencyError and ComfyUINotFoundError to prevent program crashes
• Complete Parameter Passing: Improved parameter passing for create_chat_completion(), supporting presence_penalty, typical_p, mirostat and other parameters

IV. Preset Template Optimization

• Template Optimization: Based on Seedance2.0 model prompts, added emotion and expression-related prompt content. Note: wan and other open-source models may not be suitable, generated content may not achieve expected results, please test and adjust yourself. (Same applies to previously adjusted image and audio preset templates)

v3.1.3 (2026-06-06)

I. Model Support Optimization

• Qwen3.6 Model Support: Added Qwen3.6 series model support, including Qwen3.6-27B and Qwen3.6-35B-A3B large VRAM models, optimized context length configuration (default 8192) and GPU layer allocation (not tested due to limited personal hardware, please understand if runtime errors occur)
• Gemma4 Model Support: Improved Gemma-4 series model support, including Gemma-4-E2B, Gemma-4-E4B, and Gemma4-26B-A4B model automatic recognition and loading

II. Preset Template Adjustment

• Template Optimization: 1.Optimized repetitive instruction content, resolved text generation content confusion caused by logic disorder 2.Restructured output callback to JSON output format for precise image generation control 3.Preset templates now feature intelligent category processing. Please add relevant keyword information such as poster, illustration, photography to prompts for model-based generation and reduced errors 4.Illustrious preset template added ToriiGate model exclusive optimization, ToriiGate model recommended
• Template Addition: Added Ideogram-4 model exclusive preset template (usage according to model requirements, generated results may not be ideal, please test yourself, recommended to use qwen3 series models)

v3.1.2 (2026-05-31)

I. Node Optimization

• Enhanced MoE Model Support: Optimized tensor split logic for Mixture of Experts models, improved expert layer allocation for better performance
• Added Example Output Interface: New example output interface to help users write better prompts based on preset template examples, improving generated text quality

II. Dependency Optimization

• Dependency Detection Prompt: New dependency detection feature to notify users if required dependencies are installed, avoiding missing dependency issues during use

III. Preset Template Adjustment

• Template Optimization: Image templates optimized for shot descriptions and AI-compatible logic descriptions; video templates optimized for content coherence and character expression descriptions; audio templates optimized for emotional expression of audio content (not yet tested, feedback welcome, templates continuously optimized)

v3.1.1 (2026-05-10)

I. Template Optimization

• Template Content Adjustment: Add frame sequence analysis template, optimize video reverse prompt template, enhance facial expression and detail descriptions for all templates involving characters, optimize ERNIE_IMAGE and QWEN_IMAGE_2512 output text classification confusion problem, and adjust some templates to be more suitable for European and American audiences (currently still updating, not completed yet, please understand if the generated results are not ideal)
• Inference Time Increase: To achieve more comprehensive prompt generation, the templates output more text content, which will increase the inference time (the amount of text content determines the inference time), please adjust parameters or use custom templates according to actual situation.

II. Code Maintenance

• Bug Fixes: Fix known issues found in multi-image transmission and video processing
• Template Optimization: Optimize template output format, fix output content mismatch problem (some templates only have one output format, switching after output results are consistent)

v3.1 (2026-04-30)

I. Streamlining and Optimization

• Node Streamlining: Remove all nodes that did not meet expectations, remove impractical templates, remove acceleration modules with unsatisfactory results
• Completion Optimization: Add parameter optimization for Intel graphics cards

II. Code Optimization

• Code Simplification: Optimize code structure, reduce redundancy and complexity, improve runtime efficiency, fix TTS voice mismatch and qwen3.5 output issues

III. Template Enhancement

• New Optimized Templates: Add Illustrious, Anima, ERNIE-Image dedicated preset templates, strengthen video-related template content, adjust qwen-image2512 template positioning

IV. Documentation Simplification

• Documentation Simplification: Simplify supporting documentation content, improve practicality, remove redundant and unnecessary explanatory content

v3.0 (2026-04-19)

I. Prompt Format Optimization

• JSON Format Output: Multi-image input node prompt output changed to JSON format, including complete task type, theme, narrative style, content focus, target audience, story length, video model type, and specific content fields

II. Preset Template Adjustment and Optimization

• Streamlining Optimization: Remove impractical templates, retain frequently used and stable templates
• Classification Reorganization: Optimize template positioning, reintegrate into a clearer classification system
• Portrait Template Enhancement: Add portrait-specific templates, optimize portrait feature capture, expression depiction, pose description and other functions
• Poster Template Enhancement: Strengthen poster template's layout, font style, color matching and other design element support

III. New Model Support

• Gemma4 Model Support: Add Google Gemma 4 series multimodal model support (quantized versions only). Non-quantized Gemma 4 versions cannot be compatible due to dependency limitations and have unsatisfactory performance, so support is not added
• LFM2.5-VL Model Support: Add LFM 2.5 Vision series model support (quantized versions only), expand visual understanding capabilities

IV. Conflict Dependency Handling

• Conflict Dependency Handling: Conflict dependencies embedded in the plugin to avoid conflicts with other user plugins, ensuring stable plugin operation

v2.0 (2026-04-03)

I. Modular Inference Architecture

• Smart Hardware Adaptation: Improve NVIDIA/AMD/CPU three-end adaptation, automatically adjust parameters based on VRAM size
• Error Recovery Mechanism: Enhance error handling and recovery capabilities to ensure inference continuity and stability

II. Image Inference Node Optimization and Adjustment

• Unified Inference Node: Integrate text, image, and audio processing capabilities, support 5 inference modes (text generation, image understanding, audio to text, text to audio, video understanding), achieve cross-modal understanding and generation
• API Model Node System: Including API configuration management node, API configuration selector, API configuration manager, and API model switcher, supports multiple API providers including online OpenAI, offline Ollama, llms-py, vllm-omni (requires Linux), enabling remote deployment and distributed inference
• Parameter Configuration Node: Add more efficient inference parameter configurations such as Flash Attention and KV Cache to improve inference speed

III. ASR+TTS Audio Processing System

• ASR Model Loader: Supports Qwen3-ASR speech recognition model, supports multilingual recognition, audio reverse engineering, and timestamp generation
• TTS Model Loader: Supports Qwen3-TTS speech synthesis model, supports multiple voice selection and emotion control
• Forced Aligner Model Loader: Supports Qwen3-ForcedAligner model, achieves high-precision audio-text alignment and timestamp generation
• Forced Aligner Inference Node: Processes audio-text alignment, generates precise timestamp information
• Multi-Model TTS Node: Supports multi-role dialogue synthesis, can assign different voices to different roles
• TTS Alignment Node: Optimizes audio alignment effect of TTS output

IV. Video Processing System

• Video Loader Node: Supports video file input, extracts video frame sequence for analysis
• Video Frame Sampling: Supports two modes: automatic uniform sampling and manual frame index specification
• Video Understanding Mode: Supports video content analysis, storyboard decomposition, video reverse engineering, and subtitle generation
• Video to Audio and Subtitles: Extract audio from video and generate synchronized subtitles

V. New Audio Preset Templates

• Audio Subtitle Conversion: Convert audio content to standard format subtitles, including time codes and synchronized text
• Video Audio Subtitle Generation: Generate complete subtitles with audio descriptions based on video content
• Text to Audio: Convert text to natural speech, supports multi-person dialogue synthesis and voice selection
• Audio Analysis: Analyze audio content, extract key information and emotional features

VI. Multi-Image Input Node Interface Expansion

• Double the multi-image input node interfaces, support more image inputs to meet the creation of richer and more coherent story content

v1.4.0 (2026-02-23)

• Add Multi-Image Input node with the following features:
  • Dual-mode operation: Image mode analyzes multiple images and creates stories, Text mode generates prompts through option settings
  • Multi-image input support: Supports 1-6 images input, automatic preprocessing and encoding
  • Rich content creation types: Supports 10 types including coherent story, storyboard description, scene analysis, character development, emotional progression, creative writing, script writing, advertising copy, product introduction, educational content
  • Flexible length control: Supports four length options - Short (200 words), Medium (400 words), Detailed (600 words), Complete (1000 words)
  • Multilingual support: Supports Chinese and English output
  • Rich theme selection: Supports 12 themes including Adventure, Romance, Mystery, Sci-Fi, Fantasy, Daily Life, History, Future Technology, Business Marketing, Educational Popular Science, Entertainment Comedy
  • Diverse narrative styles: Supports 4 styles including First Person, Third Person, Omniscient Perspective, Multiple Perspectives Switching
  • Content focus control: Supports 6 focuses including Balanced Development, Emphasize Plot, Emphasize Characters, Emphasize Emotion, Emphasize Visual, Emphasize Dialogue
  • Target audience customization: Supports 5 audiences including General Public, Teenagers, Children, Professionals, Specific Groups
  • Video model optimization: Optimize prompt format for different video generation models such as WAN2.2, LTX2, General Video, Custom
  • Custom prompts: Supports adding custom prompts to guide content creation
  • Image description control: Option to include description of each image before the story
• Add Multi-Image Input node usage documentation link

v1.3.0 (2026-02-08)

• Add multiple preset prompt templates: Chinese-English bilingual prompt generation, ultra-high definition image prompt reverse engineering
• Optimize model loading and inference process, improve runtime efficiency
• Improve Chinese translation, enhance localization support
• Add video interface, support video input, add templates for video reverse engineering function
• Add OCR enhancement function, support poster text recognition and style restoration, adapt to prompt reverse engineering needs
• Implement intelligent model detection system, automatically discover and support new VL models added by llama_cpp_python
• Optimize model name inference logic, automatically generate model names according to ChatHandler naming rules
• Expand model support list to ensure backward compatibility with all previously supported models
• Implement model list deduplication function, keep interface clean and organized
• Add support for multiple models
• Add dynamic support function, llama_cpp_python updates and releases support for new model versions, models can be downloaded and used immediately
• Fix known bugs

v1.2.0 (2026-01-29)

• Refactor file directory, delete old version files during installation, do not overwrite
• Add comprehensive preset prompt templates for professional AI models:
  • ZIMAGE - Turbo: Optimized for Z-Image-Turbo model, supports 8-step Turbo inference to quickly generate 1080P HD images
  • FLUX2 - Klein: Designed for FLUX.2 Klein model, creates concise and expressive prompts
  • LTX-2: Customized for LTX-2 video generation model, supports dynamic video prompts, can generate high-quality, audio-visual synchronized 4K videos
  • Qwen - Image Layered: Created for Qwen-Image-Layered model, supports detailed layered prompts, handles complex compositions and multiple elements
  • Qwen - Image Edit Combined: Edit prompt enhancer for image editing tasks
  • Qwen - Image Dual: Designed for Qwen Image 2512 model, supports high-resolution generation capabilities
  • Video - Reverse Prompt: Video reverse prompt generator, creates detailed video descriptions based on video content (600-1000 words)
  • WAN - T2V: Film director style template, adds film elements (time, light source, light intensity, light angle, color tone, shooting angle, lens size, composition)
  • WAN - I2V: Video description prompt rewriting expert, emphasizes dynamic content
  • WAN - FLF2V: Prompt optimizer, optimizes prompts based on first and last frame images of video, emphasizes motion information and camera movement
• Optimize Chinese-English switching function, improve language adaptability
• Provide bilingual preset templates (English and Chinese), better compatibility for different language models (exclusive presets have word limits to meet model generation requirements while ensuring efficient results. If not meeting requirements, please input in preset box or use external custom)
• Add Chinese-English switching function for generation results

v1.1.0 (2026-01-24)

• Refactor node file directory
• Add parameter recommendation settings documentation to help users understand the impact of each parameter on generation results
• Add support types for MiniCPM-V-4.5, LFM2.5-VL-1.6B, GLM-4.6V models
• Add Chinese-English switching function for easy switching between different types of reverse engineering models
• Model loading only supports .gguf format model files

v1.0.0 (2026-01-17)

• Add support type for llama-joycaption reverse engineering model
• Add mmproj model switch to support text-only generation
• Add session cleanup node (releases resources occupied by current conversation, reduces occurrence of no results)
• Add model unload node (reduces VRAM usage)
• Add hardware optimization module, adapts to different performance hardware, improves inference speed, ensures smooth use on different hardware