vLLM クイックスタート：高性能 LLM サービング - 2026年

vLLM Quickstart: High-Performance LLM Serving - in 2026 - Rost Glukhov | Personal site and technical blog Page content vLLM is a high-throughput, memory-efficient inference and serving engine for Large Language Models (LLMs) developed by UC Berkeley’s Sky Computing Lab. With its revolutionary PagedAttention algorithm, vLLM achieves 14-24x higher throughput than traditional serving methods, making it the go-to choice for production LLM deployments. To see how vLLM fits among Ollama, Docker Model Runner, LocalAI and cloud providers—including cost and infrastructure trade-offs—see LLM Hosting: Local, Self-Hosted & Cloud Infrastructure Compared . What is vLLM? vLLM (virtual LLM) is an open-source library for fast LLM inference and serving that has quickly become the industry standard for production deployments. Released in 2023, it introduced PagedAttention , a groundbreaking memory management technique that dramatically improves serving efficiency. Key Features High Throughput Performance : vLLM delivers 14-24x higher throughput compared to HuggingFace Transformers with the same hardware. This massive performance gain comes from continuous batching, optimized CUDA kernels, and the PagedAttention algorithm that eliminates memory fragmentation. OpenAI API Compatibility : vLLM includes a built-in API server that’s fully compatible with OpenAI’s format. This allows seamless migration from OpenAI to self-hosted infrastructure without changing application code. Simply point your API client to vLLM’s endpoint and it works transparently. PagedAttention Algorithm : The core innovation behind vLLM’s performance is PagedAttention, which applies the concept of virtual memory paging to attention mechanisms. Instead of allocating contiguous memory blocks for KV caches (which leads to fragmentation), PagedAttention divides memory into fixed-size blocks that can be allocated on-demand. This reduces memory waste by up to 4x and enables much larger batch sizes. Continuous Batching : Unlike static batching where you wait for all sequences to complete, vLLM uses continuous (rolling) batching. As soon as one sequence finishes, a new one can be added to the batch. This maximizes GPU utilization and minimizes latency for incoming requests. Multi-GPU Support : vLLM supports tensor parallelism and pipeline parallelism for distributing large models across multiple GPUs. It can efficiently serve models that don’t fit in a single GPU’s memory, supporting configurations from 2 to 8+ GPUs. Wide Model Support : Compatible with popular model architectures including LLaMA, Mistral, Mixtral, Qwen, Phi, Gemma, and many others. Supports both instruction-tuned and base models from HuggingFace Hub. When to Use vLLM vLLM excels in specific scenarios where its strengths shine: Production API Services : When you need to serve an LLM to many concurrent users via API, vLLM’s high throughput and efficient batching make it the best choice. Companies running chatbots, code assistants, or content generation services benefit from its ability to handle hundreds of requests per second. High-Concurrency Workloads : If your application has many simultaneous users making requests, vLLM’s continuous batching and PagedAttention enable serving more users with the same hardware compared to alternatives. Cost Optimization : When GPU costs are a concern, vLLM’s superior throughput means you can serve the same traffic with fewer GPUs, directly reducing infrastructure costs. The 4x memory efficiency from PagedAttention also allows using smaller, cheaper GPU instances. Kubernetes Deployments : vLLM’s stateless design and container-friendly architecture make it ideal for Kubernetes clusters. Its consistent performance under load and straightforward resource management integrate well with cloud-native infrastructure. When NOT to Use vLLM : For local development, experimentation, or single-user scenarios, tools like Ollama or llama.cpp provide better user experience with simpler setup. vLLM’s complexity is justified when you need its performance advantages for production workloads. How to Install vLLM Prerequisites Before installing vLLM, ensure your system meets these requirements: GPU : NVIDIA GPU with compute capability 7.0+ (V100, T4, A10, A100, H100, RTX 20/30/40 series) CUDA : Version 11.8 or higher Python : 3.8 to 3.11 VRAM : Minimum 16GB for 7B models, 24GB+ for 13B, 40GB+ for larger models Driver : NVIDIA driver 450.80.02 or newer Installation via pip The simplest installation method is using pip. This works on systems with CUDA 11.8 or newer: # Create a virtual environment (recommended) python3 -m venv vllm-env source vllm-env/bin/activate # Install vLLM pip install vllm # Verify installation python -c "import vllm; print(vllm.__version__)" For systems with different CUDA versions, install the appropriate wheel: # For CUDA 12.1 pip install vllm == 0.4.2+cu121 -f https://github.com/vllm-project/vllm/releases # For CUDA 11.8 pip install vllm == 0.4.2+cu118 -f https://github.com/vllm-project/vllm/releases Installation with Docker Docker provides the most reliable deployment method, especially for production: # Pull the official vLLM image docker pull vllm/vllm-openai:latest # Run vLLM with GPU support docker run --runtime nvidia --gpus all \ -v ~/.cache/huggingface:/root/.cache/huggingface \ -p 8000:8000 \ --ipc = host \ vllm/vllm-openai:latest \ --model mistralai/Mistral-7B-Instruct-v0.2 The --ipc=host flag is important for multi-GPU setups as it enables proper inter-process communication. Building from Source For the latest features or custom modifications, build from source: git clone https://github.com/vllm-project/vllm.git cd vllm pip install -e . vLLM Quickstart Guide Running Your First Model Start vLLM with a model using the command-line interface: # Download and serve Mistral-7B with OpenAI-compatible API python -m vllm.entrypoints.openai.api_server \ --model mistralai/Mistral-7B-Instruct-v0.2 \ --port 8000 vLLM will automatically download the model from HuggingFace Hub (if not cached) and start the server. You’ll see output indicating the server is ready: INFO: Started server process [12345] INFO: Waiting for application startup. INFO: Application startup complete. INFO: Uvicorn running on http://0.0.0.0:8000 Making API Requests Once the server is running, you can make requests using the OpenAI Python client or curl: Using curl : curl http://localhost:8000/v1/completions \ -H "Content-Type: application/json" \ -d '{ "model": "mistralai/Mistral-7B-Instruct-v0.2", "prompt": "Explain what vLLM is in one sentence:", "max_tokens": 100, "temperature": 0.7 }' Using OpenAI Python Client : from openai import OpenAI # Point to your vLLM server client = OpenAI( base_url = "http://localhost:8000/v1" , api_key = "not-needed" # vLLM doesn't require authentication by default ) response = client . completions . create( model = "mistralai/Mistral-7B-Instruct-v0.2" , prompt = "Explain what vLLM is in one sentence:" , max_tokens = 100 , temperature = 0.7 ) print(response . choices[ 0 ] . text) Chat Completions API : response = client . chat . completions . create( model = "mistralai/Mistral-7B-Instruct-v0.2" , messages = [ { "role" : "system" , "content" : "You are a helpful assistant." }, { "role" : "user" , "content" : "What is PagedAttention?" } ], max_tokens = 200 ) print(response . choices[ 0 ] . message . content) Advanced Configuration vLLM offers numerous parameters to optimize performance: python -m vllm.entrypoints.openai.api_server \ --model mistralai/Mistral-7B-Instruct-v0.2 \ --port 8000 \ --gpu-memory-utilization 0.95 \ # Use 95% of GPU memory --max-model-len 8192 \ # Maximum sequence length --tensor-parallel-size 2 \ # Use 2 GPUs with tensor parallelism --dtype float16 \ # Use FP16 precision --max-num-seqs 256 # Maximum batch size Key Parameters Explained : --gpu-memory-utilization : How much GPU memory to use (0.90 = 90%). Higher values allow larger batches but leave less margin for memory spikes. --max-model-len : Maximum context length. Reducing this saves memory for larger batches. --tensor-parallel-size : Number of GPUs to split the model across. --dtype : Data type for weights (float16, bfloat16, or float32). FP16 is usually optimal. --max-num-seqs : Maximum number of sequences to process in a batch. vLLM vs Ollama Comparison Both vLLM and Ollama are popular choices for local LLM hosting, but they target different use cases. Understanding when to use each tool can significantly impact your project’s success. Performance and Throughput vLLM is engineered for maximum throughput in multi-user scenarios. Its PagedAttention and continuous batching enable serving hundreds of concurrent requests efficiently. Benchmarks show vLLM achieving 14-24x higher throughput than standard implementations and 2-4x higher than Ollama under high concurrency. Ollama optimizes for single-user interactive use with focus on low latency for individual requests. While it doesn’t match vLLM’s multi-user throughput, it provides excellent performance for development and personal use with faster cold-start times and lower idle resource consumption. Ease of Use Ollama wins decisively on simplicity. Installation is a single command ( curl | sh ), and running models is as simple as ollama run llama2 . It includes a model library with quantized versions optimized for different hardware profiles. The user experience resembles Docker – pull, run, and go. vLLM requires more setup: Python environment management, CUDA installation, understanding of serving parameters, and manual model specification. The learning curve is steeper, but you gain fine-grained control over performance optimization. This complexity is warranted for production deployments where you need to squeeze maximum performance from your hardware. API and Integration vLLM provides OpenAI-compatible REST APIs out of the box, making it a drop-in replacement for OpenAI’s API in existing applications. This is crucial for migrating produ