Repositories list

• vllm

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  amdcudainference+ 17pytorchtransformerllamagptrocmmodel-servingtpu+ 10

  Python

  •

  Apache License 2.0

  •6.3k•42k•1.4k•495•Updated Mar 16, 2025Mar 16, 2025

• llm-compressor

  Public
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

  sparsitycompressionquantization

  Python

  •

  Apache License 2.0

  •98•1.1k•32•33•Updated Mar 16, 2025Mar 16, 2025

• aibrix

  Public
  Cost-efficient and pluggable Infrastructure components for GenAI inference

  Jupyter Notebook

  •

  Apache License 2.0

  •293•3.2k•145•12•Updated Mar 15, 2025Mar 15, 2025

• vllm-spyre

  Public
  Community maintained hardware plugin for vLLM on Spyre

  Python

  •

  Apache License 2.0

  •7•12•11•5•Updated Mar 15, 2025Mar 15, 2025

• buildkite-ci

  Public
  HCL

  •20•8•0•4•Updated Mar 14, 2025Mar 14, 2025

• production-stack

  Public
  vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

  Python

  •

  Apache License 2.0

  •105•794•34•17•Updated Mar 14, 2025Mar 14, 2025

• vllm-ascend

  Public
  Community maintained hardware plugin for vLLM on Ascend

  inferencetransformermodel-serving+ 6mlopsascendllmllmopsllm-servingvllm

  Python

  •

  Apache License 2.0

  •53•332•60•21•Updated Mar 14, 2025Mar 14, 2025

• flash-attention

  Public
  Fast and memory-efficient exact attention

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •1.5k•49•0•8•Updated Mar 11, 2025Mar 11, 2025

• vllm-project.github.io

  Public
  HTML

  •13•6•0•2•Updated Mar 10, 2025Mar 10, 2025

• FlashMLA

  Public
  C++

  •

  MIT License

  •792•3•0•0•Updated Feb 25, 2025Feb 25, 2025

• vllm-project.github.io-static

  Public archive
  HTML

  •

  MIT License

  •7•8•0•1•Updated Feb 7, 2025Feb 7, 2025

• vllm_allocator_adaptor

  Public
  An adaptor to allow Python allocator for PyTorch pluggable allocator

  C++

  •

  Apache License 2.0

  •1•2•0•0•Updated Jan 5, 2025Jan 5, 2025

• media-kit

  Public
  vLLM Logo Assets

  0•2•0•0•Updated Dec 12, 2024Dec 12, 2024

• vllm-nccl

  Public archive
  Manages vllm-nccl dependency

  Python

  •

  Apache License 2.0

  •3•17•2•0•Updated Jun 3, 2024Jun 3, 2024

• dashboard

  Public
  vLLM performance dashboard

  Python

  •

  Apache License 2.0

  •5•23•0•0•Updated Apr 26, 2024Apr 26, 2024