Skip to content

Issues: triton-inference-server/server

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

662 Open 3,244 Closed
662 Open 3,244 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

AWS IRSA instead credential propagation.
#8090 opened Mar 22, 2025 by yuriyyurov
Readiness probes not working for onnx-tensorrt models
#8085 opened Mar 21, 2025 by samueltrautwein
--no-container-build not work when build with --backend=onnxruntime option
#8084 opened Mar 21, 2025 by JamesPoon
GPU VRAM Leak with Python Backend BLS Requests to ORT Backend
#8083 opened Mar 21, 2025 by WoodieDudy
genai-perf out of bounds error when choices array is null when setting "include_usage": true
#8082 opened Mar 21, 2025 by sre42
Cannot specify target platform/os for compose.py
#8081 opened Mar 21, 2025 by haruyama480
TRITON_AWS_MOUNT_DIRECTORY becomes useless because of the random directory name
#8077 opened Mar 19, 2025 by ShuaiShao93
Tokens streamed out of order from OpenAI compatible frontend
#8073 opened Mar 14, 2025 by njaramish
Suggesting using SavedModelBundleLite to reduce RAM usage by 40% in Tensorflow backend
#8067 opened Mar 13, 2025 by vdel
build triton from source --no-container-build Error
#8066 opened Mar 13, 2025 by ronghuaiyang
feature request: distinct prometheus metrics for streamed vs non-streamed requests
#8063 opened Mar 12, 2025 by MadDanWithABox
Building triton server python_backend from source
#8060 opened Mar 10, 2025 by mritunjaysharma394
[feature request] Real-time streaming inference load generation by perf_analyzer
#8059 opened Mar 8, 2025 by vadimkantorov
OpenAI Frontend Batch Support
#8058 opened Mar 7, 2025 by Loc8888
CUDA Race Condition in TensorRT GEMM Kernel with Triton Inference Server load tensorRT model
#8057 opened Mar 7, 2025 by neezeeyee
Versioning for ensemble models and/or config.pbtxt files
#8056 opened Mar 5, 2025 by ghicks-novaprime
2
Bazel support and tag release for individual repos
#8049 opened Mar 4, 2025 by arpit15
RFE: Function calling in OpenAI Frontend enhancement New feature or request openai OpenAI related
#8048 opened Mar 3, 2025 by thehumit
1
How to Send FP16 Input Tensors Using gRPC in C# for NVIDIA Triton Inference Server?
#8044 opened Feb 28, 2025 by Madihaa-Shaikh
Python Backend GPU Tensor Support on Windows - A Must-Have!
#8041 opened Feb 27, 2025 by mhbassel
Multibyte UTF-8 Characters Broken in Streaming Mode (� Substitution)
#8039 opened Feb 27, 2025 by Nurgl
2
[Question] Triton Inference server vLLM backend vs vLLM serve
#8036 opened Feb 26, 2025 by pradghos
Segment fault crash due to race condition of request cancellation (with fix proposal) bug Something isn't working
#8034 opened Feb 25, 2025 by lunwang-ttd
7
Triton llm openai langgraph toolcall
#8033 opened Feb 25, 2025 by GGN1994
Python backend without GIL
#8032 opened Feb 25, 2025 by zeruniverse
ProTip! Exclude everything labeled bug with -label:bug.