Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add OpenRouter Support #8

Open
crmne opened this issue Mar 12, 2025 · 0 comments · May be fixed by #29
Open

Add OpenRouter Support #8

crmne opened this issue Mar 12, 2025 · 0 comments · May be fixed by #29
Labels
new provider New provider integration

Comments

@crmne
Copy link
Owner

crmne commented Mar 12, 2025

Context

I'm hitting Anthropic rate limits constantly with my own usage, and several community members have requested OpenRouter integration. This would allow access to a wider range of models through a single API key while maintaining RubyLLM's unified interface.

Benefits

  • Single API key to access models across providers
  • Potential cost savings through OpenRouter's pricing
  • Simplified rate limit management
  • Access to exclusive models not available through direct provider integrations
  • Fallback capabilities when primary providers are at capacity

Implementation Considerations

  • OpenRouter largely follows OpenAI's API structure, so we can likely adapt our existing OpenAI provider implementation
  • Need to handle model ID mapping/translation to keep the model selection experience consistent
  • Should implement proper error handling for OpenRouter-specific cases
  • Will need to update the Models registry to include OpenRouter-accessible models

Scope

Initial implementation should focus on:

  • Chat completion support (highest priority)
  • Embeddings support
  • Model listing
  • Full streaming support
  • Tool use

Image generation through DALL-E can be a second phase.

Next Steps

  • Research OpenRouter API documentation
  • Create a new Provider module for OpenRouter
  • Implement proper model discovery and capabilities
  • Test with complex conversations, tools, and streaming
@khasinski khasinski linked a pull request Mar 16, 2025 that will close this issue
@crmne crmne linked a pull request Mar 23, 2025 that will close this issue
@crmne crmne added the new provider New provider integration label Mar 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new provider New provider integration
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant