Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for per-request metadata/headers in GenerativeModel.generate_content #698

Open
chitalian opened this issue Mar 5, 2025 · 4 comments · May be fixed by #704
Open

Add support for per-request metadata/headers in GenerativeModel.generate_content #698

chitalian opened this issue Mar 5, 2025 · 4 comments · May be fixed by #704
Labels
p3 status:triaged Issue/PR triaged to the corresponding sub-team type:feature request New feature request/enhancement

Comments

@chitalian
Copy link

Description of the feature request:

Currently, the Gemini Python SDK only allows setting metadata (headers) at the client configuration level through default_metadata. This makes it difficult to set different headers for different requests, particularly when integrating with services like Helicone that require user-specific headers.

Other LLM client libraries (OpenAI, Anthropic) support setting headers on a per-request basis through parameters like extra_headers. Adding similar functionality to the Gemini SDK would improve flexibility and make it easier to integrate with proxy services.

What problem are you trying to solve with this feature?

When using the Gemini API through a proxy service like Helicone, it's important to be able to set user-specific headers (like Helicone-User-Id) on a per-request basis. Currently, this requires creating a new client instance for each user or reconfiguring the client before each request, which is inefficient and cumbersome.

Ideally, I would like to be able to do something like this:

config = {
    "messages": [
        {"role": "system", "content": system_msg},
        {"role": "user", "content": human_msg},
    ],
    "response_model": output_structure,
}

user_id = user_id_var.get()
if user_id:
    config["extra_headers"] = {}
    config["extra_headers"]["Helicone-User-Id"] = user_id

response = client.chat.completions.create(**config)

This would allow setting different headers for different requests without having to reconfigure the client or create multiple client instances.

Any other information you'd like to share?

Proposed implementation:

The GenerativeModel.generate_content method could be modified to accept an additional parameter like metadata or extra_headers that would be passed to the underlying client method. This would allow users to set headers on a per-request basis while maintaining backward compatibility.

Example implementation:

def generate_content(
    self,
    contents: content_types.ContentsType,
    *,
    generation_config: generation_types.GenerationConfigType | None = None,
    safety_settings: safety_types.SafetySettingOptions | None = None,
    stream: bool = False,
    tools: content_types.FunctionLibraryType | None = None,
    tool_config: content_types.ToolConfigType | None = None,
    request_options: helper_types.RequestOptionsType | None = None,
    extra_headers: dict[str, str] | None = None,  # New parameter
) -> generation_types.GenerateContentResponse:
    # ... existing code ...
    
    if request_options is None:
        request_options = {}
    
    # Convert extra_headers to metadata format if provided
    metadata = []
    if extra_headers:
        metadata = [(k, v) for k, v in extra_headers.items()]
    
    # Add metadata to request_options
    if metadata:
        request_options["metadata"] = metadata
    
    # ... rest of the method ...

This would provide a more consistent experience across different LLM client libraries and make it easier to integrate with proxy services.

@MarkDaoust MarkDaoust added the type:feature request New feature request/enhancement label Mar 5, 2025
@julurisaichandu
Copy link

I would like to work on this @chitalian , could you assign me this?

@mahimairaja
Copy link

Hi, I would like to work on this issue

@mahimairaja
Copy link

@MarkDaoust Can you please assign this issue to me

drona-gyawali added a commit to drona-gyawali/generative-ai-python that referenced this issue Mar 7, 2025
@julurisaichandu
Copy link

julurisaichandu commented Mar 7, 2025

can we just work on the issue without getting assigned @mahimairaja @MarkDaoust?
because I already see a PR from someone for this issue and the issue wasn't even assigned to them

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p3 status:triaged Issue/PR triaged to the corresponding sub-team type:feature request New feature request/enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants