Bug: ai-proxy does not correctly escape AWS Bedrock inference profile ARN in signature and request

[muscionig]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.9.0

Current Behavior

When using AWS Bedrock inference profiles, which are formatted as: arn:aws:bedrock:us-east-1:<account_id>:application-inference-profile/<profile_id>, kong is unable to route the request.
I have experience two behaviors:

1. if the ai-proxy is configured with the unescaped ARN, Kong fails with:

{"error":{"message":"transformation failed from type bedrock://llm/v1/chat: no generation candidates received from Bedrock, or max_tokens too short"}}

2. when configured with the escaped one (arn%3Aaws%3Abedrock%3Aus-east-1%3A<account_id>%3Aapplication-inference-profile%2F<profile_id>), Kong fails with:

{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method."}

Expected Behavior

Kong should correctly format and escape the ARN when generating the SigV4 signature and constructing the request URL to match AWS’s expected format.

• The Converse API requires the URL to be formatted as:

  /model/<model-id/inference-profile>/converse
  

• The AWS CLI (in debug mode) shows that this URL is then encoded when making the request to Bedrock.

Steps To Reproduce

1. Create the plugin with the unescaped model_name ARN

   • Example configuration (following the kong documentation):

     curl -X POST http://localhost:8001/services/bedrock-service/plugins \
      --header 'Content-Type: application/json' \
      --header 'accept: application/json' \
      --data '{
      "name": "ai-proxy",
      "instance_name": "ai-proxy-bedrock",
      "config": {
        "route_type": "llm/v1/chat",
        "auth": {
          "allow_override": false,
          "aws_access_key_id": "<YOUR_AWS_ACCESS_KEY_ID>",
          "aws_secret_access_key": "<YOUR_AWS_SECRET_ACCESS_KEY>"
        },
        "model": {
          "provider": "bedrock",
          "name": "arn:aws:bedrock:us-east-1:<account_id>:application-inference-profile/<profile_id>",
          "options": {
            "bedrock": {
              "aws_region": "us-east-1"
            }
          }
        }
      }
      }'
     

     This will fail with the transformation error reported above.

2. Create the plugin with the escaped model_name ARN

   • Example configuration (following the kong documentation):

     curl -X POST http://localhost:8001/services/bedrock-service/plugins \
      --header 'Content-Type: application/json' \
      ...
          "provider": "bedrock",
          "name": "arn%3Aaws%3Abedrock%3Aus-east-1%3A<account_id>%3Aapplication-inference-profile%2F<profile_id>",
      ...
      }'
     

     This will fail with the sigv4 error.

Anything else?

• I did check the aws cli with the inference profile to rule out authentication errors with the inference profile, and the profile works using the converse endpoint.
• I verified the URL behavior by enabling the File Log plugin on /dev/stdout.

[muscionig]

I have attempted a fix in PR #14310.

While going through the source code I noticed that the plugin allows for setting and overriding the upstream_path. I did try that, but the overridden path is then used for the sigv4 authentication.

Another possible way would be to allow the inference profile by recommending the model.options.upstream_path edits, and by adding ngx.unescape_uri(parsed_url.path) in the authentication process.

[tysoekong]

Hey @muscionig thanks so much for this, I actually hadn't seen that you can just specify the model ARN directly for Bedrock-converse, I thought it was just for InvokeModel, so I didn't check it. But yeah I had this same problem with "upstream_path" which is:

1. fixed but
2. getting removed soon, the field is not needed with the next release

I will either ask if your fix can be merged directly, or I will bundle it into the next 3.10 main and tag you as contributor (because we have a giant PR of fixes coming already, it's quicker).

[muscionig]

Hi @tysoekong, thanks for the update and for considering my changes! I’m totally fine with the fix being integrated into a different PR and with the collaborator approach.

Is the PR public? I’d love to test it in the meantime to ensure everything works as expected.

[ttyS0e]

Hey @muscionig

I'm not sure what you've found is entirely the problem.

When I have fixed URL escaping in just the profile ARN:

2025/03/13 00:12:57 [notice] 2603#0: [kong] bedrock.lua:configure_request:654 [ai-proxy]----------------------------------------------------------------------+
2025/03/13 00:12:57 [notice] 2603#0: |"/arn%3Aaws%3Abedrock%3Aus-west-1%3A123456789012%3Aprompt/PROMPT12345:1"                                                |
2025/03/13 00:12:57 [notice] 2603#0: +------------------------------------------------------------------------------------------------------------------------+

I still get the "no candidates received". Adding a pre-function to log the output, Bedrock is returning:

{"Output":{"__type":"com.amazon.coral.service#UnknownOperationException"},"Version":"1.0"}

I think that this SHOULD work even without any of the escaping (i.e. setting model.name to arn:aws:bedrock:us-west-1:123456789012:prompt/PROMPT12345:1 but actually there's some other issue here. I am not sure what it won't accept about this request path, I need to ask AWS. Searching says it might be related to the wrong Content-Type ?

[muscionig]

Hi @ttyS0e,

It’s definitely possible there’s more to this than what I’ve found. Here’s how I debugged it:

1. I created an inference-profile via the API (since it’s not yet available in the console).
2. I invoked the profile using:

   aws bedrock-runtime converse --model-id <inference-profile-arn> [...] --debug

   This outputs the URL the CLI uses, which is URL-encoded (as mentioned in the main description).
3. On Kong:
   • Using the ARN as <model-id> without escaping results in:

     {"error":{"message":"transformation failed from type bedrock://llm/v1/chat: no generation candidates received from Bedrock, or max_tokens too short"}}

   • Using the URL-encoded ARN returns:

     {"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."}

     This error typically occurs when the request reaches an AWS service endpoint, but SigV4 signing is incorrect. That’s why I suspected the issue lies in escaping (assuming this request is hitting bedrock).

I also found a useful example of how SigV4 should handle URL escaping in the aws-sdk-v2 for Go:
aws/aws-sdk-go-v2#L31.

My hypothesis is that Kong is trying to route after the / in the ARN, but I could be entirely mistaken.

EDIT: looking at your logs, I would try to escape the remaining / before PROMPT12345:1 to see if you get the same behavior I am experiencing.