Skip to content

chore: update bedrock IAM permissions in docs #1166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
whysumedh wants to merge 1 commit into
base: main
Choose a base branch
from
Open

chore: update bedrock IAM permissions in docs #1166

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions aigw/integrations/libraries/claude-code-bedrock.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ Enter your AWS credentials:

Create a slug like `bedrock-prod`.

<Note>
**IAM permissions:** the credentials (or assumed role) you use here need `bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream` **and** `bedrock:CountTokens`. Claude Code calls the count-tokens API on every turn, so a role without it fails with `is not authorized to perform: bedrock:CountTokens`. Add `bedrock:GetInferenceProfile` if your model ID is an inference profile ARN. See [Connect Bedrock with Amazon Assumed Role](/aigw/product/model-catalog/connect-bedrock-with-amazon-assumed-role) for the full policy.
</Note>

<Frame>
<img src="/images/product/model-catalog/create-provider-page.png" width="500"/>
</Frame>
Expand Down
97 changes: 96 additions & 1 deletion aigw/product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ This role you create will be used by the AI Gateway to execute InvokeModel comma

### Create a permission policy in your AWS account using the following JSON

These actions cover standard inference — chat completions, the Messages API, embeddings, image generation, and token counting:

```json
{
"Version": "2012-10-17",
Expand All @@ -24,14 +26,107 @@ This role you create will be used by the AI Gateway to execute InvokeModel comma
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
"bedrock:InvokeModelWithResponseStream",
"bedrock:CountTokens"
],
"Resource": "*"
}
]
}
```

<Note>
`bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` also authorize the Converse and ConverseStream APIs, which the AI Gateway uses for most models.

`bedrock:CountTokens` backs the `/v1/messages/count_tokens` endpoint. Clients such as Claude Code call it on every turn, so omitting it shows up as `is not authorized to perform: bedrock:CountTokens`. It is a recent addition to Bedrock — roles created before it existed need it added explicitly.
</Note>

### Additional permissions by feature

Add these only if you use the corresponding feature — everything else works with the three actions above.

| Feature | Additional actions |
| :-- | :-- |
| [Inference profiles](/aigw/integrations/llms/bedrock/aws-bedrock#inference-profiles) (the model ID is an inference profile ARN) | `bedrock:GetInferenceProfile` |
| Model discovery (listing Bedrock models through the AI Gateway) | `bedrock:ListFoundationModels` |
| [Bedrock Guardrails](/aigw/integrations/guardrails/bedrock-guardrails) | `bedrock:ApplyGuardrail` |
| [Reranking](/aigw/integrations/llms/bedrock/rerank) | `bedrock:Rerank` |
| [Batch inference](/aigw/integrations/llms/bedrock/batches) | `bedrock:CreateModelInvocationJob`, `bedrock:GetModelInvocationJob`, `bedrock:ListModelInvocationJobs`, `bedrock:StopModelInvocationJob`, plus S3 access to the input/output bucket and `iam:PassRole` for the Bedrock service role — see the [full batch policy](/aigw/integrations/llms/bedrock/batches#information-about-permissions-and-iam-roles) |
| [Fine-tuning](/aigw/integrations/llms/bedrock/fine-tuning) | `bedrock:CreateModelCustomizationJob`, `bedrock:GetModelCustomizationJob`, `bedrock:ListModelCustomizationJobs`, `bedrock:StopModelCustomizationJob`, plus S3 access to the training-data bucket and `iam:PassRole` |
| [Files](/aigw/integrations/llms/bedrock/files) | `s3:PutObject`, `s3:GetObject`, `s3:GetObjectAttributes`, `s3:AbortMultipartUpload`, `s3:ListBucket` on your bucket |


<Accordion title="Combined policy with every optional action">
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BedrockInference",
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream",
"bedrock:CountTokens",
"bedrock:ApplyGuardrail",
"bedrock:Rerank",
"bedrock:ListFoundationModels",
"bedrock:GetFoundationModel",
"bedrock:GetInferenceProfile"
],
"Resource": "*"
},
{
"Sid": "BedrockBatchAndFineTuning",
"Effect": "Allow",
"Action": [
"bedrock:CreateModelInvocationJob",
"bedrock:GetModelInvocationJob",
"bedrock:ListModelInvocationJobs",
"bedrock:StopModelInvocationJob",
"bedrock:CreateModelCustomizationJob",
"bedrock:GetModelCustomizationJob",
"bedrock:ListModelCustomizationJobs",
"bedrock:StopModelCustomizationJob",
"bedrock:TagResource",
"bedrock:UntagResource",
"bedrock:ListTagsForResource"
],
"Resource": "*"
},
{
"Sid": "S3ForFilesAndJobIO",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::<bucket>",
"arn:aws:s3:::<bucket>/*"
]
},
{
"Sid": "PassBedrockServiceRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::<account_id>:role/<bedrock_service_role>",
"Condition": {
"StringEquals": {
"iam:PassedToService": "bedrock.amazonaws.com"
}
}
}
]
}
```

Replace `<bucket>`, `<account_id>` and `<bedrock_service_role>` with your own values, and narrow each `Resource` to the model, inference profile and job ARNs you actually use.
</Accordion>

### Create a new IAM role
Choose *AWS account* as the trusted entity type. If you set an external ID be sure to copy it, we will need it later.

Expand Down
97 changes: 96 additions & 1 deletion aigw/product/model-catalog/connect-bedrock-with-amazon-assumed-role.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ This role you create will be used by the AI Gateway to execute InvokeModel comma

### Create a permission policy in your AWS account using the following JSON

These actions cover standard inference — chat completions, the Messages API, embeddings, image generation, and token counting:

```json
{
"Version": "2012-10-17",
Expand All @@ -25,14 +27,107 @@ This role you create will be used by the AI Gateway to execute InvokeModel comma
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
"bedrock:InvokeModelWithResponseStream",
"bedrock:CountTokens"
],
"Resource": "*"
}
]
}
```

<Note>
`bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` also authorize the Converse and ConverseStream APIs, which the AI Gateway uses for most models.

`bedrock:CountTokens` backs the `/v1/messages/count_tokens` endpoint. Clients such as Claude Code call it on every turn, so omitting it shows up as `is not authorized to perform: bedrock:CountTokens`. It is a recent addition to Bedrock — roles created before it existed need it added explicitly.
</Note>

### Additional permissions by feature

Add these only if you use the corresponding feature — everything else works with the three actions above.

| Feature | Additional actions |
| :-- | :-- |
| [Inference profiles](/aigw/integrations/llms/bedrock/aws-bedrock#inference-profiles) (the model ID is an inference profile ARN) | `bedrock:GetInferenceProfile` |
| Model discovery (listing Bedrock models through the AI Gateway) | `bedrock:ListFoundationModels` |
| [Bedrock Guardrails](/aigw/integrations/guardrails/bedrock-guardrails) | `bedrock:ApplyGuardrail` |
| [Reranking](/aigw/integrations/llms/bedrock/rerank) | `bedrock:Rerank` |
| [Batch inference](/aigw/integrations/llms/bedrock/batches) | `bedrock:CreateModelInvocationJob`, `bedrock:GetModelInvocationJob`, `bedrock:ListModelInvocationJobs`, `bedrock:StopModelInvocationJob`, plus S3 access to the input/output bucket and `iam:PassRole` for the Bedrock service role — see the [full batch policy](/aigw/integrations/llms/bedrock/batches#information-about-permissions-and-iam-roles) |
| [Fine-tuning](/aigw/integrations/llms/bedrock/fine-tuning) | `bedrock:CreateModelCustomizationJob`, `bedrock:GetModelCustomizationJob`, `bedrock:ListModelCustomizationJobs`, `bedrock:StopModelCustomizationJob`, plus S3 access to the training-data bucket and `iam:PassRole` |
| [Files](/aigw/integrations/llms/bedrock/files) | `s3:PutObject`, `s3:GetObject`, `s3:GetObjectAttributes`, `s3:AbortMultipartUpload`, `s3:ListBucket` on your bucket |


<Accordion title="Combined policy with every optional action">
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BedrockInference",
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream",
"bedrock:CountTokens",
"bedrock:ApplyGuardrail",
"bedrock:Rerank",
"bedrock:ListFoundationModels",
"bedrock:GetFoundationModel",
"bedrock:GetInferenceProfile"
],
"Resource": "*"
},
{
"Sid": "BedrockBatchAndFineTuning",
"Effect": "Allow",
"Action": [
"bedrock:CreateModelInvocationJob",
"bedrock:GetModelInvocationJob",
"bedrock:ListModelInvocationJobs",
"bedrock:StopModelInvocationJob",
"bedrock:CreateModelCustomizationJob",
"bedrock:GetModelCustomizationJob",
"bedrock:ListModelCustomizationJobs",
"bedrock:StopModelCustomizationJob",
"bedrock:TagResource",
"bedrock:UntagResource",
"bedrock:ListTagsForResource"
],
"Resource": "*"
},
{
"Sid": "S3ForFilesAndJobIO",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::<bucket>",
"arn:aws:s3:::<bucket>/*"
]
},
{
"Sid": "PassBedrockServiceRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::<account_id>:role/<bedrock_service_role>",
"Condition": {
"StringEquals": {
"iam:PassedToService": "bedrock.amazonaws.com"
}
}
}
]
}
```

Replace `<bucket>`, `<account_id>` and `<bedrock_service_role>` with your own values, and narrow each `Resource` to the model, inference profile and job ARNs you actually use.
</Accordion>

### Create a new IAM role
Choose *AWS account* as the trusted entity type. If you set an external ID be sure to copy it, we will need it later.

Expand Down
2 changes: 1 addition & 1 deletion aigw/self-hosting/hybrid-deployments/aws/eks.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -584,7 +584,7 @@ cat >bedrock-access-policy.json <<EOF
"Statement": [
{
"Effect": "Allow",
"Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
"Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream", "bedrock:CountTokens"],
"Resource": [
"arn:aws:bedrock:<region>::foundation-model/<model-id>",
"arn:aws:bedrock:<region>:<account-id>:provisioned-model/<model-id>"
Expand Down
4 changes: 4 additions & 0 deletions integrations/libraries/claude-code-bedrock.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ Enter your AWS credentials:

Create a slug like `bedrock-prod`.

<Note>
**IAM permissions:** the credentials (or assumed role) you use here need `bedrock:InvokeModel`, `bedrock:InvokeModelWithResponseStream` **and** `bedrock:CountTokens`. Claude Code calls the count-tokens API on every turn, so a role without it fails with `is not authorized to perform: bedrock:CountTokens`. Add `bedrock:GetInferenceProfile` if your model ID is an inference profile ARN. See [Connect Bedrock with Amazon Assumed Role](/product/model-catalog/connect-bedrock-with-amazon-assumed-role) for the full policy.
</Note>

<Frame>
<img src="/images/product/model-catalog/create-provider-page.png" width="500"/>
</Frame>
Expand Down
4 changes: 4 additions & 0 deletions integrations/llms/bedrock/count-tokens.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ Portkey supports the [AWS Bedrock CountTokens API](https://docs.aws.amazon.com/b

The token count returned by this operation matches the token count that would be charged if the same input were sent to the model in an `InvokeModel` or `Converse` request.

<Note>
Your Bedrock credentials need the `bedrock:CountTokens` IAM action on the foundation models you call. It is separate from `bedrock:InvokeModel`, so roles created before this action existed will fail with `is not authorized to perform: bedrock:CountTokens`. See [Connect Bedrock with Amazon Assumed Role](/product/model-catalog/connect-bedrock-with-amazon-assumed-role) for the full policy.
</Note>

## Using Count Tokens with Portkey

Portkey supports Bedrock's CountTokens endpoint through the Anthropic-compatible format, allowing you to use the same API signature across Bedrock, Vertex AI, and Anthropic providers.
Expand Down
97 changes: 96 additions & 1 deletion product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ This role you create will be used by Portkey to execute InvokeModel commands on

### Create a permission policy in your AWS account using the following JSON

These actions cover standard inference — chat completions, the Messages API, embeddings, image generation, and token counting:

```json
{
"Version": "2012-10-17",
Expand All @@ -34,18 +36,111 @@ This role you create will be used by Portkey to execute InvokeModel commands on
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
"bedrock:InvokeModelWithResponseStream",
"bedrock:CountTokens"
],
"Resource": "*"
}
]
}
```

<Note>
`bedrock:InvokeModel` and `bedrock:InvokeModelWithResponseStream` also authorize the Converse and ConverseStream APIs, which Portkey uses for most models.

`bedrock:CountTokens` backs the `/v1/messages/count_tokens` endpoint. Clients such as Claude Code call it on every turn, so omitting it shows up as `is not authorized to perform: bedrock:CountTokens`. It is a recent addition to Bedrock — roles created before it existed need it added explicitly.
</Note>

<Frame>
<img src="/images/product/ai-gateway/create-policy.png"/>
</Frame>

### Additional permissions by feature

Add these only if you use the corresponding feature — everything else works with the three actions above.

| Feature | Additional actions |
| :-- | :-- |
| [Inference profiles](/integrations/llms/bedrock/aws-bedrock#inference-profiles) (the model ID is an inference profile ARN) | `bedrock:GetInferenceProfile` |
| Model discovery (listing Bedrock models through Portkey) | `bedrock:ListFoundationModels` |
| [Bedrock Guardrails](/integrations/guardrails/bedrock-guardrails) | `bedrock:ApplyGuardrail` |
| [Reranking](/integrations/llms/bedrock/rerank) | `bedrock:Rerank` |
| [Batch inference](/integrations/llms/bedrock/batches) | `bedrock:CreateModelInvocationJob`, `bedrock:GetModelInvocationJob`, `bedrock:ListModelInvocationJobs`, `bedrock:StopModelInvocationJob`, plus S3 access to the input/output bucket and `iam:PassRole` for the Bedrock service role — see the [full batch policy](/integrations/llms/bedrock/batches#information-about-permissions-and-iam-roles) |
| [Fine-tuning](/integrations/llms/bedrock/fine-tuning) | `bedrock:CreateModelCustomizationJob`, `bedrock:GetModelCustomizationJob`, `bedrock:ListModelCustomizationJobs`, `bedrock:StopModelCustomizationJob`, plus S3 access to the training-data bucket and `iam:PassRole` |
| [Files](/integrations/llms/bedrock/files) | `s3:PutObject`, `s3:GetObject`, `s3:GetObjectAttributes`, `s3:AbortMultipartUpload`, `s3:ListBucket` on your bucket |


<Accordion title="Combined policy with every optional action">
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BedrockInference",
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream",
"bedrock:CountTokens",
"bedrock:ApplyGuardrail",
"bedrock:Rerank",
"bedrock:ListFoundationModels",
"bedrock:GetFoundationModel",
"bedrock:GetInferenceProfile"
],
"Resource": "*"
},
{
"Sid": "BedrockBatchAndFineTuning",
"Effect": "Allow",
"Action": [
"bedrock:CreateModelInvocationJob",
"bedrock:GetModelInvocationJob",
"bedrock:ListModelInvocationJobs",
"bedrock:StopModelInvocationJob",
"bedrock:CreateModelCustomizationJob",
"bedrock:GetModelCustomizationJob",
"bedrock:ListModelCustomizationJobs",
"bedrock:StopModelCustomizationJob",
"bedrock:TagResource",
"bedrock:UntagResource",
"bedrock:ListTagsForResource"
],
"Resource": "*"
},
{
"Sid": "S3ForFilesAndJobIO",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectAttributes",
"s3:AbortMultipartUpload"
],
"Resource": [
"arn:aws:s3:::<bucket>",
"arn:aws:s3:::<bucket>/*"
]
},
{
"Sid": "PassBedrockServiceRole",
"Effect": "Allow",
"Action": "iam:PassRole",
"Resource": "arn:aws:iam::<account_id>:role/<bedrock_service_role>",
"Condition": {
"StringEquals": {
"iam:PassedToService": "bedrock.amazonaws.com"
}
}
}
]
}
```

Replace `<bucket>`, `<account_id>` and `<bedrock_service_role>` with your own values, and narrow each `Resource` to the model, inference profile and job ARNs you actually use.
</Accordion>

### Create a new IAM role
Choose *AWS account* as the trusted entity type. If you set an external ID be sure to copy it, we will need it later.

Expand Down
Loading
Loading