# Portkey Docs

> Comprehensive documentation for Portkey's AI Gateway, Guardrails, Observability, Prompts, and Governance features.

## Docs

- [How to Contribute](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/README.md)
- [Get cache hit latency data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-cache-hit-latency-data.md)
- [Get cache hit rate data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-cache-hit-rate-data.md)
- [Get cost data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-cost-data.md)
- [Get error rate data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-error-rate-data.md)
- [Get errors data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-errors-data.md)
- [Get feedback data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-feedback-data.md)
- [Get feedback per ai models data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-feedback-per-ai-models-data.md)
- [Get feedback score distribution data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-feedback-score-distribution-data.md)
- [Get latency data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-latency-data.md)
- [Get requests data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-requests-data.md)
- [Get requests per user data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-requests-per-user-data.md)
- [Get rescued requests data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-rescued-requests-data.md)
- [Get status code data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-status-code-data.md)
- [Get tokens data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-tokens-data.md)
- [Get unique status code data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-unique-status-code-data.md)
- [Get users data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-users-data.md)
- [Get weighted feedback data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/graphs-time-series-data/get-weighted-feedback-data.md)
- [Get Metadata Grouped Data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/groups-paginated-data/get-metadata-grouped-data.md)
- [Get Model Grouped Data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/groups-paginated-data/get-model-grouped-data.md)
- [Get User Grouped Data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/groups-paginated-data/get-user-grouped-data.md)
- [Get User Grouped Data](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/analytics/summary/get-all-cache-data.md)
- [Create API Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/api-keys/create-api-key.md)
- [Delete an API Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/api-keys/delete-an-api-key.md)
- [List API Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/api-keys/list-api-keys.md)
- [Retrieve an API Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/api-keys/retrieve-an-api-key.md)
- [Update API Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/api-keys/update-api-key.md)
- [List Audit Logs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/audit-logs/list-audit-logs.md)
- [Create Config](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/configs/create-config.md)
- [Delete Config](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/configs/delete-config.md)
- [List Configs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/configs/list-configs.md)
- [Retrieve Config](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/configs/retrieve-config.md)
- [Update Config](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/configs/update-config.md)
- [Create Guardrail](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/guardrails/create-guardrail.md): Creates a new guardrail with specified checks and actions
- [Delete Guardrail](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/guardrails/delete-guardrail.md): Deletes an existing guardrail
- [List Guardrails](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/guardrails/list-guardrails.md): Retrieves a paginated list of guardrails for the specified workspace or organisation
- [Retrieve Guardrail](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/guardrails/retrieve-guardrail.md): Retrieves details of a specific guardrail by ID or slug
- [Update Guardrail](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/guardrails/update-guardrail.md): Updates an existing guardrail's name, checks, or actions
- [Create Integration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/create-integration.md)
- [Delete Integration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/delete-integration.md)
- [List Integrations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/list-integrations.md)
- [Delete Custom Model](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/models/delete-custom-model.md): Removes multiple custom models from an integration by their slugs.
- [List Model Access](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/models/list-model-access.md): Retrieves all model access for a specific integration with their configuration and pricing details.
- [Update Model Access](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/models/update-model-access.md): Updates model access, pricing configuration, and settings for multiple models in an integration. Can enable/disable models and configure custom pricing.
- [Retrieve Integration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/retrieve-integration.md)
- [Update Integration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/update-integration.md)
- [List Workspace Access](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/workspaces/list-workspace-access.md): Retrieves workspace access configuration for an integration, including usage limits and rate limits.
- [Update Workspace Access](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/integrations/workspaces/update-workspace-access.md): Updates workspace access permissions, usage limits, and rate limits for an integration. Can configure global workspace access or per-workspace settings.
- [Create Prompt Collection](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/collections/create-collection.md): Creates a new collection in the specified workspace
- [Delete Prompt Collection](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/collections/delete-collection.md): Deletes a collection
- [List Prompt Collections](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/collections/list-collections.md): Lists all collections in the specified workspace
- [Retrieve Prompt Collection](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/collections/retrieve-collection.md): Retrieves details of a specific collection
- [Update Prompt Collection](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/collections/update-collection.md): Updates a collection's details
- [Create Prompt](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/create-prompt.md)
- [Delete Prompt](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/delete-prompt.md)
- [Create Label](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/labels/create-label.md): Creates a new label in the system
- [Delete Label](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/labels/delete-label.md): Deletes a label
- [List Labels](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/labels/list-labels.md): Returns a list of labels based on filters
- [Retrieve Label](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/labels/retrieve-label.md): Returns a specific label by its ID
- [Update Label](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/labels/update-label.md): Updates an existing label
- [List Prompt Versions](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/list-prompt-versions.md)
- [List Prompts](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/list-prompts.md)
- [Create Partial](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/partials/create-partial.md)
- [Delete Partial](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/partials/delete-partial.md)
- [List Partial Versions](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/partials/list-partial-versions.md)
- [List Partials](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/partials/list-partials.md)
- [Publish Partial](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/partials/publish-partial.md)
- [Retrieve Partial](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/partials/retrieve-partial.md)
- [Update Partial](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/partials/update-partial.md)
- [Publish Prompt](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/publish-prompt.md)
- [Retrieve Prompt](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/retrieve-prompt.md)
- [Retrieve Prompt Version](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/retrieve-prompt-version.md)
- [Update Prompt](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/update-prompt.md)
- [Update Prompt Version](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/prompts/update-prompt-version.md)
- [Create Provider](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/providers/create-provider.md)
- [Delete Provider](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/providers/delete-provider.md)
- [List Providers](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/providers/list-providers.md)
- [Retrieve Provider](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/providers/retrieve-provider.md)
- [Update Provider](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/providers/update-provider.md)
- [Delete a user invite](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/user-invites/delete-a-user-invite.md)
- [Invite a User](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/user-invites/invite-a-user.md): Send an invite to user for your organization
- [Resend a user invite](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/user-invites/resend-a-user-invite.md): Resend an invite to user for your organization
- [Retrieve all user invite](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/user-invites/retrieve-all-user-invites.md)
- [Retrieve an user invite](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/user-invites/retrieve-an-invite.md)
- [Remove a user](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/users/remove-a-user.md)
- [Retrieve a user](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/users/retrieve-a-user.md)
- [Retrieve all users](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/users/retrieve-all-users.md)
- [Update a user](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/users/update-a-user.md)
- [Create Virtual Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/virtual-keys/create-virtual-key.md)
- [Delete Virtual Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/virtual-keys/delete-virtual-key.md)
- [List Virtual Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/virtual-keys/list-virtual-keys.md)
- [Retrieve Virtual Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/virtual-keys/retrieve-virtual-key.md)
- [Update Virtual Key](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/virtual-keys/update-virtual-key.md)
- [Add a Workspace Member](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspace-members/add-a-workspace-member.md)
- [Remove Workspace Member](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspace-members/remove-workspace-member.md)
- [Retrieve a Workspace Member](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspace-members/retrieve-a-workspace-member.md)
- [Retrieve all Workspace Member](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspace-members/retrieve-all-workspace-members.md)
- [Update Workspace Member](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspace-members/update-workspace-member.md)
- [Create Workspace](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspaces/create-workspace.md)
- [Delete a Workspace](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspaces/delete-a-workspace.md)
- [Retrieve a Workspace](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspaces/retrieve-a-workspace.md)
- [Retrieve all Workspaces](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspaces/retrieve-all-workspaces.md)
- [Update Workspace](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/control-plane/workspaces/update-workspace.md)
- [Create Feedback](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/feedback/create-feedback.md): This endpoint allows users to submit feedback for a particular interaction or response.
- [Update Feedback](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/feedback/update-feedback.md): This endpoint allows users to update existing feedback.
- [Insert a Log](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/insert-a-log.md): Submit one or more log entries
- [Cancel a Log Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/log-exports-beta/cancel-a-log-export.md)
- [Create a Log Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/log-exports-beta/create-a-log-export.md)
- [Download a Log Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/log-exports-beta/download-a-log-export.md)
- [List a Log Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/log-exports-beta/list-log-exports.md)
- [Retrieve a Log Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/log-exports-beta/retrieve-a-log-export.md)
- [Start a Log Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/log-exports-beta/start-a-log-export.md)
- [Update a Log Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/data-plane/logs/log-exports-beta/update-a-log-export.md)
- [Errors](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/error.md): Error codes returned by the Admin API and how to resolve them.
- [Introduction](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/introduction.md): Manage your Portkey organization and workspaces programmatically
- [OpenAPI Specification](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/admin-api/open-api-specification.md)
- [Create Assistant](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/assistants/create-assistant.md)
- [Delete Assistant](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/assistants/delete-assistant.md)
- [List Assistant](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/assistants/list-assistants.md)
- [Modify Assistant](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/assistants/modify-assistant.md)
- [Retrieve Assistant](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/assistants/retrieve-assistant.md)
- [Create Message](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/messages/create-message.md)
- [Delete Message](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/messages/delete-message.md)
- [List Message](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/messages/list-messages.md)
- [Modify Message](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/messages/modify-message.md)
- [Retrieve Message](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/messages/retrieve-message.md)
- [List Run Steps](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/run-steps/list-run-steps.md)
- [Retrieve Run Steps](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/run-steps/retrieve-run-steps.md)
- [Cancel Run](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/runs/cancel-run.md)
- [Create Run](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/runs/create-run.md)
- [Create thread and Run](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/runs/create-thread-and-run.md)
- [list Run](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/runs/list-runs.md)
- [Modify Run](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/runs/modify-run.md)
- [Retrieve Run](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/runs/retrieve-run.md)
- [Submit Tool Outputs to Run](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/runs/submit-tool-outputs-to-run.md)
- [Create Thread](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/threads/create-thread.md)
- [Delete Thread](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/threads/delete-thread.md)
- [Modify Thread](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/threads/modify-thread.md)
- [Retrieve Thread](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/assistants-api/threads/retrieve-thread.md)
- [Create Speech](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/audio/create-speech.md)
- [Create Transcription](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/audio/create-transcription.md)
- [Create Translation](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/audio/create-translation.md)
- [Authentication](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/authentication.md)
- [Cancel Batch](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/batch/cancel-batch.md)
- [Create Batch](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/batch/create-batch.md)
- [List Batch](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/batch/list-batch.md)
- [Retrieve Batch](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/batch/retrieve-batch.md)
- [Chat](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/chat.md)
- [Completions](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/completions.md)
- [Gateway Config Object](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/config-object.md)
- [Embeddings](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/embeddings.md)
- [Errors](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/error-codes.md)
- [Delete File](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/files/delete-file.md)
- [List Files](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/files/list-files.md)
- [Retrieve File](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/files/retrieve-file.md)
- [Retrieve File Content](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/files/retrieve-file-content.md)
- [Upload File](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/files/upload-file.md)
- [Cancel Fine-tuning](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/fine-tuning/cancel-fine-tuning.md)
- [Create Fine-tuning Job](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/fine-tuning/create-fine-tuning-job.md): Finetune a provider model
- [List Fine-tuning Checkpoints](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/fine-tuning/list-fine-tuning-checkpoints.md)
- [List Fine-tuning Events](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/fine-tuning/list-fine-tuning-events.md)
- [List Fine-tuning Jobs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/fine-tuning/list-fine-tuning-jobs.md)
- [Retrieve Fine-tuning Job](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/fine-tuning/retrieve-fine-tuning-job.md)
- [Gateway to Other APIs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/gateway-for-other-apis.md): Access any custom provider endpoint through Portkey API
- [Headers](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/headers.md): Header requirements and options for the Portkey API
- [Create Image](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/images/create-image.md)
- [Create Image Edit](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/images/create-image-edit.md)
- [Create Image Variation](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/images/create-image-variation.md)
- [Introduction](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/introduction.md): This documentation provides detailed information about the various ways you can access and interact with Portkey - **a robust AI gateway** designed to simplify and enhance your experience with Large Language Models (LLMs) like OpenAI's GPT models.
- [Models](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/models/models.md): Lists the currently available models that can be used through Portkey, and provides basic information about each one.
- [Moderations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/moderations.md)
- [OpenAPI Specification](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/open-api-specification.md)
- [Prompt Completions](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/prompts/prompt-completion.md): Execute your saved prompt templates on Portkey
- [Prompt Render](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/prompts/render.md): Renders a prompt template with its variable values filled in
- [Response Schema](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/response-schema.md)
- [Delete a Response](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/responses/delete-response.md)
- [Create a Response](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/responses/responses.md)
- [List Input Items](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/responses/retrieve-inputs.md)
- [Get a Response](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/responses/retrieve-response.md)
- [Supported Providers](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/inference-api/supported-providers.md)
- [C# (.NET)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/sdk/c-sharp.md): Integrate Portkey in your `.NET` app easily using the OpenAI library and get advanced monitoring, routing, and enterprise features.
- [Supported SDKs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/sdk/list.md): Find the best way to use Portkey in your preferred language.
- [Node.js](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/sdk/node.md): Official Portkey Node.js SDK – robust, modern, and fully typed integration for JavaScript and TypeScript developers.
- [Python](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/api-reference/sdk/python.md): Official Portkey Python SDK to help take your AI apps to production
- [December](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2024/dec.md)
- [November](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2024/nov.md)
- [October](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2024/oct.md)
- [April](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/apr.md)
- [August](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/august.md)
- [February](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/feb.md)
- [January](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/jan.md)
- [July](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/july.md)
- [June](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/june.md)
- [March](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/mar.md)
- [May](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/may.md)
- [September](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/2025/september.md)
- [Data Service](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/data-service.md)
- [Enterprise Gateway](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/enterprise.md)
- [Helm Chart](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/helm-chart.md)
- [Node.js](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/node-sdk-changelog.md)
- [Latest Updates](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/product.md)
- [Python](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/changelog/python-sdk-changelog.md)
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started.md)
- [101 on Portkey's Gateway Configs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/101-on-portkey-s-gateway-configs.md): You are likely familiar with how to make an API call to GPT4 for chat completions.
- [A/B Test Prompts and Models](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/a-b-test-prompts-and-models.md): A/B testing with large language models in production is crucial for driving optimal performance and user satisfaction.
- [Function Calling](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/function-calling.md): Get the LLM to interact with external APIs!
- [Getting started with AI Gateway](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/getting-started-with-ai-gateway.md)
- [Image Generation](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/image-generation.md)
- [Llama 3 on Groq](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/llama-3-on-groq.md)
- [Return Repeat Requests from Cache](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/return-repeat-requests-from-cache.md): If you have multiple users of your GenAI app triggering the same or similar queries to your models, fetching LLM response from the models can be slow and expensive.
- [Tackling Rate Limiting](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/tackling-rate-limiting.md): LLMs are **costly** to run. As their usage increases, the providers have to balance serving user requests v/s straining their GPU resources too thin. They generally deal with this by putting _rate limits_ on how many requests a user can send in a minute or in a day.
- [Trigger Automatic Retries on LLM Failures](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/getting-started/trigger-automatic-retries-on-llm-failures.md)
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations.md)
- [Anyscale](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/anyscale.md): Portkey helps bring Anyscale APIs to production with its abstractions for observability, fallbacks, caching, and more. Use the Anyscale API **through** Portkey for.
- [Deepinfra](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/deepinfra.md)
- [Groq](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/groq.md)
- [Introduction to GPT-4o](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/introduction-to-gpt-4o.md)
- [Langchain](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/langchain.md)
- [Llama 3 on Portkey + Together AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/llama-3-on-portkey-+-together-ai.md): Try out the new Llama 3 model directly using the OpenAI SDK
- [Mistral](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/mistral.md): Portkey helps bring Mistral's APIs to production with its observability suite & AI Gateway.
- [Mixtral 8x22b](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/mixtral-8x22b.md)
- [Sync Open WebUI Feedback → Portkey](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/openwebui-to-portkey.md): How to export thumbs-up/down from Open WebUI and ingest into Portkey using a one-file Python or Node script.
- [Segmind](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/segmind.md)
- [Vercel AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/integrations/vercel-ai.md): Portkey is a control panel for your Vercel AI app. It makes your LLM integrations prod-ready, reliable, fast, and cost-efficient.
- [Prompts](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/prompts.md)
- [Build a chatbot using Portkey's Prompt Templates](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/prompts/build-a-chatbot-using-portkeys-prompt-templates.md): Portkey's prompt templates offer a powerful solution for testing and building chatbots.
- [Optimizing Prompts for Customer Support using Portkey | LLama Prompt Ops Integration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/prompts/llama-prompts.md)
- [Building an LLM-as-a-Judge System for AI (Customer Support) Agent](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/prompts/llm-as-a-judge.md)
- [Ultimate AI SDR](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/prompts/ultimate-ai-sdr.md): Building a sophisticated AI SDR agent leveraging internet search and evals to draft personalized outreach emails in 15 seconds
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases.md)
- [Build an article suggestion app with Supabase pgvector, and Portkey](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/build-an-article-suggestion-app-with-supabase-pgvector-and-portkey.md): Consider that you have list of support articles that you want to suggest it to users when users search for it. You want to suggest as best fit as possible. With the availability of tools like Large Language Model (LLMs) and Vector Databases, the approach towards suggestions & recommendation systems…
- [Comparing Top10 LMSYS Models with Portkey](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/comparing-top10-lmsys-models-with-portkey.md)
- [Comparing DeepSeek Models Against OpenAI, Anthropic & More Using Portkey](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/deepseek-r1.md)
- [Detecting Emotions with GPT-4o](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/emotions-with-gpt-4o.md)
- [Enforcing JSON Schema with Anyscale & Together](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/enforcing-json-schema-with-anyscale-and-together.md): Get the LLM to adhere to your JSON schema using Anyscale & Together AI's newly introduced JSON modes
- [Unified LLM API with Automatic Failover & Error Handling](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/enterprise-ready-unified-api.md): Build a single API interface with automatic failover and unified error handling across OpenAI, Anthropic, and AWS Bedrock
- [Fallback from SDXL to Dall-e-3](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/fallback-from-sdxl-to-dall-e-3.md): Generative AI models have revolutionized text generation and opened up new possibilities for developers.
- [Testing Application Resilience with Fallbacks](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/fallbacks-test.md)
- [Few-Shot Prompting](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/few-shot-prompting.md): LLMs are highly capable of following a given structure. By providing a few examples of how the assistant should respond to a given prompt, the LLM can generate responses that closely follow the format of these examples.
- [How to use Portkey Guardrails for PII Protection](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/guardrail-pii-use-case.md): Portkey Guardrails is a powerful tool for protecting Personally Identifiable Information (PII) in your AI applications. This guide provides a comprehensive walkthrough of implementing PII protection using Portkey's guardrail capabilities.
- [How to use OpenAI SDK with Portkey Prompt Templates](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/how-to-use-openai-sdk-with-portkey-prompt-templates.md): Portkeys Prompt Playground allows you to test and tinker with various hyperparameters without any external dependencies and deploy them to production seamlessly. Moreover, all team members can use the same prompt template, ensuring that everyone works from the same source of truth.
- [Web Search for Any LLM in LibreChat](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/librechat-web-search.md)
- [Multi-Tenant LLM Access with Enterprise Control for Customer-Facing Apps](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/multi-tenant-ai-feature.md)
- [Portkey with OpenAI Computer Use](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/openai-computer-use.md): Leverage Portkey with OpenAI's Computer Use tool for automated browser interactions with enterprise-grade observability and controls
- [How to Run Structured Output Evals at Scale](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/run-batch-evals.md)
- [Run Portkey on Prompts from Langchain Hub](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/run-portkey-on-prompts-from-langchain-hub.md): Writing the right prompt is often hard to get a quality LLM response. You want the prompt to be specialized and exhaustive enough for your problem. There is a high chance someone else might’ve stumbled across a similar situation and written the prompt you’ve been figuring out all this while.
- [Setting up resilient Load balancers with failure-mitigating Fallbacks](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/setting-up-resilient-load-balancers-with-failure-mitigating-fallbacks.md): Companies often face challenges of scaling their services efficiently as the traffic to their applications grow - when you’re consuming APIs, the first point of failure is that if you hit the API too much, you can get rate limited. Loadbalancing is a proven way to scale usage horizontally without ov…
- [Setup OpenAI -> Azure OpenAI Fallback](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/setup-openai-greater-than-azure-openai-fallback.md): Portkey Fallbacks can automatically switch your app's requests from one LLM provider to another, ensuring reliability by allowing you to fallback among multiple LLMs.
- [Smart Fallback with Model-Optimized Prompts](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/smart-fallback-with-model-optimized-prompts.md): Portkey can help you easily create fallbacks from one LLM to another, making your application more reliable. While Fallback ensures reliability, it also means that you'll be running a prompt optimized for one LLM on another, which can often lead to significant differences in the final output.
- [Tracking LLM Costs Per User with Portkey](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/use-cases/track-costs-using-metadata.md): Monitor and analyze user-level LLM costs across 1600+ models using Portkey's metadata and analytics API.
- [4. Advanced Strategies for Performance Improvement](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/advanced-strategies.md)
- [5. Architectural Considerations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/architectural-considerations.md)
- [10. Conclusion and Key Takeaways](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/conclusion-and-key-takeaways.md): Summarizing the key strategies for LLM cost optimization and performance improvement
- [7. Cost Effective Development Practices](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/cost-effective-development.md)
- [Executive Summary](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/executive-summary.md): Overview of LLM cost optimization and performance improvement strategies
- [3. FrugalGPT Techniques for Cost Optimization](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/frugalgpt-techniques.md)
- [9. Future Trends in LLM Cost Optimization](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/future-trends.md)
- [1. Introduction](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/introduction.md): An overview of the challenges and opportunities in LLM cost optimization
- [2. Understanding LLM Cost Drivers](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/llm-cost-drivers.md): An overview of the factors that influence costs in Large Language Model applications
- [6. Operational Best Pracitces](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/operational-best-practices.md)
- [8. User Education and Change Management](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/guides/whitepapers/optimizing-llm-costs/user-education.md)
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents.md): Portkey helps bring your agents to production
- [Agno AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/agno-ai.md): Use Portkey with Agno to build production-ready autonomous AI agents
- [Autogen](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/autogen.md): Use Portkey with Autogen to take your AI Agents to production
- [Bring Your own Agents](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/bring-your-own-agents.md): You can also use Portkey if you are doing custom agent orchestration!
- [Control Flow](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/control-flow.md): Use Portkey with Control Flow to take your AI Agents to production
- [CrewAI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/crewai.md): Use Portkey with CrewAI to take your AI Agents to production
- [Langchain Agents](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/langchain-agents.md)
- [LangGraph](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/langgraph.md): Use Portkey with LangGraph to take your AI agent workflows to production
- [Langroid](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/langroid.md)
- [LiveKit](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/livekit.md): Build production-ready voice AI agents with Portkey's enterprise features
- [Llama Agents by Llamaindex](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/llama-agents.md): Use Portkey with Llama Agents to take your AI Agents to production
- [OpenAI Agents SDK (Python)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/openai-agents.md): Use Portkey with OpenAI Agents SDK to take your AI Agents to production
- [OpenAI Agents SDK (TypeScript)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/openai-agents-ts.md): Use Portkey with OpenAI Agents SDK to take your AI Agents to production
- [OpenAI Swarm](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/openai-swarm.md): The Portkey x Swarm integration brings advanced AI gateway capabilities, full-stack observability, and reliability features to build production-ready AI agents.
- [Phidata](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/phidata.md): Use Portkey with Phidata to take your AI Agents to production
- [Pydantic AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/pydantic-ai.md): Use Portkey with PydanticAI to take your AI Agents to production
- [Strands Agents](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/agents/strands.md): Use Portkey with AWS's Strands Agents to take your AI Agents to production
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/ai-apps.md)
- [Microsoft Azure](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/cloud/azure.md): Discover how you can build your Gen AI platform on Azure using Portkey
- [Integrations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/ecosystem.md)
- [Acuvity](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/acuvity.md): Acuvity is model agnostic GenAI security solution. It is built to secure existing and future GenAI models, apps, services, tools, plugins and more.
- [Aporia](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/aporia.md)
- [Azure Guardrails](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/azure-guardrails.md): Integrate Microsoft Azure's powerful content moderation services & PII guardrails with Portkey
- [AWS Bedrock Guardrails](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/bedrock-guardrails.md): Secure your AI applications with AWS Bedrock's guardrail capabilities through Portkey.
- [Bring Your Own Guardrails](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/bring-your-own-guardrails.md): Integrate your custom guardrails with Portkey using webhooks
- [Javelin](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/javelin.md): Javelin provides comprehensive AI security guardrails including Trust & Safety, Prompt Injection Detection, and Language Detection for enterprise AI agents & applications.
- [Lasso Security](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/lasso.md): Lasso Security protects your GenAI apps from data leaks, prompt injections, and other potential risks, keeping your systems safe and secure.
- [Mistral](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/mistral.md): Mistral moderation service helps detect and filter harmful content across multiple policy dimensions to secure your AI applications.
- [Palo Alto Networks Prisma AIRS](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/palo-alto-panw-prisma.md): Comprehensive AI security platform providing runtime protection against prompt injections, data leakage, and AI-specific threats
- [Pangea](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/pangea.md): Pangea AI Guard helps analyze and redact text to prevent model manipulation and malicious content.
- [Patronus AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/patronus-ai.md): Patronus excels in industry-specific guardrails for RAG workflows.
- [Pillar](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/pillar.md)
- [Prompt Security](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/prompt-security.md): Prompt Security detects and protects against prompt injection, sensitive data exposure, and other AI security threats.
- [Replace Custom Regex Patterns](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/guardrails/regex.md): Redact custom patterns with Regex in Portkey.
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries.md)
- [Anthropic Computer Use](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/anthropic-computer-use.md)
- [AnythingLLM](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/anythingllm.md): Add usage tracking, cost controls, and security guardrails to your AnythingLLM deployment
- [Autogen (DEPRECATED)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/autogen.md): AutoGen is a framework that enables the development of LLM applications using multiple agents that can converse with each other to solve tasks.
- [Claude Code](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/claude-code.md): Integrate Portkey with Claude Code for enterprise-grade AI coding assistance with observability and governance
- [Cline](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/cline.md): Add enterprise-grade observability, cost tracking, and governance to your Cline AI coding assistant
- [OpenAI Codex CLI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/codex.md): Add usage tracking, cost controls, and security guardrails to your Codex CLI deployment
- [Cursor](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/cursor.md): Learn how to integrate Portkey's enterprise features with Cursor for enhanced observability, reliability and governance.
- [DSPy](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/dspy.md): Integrate DSPy with Portkey for production-ready LLM pipelines
- [GitHub Copilot](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/github-copilot.md): Use Portkey with GitHub Copilot for centralized governance, observability, and cost controls.
- [Goose](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/goose.md): Add usage tracking, cost controls, and security guardrails to your Goose AI agent
- [Instructor](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/instructor.md): With Portkey, you can confidently take your Instructor pipelines to production and get complete observability over all of your calls + make them reliable - all with a 2 LOC change!
- [Jan](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/janhq.md): Add usage tracking, cost controls, and security guardrails to your Jan deployment
- [Langchain (JS/TS)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/langchain-js.md): Portkey adds core production capabilities to any Langchain app.
- [Langchain (Python)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/langchain-python.md): Supercharge Langchain apps with Portkey: Multi-LLM, observability, caching, reliability, and prompt management.
- [Langflow](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/langflow.md): Add enterprise-grade features to your Langflow AI workflows with Portkey
- [LibreChat](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/librechat.md): Cost tracking, observability, and more for LibreChat
- [LlamaIndex (Python)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/llama-index-python.md): The **Portkey x LlamaIndex** integration brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** to apps built on LlamaIndex.
- [Microsoft Semantic Kernel](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/microsoft-semantic-kernel.md)
- [MindsDb](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/mindsdb.md): Integrate MindsDB with Portkey to build enterprise-grade AI use-cases
- [MongoDB](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/mongodb.md)
- [n8n](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/n8n.md): Add usage tracking, cost controls, and security guardrails to your n8n workflows
- [OpenAI Agent Builder (TS)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/openai-agent-builder.md): Use Portkey with OpenAI Agent Builder to add observability, reliability, and production features to your visual agent workflows
- [OpenAI Agent Builder (Python)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/openai-agent-builder-python.md): Use Portkey with OpenAI Agent Builder to add observability, reliability, and production features to your visual agent workflows
- [Portkey with Any OpenAI Compatible Project](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/openai-compatible.md): Learn how to integrate Portkey's enterprise features with any OpenAI Compliant project for enhanced observability, reliability and governance.
- [Open WebUI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/openwebui.md): Enterprise-grade cost tracking, observability, and more for Open WebUI
- [Image Genration with OpenWebUI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/openwebui/image-generation-with-openwebui.md)
- [Promptfoo](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/promptfoo.md): Portkey brings advanced **AI gateway** capabilities, full-stack **observability**, and **prompt management** + **versioning** to your **Promptfoo** projects. This document provides an overview of how to leverage the strengths of both the platforms to streamline your AI development workflow.
- [Roo Code](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/roo-code.md): Add enterprise-grade observability, cost tracking, and governance to your Roo AI coding assistant
- [Supabase](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/supabase.md)
- [ToolJet](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/tooljet.md): ToolJet is a low-code platform that lets you build apps by connecting APIs and data sources, with Portkey integration adding AI features like chat interfaces and automation.
- [Vercel](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/vercel.md): Integrate Portkey with Vercel AI SDK for production-ready and reliable AI apps
- [Zed](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/libraries/zed.md): Learn how to integrate Portkey's enterprise features with Zed for enhanced observability, reliability and governance.
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms.md): Portkey connects with all major LLM providers and orchestration frameworks.
- [AI21](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/ai21.md)
- [Anthropic](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/anthropic.md)
- [Computer use tool](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/anthropic/computer-use.md)
- [Count Tokens](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/anthropic/count-tokens.md)
- [Prompt Caching](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/anthropic/prompt-caching.md)
- [Remote MCP Support](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/anthropic/remote-mcp.md)
- [Anyscale](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/anyscale-llama2-mistral-zephyr.md): Integrate Anyscale endpoints with Portkey seamlessly and make your OSS models production-ready
- [AWS SageMaker](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/aws-sagemaker.md): Route to your AWS Sagemaker models through Portkey
- [Azure AI Foundry](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/azure-foundry.md): Learn how to integrate Azure AI Foundry with Portkey to access a wide range of AI models with enhanced observability and reliability features.
- [Azure OpenAI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/azure-openai/azure-openai.md): Azure OpenAI is a great alternative to accessing the best models including GPT-4 and more in your private environments. Portkey provides complete support for Azure OpenAI.
- [Batches](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/azure-openai/batches.md): Perform batch inference with Azure OpenAI
- [Files](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/azure-openai/files.md): Upload files to Azure OpenAI
- [Fine-tune](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/azure-openai/fine-tuning.md): Fine-tune your models with Azure OpenAI
- [AWS Bedrock](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/bedrock/aws-bedrock.md)
- [Batches](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/bedrock/batches.md): Perform batch inference with Bedrock
- [AWS Bedrock Knowledge Bases](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/bedrock/bedrock-knowledgebase.md): Create, manage, and connect your LLMs to organizational data using AWS Bedrock Knowledge Bases through Portkey.
- [Embeddings](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/bedrock/embeddings.md): Get embeddings from Bedrock
- [Files](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/bedrock/files.md): Upload files to S3 for Bedrock batch inference
- [Fine-tune](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/bedrock/fine-tuning.md): Fine-tune your models with Bedrock
- [Prompt Caching on Bedrock](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/bedrock/prompt-caching.md)
- [Bring Your Own LLM](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/byollm.md): Integrate your privately hosted LLMs with Portkey for unified management, observability, and reliability.
- [Cerebras](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/cerebras.md)
- [Cohere](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/cohere.md)
- [Dashscope](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/dashscope.md): Integrate Dashscope with Portkey for seamless completions, prompt management, and advanced features like streaming, function calling, and fine-tuning.
- [Deepbricks](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/deepbricks.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [Deepbricks](https://deepbricks.ai/).
- [Deepgram](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/deepgram.md): Portkey provides a robust and secure gateway to use and observe Deepgrm's Speech-to-Text API.
- [Deepinfra](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/deepinfra.md)
- [DeepSeek](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/deepseek.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including DeepSeek models.
- [Featherless AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/featherless.md)
- [Fireworks](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/fireworks.md)
- [Files](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/fireworks/files.md): Upload files to Fireworks
- [Fine-tune](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/fireworks/fine-tuning.md): Fine-tune your models with Bedrock
- [Google Gemini](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/gemini.md)
- [Github](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/github.md)
- [Groq](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/groq.md)
- [Hugging Face](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/huggingface.md)
- [Inference.net](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/inference.net.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including the models hosted on [Inference.net](https://www.inference.net/).
- [Jina AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/jina-ai.md)
- [Lambda Labs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/lambda.md): Integrate Lambda with Portkey AI for seamless completions, prompt management, and advanced features like streaming and function calling.
- [Lemonfox-AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/lemon-fox.md): Integrate LemonFox with Portkey for seamless completions, prompt management, and advanced features like streaming, function calling, and fine-tuning.
- [Lepton AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/lepton.md)
- [Lingyi (01.ai)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/lingyi-01.ai.md)
- [LocalAI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/local-ai.md)
- [Mistral AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/mistral-ai.md)
- [Monster API](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/monster-api.md): MonsterAPIs provides access to generative AI model APIs at 80% lower costs. Connect to MonsterAPI LLM APIs seamlessly through Portkey's AI gateway.
- [Moonshot](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/moonshot.md)
- [Ncompass](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/ncompass.md)
- [Nebius](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/nebius.md)
- [Nomic](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/nomic.md)
- [Novita AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/novita-ai.md)
- [Nscale (EU Sovereign)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/nscale.md)
- [Ollama](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/ollama.md)
- [OpenAI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openai.md): Learn to integrate OpenAI with Portkey, enabling seamless completions, prompt management, and advanced functionalities like streaming, function calling and fine-tuning.
- [Batches](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openai/batches.md): Perform batch inference with OpenAI
- [Files](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openai/files.md): Upload files to OpenAI
- [Fine-tune](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openai/fine-tuning.md): Fine-tune your models with OpenAI
- [Prompt Caching](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openai/prompt-caching-openai.md)
- [Remote MCP Support](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openai/remote-mcp.md)
- [Structured Outputs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openai/structured-outputs.md): Structured Outputs ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Portkey supports OpenAI's Structured Outputs feature out of the box with our SDKs & APIs.
- [OpenRouter](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/openrouter.md)
- [Perplexity AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/perplexity-ai.md)
- [Predibase](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/predibase.md)
- [Recraft AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/recraft-ai.md)
- [Reka AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/reka-ai.md)
- [Replicate](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/replicate.md)
- [SambaNova](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/sambanova.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including [SambaNova AI](https://sambanova.ai/).
- [Segmind](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/segmind.md)
- [SiliconFlow](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/siliconflow.md)
- [Snowflake Cortex](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/snowflake-cortex.md)
- [Stability AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/stability-ai.md)
- [Suggest a new integration!](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/suggest-a-new-integration.md)
- [Together AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/together-ai.md)
- [Triton](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/triton.md): Integrate Trtiton-hosted custom models with Portkey and take them to production
- [Upstage AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/upstage.md): Integrate Upstage with Portkey AI for seamless completions, prompt management, and advanced features like streaming and embedding.
- [Google Vertex AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/vertex-ai.md)
- [Batches](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/vertex-ai/batches.md): Perform batch inference with Vertex AI
- [Controlled Generations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/vertex-ai/controlled-generations.md): Controlled Generations ensure that the model always follows your supplied [JSON schema](https://json-schema.org/overview/what-is-jsonschema). Portkey supports Vertex AI's Controlled Generations feature out of the box with our SDKs & APIs.
- [Embeddings](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/vertex-ai/embeddings.md): Get embeddings from Vertex AI
- [Files](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/vertex-ai/files.md): Upload files to Google Cloud Storage for Vertex AI fine-tuning and batch inference
- [Fine-tune](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/vertex-ai/fine-tuning.md): Fine-tune your models with Vertex AI
- [vLLM](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/vllm.md): Integrate vLLM-hosted custom models with Portkey and take them to production
- [Voyage AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/voyage-ai.md): Portkey provides a robust and secure gateway to facilitate the integration of various Large Language Models (LLMs) into your applications, including Voyage AI's embedding and Re-rank endpoints.
- [Workers AI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/workers-ai.md)
- [xAI (Grok)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/x-ai.md): Portkey supports xAI's chat completions, completions, and embeddings APIs.
- [ZhipuAI / ChatGLM / BigModel](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/llms/zhipu.md)
- [Claude Desktop](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-clients/claude.md): Connect your MCP servers to Claude Desktop through Portkey's secure MCP gateway
- [Claude Code](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-clients/claude-code.md): Connect your MCP servers to Claude Code CLI through Portkey's secure MCP gateway
- [Cursor](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-clients/cursor.md): Connect your MCP servers to Cursor IDE through Portkey's secure MCP gateway
- [LibreChat](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-clients/librechat.md): Connect your MCP servers to LibreChat through Portkey's secure MCP gateway
- [VS Code](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-clients/vs-code.md): Connect your MCP servers to Visual Studio Code through Portkey's secure MCP gateway
- [Atlassian Remote MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/atlassian-mcp-server.md): The Atlassian Remote MCP Server is a cloud-based bridge between your Atlassian Cloud site and MCP-compatible clients, enabling secure interaction with Jira, Confluence, and Compass in real time.
- [Cerebras Code MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/cerebras-code-mcp-server.md): The **Cerebras Code MCP Server (v1.3.3)** integrates high-speed code generation and editing capabilities into AI-assisted IDEs such as Claude Code, Cline, and Cursor. It allows you to use your preferred AI (Claude, Qwen, etc.) for planning and strategy, and delegate the actual code-writing and modif…
- [Datadog MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/datadog-mcp-server.md): The Datadog MCP server enables AI agents to interact with Datadog monitoring, dashboards, metrics, logs, and alerts through MCP. Built for conversational and automated observability workflows.
- [Figma MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/figma-mcp-server.md): The Figma MCP server connects Figma's design environment to MCP clients, enabling agents to translate designs into code, extract variables, and align output with your design system.
- [Firebase MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/firebase-mcp-server.md): The Firebase MCP server gives AI agents programmatic access to Firebase projects, including Auth user management, Firestore data access, Storage rules, and GraphQL features via the Firebase CLI.
- [Firecrawl MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/firecrawl-mcp-server.md): The Firecrawl MCP server enables AI agents to perform web scraping, crawling, search, extraction, and research through MCP. Built for content discovery and information retrieval workflows.
- [GitHub MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/github-mcp-server.md): The GitHub MCP server enables AI agents to safely interact with repositories, issues, pull requests, commits, and code search through MCP. Built for conversational and automated engineering workflows.
- [Gitlab MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/gitlab-mcp-server.md): The GitLab MCP server enables AI agents to interact with GitLab—the DevOps platform for software development, security, and operations.
- [Grafana MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/grafana-mcp-server.md): The Grafana MCP server enables AI agents to interact with Grafana dashboards, metrics, logs, incidents, alerts, and monitoring tools through MCP. Built for observability and monitoring workflows.
- [Kubernetes MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/kubernetes-mcp-server.md): The Kubernetes MCP server provides a flexible Model Context Protocol (MCP) interface for managing and interacting with Kubernetes and OpenShift clusters.
- [Linear MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/linear-mcp-server.md): The Linear MCP server provides a standardized interface for AI models and agents to securely access and manage data from your Linear workspace.
- [Notion MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/notion-mcp-server.md): The Notion MCP server enables AI agents to interact with Notion, a versatile workspace that combines note-taking, project management, knowledge bases, and relational databases. Through MCP, assistants can programmatically retrieve, search, and update pages, blocks, and databases—turning Notion into…
- [Open Library MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/open-library-mcp-server.md): The Open Library MCP server provides AI agents with access to Open Library's book and author catalog through a simple read-only API with no authentication required.
- [Playwright MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/playwright-mcp-server.md): The Playwright MCP server provides browser automation capabilities using Playwright through the Model Context Protocol (MCP), enabling deterministic web interaction via accessibility trees rather than screenshots.
- [Postman MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/postman-mcp-server.md): The Postman MCP Server connects Postman to AI tools, enabling assistants and agents to securely access and interact with Postman workspaces, collections, environments, and APIs through natural language. It bridges Postman's API platform with AI-driven automation, allowing developers to query, evalua…
- [Qdrant MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/qdrant-mcp-server.md): The Qdrant MCP server enables AI agents to store and retrieve semantic information using vector search through MCP. Built for persistent memory, contextual knowledge, and efficient semantic retrieval workflows.
- [Stripe MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/stripe-mcp-server.md): The Stripe MCP server exposes Stripe's payments and billing operations to AI agents through the Model Context Protocol, providing secure access to customers, products, pricing, invoices, and more.
- [Vectara MCP server](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/mcp-servers/vectara-mcp-server.md): The Vectara MCP server provides AI agents with access to Vectara's semantic search and retrieval-augmented generation (RAG) APIs.
- [Submit an Integration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/partner.md)
- [Exa Online Search](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/plugins/exa.md): Transform offline LLMs into online models with real-time internet search capabilities.
- [Arize Phoenix](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/arize.md): Extend Portkey’s powerful AI Gateway with Arize Phoenix for unified LLM observability, tracing, and analytics across your ML stack.
- [FutureAGI](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/future-agi.md): Integrate FutureAGI with Portkey for automated LLM evaluation and comprehensive observability
- [HoneyHive](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/honeyhive.md): Integrate HoneyHive observability with Portkey's AI gateway for comprehensive LLM monitoring and advanced routing capabilities
- [Langfuse](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/langfuse.md): Integrate Langfuse observability with Portkey's AI gateway for comprehensive LLM monitoring and advanced routing capabilities
- [Langsmith](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/langsmith.md): Integrate LangSmith observability with Portkey's AI gateway for comprehensive LLM monitoring and advanced routing capabilities
- [Pydantic Logfire](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/logfire.md): Modern Python observability with automatic OpenAI instrumentation and intelligent gateway routing
- [MLflow Tracing](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/ml-flow.md): Enhance LLM observability with automatic tracing and intelligent gateway routing
- [OpenLIT](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/openlit.md): Simplify AI development with OpenTelemetry-native observability and intelligent gateway routing
- [OpenTelemetry Python SDK](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/opentelemetry-python-sdk.md): Direct OpenTelemetry instrumentation with full control over traces and intelligent gateway routing
- [Phoenix(Arize) Open-Telemetry](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/phoenix.md): AI observability and debugging platform with OpenInference instrumentation and intelligent gateway routing
- [Traceloop (OpenLLMetry)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/tracing-providers/traceloop.md)
- [Milvus](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/vector-databases/milvus.md)
- [Qdrant](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/integrations/vector-databases/qdrant.md)
- [Portkey Features](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/introduction/feature-overview.md): Explore the powerful features of Portkey
- [Make Your First Request](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/introduction/make-your-first-request.md): Integrate Portkey and analyze your first LLM call in 2 minutes!
- [What is Portkey?](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/introduction/what-is-portkey.md): Portkey AI is a comprehensive platform designed to streamline and enhance AI integration for developers and organizations. It serves as a unified interface for interacting with over 250 AI models, offering advanced tools for control, visibility, and security in your Generative AI apps.
- [Configure API Key Access Permissions for Workspaces](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/configure-api-key-access-permissions.md)
- [Configure Logs Access Permissions for Workspace](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/configure-logs-access-permissions-in-workspace.md)
- [Configure Model Access Permissioning for AI Integrations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/configure-model-access-permissioning-for-ai-integratinon.md)
- [Configure Virtual Key Access Permission for Workspaces](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/configure-virtual-key-access-permissions.md)
- [Configure Workspace Provisioning for Your AI Integrations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/configure-workspace-provisioning-for-your-ai-integrations.md)
- [Configure Request Logging](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/configuring-request-logging.md): Control what data is stored for all LLM requests in your organization
- [Enforce Budget Limits and Rate Limits for Your API Keys](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforce-budget-and-rate-limit.md): Configure budget and rate limits on API keys to effectively manage AI spending and usage across your organization
- [Enforce Budget Limits on Your AI Provider](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforce-budget-limits-on-your-ai-provider.md)
- [Enforcing Default Configs on API Keys](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforce-default-config.md): Learn how to attach default configs to API keys for enforcing governance controls across your organization
- [Enforcing Org Level Guardrails](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforce-orgnization-level-guardrails.md)
- [Enforce Rate Limits on Your AI Provider](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforce-rate-limits-on-your-ai-provider.md)
- [Enforce Workspace Budget and Rate Limits](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforce-workspace-budget-limts-and-rate-limits.md): Configure budget and rate limits at the workspace level to effectively manage AI spending and resource allocation
- [Enforcing Workspace Level Guardrails](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforce-workspace-level-guardials.md)
- [Enforcing Request Metadata](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/administration/enforcing-request-metadata.md)
- [AI Gateway](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway.md): The world's fastest AI Gateway with advanced routing & integrated Guardrails.
- [Automatic Retries](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/automatic-retries.md): LLM APIs often have inexplicable failures. With Portkey, you can rescue a substantial number of your requests with our in-built automatic retries feature.
- [Unified Batch Inference](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/batches.md): Run large‑scale inference jobs through one consistent endpoint
- [Cache (Simple & Semantic)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/cache-simple-and-semantic.md)
- [Canary Testing](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/canary-testing.md): You can use Portkey's AI gateway to also canary test new models or prompts in different environments.
- [Circuit Breaker](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/circuit-breaker.md): Configure per-strategy circuit protection and failure handling
- [Conditional Routing](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/conditional-routing.md)
- [Configs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/configs.md): This feature is available on all Portkey plans.
- [Fallbacks](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/fallbacks.md)
- [Files](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/files.md): Upload files to Portkey and reuse the content in your requests
- [Fine-tuning](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/fine-tuning.md): Run your fine-tuning jobs with Portkey Gateway
- [Load Balancing](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/load-balancing.md): Load Balancing feature efficiently distributes network traffic across multiple LLMs.
- [Multimodal Capabilities](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/multimodal-capabilities.md)
- [Function Calling](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/multimodal-capabilities/function-calling.md): Portkey's AI Gateway supports function calling capabilities that many foundational model providers offer. In the API call you can describe functions and the model can choose to output text or this function name with parameters.
- [Image Generation](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/multimodal-capabilities/image-generation.md): Portkey's AI gateway supports image generation capabilities that many foundational model providers offer.
- [Speech-to-Text](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/multimodal-capabilities/speech-to-text.md): Portkey's AI gateway supports STT models like Whisper by OpenAI.
- [Text-to-Speech](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/multimodal-capabilities/text-to-speech.md): Portkey's AI gateway currently supports text-to-speech models on `OpenAI` and `Azure OpenAI`.
- [Thinking Mode](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/multimodal-capabilities/thinking-mode.md)
- [Vision](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/multimodal-capabilities/vision.md): Portkey's AI gateway supports vision models like GPT-4V by OpenAI, Gemini by Google and more.
- [Realtime API](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/realtime-api.md): Use OpenAI's Realtime API with logs, cost tracking, and more!
- [Remote MCP](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/remote-mcp.md): Portkey's AI gateway has MCP server support that many foundational model providers offer.
- [Request Timeouts](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/request-timeouts.md): Manage unpredictable LLM latencies effectively with Portkey's **Request Timeouts**.
- [Strict OpenAI Compliance](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/strict-open-ai-compliance.md)
- [Universal API](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/universal-api.md): Portkey's Universal API provides a consistent interface to integrate a wide range of modalities (text, vision, audio) and LLMs (hosted OR local) into your apps.
- [Virtual Keys](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/virtual-keys.md): Portkey's virtual key system allows you to securely store your LLM API keys in our vault, utilizing a unique virtual identifier to streamline API key management.
- [Connect Bedrock with Amazon Assumed Role](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/virtual-keys/bedrock-amazon-assumed-role.md): How to create a new integration for Bedrock using Amazon Assumed Role Authentication
- [Budget Limits](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/virtual-keys/budget-limits.md): Budget Limits lets you set cost limits on providers/integrations
- [Rate Limits](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/ai-gateway/virtual-keys/rate-limits.md): Set Rate Limts to your Integrations/Providers
- [Enterprise Offering](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering.md)
- [Access Control Management](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/access-control-management.md): With customizable user roles, API key management, and comprehensive audit logs, Portkey provides the flexibility and control needed to ensure secure collaboration & maintain a strong security posture
- [OTel Integration (Analytics Data)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/analytics-logs-export.md)
- [Audit Logs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/audit-logs.md): Track and monitor all administrative activities across your Portkey organization with comprehensive audit logging.
- [Budget Limits](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/budget-limits.md)
- [Enterprise Components](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/components.md)
- [KMS Integration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/kms.md): Customers can bring their own encryption keys to Portkey AI to encrypt data at storage.
- [Logs Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/logs-export.md)
- [Org Management](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management.md): A high-level introduction to Portkey's organization management structure and key concepts.
- [API Keys (AuthN and AuthZ)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/api-keys-authn-and-authz.md): Discover how Admin and Workspace API Keys are used to manage access and operations in Portkey.
- [JWT Authentication](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/jwt.md): Configure JWT-based authentication for your organization in Portkey
- [Organizations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/organizations.md): Understand the role and features of Organizations, the highest level of abstraction in Portkey's structure.
- [Azure Entra](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/scim/azure-ad.md): Setup Azure Entra for SCIM provisioning with Portkey.
- [Okta](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/scim/okta.md): Set up Okta for SCIM provisioning with Portkey.
- [Overview](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/scim/scim.md): SCIM integration with Portkey.
- [SSO](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/sso.md): SSO support for enterprises
- [User Roles & Permissions](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/user-roles-and-permissions.md): Learn about Portkey's comprehensive role-based access control system across Organizations and Workspaces.
- [Workspaces](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/org-management/workspaces.md): Explore Workspaces, the sub-organizational units that enable granular project and team management.
- [Security @ Portkey](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/enterprise-offering/security-portkey.md): Portkey AI provides a secure, reliable AI gateway for the seamless integration and management of large language models (LLMs).
- [Guardrails](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/guardrails.md): Ship to production confidently with Portkey Guardrails on your requests & responses
- [Creating Raw Guardrails (in JSON)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/guardrails/creating-raw-guardrails-in-json.md): With the raw Guardrails mode, we let you define your Guardrail checks & actions however you want, directly in code.
- [Guardrails for Embedding Requests](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/guardrails/embedding-guardrails.md): Apply security and data validation measures to vector embedding requests to protect sensitive information and ensure data quality.
- [List of Guardrail Checks](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/guardrails/list-of-guardrail-checks.md)
- [PII Redaction](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/guardrails/pii-redaction.md): Replace any sensitive data in requests with standard identifiers
- [MCP](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/mcp.md)
- [MCP Gateway](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/mcp-gateway.md): The universal adapter for Model Context Protocol—connect once, access every MCP tool .
- [Architecture](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/mcp-gateway/architecture.md)
- [Deployment](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/mcp-gateway/deployment.md)
- [MCP Clients](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/mcp-gateway/mcp-clients.md)
- [MCP Hub](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/mcp-gateway/mcp-hub.md): Portkey's MCP Hub to connect 1600+ LLMs with any Remote MCP Server
- [Quickstart](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/mcp-gateway/quickstart.md)
- [Model Catalog](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog.md): A single pane to view and manage every AI provider and model in your organization. It provides centralized governance, discovery, and usage controls for all your AI resources.
- [Budget Limits](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/budget-limits.md)
- [Connect Bedrock with Amazon Assumed Role](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/connect-bedrock-with-amazon-assumed-role.md): How to create an integrate Bedrock using Amazon Assumed Role Authentication on Portkey
- [Adding Custom Models](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/custom-models.md): Learn how to add custom and fine-tuned models to your Portkey Model Catalog for seamless integration.
- [Integrations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/integrations.md): Securely store and manage AI provider credentials across your organization with centralized governance controls
- [Overriding Model Details](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/model-overrides.md): Learn how to set custom pricing for any base model in your Portkey Model Catalog to reflect your specific costs.
- [AI Model Provisioning](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/model-provisioning.md)
- [Rate Limits](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/rate-limits.md)
- [Workspace Provisioning](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/model-catalog/workspace-provisioning.md)
- [Observability (OpenTelemetry)](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability.md): Gain real-time insights, track key metrics, and streamline debugging with our comprehensive observability suite.
- [Analytics](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/analytics.md)
- [Auto-Instrumentation [BETA]](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/auto-instrumentation.md): Portkey's auto-instrumentation allows you to instrument tracing and logging for multiple LLM/Agent frameworks and view the logs, traces, and metrics in a single place.
- [Budget Limits](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/budget-limits.md)
- [Model Pricing and Cost Management](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/cost-management.md): Learn how Portkey handles pricing data, cost calculations, and pricing updates across different deployment modes.
- [Feedback](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/feedback.md): Portkey's Feedback APIs provide a simple way to get weighted feedback from customers on any request you served, at any stage in your app.
- [Filters](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/filters.md)
- [Logs](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/logs.md): The Logs section presents a chronological list of all the requests processed through Portkey.
- [Logs Export](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/logs-export.md): Easily access your Portkey logs data for further analysis and reporting
- [Metadata](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/metadata.md): Add custom context to your AI requests for better observability and analytics
- [OpenTelemetry for LLM Observability](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/opentelemetry.md): Leverage OpenTelemetry with Portkey for comprehensive LLM application observability, combining gateway insights with full-stack telemetry.
- [Supported OTel Libraries](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/opentelemetry/list-of-supported-otel-instrumenters.md): Portkey works with any OpenTelemetry-compatible instrumentation. Here are some popular options.
- [Tracing](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/observability/traces.md): The **Tracing** capabilities in Portkey empowers you to monitor the lifecycle of your LLM requests in a unified, chronological view.
- [Open Source](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/open-source.md)
- [Feature Comparison](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/product-feature-comparison.md): Comparing Portkey's Open-source version and Dev, Pro, Enterprise plans.
- [Prompt Engineering Studio](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio.md)
- [Prompt API](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-api.md): Learn how to integrate Portkey's prompt templates directly into your applications using the Prompt API
- [Guides](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-guides.md): Learn how to get the most out of Portkey Prompts with these practical guides
- [Integrations](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-integration.md)
- [Prompt Library](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-library.md)
- [Prompt Observability](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-observability.md)
- [Prompt Partials](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-partial.md): With Prompt Partials, you can save your commonly used templates (which could be your instruction set, data structure explanation, examples etc.) separately from your prompts and flexibly incorporate them wherever required.
- [Prompt Playground](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-playground.md)
- [Prompt Versioning & Labels](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/prompt-versioning.md)
- [Tool Library](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/product/prompt-engineering-studio/tool-library.md)
- [Enterprise Architecture](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/self-hosting/hybrid-deployments/architecture.md): Comprehensive guide to Portkey's hybrid deployment architecture for enterprises
- [EKS](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/self-hosting/hybrid-deployments/aws/eks.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software in a hybrid mode on Amazon EKS clusters, designed to meet the needs of large-scale, mission-critical applications. It includes specific recommendations for component sizing, high availability, and…
- [AWS Marketplace](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/self-hosting/hybrid-deployments/aws/marketplace.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software using AWS Marketplace.
- [Azure](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/self-hosting/hybrid-deployments/azure.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software on Azure Kubernetes Service (AKS), tailored to meet the needs of large-scale, mission-critical applications. It includes specific recommendations for component sizing, high availability, and integ…
- [GCP](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/self-hosting/hybrid-deployments/gcp.md): This enterprise-focused document provides comprehensive instructions for deploying the Portkey software on Google Cloud Platform (GCP), tailored to meet the needs of large-scale, mission-critical applications.
- [Prometheus Metrics](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/self-hosting/prometheus-metrics.md): Comprehensive monitoring and observability for Portkey Enterprise Gateway through Prometheus metrics
- [Common Errors & Resolutions](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/support/common-errors-and-resolutions.md): Since Portkey functions as a gateway - you may encounter Portkey-related, as well as non-Portkey related errors while using our services.
- [Contact Us](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/support/contact-us.md)
- [Developer Forum](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/support/developer-forum.md): Are you navigating the challenging journey of transitioning LLMs from prototype stages to full-scale production? You're not alone. As this frontier of technology continues to expand, the roadmap isn't always clear. Best practices, guidelines, and efficient methodologies are still on the horizon.
- [December '23 Migration](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/support/portkeys-december-migration.md)
- [Upgrade to Model Catalog](https://portkey-docs-mintlify-bedrock-guardrails-docs-36443.mintlify.app/support/upgrade-to-model-catalog.md): Learn how to upgrade to Model Catalog to replace Virtual Keys

## OpenAPI Specs

- [openapi](https://raw.githubusercontent.com/Portkey-AI/openapi/refs/heads/master/openapi.yaml)