Configuration Reference

Configuration Reference

This document provides a comprehensive reference for all configuration files in the MCP Gateway Registry project. Each configuration file serves a specific purpose in the authentication and operation of the system.

Configuration Files Overview

Main Environment Configuration

File: .env (Project root) Purpose: Core project settings, registry URLs, and primary authentication credentials.

Authentication Provider Selection

The MCP Gateway Registry supports multiple authentication providers. Choose one by setting the AUTH_PROVIDER environment variable:

• keycloak: Open-source identity and access management with individual agent audit trails
• cognito: Amazon managed authentication service

Based on your selection, configure the corresponding provider-specific variables below.

Core Variables

Keycloak Configuration (if AUTH_PROVIDER=keycloak)

Note: Getting Keycloak Client IDs and Secrets

The client IDs and secrets are automatically generated when you run the Keycloak initialization script:

cd keycloak/setup
./init-keycloak.sh

The script will:

1. Create the clients with the IDs you specify (mcp-gateway-web and mcp-gateway-m2m)
2. Generate secure random secrets for each client
3. Display the generated secrets at the end of the script output
4. Save them to a file for your reference

To retrieve existing client secrets from a running Keycloak instance:

# Method 1: Use the helper script (Recommended)
cd keycloak/setup
export KEYCLOAK_ADMIN_PASSWORD="your-admin-password"
./get-all-client-credentials.sh
# This will display the secrets and save them to .oauth-tokens/keycloak-client-secrets.txt

# Method 2: Using Keycloak Admin Console (Web UI)
# 1. Navigate to https://your-keycloak-url/admin
# 2. Login with admin credentials
# 3. Select your realm (mcp-gateway)
# 4. Go to Clients → Select your client
# 5. Go to Credentials tab
# 6. Copy the Secret value

# Method 3: Check the original initialization output
# The init-keycloak.sh script saves secrets to keycloak-client-secrets.txt
cat keycloak/setup/keycloak-client-secrets.txt

Amazon Cognito Configuration (if AUTH_PROVIDER=cognito)

Session Cookie Security Configuration

CRITICAL: These settings control how session cookies are transmitted and shared. Incorrect configuration will cause login failures.

SESSION_COOKIE_SECURE - Critical for Your Environment

YOU MUST SET THIS CORRECTLY OR LOGIN WILL FAIL:

For Local Development (localhost via HTTP):

SESSION_COOKIE_SECURE=false  # MUST be false

• Localhost runs over HTTP (not HTTPS)
• Cookies with secure=true are ONLY sent over HTTPS
• Setting this to true on localhost = login will fail ❌

For Production with HTTPS:

SESSION_COOKIE_SECURE=true  # MUST be true

• Production deployments use HTTPS
• Cookies must have secure=true to prevent session hijacking
• Setting this to false in production = security vulnerability ❌

SESSION_COOKIE_DOMAIN - When to Set This

Most deployments should leave this EMPTY (default behavior = safest):

SESSION_COOKIE_DOMAIN=  # Empty string or unset

Only set this if you need cross-subdomain authentication:

Important Security Notes:

• Empty domain = cookie scoped to exact host only (safest)
• Set domain only when you control ALL subdomains
• Never set to public suffixes (.com, .net, .ddns.net)
• Domain must start with a dot (.example.com)

See Also: Cookie Security Design Documentation for detailed security analysis and deployment scenarios.

Optional Variables

Storage Backend Configuration

The MCP Gateway Registry supports three storage backends for servers, agents, and scopes management.

⚠️ DEPRECATION WARNING: File-based storage is deprecated and will be removed in a future release. MongoDB CE is now the recommended backend for local development and testing.

Backend Options:

File Backend (Deprecated)

• Status: DEPRECATED - Will be removed in a future release
• Migration Path: Switch to MongoDB CE for local development or DocumentDB for production
• Pros: Simple, no external dependencies, human-readable JSON files
• Cons: Limited concurrent writes, no distributed access, FAISS-based vector search, deprecated

STORAGE_BACKEND=file  # DEPRECATED - Use mongodb-ce instead

Data stored in:

• Servers: ~/mcp-gateway/servers/*.json
• Agents: ~/mcp-gateway/agents/*.json
• Scopes: auth_server/scopes.yml
• Security scans: ~/mcp-gateway/security_scans/*.json

MongoDB CE Backend (Recommended for Local Development)

• Status: RECOMMENDED for all local development and testing
• Best for: Local development, feature development, testing, CI/CD pipelines
• Pros: Docker-based, no cloud dependencies, replica set support, application-level vector search, production-like environment
• Cons: Limited to ~10,000 documents, O(n) vector search performance (acceptable for development)

STORAGE_BACKEND=mongodb-ce
DOCUMENTDB_HOST=mongodb       # Docker service name
DOCUMENTDB_PORT=27017
DOCUMENTDB_DATABASE=mcp_registry
DOCUMENTDB_NAMESPACE=default
DOCUMENTDB_USE_TLS=false      # No TLS for local dev

MongoDB Collections Created:

• mcp_servers_{namespace} - Server definitions
• mcp_agents_{namespace} - A2A agent cards
• mcp_scopes_{namespace} - Authorization scopes
• mcp_embeddings_1536_{namespace} - Vector embeddings (1536 dimensions)
• mcp_security_scans_{namespace} - Security scan results
• mcp_federation_config_{namespace} - Federation configuration

First-Time MongoDB CE Setup:

# 1. Start MongoDB container
docker-compose up -d mongodb
sleep 5

# 2. Initialize collections and indexes
docker-compose up mongodb-init

# 3. Verify setup
docker exec mcp-mongodb mongosh --eval "use mcp_registry; show collections"

# 4. Switch backend and restart
export STORAGE_BACKEND=mongodb-ce
docker-compose restart registry

DocumentDB Backend (Production, Recommended)

• Best for: Production deployments, high concurrency, large-scale systems
• Pros: Native HNSW vector search, distributed storage, AWS-managed, clustering support
• Cons: Requires AWS infrastructure, uses AWS pricing

STORAGE_BACKEND=documentdb
DOCUMENTDB_HOST=cluster.docdb.amazonaws.com
DOCUMENTDB_PORT=27017
DOCUMENTDB_DATABASE=mcp_registry
DOCUMENTDB_NAMESPACE=production
DOCUMENTDB_USERNAME=admin
DOCUMENTDB_PASSWORD=<secure-password>
DOCUMENTDB_USE_TLS=true
DOCUMENTDB_TLS_CA_FILE=global-bundle.pem
DOCUMENTDB_REPLICA_SET=rs0

DocumentDB Collections Created: Same as MongoDB CE (above), but with native HNSW vector indexes for sub-100ms semantic search.

First-Time DocumentDB Setup:

# 1. Deploy DocumentDB cluster via Terraform
cd terraform/aws-ecs
terraform apply

# 2. Collections and indexes are created automatically on first application startup

# 3. Verify setup (from bastion host or EC2 with access)
mongosh --host <cluster-endpoint> \
        --username admin \
        --password <password> \
        --tls \
        --tlsCAFile global-bundle.pem \
        --eval "use mcp_registry; show collections"

Important Notes:

• MongoDB CE uses application-level vector search (Python cosine similarity)
• DocumentDB uses native HNSW vector indexes for production performance
• Both backends use the same repository code (DocumentDBServerRepository, etc.)
• auth_server/scopes.yml is no longer the source of truth when using mongodb-ce or documentdb backend

Switching Between Backends:

You can switch between backends at any time by changing STORAGE_BACKEND:

# Switch to file backend
export STORAGE_BACKEND=file
docker-compose restart registry

# Switch to MongoDB CE backend
export STORAGE_BACKEND=mongodb-ce
docker-compose restart registry

# Switch to DocumentDB backend
export STORAGE_BACKEND=documentdb
docker-compose restart registry

For AWS ECS Deployments: See terraform/aws-ecs/README.md for automated Terraform deployment with DocumentDB.

For Detailed Architecture: See Storage Architecture: MongoDB CE & AWS DocumentDB for comprehensive implementation details.

Container Registry Configuration (Optional - for CI/CD and local builds)

Note: Container Registry Credentials (Completely Optional)

These credentials are entirely optional and only needed if you want to:

• Publish container images: Automatically via GitHub Actions or manually via scripts
• Contribute pre-built containers: For easier deployment by other users

What happens if these are not configured:

• ✅ The MCP Gateway Registry will work perfectly - all core functionality remains intact
• ✅ GitHub Actions will succeed - builds will complete successfully, just without publishing to Docker Hub
• ✅ Local development is unaffected - no scripts will fail or produce errors
• ✅ Only container publishing is skipped - everything else continues normally

When you might want to configure these:

• Contributing to the project: Publishing official container images
• Custom deployments: Creating your own container registry for internal use
• Development workflow: Testing container builds locally

How to obtain credentials (only if needed):

• Docker Hub: Get access token from Docker Hub Security Settings
• GitHub Container Registry: Generate Personal Access Token with packages:write scope from GitHub Token Settings

Setup instructions (only if publishing containers):

• In GitHub Actions: Add DOCKERHUB_USERNAME and DOCKERHUB_TOKEN as repository secrets
• For local builds: Add credentials to your .env file and use scripts/publish_containers.sh
• GITHUB_TOKEN: Automatically provided in GitHub Actions, manually generated for local use

Organization vs Personal Account Publishing:

• Personal Account (Free): Leave DOCKERHUB_ORG and GITHUB_ORG empty
  • Images published as: username/image-name
  • Example: aarora79/registry:latest
• Organization Account (Paid for Docker Hub): Set organization names
  • Images published as: organization/image-name
  • Example: mcpgateway/registry:latest

Keycloak Setup and Configuration

When using Keycloak as your authentication provider, the system provides comprehensive setup scripts and configuration options:

Initial Setup

Run the Keycloak initialization script to set up the realm, clients, and groups:

cd keycloak/setup
./init-keycloak.sh

This script will:

1. Create the mcp-gateway realm
2. Set up web and M2M clients with proper configurations
3. Create necessary groups (mcp-servers-unrestricted, mcp-servers-restricted)
4. Configure group mappers for JWT token claims
5. Create initial admin and test users

Service Account Management

For individual AI agent audit trails, create service accounts:

# Create individual agent service account
./setup-agent-service-account.sh --agent-id sre-agent --group mcp-servers-unrestricted

# Create shared M2M service account
./setup-m2m-service-account.sh

Token Generation

Generate tokens for Keycloak authentication:

# Generate M2M token for ingress
uv run python credentials-provider/token_refresher.py

# Generate agent-specific token
uv run python credentials-provider/token_refresher.py --agent-id sre-agent

For detailed Keycloak integration documentation, see Keycloak Integration Guide.

OAuth Environment Configuration

File: credentials-provider/oauth/.env Purpose: OAuth provider credentials for ingress and egress authentication flows.

Ingress Authentication

For Keycloak (if AUTH_PROVIDER=keycloak)

For Cognito (if AUTH_PROVIDER=cognito)

Egress Authentication (Optional)

Support for multiple OAuth provider configurations using numbered suffixes (_1, _2, _3, etc.):

Supported Providers

• Google: Gmail, Drive, Calendar services
• GitHub: Repository and issue management
• Microsoft: Office 365, Teams integration
• Bedrock AgentCore: AWS AgentCore services

AgentCore Environment Configuration

File: credentials-provider/agentcore-auth/.env Purpose: Amazon Bedrock AgentCore authentication configuration with support for multiple gateways.

Shared Configuration

Gateway-Specific Configurations

Support for multiple gateways using numbered suffixes (_1, _2, _3, etc., up to _100). Each configuration set requires all four parameters:

Example Configuration:

# Configuration Set 1
AGENTCORE_CLIENT_ID_1=your_client_id_here
AGENTCORE_CLIENT_SECRET_1=your_client_secret_here
AGENTCORE_GATEWAY_ARN_1=arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/my-gateway-1
AGENTCORE_SERVER_NAME_1=my-gateway-1

# Configuration Set 2
AGENTCORE_CLIENT_ID_2=your_client_id_here
AGENTCORE_CLIENT_SECRET_2=your_client_secret_here
AGENTCORE_GATEWAY_ARN_2=arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/my-gateway-2
AGENTCORE_SERVER_NAME_2=my-gateway-2

OAuth2 Providers Configuration

File: auth_server/oauth2_providers.yml Purpose: OAuth2 provider definitions for web-based authentication flows.

Keycloak Provider Configuration

When using Keycloak as the authentication provider, the following configuration is used:

General Provider Configuration Fields

Supported Providers

• Keycloak: Open-source identity and access management
• Amazon Cognito: Amazon managed authentication service
• GitHub: Repository and development services (planned)
• Google: Google Workspace and consumer services (planned)

Scopes Configuration

File: auth_server/scopes.yml Purpose: Fine-grained access control (FGAC) scope definitions.

Scope Categories

• MCP Servers: Individual server access (mcp-servers-{name}/read, mcp-servers-{name}/execute)
• Unrestricted: Global access (mcp-servers-unrestricted/read, mcp-servers-unrestricted/execute)
• Admin: Administrative functions (admin/registry, admin/users)

OAuth Providers Mapping

File: credentials-provider/oauth/oauth_providers.yaml Purpose: Provider-specific OAuth endpoint configurations and metadata.

Provider Fields

Docker Compose Configuration

File: docker-compose.yml Purpose: Container orchestration for development and deployment.

Services

• registry: Main MCP Gateway Registry service
• auth-server: OAuth2 authentication server
• frontend: Web interface (React application)

Key Configuration

• Environment variable injection from .env files
• Port mappings for local development
• Volume mounts for persistent data
• Health checks and restart policies

Configuration Security

Best Practices

1. Never commit real credentials to version control
2. Use environment variables for sensitive data
3. Rotate credentials regularly especially for production
4. Limit scope permissions to minimum required access
5. Monitor credential usage through logging and audit trails

File Permissions

• .env files should have 600 permissions (readable only by owner)
• Configuration directories should have 700 permissions
• Generated token files are automatically secured with 600 permissions

Troubleshooting

Common Issues

1. Login redirects back to login page

   • Most Common Cause: SESSION_COOKIE_SECURE=true but accessing via HTTP
   • Solution for localhost: Set SESSION_COOKIE_SECURE=false in .env
   • Solution for production: Ensure HTTPS is properly configured
   • Check: Browser dev tools → Application → Cookies (cookie should be present)
   • Check: Server logs for Auth server setting session cookie: secure=...

2. Missing environment variables: Check that all required variables are set in the appropriate .env files

3. Invalid credentials: Verify OAuth client IDs and secrets with providers

4. Network connectivity: Ensure firewall rules allow OAuth callback URLs

5. Token expiration: Use the credential refresh scripts to update expired tokens

6. Scope mismatches: Verify requested OAuth scopes match provider configurations

7. Session cookie not being sent by browser

   • Check cookie domain matches your hostname
   • Verify SESSION_COOKIE_DOMAIN is empty for single-domain deployments
   • Check browser third-party cookie settings
   • Inspect cookie attributes in browser dev tools

Validation Commands

# Validate OAuth configuration
cd credentials-provider
./generate_creds.sh --verbose

# Test MCP gateway connectivity
cd tests
./tests/mcp_cmds.sh ping

# Check configuration files
python -c "import yaml; yaml.safe_load(open('file.yml'))"  # YAML validation

Log Files

• OAuth flows: .oauth-tokens/ directory contains generated tokens and logs
• Registry operations: Check registry.log for service-level issues
• Authentication: Check auth.log for OAuth and FGAC issues