OpenStack Keystone in Rust

The legacy Keystone identity service (written in Python and maintained upstream by OpenStack Foundation) has served the OpenStack ecosystem reliably for years. It handles authentication, authorization, token issuance, service catalog, project/tenant management, and federation services across thousands of deployments. However, as we embarked on adding next-generation identity features—such as native WebAuthn (“passkeys”), modern federation flows, direct OIDC support, JWT login, workload authorization, restricted tokens and service-accounts—it became clear that certain design and performance limitations of the Python codebase would hamper efficient implementation of these new features.

Consequently, we initiated a project termed “Keystone-NG”: a Rust-based component that augments rather than fully replaces the existing Keystone service. The original plan was to implement only the new feature-set in Rust and route those new API paths to the Rust component, while keeping the core Python Keystone service in place for existing users and workflows.

As development progressed, however, the breadth of new functionality (and the opportunity to revisit some of the existing limitations) led to a partial re-implementation of certain core identity flows in Rust. This allows us to benefit from Rust’s memory safety, concurrency model, performance, and modern tooling, while still preserving the upstream Keystone Python service as the canonical “master” identity service, routing only the new endpoints and capabilities through the Rust component.

In practice, this architecture means:

• The upstream Python Keystone remains the main identity interface, preserving backward compatibility, integration with other OpenStack services, existing user workflows, catalogs, policies and plugins.

• The Rust “Keystone-NG” component handles new functionality, specifically:

  • Native WebAuthN (passkeys) support for passwordless / phishing-resistant MFA

  • A reworked federation service, enabling modern identity brokering and advanced federation semantics OIDC (OpenID Connect) Direct in Keystone, enabling Keystone to act as an OIDC Provider or integrate with external OIDC identity providers natively JWT login flows, enabling stateless, compact tokens suitable for new micro-services, CLI, SDK, and workload-to-workload scenarios

  • Workload Authorization, designed for service-to-service authorization in cloud native contexts (not just human users)

  • Restricted Tokens and Service Accounts, which allow fine-grained, limited‐scope credentials for automation, agents, and service accounts, with explicit constraints and expiry

By routing only the new flows through the Rust component we preserve the stability and ecosystem compatibility of Keystone, while enabling a forward-looking identity architecture. Over time, additional identity flows may be migrated or refactored into the Rust component as needed, but our current objective is to retain the existing Keystone Python implementation as the trusted, mature baseline and incrementally build the “Keystone-NG” Rust service as the complement.

We believe this approach allows the best of both worlds: the trusted maturity of Keystone’s Python code-base, combined with the modern, high-safety, high-performance capabilities of Rust where they matter most.

Compatibility

Highest priority is to ensure that this implementation is compatible with the original python Keystone: authentication issued by Rust implementation is accepted by the Python Keystone and vice versa. At the same time it is expected, that the new implementation may implement new features not supported by the Python implementation. In this case, it is still expected that such features do not break authentication flows. It must be possible to deploy Python and Rust implementation in parallel and do request routing on the web server level.

Database

Adding new features most certainly require having database changes. It is not expected that such changes interfere with the Python implementation to ensure it is working correctly.

API

Also here it is expected that new API resources are going to be added. As above it is not expected that such changes interfere with the Python implementation to ensure it is still working correctly and existing clients will not break.

Installation

The easiest way to get started with the keystone-ng is using the container image. It is also possible to use the compiled binary. It can be either compiled locally or downloaded from the project artifacts.

Using pre-compiled binaries

As of the moment of writing there were no releases. Due to that there are no pre-compiled binaries available yet. Every release of the project would include the pre-compiled binaries for a variety of platforms.

Compiling

In order to compile the keystone-ng it is necessary to have the rust compiler available. It may be installed from the system packages or using the rustup.rs

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Afterwards in the root of the project source tree following command may be executed to invoke the cargo

cargo build --release

It produces 2 binaries:

• target/release/keystone (the api server)

• target/release/keystone-db (the database management tool)

Currently keystone depends on the openssl (as a transitive dependency). Depending on the environment it may be a statically linked or dynamically. There are signals that that may be not necessary anymore once all dependencies transition to the use of rustls.

Using containers

It is possible to run Keystone-ng inside containers. A sample Dockerfile is present in the project source tree to build container image with the Keystone and the keystone-db utility. When no ready image is available it can be build like that:

docker build . -t keystone:rust

Since keystone itself communicates with the database and OpenPolicyAgent those must be provided separately. docker-compose.yaml demonstrates how this can be done.

docker run -v /etc/keystone/:/etc/keystone -p 8080:8080 ghcr.io/openstack-experimental/keystone:main -v /etc/keystone/keystone.conf

Database migrations

Rust Keystone is using different ORM and implements migration that co-exist together with alembic migrations of the python Keystone. It also ONLY manages the database schema additions and does NOT include the original database schema. Therefore it is necessary to apply both migrations.

keystone-db --config /etc/keystone/keystone.conf

It is important to also understand that the DB_URL may differ between python and rust due to the optional presence of the preferred database driver in the url. keystone-ng will ignore the the driver in the application itself, but the migration may require user to manually remove it since it is being processed by the ORM itself and not by the keystone-ng code.

OpenPolicyAgent

keystone-ng relies on the OPA for policy enforcement. Default policies are provided with the project and can be passed directly to the OPA process or compiled into the bundle.

opa run -s policies

NOTE: by default OPA process listens on the localhost only what lead to unavailability to expose it between containers. Please use -a 0.0.0.0:8181 to start listening on all interfaces.

Parallel installation with the python Keystone

Since Keystone-NG is only an addition and is not a drop-in replacement for the python Keystone it is necessary to deploy both versions together.

With the python Keystone no changes on the deployment strategy should be necessary. Whichever WSGI method is used to run the Keystone it stays this way and continues listening on the expected port.

The rust Keystone is deployed in parallel to it (usually on the same hardware) and by default it listens on the port 8080.

Next required step is to alter the http proxy server configuration. In the devstack this is usually the Apache webserver. Some operators may run nginx or haproxy in front of the default webserver with the Keystone. Depending on the preferred functionality (i.e. whether the token validation should be performed by the python or the rust implementation) redirects can be implemented. This way it is possible to decide for every single API call individually whether it should be served by python or rust implementation.

Nginx sample configuration

server {
    listen 443 ssl;
    server_name devstack.v6.rocks;

    ...

    # by default in devstack services are exposed with the url path style.
    location /identity/v4 {
      server http://localhost:8080;
    }
    proxy_pass http://<192.168.1.1>;
}

Apache sample configuration

<VirtualHost *:443>
    ServerName devstack..v6.rocks

    ...
    # Rust /v4 API
    ProxyPass "/identity/v4" http://localhost:8080/v4 retry=0
    # Python /v3 APIs are served by the uwsgi app
    ProxyPass "/identity/v3" "unix:/var/run/uwsgi/keystone-api.socket|uwsgi://uwsgi-uds-keystone-api/v3" retry=0
    # We want discovery URL to be served by Rust. The same way any /v3/ API can
    # be forwarded to rust version (where supported).
    ProxyPass "/identity" http://localhost:8080 retry=0
</VirtualHost>

Architecture

Keystone requires 2 additional components to run:

• database (the same as the py-keystone uses)

• OpenPolicyAgent, that implements API policy enforcement

architecture-beta

    service db(database)[Database]
    service keystone(server)[Keystone]
    service opa(server)[OpenPolicyAgent]

    db:L -- R:keystone
    opa:L -- T:keystone

Database

Python keystone uses the sqlalchemy as ORM and the migration tool. It cannot be used from Rust efficiently, therefore keystone-ng uses the sea-orm which provides async support natively and also allows database type abstraction. Current development focuses on the PostgreSQL database. The MySQL should be supported, but is not currently tested against.

New API and resources are being added. This requires database changes. sea-orm also comed with the migration tools. However there is a slight difference between sqlalchemy and sea-orm. The later suggests doing database schema first. In the next step object types are created out of the database. That means that the database migration must be written first and cannot be automatically generated from the code (easily, but there is a way). Current migrations do not create database schema that is managed by the py-keystone. Therefore in order to get a fully populated database schema it is necessary to apply keystone-manage db_sync and keystone-db up independently.

Target of the keystone-ng is to be deployed in pair with the python keystone of “any” version. Due to that it is not possible to assume the state of the database, nor to apply any changes to the schema manaaged by the py-keystone. A federation rework assumes model change. To keep it working with the python-keystone artificial table entries may be created (in the example when a new identity provider is being created automatically sanitized entries are being added for the legacy identity provider and necessary protocols) A federation rework assumes model change. To keep it working with the python-keystone artificial table entries may be created (in the example when a new identity provider is being created automatically sanitized entries are being added for the legacy identity provider together with necessary idp protocols).

Fernet

keystone-ng uses the same mechanism for tokens to provide compatibility. The fernet-keys repository must be provided in the runtime (i.e. by mounting them as a volume into the container). There is no tooling to create or rotate keys as the py-keystone does.

Architecture Decision Records

Keystone project uses Architecture Decision Records to describe why the software is built how it is built.

1. Record architecture decisions

Date: 2025-11-03

Status

Accepted

Context

We need to record the architectural decisions made on this project.

Decision

We will use Architecture Decision Records, as described by Michael Nygard.

Consequences

See Michael Nygard’s article, linked above. For a lightweight ADR toolset, see Nat Pryce’s adr-tools.

2. Open Policy Agent

Date: 2025-11-03

Status

Accepted

Context

Use of oslo.policy is not easily possible from Rust. In addition to that during the OpenStack Summit 2025 it was shown how Open Policy Agent can be used to further improve the policy control in OpenStack. As such the Keystone implement the policy enforcement using the OPA with the following rules:

1. List operation MUST receive the all query parameters of the operation in the target.

2. For Show operation the policy MUST receive the current record as the target (fetch the record and pass it into the policy engine).

3. Update operation MUST receive current and new state of the resource (first the current resource is fetched and passed together with the new state [current, target] to the policy engine).

4. Create operation works similarly as current oslo.policy with the desired state passed to the policy engine.

5. Delete operation MUST pass the current resource state of the resource into the policy engine.

Decision

The only policy enforcement engine supported in the Keystone is Open Policy Engine.

Consequences

• Policy evaluation requires external service (OPA) to be running.

• When covering existing functionality of the python Keystone policies SHOULD be converted as is and do not introduce a changed flow.

Standardized Policy Input Structure

The PolicyEnforcer interface is standardized with the following signature:

#![allow(unused)]
fn main() {
async fn enforce(
    &self,
    policy_name: &'static str,
    credentials: &ValidatedSecurityContext,
    target: Value,
    existing: Option<Value>,
) -> Result<PolicyEvaluationResult, PolicyError>;
}

The OPA input document structure is:

{
  "credentials": { "user_id": "...", "roles": [...], "domain_id": "...", ... },
  "target": {
    "<resource>": <object or null>
  },
  "existing": {
    "<resource>": <object or null>
  }
}

The <resource> key matches the REST resource type: user, group, role, project, instance, idp, mapping, restriction, assignment, etc. This prevents field name collisions between policies and ensures each resource’s data is properly isolated.

Field Semantics Per Operation

The <resource> key matches the REST resource type:

• user, group, role, project, instance, idp, mapping, restriction, assignment, etc.
• This isolates data and prevents field name collisions between different resource schemas.

Policies read fields as input.target.user.domain_id, input.existing.restriction.user_id, input.target.instance.name, etc.

Examples:

• Create user: {"target": {"user": request_payload}}
• Update restriction: {"target": {"restriction": patch}, "existing": {"restriction": stored}}
• Show group: {"target": {"group": stored_object}}
• List instances: {"target": {"instance": query_params}}

Implementation Details

The handler-side contract for enforce():

• Create: Pass serde_json::to_value(request_object)? as target, None as existing
• Update: Pass serde_json::to_value(patch)? as target, Option::from(stored_object).map(serde_json::to_value) as existing
• Show: Pass serde_json::to_value(stored_object)? as target, None as existing
• Delete: Pass serde_json::to_value(stored_object)? as target, None as existing
• List: Pass serde_json::to_value(query_params)? as target, None as existing

Policy Evaluation Guidelines

Ownership predicates that need to work across create/show/delete/update should resolve the domain_id from either target or existing:

# Resolve domain_id from target or existing depending on operation
resource_domain_id := input.target.domain_id if {
    input.target.domain_id
}
resource_domain_id := input.existing.domain_id if {
    input.existing.domain_id
}

own_resource if {
    resource_domain_id != null
    resource_domain_id == input.credentials.domain_id
}

Validation rules (checking user-provided data for referential integrity, e.g., that domain/project/role IDs exist) should read from input.target for both create and update operations, as target carries the user’s request in both cases.

Notes

• The input.existing field is Value::Null when passed as None from the handler, not an absent key. Policies can check input.existing == null.

• The input.target field is never null except deliberately (e.g., when no target object is relevant). For operations where the object is the existing stored resource, target carries that object.

• Policy tests (*_test.rego) should use the same input structure as handlers:

  • Create tests: "target": { "binding": { ... } }
  • Update tests: "target": { "binding": { ... } }, "existing": { "binding": { ... } }
  • Show/Delete tests: "target": { "binding": { ... } }

3. Sea ORM

Date: 2025-11-03

Status

Accepted

Context

Sea ORM provides a nice full async support while at the same time enables simultaneous support for multiple database backends (PostgreSQL, MySQL, …).

Decision

Sea ORM is used as the ORM library for the database communication.

Consequences

No known

4. v4 API

Date: 2025-11-03

Status

Accepted

Context

New authentication workflows cannot be covered with the existing Keystone v3 API. As such it is required to add new API methods and eventually change the existing. As such not to break compatibility and provide a relatively easy way to route the traffic allowing both project (python and rust) to co-exist a new v4 API version should be introduced.

Decision

• All new auth methods MUST be implemented in v4.

• Known issues with the v3 API SHOULD be addressed in the v3.

Consequences

• New extended functionality will not be available in the v3.

• Certain necessary changes may be ported to the v3 (including python) to implement backwards compatibility. Example: acknowledge new token payload issued with v4 with the python Keystone.

5. Passkey Auth

Date: 2025-11-03

Status

Accepted

Context

Nowadays password-less authentication becomes standard. In OpenStack it is at the moment not implemented whether on the API level (for the CLI) nor on the UI.

Webauthn is a well accepted standard for implementing password-less authentication with the help of hardware or software authenticators. Keystone should implement support for new authentication methods relying on the webauthn.

Decision

Introduce webauthn support in Keystone. This requires adding new database tables and introduction of the additional flows to allow user registering authenticators.

• webauthn_credential table describes the authenticators of the user (user_id as the primary key).

• webauthn_state table stores authentication and registration states according to the standard.

• User should be able to request the desired scope in the authentication initialization request. In this case a scoped token is returned when user has the required access.

• To prevent attacks authentication requests for not existing users or users without registered authenticators MUST return fake (but valid) authentication state.

Consequences

New authentication method allows users to get valid token without requiring user to pass any secrets on the wire. Overall security of the system is increased.

6. Federation IDP

Date: 2025-11-03

Status

Accepted

Context

OIDC requires the server side to know the Identity provider details. Python Keystone relies on the external software to implement the OIDC flow only receiving the final data once the flow completes. Certain flows are triggered by the Service side (i.e. back-channel-logout). In addition to that relying on the external software does not allow any seld-service for the customer.

v3 currently provides limited OIDC support, but it is not possible to extend it in a backward compatible way.

As such OIDC support must be implemented natively in Keystone.

Decision

Keystone implement OIDC support natively without relying on the 3rd-party software. New APIs must provide self-service capabilities. Identity providers may be global (i.e. a social login) or dedicated (i.e. private Okta tenant).

A new set of APIs and database tables is added to Keystone for implementing new functionality. Existing DB constraints MUST not be deleted and only additive changes can be implemented to allow parallel deployment of python and rust Keystones for the smooth transition. “Virtual” database entries MUST be inserted for the old-style identity provider to guarantee the co-existence.

Global/private identity providers are implemented using the optional domain_id attribute. When empty the identity provider is treated as global (shared) and is correspondingly visible to every user of the cloud. Private IdPs SHOULD be only visible to the users of the domain. Corresponding rules MUST be implemented on the policy level to allow customization by the CSP.

The IdP specifies client_id and client_secret (when necessary). client_secret MUST not be retrievable. It can only be set during create or update operations. It MUST be also possible to specify JWKS urls when the identity provider does not implement metadata discovery.

Consequences

New APIs must be implemented in the CLI.

7. Federation Mapping

Date: 2025-11-03

Status

Accepted

Context

OIDC protocol describes how the user data is being passed to the service provider. It is necessary to translate this information to the Keystone data model. In v3 “mapping” is being used to describe this translation. The data model of the v3 mapping is, however, unnecessary complex (due to historical reasons). HashiCorp vault describes “mapping“s as “roles” for such translations. Since it is not possible to use the term “role” the mapping should continue to be used instead. The model of the Vault role provides a nice and easy reference for Keystone.

Decision

“Mapping” (attribute mapping) MUST describe how the information from OIDC claims need to be translated into the Keystone data model. It MUST also describe user defined bounds to allow use restriction.

When domain_id is not being set on the IdP level it MUST be defined either on the mapping entry, or the mapping MUST define domain_id_claim to extract the information about domain relation of the user. domain_id MUST be immutable property of the mapping to prevent moving it to the foreign domain.

Mapping MUST have the name attribute that is unique within the domain.

default_mapping_name property SHOULD be specified on the IdP level to provide a default for when the user does not explicitly specify which mapping should be used.

Consequences

• Mappings MUST be configured carefully to prevent login of users across the domain borders. bound_xxx should be used extensively to guard this.

8. Workload Federation

Date: 2025-11-03

Status

Accepted

Context

It is often desired to access the OpenStack cloud from workloads (i.e. GitHub workflow, Zuul job, etc). Usually such services provide a JWT issued by the platform which the service provider can trust. This is very similar (and technically relates) to the OIDC standard.

In the JWT flow the “user” is exchanging a JWT token issued by the trusted IdP for a Keystone token. This authentication response includes a token and a service catalog to provide a known OpenStack usage scenario.

Decision

OIDC mappings MUST specify a type which is oidc or jwt to specify the flow they define. A jwt type mapping can be only used in the JWT flow.

The new authentication API includes the IdP ID. The authentication request does not support the Json request body and uses a generic authorization: bearer <jwt> header and openstack-mapping-name: <mapping_name> to request the information. Depending on the mapping configuration the desired authorization scope is returned. The flow does not support explicitly requesting the scope beyond what is described by the mapping.

Consequences

• A new API to exchange JWT token for the Keystone token is added.

• JWT auth must provide the mapping name.

• The mapping SHOULD point to some for of the technical user.

9. Auth token revocation

Date: 2025-11-18

Status

Accepted

Context

Issued tokens are having certain configurable validity. In cases when a user need to be disabled, the project deactivated, or simply to prevent the token use after the work has been completed it is necessary to provide the possibility to invalidate the tokens. Python Keystone provides this possibility and so it is necessary to implement it in the same way.

Since original functionality is not explicitly documented this ADR will become the base of such information.

Decision

Fernet token revocation is implemented based on the revocation_event database table.

The table has following fields:

    pub id: i32,
    pub domain_id: Option<String>,
    pub project_id: Option<String>,
    pub user_id: Option<String>,
    pub role_id: Option<String>,
    pub trust_id: Option<String>,
    pub consumer_id: Option<String>,
    pub access_token_id: Option<String>,
    pub issued_before: DateTime,
    pub expires_at: Option<DateTime>,
    pub revoked_at: DateTime,
    pub audit_id: Option<String>,
    pub audit_chain_id: Option<String>,

Token revocation

When a revocation of a valid token is being requested the record with the following information is being inserted into the database:

• audit_id is populated with the first entry of the token audit_ids list. When this list is empty an error is being returned.
• issued_before is set to the current time with the UTC timezone.
• revoked_at is set to the current time with the UTC timezone.
• other fields are left empty.

Revocation check

A token validation for being revoked is performed based on the presence of the revocation events in the revocation_event table matching the expanded token properties. This means that before the token revocation is being checked additional database queries for expanding the scope information including the roles the token is granting are performed.

Following conditions are combined with the AND condition:

• First element of the token’s audit_ids property is compared against the database record. When this list is empty an error is being returned.
• token.project_id is compared against the database record when present.
• token.user_id is compared against the database record when present.
• token.trustor_id is compared against the database record user_id when present.
• token.trustee_id is compared against the database record user_id when present.
• token.trust_id is compared against the database record trust_id when present.
• token.issued_at is compared against the database record with revocation_event.issued_before >= token.issued_at.

Python version of the Keystone applies additional match verification for the selected data on the server side and not in the database query.

• When revocation_event.domain_id is set it is compared against token.domain_id and token.identity_domain_id.
• When revocation_event.role_id is present it is compared against every of the token.roles.

After the first non matching result further evaluation is being stopped. Logically there does not seem to be a reason for such handling and it looks to be an evolutionary design decision. Following checks can be added into the single database query with a different logic only comparing the corresponding fields when the column is not empty.

While following checks allow much higher details of the revocation events in the context of the usual fernet token revocation it is only going to match on the audit_id and issued_before.

Revocation table purge

In the python Keystone there is no automatic cleanup handling. Due to that expired records are removed during the revocation check. Records to be expired are selected using the following logic.

• expire_delta = CONF.token.expiration + CONF.token.expiration_buffer
• oldest = utc.now() - expire_delta
• DELETE from revocation_event WHERE revoked_at < oldest

When both python and rust Keystone versions are deployed in parallel and both try to delete expired records errors can occur. However, if only rust version is validating the tokens python version will not perform any backups. Additionally no errors were reported yet in installations with multiple Keystone instances. Therefore it is necessary for the rust implementation to do periodic cleanup. It should be exexcuted with the following query filter: revoked_at < (now - (expiration + expiration_buffer)). Such implementation must be made optional with possibility to disable this behavior using the config file.

Consequences

• Database table with the revocation events must be periodically cleaned up.

• Token validation processing time is increased with the database lookup.

• Expired revocation records are optionally periodically cleaned by the rust implementation.

10. PCI-DSS requirement: Invalid authentication attempts are limited

Date: 2025-11-27

Status

Accepted

Context

PCI-DSS contains the following requirement to the IAM system:

Invalid authentication attempts are limited by:

• Locking out the user ID after not more than 10 attempts.
• Setting the lockout duration to a minimum of 30 minutes or until the user’s identity is confirmed.

Python Keystone implements this requirement with the help of the conf.security_compliance.lockout_duration during the login attempt to identify whether the user is currently temporarily disabled:

    def _is_account_locked(self, user_id, user_ref):
        """Check if the user account is locked.

        Checks if the user account is locked based on the number of failed
        authentication attempts.

        :param user_id: The user ID
        :param user_ref: Reference to the user object
        :returns Boolean: True if the account is locked; False otherwise

        """
        ignore_option = user_ref.get_resource_option(
            options.IGNORE_LOCKOUT_ATTEMPT_OPT.option_id
        )
        if ignore_option and ignore_option.option_value is True:
            return False

        attempts = user_ref.local_user.failed_auth_count or 0
        max_attempts = CONF.security_compliance.lockout_failure_attempts
        lockout_duration = CONF.security_compliance.lockout_duration
        if max_attempts and (attempts >= max_attempts):
            if not lockout_duration:
                return True
            else:
                delta = datetime.timedelta(seconds=lockout_duration)
                last_failure = user_ref.local_user.failed_auth_at
                if (last_failure + delta) > timeutils.utcnow():
                    return True
                else:
                    self._reset_failed_auth(user_id)
        return False

Decision

For compatibility reasons rust implementation must adhere to the requirement.

During password authentication before validating the password following check must be applied part of the locked account verification:

• When conf.security_compliance.lockout_duration and conf.security_compliance.lockout_failure_attempts are not set the account is NOT locked.

• When user_options.IGNORE_LOCKOUT_ATTEMPT is set user account is NOT locked

• When user.failed_auth_count >= conf.security_compliance.lockout_failure_attempts the account is locked.

• When user.failed_auth_at + conf.security_compliance.lockout_duration > now() account is locked. When the time is < now() - reset the counters in the database.

• Otherwise the account is NOT locked.

After the authentication is success the user.failed_auth_at and user.failed_auth_count are being reset. In the case of failed authentication such attempt sets the mentioned properties correspondingly.

Consequences

• Authentication with methods other than username password are not protected.

• Reactivating the temporarily locked account can be performed by the admin or domain admin via resetting the user.failed_auth_count attribute.

11. PCI-DSS requirement: Inactive user accounts are removed/disabled

Date: 2025-11-27

Status

Accepted

Context

PCI-DSS contains the following requirement to the IAM system:

Inactive user accounts are removed or disabled within 90 days of inactivity.

Python Keystone implements this requirement with the help of the conf.security_compliance.disable_user_account_days_inactive during the login attempt to identify whether the user is currently active or deactivated:

    def enabled(self):
        """Return whether user is enabled or not."""
        if self._enabled:
            max_days = (
                CONF.security_compliance.disable_user_account_days_inactive
            )
            inactivity_exempt = getattr(
                self.get_resource_option(
                    iro.IGNORE_USER_INACTIVITY_OPT.option_id
                ),
                'option_value',
                False,
            )
            last_active = self.last_active_at
            if not last_active and self.created_at:
                last_active = self.created_at.date()
            if max_days and last_active:
                now = timeutils.utcnow().date()
                days_inactive = (now - last_active).days
                if days_inactive >= max_days and not inactivity_exempt:
                    self._enabled = False
        return self._enabled

In python Keystone there is no periodic process that deactivates inactive accounts. Instead it is calculated on demand during the login process and listint/showing user details. With the new application architecture in Rust it is possible to implement background processes that disable inactive users. This allows doing less calculations during user authentication and fetching since it is possible to rely that the background process deactivates accounts when necessary.

Decision

For compatibility reasons rust implementation must adhere to the requirement.

After successful authentication when user.enabled attribute is not true the authentication request must be rejected with http.Unauthorized.

Additional background process must be implemented to deactivate inactive accounts. For this when conf.security_compliance.disable_user_account_days_inactive is set a process should loop over all user accounts. When the user.last_active_at + disable_user_account_days_inactive < now() presence of the user.options.IGNORE_USER_INACTIVITY_OPT should be checked. When absent the account must be updated setting user.enabled to false.

Since it is technically possible that the background process is not running for any reason the same logic should be applied also when converting the identity backend data to the internal account representation and applied when the user data is reported by the backend as active. On the other hand having a separate background process helps updating account data in the backend and produce audit records on time without waiting for the on-demand logic to apply. It also allows disabling accounts in the remote identity backends that are connected with read/write mode (i.e. SCIM push).

After the successful authentication of the user with password or the federated workflow the user.last_active_at should be set to the current date time.

Consequences

• Authentication with methods other than username password are not updating the lst_active_at. Due to that the account that used i.e. application credentials for the activation for more than X days would become disabled. This requires account to perform periodic login using the password.

• It should be considered to update application credentials workflow to update the user.last_active_at attribute after successful authentication.

• It could happen that the periodic account deactivation process does not work for certain amount of time (i.e due to bugs in the code or the chosen frequency) allowing the user to login when it should have been disabled. This can be only prevented by applying the same logic during the conversion of the database entry to the internal User structure the same way like python keystone is doing.

• Administrator account can be deactivated. Separate tooling or documentation how to unlock the account must be present.

12. PCI-DSS requirement: Inactive user accounts are removed/disabled

Date: 2025-11-27

Status

Accepted

Context

PCI-DSS contains the following requirement to the IAM system:

If passwords/passphrases are used as the only authentication factor for user access (i.e., in any single-factor authentication implementation) then either:

• Passwords/passphrases are changed at least once every 90 days, OR

• The security posture of accounts is dynamically analyzed, and real-time access to resources is automatically determined accordingly.

Python Keystone implements this requirement with the help of the conf.security_compliance.password_expires_days and password.expires_at during the login attempt to identify whether the specified used password is expired. user.options.IGNORE_PASSWORD_EXPIRY_OPT option allows bypassing the expiration check.

Decision

For compatibility reasons rust implementation must adhere to the requirement.

Password expiration is performed after verification that the password is valid.

• password.expires_at_int (as epoch seconds) or the password.expires_at (as date time specifies the password expiration. When none is set password is considered as valid. Otherwise it is compared against the current time.

• During account password update operation when user is not having the user.options.IGNORE_PASSWORD_EXPIRY_OPT option enabled the current date time plus the conf.security_compliance.password_expires_days time is persisted as the password.expires_at_int property.

• Password expiration MUST NOT be enforced in the password change flow to prevent a permanent lock out.

Consequences

• Administrator account can be deactivated. Separate tooling or documentation how to unlock the account must be present.

13. OpenIDConnect federation: Expiring group membership

Date: 2025-12-09

Status

Accepted

Context

Python Keystone uses expiring group membership for the federated users https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ussuri/expiring-group-memberships.html. Every time the user authenticates using the federated login it’s group membership are persisted in the expiring_user_group_membership table instead of the user_group_membership. The table has a non nullable column last_verified which is set to the time of the last user login. The user is considered to be included as a member of the group for the period of time specified in the conf.federation.default_authorization_ttl. Once the user.last_verified + ttl < current_timestamp() the user is not considered the member of the group anymore. The intention of this mechanism is to prevent stale group memberships granting the user privileges.

Decision

For compatibility reasons rust implementation must implement the same functionality.

Every time the user authenticates the user group memberships are persisted in the expiring_user_group_membership table.

• Current group membership is being read from the database (ignoring the time limitation).

• The group membership that the user should not be having anymore are deleted.

• For the new group memberships corresponding entries are added with the current timestamp.

• For all other groups that the user is still member of corresponding records are updated to set last_verified to the current timestamp.

• Effective role assignments of the user are taking into the consideration expiring group memberships through the list_user_groups respecting the expiring membershipts (independent of the idp_id) as expiring_user_group_membership.last_verified > current_timestamp - conf.federation.default_authorization_ttl.

Consequences

• The user must login periodically to keep application credentials working when corresponding roles are granted through the expiring group membership.

• With the SCIM support the expiring membership should not be necessary.

14. Application Credentials

Date: 2025-12-12

Status

Accepted

Context

Application Credentials will have the following characteristics:

• Immutable.

• Allow for optionally setting limits, e.g. 5 Application Credentials per User or Project, to prevent abuse of the resource.

• Assigned the set of current roles the creating User has on the Project at creation time, or optionally a list of roles that is a subset of the creating User’s roles on the Project.

• Secret exposed only once at creation time in the create API response.

• Limited ability to manipulate identity objects (see Limitations Imposed)

• Support expiration.

• Are deleted when the associated User is deleted.

Application Credentials will be treated as credentials and not authorization tokens, as this fits within the keystone model and is consistent with others APIs providing application authentication. It also avoids the security and performance implications of creating a new token type that would potentially never expire and have custom validation.

Decision

For compatibility reasons rust implementation must implement the same functionality.

Application Credential Management

Users can create, list, and delete Application Credentials for themselves. For example, adding an Application Credential:

POST /v3/users/{user_id}/application_credentials

{
    "application_credential": {
        "name": "backup",
        "description": "Backup job...",
        "expires_at": "2017-11-06T15:32:17.000000",
        "roles": [
            {"name": "Member"}
        ]
    }
}

name must be unique among a User’s application credentials, but name is only guaranteed to be unique under that User. name may be useful for Consumers who want human readable config files.

description is a long description for storing information about the purpose of the Application Credential. It is mostly useful in reports or listings of Application Credential.

expires_at is when the Application Credential expires. null means that the Application Credential does not automatically expire. expires_at is in ISO Date Time Format and is assumed to be in UTC if an explicit timezone offset is not included.

roles is an optional list of role names or ids that is a subset of the roles the Creating User has on the Project to which they are scoped at creation time. Roles that the Creating User does not have on the Project are an error.

In the initial implementation, the Application Credential will assume the roles of the Creating User or the given subset and we will not implement fine-grained access controls beyond that.

Response example:

{
    "application_credential": {
        "id": "aa4541d9-0bc0-44f5-b02d-a9d922df7cbd",
        "secret": "a49670c3c18b9e079b9cfaf51634f563dc8ae3070db2...",
        "name:" "backup",
        "description": "Backup job...",
        "expires_at": "2017-11-06T15:32:17.000000",
        "project_id": "1a6f968a-cebe-4265-9b36-f3ca2801296c",
        "roles": [
            {
                "id": "d49d6689-b0fc-494a-abc6-e2e094131861",
                "name": "Member"
            }
        ]
    }
}

The id in the response is the Application Credential identifier and would be returned in get or list API calls. An id is globally unique to the cloud.

secret is a random string and only returned via the create API call. Keystone will only store a hash of the secret and not the secret itself, so a lost secret is unrecoverable. Subsequent queries of an Application Credential will not return the secret field.

roles is a list of role names and ids. It is informational and can be used by the Consumer to verify that the Application Credential inherited the roles from the User that the Consumer expected. This is not a policy enforcement, it is simply for human validation.

If the Consumer prefers to generate their own secret, they can do so and provide it in the create call. Keystone will store a hash of the given secret. Keystone will return the secret once upon creation in the same way it would if it was generated, but will not store the secret itself nor return it after the initial creation.

A Consumer can list their existing Application Credentials:

GET /v3/users/{user_id}/application_credentials

{
  "application_credentials": [
    {
        "id": "aa4541d9-0bc0-44f5-b02d-a9d922df7cbd",
        "name:" "backup",
        "description": "Backup job...",
        "expires_at": "2017-11-06T15:32:17.000000",
        "project_id": "1a6f968a-cebe-4265-9b36-f3ca2801296c",
        "roles": [
            {
                "id": "d49d6689-b0fc-494a-abc6-e2e094131861",
                "name": "Member"
            }
        ]
    }
  ]
}

A Consumer can get information about a specific existing Application Credential:

GET /v3/users/{user_id}/application_credentials/{application_credential_id}

{
  "application_credentials": [
    {
        "id": "aa4541d9-0bc0-44f5-b02d-a9d922df7cbd",
        "name:" "backup",
        "description": "Backup job...",
        "expires_at": "2017-11-06T15:32:17.000000",
        "project_id": "1a6f968a-cebe-4265-9b36-f3ca2801296c",
        "roles": [
            {
                "id": "d49d6689-b0fc-494a-abc6-e2e094131861",
                "name": "Member"
            }
        ]
    }
  ]
}

A Consumer can delete one of their own existing Application Credential to invalidate it:

DELETE /v3/users/{user_id}/application_credentials/{application_credential_id}

Note

Application Credentials that expire will be deleted. The alternative would be to allow them to accumulate for forever in the hopes that keeping them around will make investigation as to why an Application is not working easier, but the only real benefit to this is providing a different error message. More thought and feedback on this are needed, but are not essential for the first round of work.

When the Creating User for an Application Credential is deleted, or if their roles on the Project to which the Application Credential is scoped are unassigned, that Application Credential is also deleted.

Aside from deletion, Application Credentials are immutable and may not be modified. Using an Application Credential to Obtain a Token

An Application Credential can be used for authentication to request a scoped token following Keystone’s normal authorization flow. For example:

POST /v3/auth/tokens

{
    "auth": {
        "identity": {
            "methods": [
                "application_credential"
            ],
            "application_credential": {
                "id": "aa4541d9-0bc0-44f5-b02d-a9d922df7cbd",
                "secret": "a49670c3c18b9e079b9cfaf51634f563dc8ae3070db2..."
            }
        }
    }
}

Keystone will validate the Application Credential by matching a hash of the key secret associated with the id similar to how Keystone does Password authentication currently.

If the Application Credential is referred to by name, it will be necessary to provide either user_id or the combination of user_name and user_domain_name so that Keystone can look up the Application Credential for the User.

POST /v3/auth/tokens

{
    "auth": {
        "identity": {
            "methods": [
                "application_credential"
            ],
            "application_credential": {
                "name": "backup",
                "user": {
                    "id": "1a6f968a-cebe-4265-9b36-f3ca2801296c"
                },
                "secret": "a49670c3c18b9e079b9cfaf51634f563dc8ae3070db2..."
            }
        }
    }
}

As an alternative to the current use of Service Users, a Deployer could create a single Service User and an Application Credential for each service. Or even create a Nova user and then give each nova instance it’s own Application Credential. Although at this point the Application Credential does not have the ability to further limit API use, the ability to start assigning Application Credentials per-service and performing expiration and rotation may be a desirable step forward that can be further enhanced with the addition of restricting an Application Credential’s API Access.

Consequences

This would have a positive security impact:

• Instead of having a Service User for each service, all services can use a single Service User and multiple Application Credentials. This decreases the attack vector of gaining access to privileged operations by reducing the number of accounts to attack.

• Usernames and passwords are kept out of configuration files. While Application Credentials are still extremely sensitive, if compromised they do not allow attackers to glean service user password conventions from configuration.

• Application Credentials will grow the ability to have limited access, so a move to them is a step towards limited access credentials.

• Application Credentials can be gracefully rotated out of use and deleted periodically, allowing Consumers and Deployers a mechanism to prevent compromised Users without requiring swapping credentials in short amounts of time that might cause service interruption or downtime.

• Although we had long considered allowing application credentials to live beyond the lifetime of its creating user in order to allow seamless application uptime when the user leaves the team, it unfortunately poses too high a risk for abuse. Ensuring the application credential is deleted when the user is deleted or removed from the project will prevent malicious or lazy users from giving themselves access to a project when they should no longer have it.

There is an inherent risk with adding a new credential type and changing authentication details. One such risk would be the allowing of many credentials for the same User account.

End user impact

• Consumers who have Applications that monitor or interact with OpenStack Services should be able to leverage this feature to improve the overall security and manageability of their Applications.
• Consumers can gracefully rotate Application Credentials for an Application with no downtime by creating a new Application Credential, updating config files to use the new Application Credential, and finally deleting the old Application Credential.
• Consumers who do not start using Application Credentials should experience no impact.

Deployers impact

• Deployers only need to enforce security on a single Service User instead of multiple.
• Password rotation policies for Service Users no longer require immediately redeploying service configuration files. A User password change does not affect the existing Application Credential in the various service configuration files.
• Deployers can gracefully rotate Application Credentials through a deployment with no downtime.

15. Kubernetes Authentication Mechanism for Keystone

Date: 2026-02-17

Status

Accepted

Context

Currently, Keystone supports various authentication mechanisms (Password, Token, TOTP, External, etc.). As OpenStack increasingly runs alongside or underneath Kubernetes workloads, there is a need for “machine authentication” where a Kubernetes Pod can exchange its Service Account Token (JWT) for a Keystone token without managing long-lived secrets like passwords or API keys.

This implementation follows the logic used by OpenBao:

1. Trust Establishment: Keystone is configured to trust a Kubernetes API server’s JWT issuer.
2. Role Mapping: A Kubernetes Service Account (and namespace) is mapped to a specific Keystone Project/Role.
3. Validation: Keystone validates the incoming JWT against the Kubernetes TokenReview API.

Decision

We will implement a new kubernetes auth method in Keystone. This requires persistent storage to manage multiple Kubernetes clusters (backends) and the mapping of Kubernetes identities to OpenStack identities.

1. Data Model Changes

Two new tables will be introduced to the Keystone schema.

Table: kubernetes_auth_instance

This table stores the configuration for connecting to and validating tokens from external Kubernetes clusters.

Table: kubernetes_auth_role

This table maps Kubernetes-specific attributes (Namespace/ServiceAccount) to Keystone-specific token restriction (User/Project/Roles).

Token Restrictions represent here a finite mapping of the user_id (which should point to the service account user and MUST be set), the target project (based on the project_id) and the corresponding roles granted on this scope. As such it is not required to grant the user roles on the project directly and instead only specify them in the token restriction mapping.

2. Required API

Administrative API (CRUD for Configuration)

Endpoints to manage the trust relationship.

• POST /v4/k8s_auth/: Register a new Kubernetes cluster.
• GET/PATCH/DELETE /v4/k8s_auth/instances/{cluster_id}: Manage cluster config.
• POST /v4/k8s_auth/instances/{cluster_id}/roles: Create a mapping between a K8s SA/Namespace and a Keystone Project.
• GET/PATCH/DELETE /v4/k8s_auth/instances/{cluster_id}/roles/{role_name}: Manage role mappings.
• POST /v4/k8s_auth/instances/{cluster_id}/auth: Exchange K8s SA token for Keystone token.

Authentication API (The “Login” Flow)

The new authentication endpoint is exposed under /v4/k8s_auth/instances/{cluster_id}/auth and expects a json payload with a POST method.

Request Payload:

{
  "k8s_role": "web-servers-role",
  "jwt": "<jwt_from_k8s_service_account_token_volume_projection>"
}

3. Authentication Workflow

1. Lookup: Keystone receives the request, identifies the role. It fetches the associated kubernetes_auth_instance config.
2. Verification: Keystone calls the Kubernetes API (host) at the /apis/authentication.k8s.io/v1/tokenreviews endpoint using the token_reviewer_jwt or the user specified jwt when token_reviewer_jwt is unset. In the later case it is required that the service account has the system:auth-delegator ClusterRole. It can be granted with

kubectl create clusterrolebinding client-auth-delegator \
  --clusterrole=system:auth-delegator \
  --group=group1 \
  --serviceaccount=default:svcaccount1 ...

3. Validation:

• K8s returns the status of the JWT.
• Keystone verifies that the kubernetes.io/serviceaccount/service-account.name and namespace claims match the bound_service_account_names and namespaces in the kubernetes_auth_role table.

4. Token Issuance: If valid, Keystone issues a scoped token for the token_restriction_id defined in the role mapping.

Consequences

• Pros:

  • Enables seamless “Secretless” authentication for workloads running on Kubernetes.
  • Matches industry standards set by OpenBao/Vault.
  • Supports multi-tenancy by allowing multiple Kubernetes clusters to connect to one Keystone.

• Cons:

  • Keystone must have network line-of-sight to the Kubernetes API server.
  • Adds complexity to the identity backend.

16. Distributed Encrypted Storage via Raft and Fjall

Date: 2026-04-12

Status

Accepted

Context

The current implementation of keystone requires a storage back-end that provides high availability, strong consistency for identity assignments, and industry-leading security for PII and secrets. Traditional SQL databases often introduce complexity in secret management and lack native “At-Rest” encryption tied to the application’s lifecycle.

We need a solution that:

• Guarantees Consistency: Identity changes must be linearizable.

• Embedded Performance: An embedded LSM-tree to avoid external database network overhead.

• Cryptographic Sovereignty: Data must be encrypted before it hits the log or the disk, ensuring a “Zero-Knowledge” storage layer.

Decision

We will implement a distributed storage engine using OpenRaft for consensus and Fjall as the local State Machine and Log Store. The architecture will follow the “Vault-style” encryption model.

1. The Storage Stack

• Consensus: openraft (Rust) for managing cluster membership and log replication.

• LSM-Tree: fjall for high-performance, disk-backed storage of the state machine.

• Serialization: rmp-serde (MessagePack) for compact binary representation of log entries.

2. The Cryptographic Barrier

To ensure data is never stored in plain-text on disk:

• AEAD Encryption: Use AES-256-GCM for all payloads.

• Log Binding: The Raft Index will be used as Associated Data (AD) for log entries to prevent replay attacks.

• Storage Binding: The Primary Key (e.g., UserID) will be used as AD for FjallDB entries to prevent key-substitution attacks.

• Key Hierarchy: A Master Key (KEK) provided via Environment/HSM will wrap a volatile Data Encryption Key (DEK) kept in memory.

3. Data Flow

• Write Path:

  API receives a request → Serialize to MsgPack → Encrypt → Propose to OpenRaft.

  Apply step: Decrypt using Raft Index → Re-encrypt for storage → Write to Fjall.

• Read Path:

  Linearizable Read: Follower queries Leader for ReadIndex → Follower waits for local apply → Decrypt from Fjall → Return over mTLS.

Technical Specifications

gRPC Definitions

The internal Raft communication will use an opaque binary payload to keep the consensus layer decoupled from the IAM logic.

Protocol Buffers

#![allow(unused)]
fn main() {

message RaftEntry {
  uint64 term = 1;
  uint64 index = 2;

  // Optional Membership config.
  Membership membership = 3;

  // Optional Store request.
  // [12b Nonce][Ciphertext][16b Tag] }
  optional bytes app_data = 4;
}
}

Type Configurations (openraft)

#![allow(unused)]
fn main() {
openraft::declare_raft_types!(
    pub KeystoneConfig:
        D = EncryptedBlob, // Vec<u8> wrapper
        R = Response, // Ephemeral, plain-text over mTLS
        NodeId = u64,
        Node = BasicNode,
);
}

Consequences

Positive

• Security: Compromising the disk or the Raft log does not leak user secrets.

• Performance: Fjall provides SSD-optimized writes and efficient prefix-seeking for IAM queries.

• Simplicity: No external dependency on Postgres/MySQL; the binary is self-contained. Operator is able to select the traditional SQL backend drivers though.

Negative / Risks

• CPU Overhead: Every write/read involves AES-GCM operations.

• Operational Complexity: Cluster forming, backup/restore operations are now part of the Keystone operations.

• Stale Reads: If not configured correctly, followers might serve stale identity data unless the ReadIndex protocol is strictly followed.

Compliance

All secret handling must implement the Zeroize trait to ensure plain-text data is wiped from RAM immediately after gRPC transmission.

ADR 0016-v2: Distributed Encrypted Storage via Raft and Fjall

Date: 2026-06-13 Last-revised: 2026-07-02 (PKCS#11/TPM KEK provisioning addendum)

Status

Proposed

Supersedes: ADR-0016 (2026-04-12)

Security review findings applied (2026-06-24):

• F1 HIGH: Raft log nonce NodeId widened from 4 to 8 bytes; counter shrunk to 4 bytes (§2.2)
• F2 MEDIUM: Audit HMAC key derivation made per-node via node_id in HKDF info (§3.1)
• F3 MEDIUM: Backup manifest AD now includes dek_version_u32 to prevent same-second swap (§7)
• F4 MEDIUM: SPIFFE mode expanded with TTL ceiling, fail-closed behaviour, SPIFFE ID pattern (§4.1)
• F5 MEDIUM: Quarantine state specified as Raft-committed and restart-persistent (§10 invariant 5)
• F6 LOW: Emergency rotation confirmation timeout now has an explicit abort + audit path (§6.2)
• F7 LOW: NodeId uniqueness check specified as fail-closed when leader is unreachable (§4.3)
• F8 LOW: Sub-key derivation notation changed from HKDF-SHA256 to HKDF-Expand (§2.1)

Addendum applied (2026-07-02): §2.5 added, specifying the concrete PKCS#11 and TPM 2.0 KEK provider mechanisms that §2.1 previously deferred (“HSM / PKCS#11 / Cloud KMS”), the kek_provider configuration schema, and new invariants 13–15 (§10).

Context

Keystone-NG requires a storage backend providing high availability, strong linearizable consistency for identity assignments, and absolute cryptographic sovereignty over PII and secrets. Traditional SQL databases lack native application-lifecycle encryption and introduce external network dependencies.

We need a solution that:

• Guarantees Consistency: Identity changes must be linearizable; a revoked user or disabled account must never be observable as active by any node.
• Embedded Performance: An embedded LSM-tree avoids external database network overhead.
• Cryptographic Sovereignty: Data must be encrypted before it touches the Raft log or disk. A full disk or log compromise must not leak plaintext payloads or user identifiers.
• Zero-Trust Transport: Intra-cluster communication must be mutually authenticated with short-lived, automatically rotated credentials.

Decision

We will implement a distributed storage engine using OpenRaft for consensus and Fjall as the local state machine and log store, following a “Vault-style” encryption model. Intra-cluster mTLS supports two modes (spiffe and tls), and local follower reads are permitted strictly for non-sensitive data via a cryptographically bound tiering system.

1. The Storage Stack

Management Interface: Admin operations are performed via the keystone-manage storage CLI, which communicates with the cluster over gRPC with mTLS enforcement (SPIFFE SVID or operator-managed TLS certificates). This gRPC interface is not exposed to the public network — it is accessible only on the internal management network to operators.

Authorization is enforced at the gRPC interceptor level using the mTLS client identity. Each management RPC has an explicit allow-list mapping SPIFFE SVID identities (or TLS SAN URIs) to permitted operations. For example, only SVIDs with a storage-operator role tag may invoke RotateDek or ClearQuarantine. Network isolation serves as a compensating control, not the sole enforcement boundary, in accordance with the zero-trust principle stated in §Context.

Rate Limiting: Management RPCs enforce per-source-IP and per-identity rate limits. RotateDekRequest is limited to 2 invocations per hour per operator; ClearQuarantineRequest is limited to 10 per hour per operator. RotateDekRequest{emergency: true} additionally requires dual-control approval: a second operator with the storage-operator role must confirm within 5 minutes via a separate ConfirmRotateDekRequest RPC. Dual-control events are recorded in the audit log with both operator identities.

Supply Chain: Core dependencies (openraft, fjall) are pinned to exact versions in Cargo.lock. New releases must pass a manual security review before upgrading. A contingency plan (fork, vendor, or replace) is maintained for each dependency. cargo-vet or equivalent is used in CI for these two crates specifically. cargo deny rules reject any transitive dependency that stores key material without implementing ZeroizeOnDrop. cargo-vet coverage is extended to all crates that directly handle key material or ciphertext: the AES-GCM provider, the mlock wrapper, the HKDF implementation, and — per the §2.5 PKCS#11/TPM addendum — the two production KEK-provider crates’ HSM/TPM client libraries: cryptoki (pinned 0.12.0, storage-crypto-pkcs11) and tss-esapi (pinned 7.7.0, storage-crypto-tpm). Both hold the unwrapped DEK transiently during wrap_dek/unwrap_dek and sit directly in the KEK call path, so they meet criteria (a) and (b) below. Any new dependency falls under extended cargo-vet coverage if it: (a) receives or stores a Zeroizing<T> value; (b) is in the AES-HKDF-KMS call path; (c) provides mlock or VirtualLock functionality; or (d) processes raw ciphertext before decryption. Pull request reviewers must verify new Cargo.toml entries against this checklist before approval.

2. The Cryptographic Barrier

2.1 Key Hierarchy

HSM / PKCS#11 / Cloud KMS
        │
        ▼
  Master Key (KEK)                ← never touches RAM as plaintext
        │
        │  AES-256-GCM unwrap
        ▼
Data Encryption Key (DEK)       ← 256-bit random key, generated at bootstrap
         │
         ├── Log DEK (LD)          ← HKDF-Expand(DEK, info="keystone-raft-log-v1", L=32)
         ├── State DEK (SD)        ← HKDF-Expand(DEK, info="keystone-fjall-state-v1", L=32)
         └── Backup DEK (BDEK)     ← HKDF-Expand(DEK, info="keystone-backup-v1" ++ dek_version_u32_be, L=32)

DEK Bootstrap: DEK generation MUST target an already-mlock’d allocation; it MUST NOT be generated into an unlocked buffer and subsequently copied. A transient unlocked copy would allow the DEK to be written to swap, bypassing the memory protection described in §9.

• KEK Provisioning (Production): KEK resides in an HSM or Cloud KMS. The KEK never enters process memory.
• KEK Provisioning (Development): An environment variable KEYSTONE_DEV_KEK may supply a hex-encoded KEK. The process refuses to start unless --dev-mode and KEYSTONE_ALLOW_ENV_KEK=1 are explicitly set. After reading KEYSTONE_DEV_KEK, the process must immediately unset it via std::env::remove_var and zero the original string. The variable must not persist in the process environment for the lifetime of the process, preventing exposure via /proc/<pid>/environ.
• DEK Derivation: The DEK is derived into isolated sub-keys via HKDF-Expand (Expand-only; the DEK is already uniformly random so HKDF-Extract is not needed) to ensure log, state, and backup ciphertexts are never encrypted under the same key context. The backup DEK (BDEK) incorporates the active dek_version_u32 into its derivation input, binding it to the current DEK epoch. This ensures backups created under different DEK rotations do not share the same BDEK, limiting the blast radius of a BDEK compromise.

2.2 Nonce Management & GCM Tags

AES-256-GCM requires strict nonce uniqueness. Truncated tags are prohibited; all tags must be 16 bytes.

• Raft Log (Log DEK): Nonce is [8-byte NodeId BE] ++ [4-byte monotonic counter BE]. NodeId occupies the full 8 bytes (matching the u64 type in §8) to prevent nonce-space collision between nodes that share the same lower 32 bits. The counter is stored durably and increments with a reservation block of 1024 to absorb crashes. On startup, the persisted counter is validated against a separately-stored high-water mark (kept in a dedicated Fjall metadata key _meta:nonce_hwm:<node_id>); if the recovered counter is less than or equal to the high-water mark, the node refuses to start and requires operator intervention, preventing nonce reuse from counter corruption. After each reservation block write, the node verifies the write by reading back and comparing; if the read-back does not match, the node treats it as a fatal storage error and halts, preventing silent nonce reuse during a live session. A warning is emitted when remaining counter space drops below 10% of the 2^31 rotation threshold.
• Fjall State Machine (State DEK): Nonce is derived via HKDF-Expand(StateDek, info=PrimaryKey || version_u32, L=12). The version field starts at 0 for new records and increments on each update. This guarantees that even if the same primary key is re-written, the nonce is unique under the current DEK. u32 is used (allowing ~4.3 billion updates per record per DEK epoch); with 90-day rotation and realistic IAM workloads, overflow is not expected. The version is stored as a 4-byte big-endian suffix alongside the ciphertext in Fjall, laid out as: [nonce_12b][ciphertext][tag_16b][version_u32_BE]. On read, the version is extracted from this suffix to determine the next increment value.

2.3 Associated Data (AD) Bindings

To prevent ciphertext substitution and data tampering, we tightly bind the context of the data to the AES-GCM envelope.

2.4 Known Limitation: Primary Key Confidentiality

Fjall stores primary keys (UserIDs, domain identifiers, etc.) as plaintext index entries. An attacker with disk access can enumerate all stored identifiers without decrypting values. All primary keys are UUIDv4 — cryptographically random identifiers with no encoded semantic content. An attacker can learn account cardinality (total count) and existence, but cannot determine personal identities, names, emails, or other PII without decrypting the associated values under the DEK. This reduces the exposure from PII disclosure to merely revealing the existence and cardinality of accounts.

This is an accepted limitation: encrypting index keys would require deterministic encryption (leaking frequency patterns) or an oblivious data structure (impractical for an LSM-tree). The AES-256-GCM AD binding ensures that even with full index enumeration, the attacker cannot move values between keys or decrypt payloads without the DEK.

Residual Correlation: LSM-tree key ordering reveals creation order, and cross-reference fields (e.g., domain_id stored as a value in user records) link identifiers — but only when the values are decrypted. An attacker with plaintext keys alone cannot perform cross-account correlation.

Access Pattern Leakage: Access pattern analysis can reveal which keys are read or written and when, correlating active accounts and revocation events. This is evaluated against the in-scope attacker (physical disk access, backup exfiltration) — the attacker cannot observe real-time access patterns on a cold disk or static backup. For deployments where an attacker can monitor hardware- level I/O patterns, this constitutes an accepted limitation.

Threat Model: This limitation is evaluated against the following attacker capabilities:

• In scope: Physical disk access, backup exfiltration. The attacker can enumerate identifiers but cannot read values, modify data, impersonate nodes, or determine account ownership without the DEK and valid mTLS credentials.
• Out of scope: Attacker with KMS access, rogue operator with root privileges, or supply-chain compromise of fjall. The legal/compliance team has reviewed this limitation against applicable regulations (GDPR pseudonymisation obligations) and confirmed it meets the organization’s privacy requirements.

2.5 PKCS#11 and TPM KEK Provisioning

§2.1 named “HSM / PKCS#11 / Cloud KMS” as the production KEK source but did not specify a mechanism. This section specifies the PKCS#11 and TPM 2.0 KekProvider implementations. Both satisfy invariant 2 (§10): the KEK itself never enters process memory in plaintext, only wrapped DEK bytes cross the provider boundary.

Provider selection: [distributed_storage] kek_provider selects between env (dev-mode only, §2.1), pkcs11, and tpm. Exactly one provider is active per node. Production deployments (dev_mode = false) MUST set this to pkcs11 or tpm; kek_provider = "env" outside dev_mode is rejected at config-validation time, before any KEK material is touched (invariant 6).

2.5.1 PKCS#11

The KEK is an AES-256 key object resident on a PKCS#11 token (a hardware HSM or, for development/CI, SoftHSM2), created with CKA_EXTRACTABLE = false and CKA_SENSITIVE = true. wrap_dek/unwrap_dek invoke CKM_AES_GCM directly against that key object (C_EncryptInit/C_Encrypt and C_DecryptInit/C_Decrypt with a CK_GCM_PARAMS structure carrying a freshly generated 12-byte nonce, DEK_WRAP_AD as additional authenticated data, and a 128-bit tag). The resulting wire format is byte-identical to EnvKek’s: [12-byte nonce][ciphertext][16-byte tag] — no downstream code (DEK bootstrap, Fjall metadata storage) needs to distinguish which KekProvider produced a wrapped blob.

• Key provisioning: creating the AES-256 key object on the token is an out-of-band operator step (e.g. pkcs11-tool --keygen or softhsm2-util --init-token for development), documented in the operator guide. The provider MAY auto-generate the key via CKM_AES_KEY_GEN on first startup if pkcs11_key_label is not found on the configured slot — this is an ergonomic convenience for fresh clusters, not a substitute for operator-controlled key ceremony in regulated deployments.
• PIN handling: the token PIN is read once at startup from pkcs11_pin_file into a SecretSlice (zeroized on Drop), used for C_Login. It is held for the duration of the init_storage call — not scrubbed the instant C_Login returns, since the same in-memory config clone is threaded through the rest of node startup — and zeroized once that call completes and the clone is dropped. The PIN is never accepted via environment variable or inline config value — only a file path, consistent with the existing tls_key_file/tls_cert_file convention (§4.2).
• Failure handling: a login failure, missing key object, or GCM tag mismatch on unwrap is fatal to node startup (or, post-startup, treated the same as any other GCM tag failure under invariant 5’s quarantine logic).

2.5.2 TPM 2.0

The KEK is a non-duplicable, TPM-resident symmetric-cipher key (fixedTPM | fixedParent | sensitiveDataOrigin, no duplication attribute). As with PKCS#11, the raw key material never leaves the TPM and never enters process memory — only the wrapped DEK crosses the boundary.

TPM 2.0 has no native AES-GCM command. TPM2_EncryptDecrypt2 supports only CFB/CBC/CTR/OFB/ECB symmetric modes; there is no AEAD primitive. The TPM provider therefore uses Encrypt-then-MAC instead of AES-GCM:

wrap_dek(dek):
  iv          = random 16 bytes
  ciphertext  = TPM2_EncryptDecrypt2(key = tpm_kek, mode = AES-256-CFB, iv, data = dek)
  tag         = TPM2_HMAC(key = tpm_hmac, data = iv ++ ciphertext ++ DEK_WRAP_AD)
  wrapped     = iv ++ ciphertext ++ tag        // [16b][32b][32b] = 80 bytes

unwrap_dek(wrapped):
  split wrapped into iv, ciphertext, tag
  expected_tag = TPM2_HMAC(key = tpm_hmac, data = iv ++ ciphertext ++ DEK_WRAP_AD)
  reject unless expected_tag == tag (constant-time compare) — never attempt
  TPM2_EncryptDecrypt2 on an unauthenticated ciphertext
  dek = TPM2_EncryptDecrypt2(key = tpm_kek, mode = AES-256-CFB, iv, data = ciphertext, decrypt = true)

This is a deliberate deviation from “AES-256-GCM for all payloads” (§2.2) scoped strictly to the TPM KEK-wrap boundary: it does not touch the Log DEK, State DEK, or Backup DEK, which remain AES-256-GCM as specified elsewhere in this ADR. tpm_kek and tpm_hmac MAY be the same TPM key object used in two different USAGE modes if the provisioning tooling supports it, or two separate persistent handles; either is acceptable provided both satisfy the non-duplicable, non-extractable attributes above.

• Key provisioning: a persistent handle (tpm_key_handle) or a saved key context (tpm_key_context_file) identifying the pre-provisioned TPM key(s). Provisioning itself (via tpm2_create/tpm2_evictcontrol or equivalent) is an out-of-band operator step, documented alongside the PKCS#11 key ceremony.
• Auth handling: if the key was provisioned with userWithAuth, the auth value is read once from tpm_auth_file into a SecretSlice (zeroized on Drop) and used to authorize the TPM session. As with the PKCS#11 PIN (§2.5.1), it is held for the duration of the init_storage call rather than scrubbed immediately after use, and zeroized once that call completes. Only a file path is accepted, never an environment variable or inline value. A key relying purely on PCR/policy session authorization may omit tpm_auth_file.
• Sample scope: the TPM provider ships with a runnable example targeting a software TPM (swtpm) for local exploration, and is not part of the required CI gate — real and virtual TPM availability in CI runners is not reliable enough to gate merges on. The PKCS#11 path (§2.5.1), backed by SoftHSM2, is the CI-gated path (§1).

3. Read Consistency and Data Tiers

To optimize read-heavy IAM workloads without sacrificing security, data is categorized into sensitivity tiers. The tier marker is prefixed to the plaintext and bound into the AES-GCM AD; altering the tier invalidates the ciphertext.

Audit data is out of scope for this Raft storage engine. Audit logging is handled by a separate external pipeline (e.g., SIEM, centralized logging) and is not subject to the tiering or read consistency model described here.

Group membership is elevated to Tier 2 (linearizable only) because it is a direct input to authorization decisions. A stale read of group membership can cause a recently removed member to still be considered part of a privileged group, producing an incorrect access grant. Callers must ensure that any Tier 1 data used in live authorization decisions is read via a linearizable path.

Configuration: Operators enable local reads via local_reads_mode = "local_for_public". Tier 2 and 3 data always require the ReadIndex protocol, ensuring revoked credentials are never exposed via a stale follower.

3.1 Audit Log Architecture

Audit events referenced in this ADR (DEK rotation, skipped re-encryption keys, quarantine recovery, emergency rotation, and operator actions) are forwarded to an external SIEM or centralized logging pipeline. The audit log is not stored within the Raft storage engine, satisfying GDPR Article 30 requirements independently of the identity data store.

Integrity: Each audit record is signed with a per-node HMAC-SHA256 key derived from the KEK via HKDF-Expand(KEK, info="keystone-audit-hmac-v1" ++ node_id_u64_be, L=32). The node_id is included in the derivation so each node holds a distinct signing key; a compromised node cannot forge audit records attributed to other nodes. The signing key is rotated on every DEK rotation, binding the HMAC key lifetime to the DEK epoch. The signature is computed over the canonical JSON representation of the audit record (including timestamp, event type, actor, and node_id), and transmitted alongside the record. An epoch tag (dek_version_u32) and the originating node_id are included in each audit record to identify which HMAC key signed it. Because the KEK never enters process memory in production (§2.1), this derivation is performed inside the HSM or Cloud KMS using a context-keyed derivation operation.

Transport: Audit records are forwarded over an authenticated channel (TCP/TLS) to the SIEM. The keystone node cannot unilaterally modify records already received by the SIEM, which enforces append-only semantics downstream.

HMAC Key Lifecycle: The epoch-tagged HMAC signing key is transmitted to the SIEM over the same authenticated channel as the audit records, bound to the dek_version_u32 epoch. This ensures the HMAC key is protected by the same transport mechanisms as the audit records themselves. The SIEM retains each epoch’s key for the duration of the audit retention period, enabling re- verification of historic records across epoch boundaries. The keystone node does not need to retain epoch keys beyond the current DEK epoch — responsibility for key lifecycle lies with the SIEM.

Availability: If the SIEM endpoint is unreachable, audit records are buffered locally (encrypted at rest with the Log DEK) and replayed on connectivity restoration. Buffer capacity is bounded to prevent disk exhaustion; if the buffer reaches 90% capacity, the node emits a CRITICAL alert. Audit buffer exhaustion is an operational concern for the audit pipeline, and does NOT affect the Raft storage engine’s availability — writes to the identity data store proceed normally regardless of SIEM connectivity or audit buffer state.

4. Intra-Cluster Transport (mTLS)

Operators select the transport mode via [storage] transport_mode.

Protocol Requirements: All mTLS connections (both SPIFFE and TLS fallback) MUST use TLS 1.3 or later. TLS 1.2 and earlier are prohibited. Permitted cipher suites are restricted to AEAD-only: TLS_AES_256_GCM_SHA384 and TLS_CHACHA20_POLY1305_SHA256. The TLS stack must enforce these settings at configuration time and refuse to start if unsupported ciphers are negotiated.

4.1 SPIFFE Mode (Default)

Managed automatically by SPIRE.

• Identity: Short-lived X.509 SVIDs rotated automatically. SVID TTL MUST NOT exceed 1 hour. Node processes enforce this by refusing to use an SVID with a remaining validity of less than 5 minutes (force-renewal window).
• SPIFFE ID Pattern: All Keystone storage node SVIDs MUST match spiffe://<trust-domain>/keystone/storage/<role>. The gRPC interceptor validates this pattern before any RPC is dispatched; connections from SVIDs that do not match are rejected with PERMISSION_DENIED.
• SPIRE Unavailability (Fail-Closed): If the SPIRE agent cannot renew an expiring SVID before it enters the force-renewal window, the node emits a CRITICAL log entry. If the SVID expires before renewal succeeds, the node refuses to accept new inbound connections and drains in-flight Raft proposals before halting. It does NOT fall back to an expired SVID.
• Trust Bundle Refresh: The SPIRE agent manages trust bundle rotation automatically. No manual intervention is required or permitted for trust bundle updates.

4.2 TLS Mode (Fallback)

Operator-managed PKI for environments without SPIRE.

• PKI Rules: Must use a dedicated Keystone Intermediate CA. Leaf certificates MUST NOT exceed 30 days validity (enforced at startup).
• Certificate Expiry Watchdog: A runtime watchdog checks remaining certificate validity at a regular interval (every hour). It logs warnings at 7 days remaining, errors at 2 days remaining, and triggers a configurable action (warn-only or shutdown) at expiry. This prevents the enforcement gap where a node starts with a valid certificate but continues operating after expiry.
• Certificate Revocation: CRL and OCSP are not implemented for the TLS fallback path. The compensating controls are: (1) the 30-day maximum leaf certificate validity limits exposure from a stolen or miss-issued certificate; (2) prompt certificate replacement is required on compromise, enforced by the operator’s PKI management procedures. This is acceptable for an internal cluster where the operator manages the Intermediate CA and has direct control over certificate issuance and replacement.

Planned Improvement: CRL and OCSP support for the TLS fallback path is planned as a future enhancement to the custom mTLS infrastructure. When implemented, it will replace the reliance on short certificate validity as the sole revocation control and provide real-time certificate status verification at connection time.

4.3 NodeId Assignment

Decision: NodeId is a manually configured u64 that must be unique within the cluster.

Rationale: NodeId is an opaque cluster-local identifier, not a secret or authentication material. Cryptographic derivation (e.g., BLAKE3) adds complexity without security benefit: mTLS already provides mutual authentication, and the collision check enforces uniqueness. A deterministic hash of cluster_id and URI provides no additional assurance over manual assignment — the operator controls the certificate and the configuration, and under zero-trust both are within the adversary boundary.

Uniqueness Enforcement: At startup, the node queries the Raft membership config and compares its (node_id, rpc_addr) against all existing members. If any existing member shares the same node_id but has a different rpc_addr, a collision is detected. The node emits a fatal log entry and refuses to start. The leader enforces the same check when processing add_learner requests. This catches both operator misconfiguration and deliberate duplication. Fail-closed: If the node cannot contact any cluster member to retrieve the membership config at startup (e.g., network partition, no quorum), it MUST refuse to start rather than skip the uniqueness check. An inability to verify uniqueness is treated the same as a detected collision. This prevents a misconfigured node from joining an isolated network segment undetected.

5. Data Flow Architectures

5.1 Write Path

The payload is subject to double-encryption to separate log concerns from state concerns. This adds approximately 2× symmetric crypto overhead per write (Log DEK decrypt + State DEK encrypt). With AES-NI, measured at ~0.2ms per operation at typical IAM payload sizes.

Client API → Serialize (MsgPack)
    │
    ▼
Encrypt with Log DEK (AD = term ++ index)
    │
    ▼
Propose to OpenRaft Leader (Replicated over mTLS)
    │
    ▼
Apply on Node:
  1. Decrypt log entry (Log DEK)
  1.5. Fetch current version for PK from Fjall (default 0 if new); increment
  2. Re-encrypt for state (State DEK, nonce = HKDF-Expand(SD, info=PK ||
        version, L=12), AD = tier ++ domain ++ pk)
  3. Write ciphertext + version suffix to Fjall DB

The state machine is idempotent under Raft replay: a crash between the version fetch (§5.1 step 1.5) and the write (§5.1 step 3) results in no persistent state change, and the replay reads the same on-disk version, deriving the same nonce and producing the same ciphertext. This invariant holds because Fjall’s batch commit is atomic and Raft entries are deterministic.

6. DEK Rotation Lifecycle

To prevent AES-256-GCM nonce exhaustion, Data Encryption Keys must be periodically rotated. Rotation is triggered either by time (configurable via [storage] dek_rotation_days, default 90 days) or by volume (when AES-GCM encryptions reach 2^31 under any sub-key).

DEK Version Tracking: Each DEK epoch is assigned a monotonically increasing dek_version_u32. The version is stored alongside the wrapped DEK in Fjall metadata (e.g., _meta:dek:current:version). The version counter used in state machine nonces (see §2.2) is also tracked per DEK epoch.

Live Background Rotation Flow:

1. Generate a fresh DEK in memory with dek_version = current + 1.
2. Wrap under the KEK and write to _meta:dek:pending via a Raft proposal, recording the committed Raft index as rotation_index in the same proposal. This guarantees the pending DEK and its version exist before any write can reference them.
3. rotation_index serves as an unambiguous boundary: log entries at index < rotation_index use the retired DEK; entries at index ≥ rotation_index use the pending DEK.
4. All new Raft writes at or after rotation_index use the pending DEK.
5. A background task re-encrypts all existing Fjall records under the new DEK atomically in key-sorted order. The re-encryption uses optimistic concurrency control (CAS on version): it reads the on-disk version, encrypts under the new DEK with version + 1, and writes only if the version on disk is unchanged. If a concurrent Raft write incremented the version, the key is either already encrypted under the new DEK (the Raft write used the pending DEK) or is retried. After 3 failed CAS attempts on the same key, the background task skips it — the key is assumed to have been updated under the new DEK by a Raft write. This CAS-on-version mechanism ensures no re-encryption can clobber a concurrent Raft mutation during rotation. Skipped keys are flagged in the post-rotation verification report (see step 8), are automatically retried on the next scheduled rotation cycle, and are emitted to an audit log entry. If a skipped key remains unverified for more than 24 hours, a CRITICAL alert is emitted and operator intervention is required.
6. During rotation, both DEKs are active. Reads use the per-record dek_version to select the correct key deterministically — they never fall back via tag verification failure. If the dek_version is missing or ambiguous, the record is treated as corrupt and quarantined rather than probing both keys. Recovery path: Records with missing or ambiguous dek_version can be recovered from the Raft log by replaying the entry with the known rotation_index boundary to determine the correct DEK epoch. If the Raft log entry is unavailable (e.g., truncated by snapshots), the record is restored from backup. The blast radius is limited to the partition containing the affected key. Operators run keystone-manage storage recover --record to trigger recovery, which emits an audit log entry. For state entries, the per-record version field is incremented during re-encryption to produce a fresh nonce.
7. Upon completion, the old DEK is atomically promoted to _meta:dek:retired:<timestamp> via a second Raft proposal, and an audit log event is recorded.
8. Post-rotation verification: A verification pass reads each record’s stored dek_version and confirms it is consistent with the new DEK epoch. Any record that cannot be verified is flagged in an operator-facing report rather than silently accepted, ensuring no records remain orphaned under the retired DEK after rotation is declared complete. Unverified records trigger an automated retry on the next rotation cycle. If verification fails for more than 24 hours consecutively, a CRITICAL alert is emitted. The DEK retirement is blocked until the operator resolves the flagged records or explicitly approves proceeding with a signed audit entry acknowledging the unverified records.

Partial Rotation Recovery: If a node restarts mid-rotation, it detects _meta:dek:pending. The leader resumes the idempotent re-encryption from the last committed progress marker stored in _meta:dek:rotation_progress before normal operations complete. The rotation_index boundary remains authoritative for determining which DEK each log entry was encrypted under.

6.2 Emergency Rotation and DEK Compromise

When a DEK is suspected or confirmed compromised, the operator triggers an emergency rotation that follows the same flow as normal rotation but with additional containment steps:

1. Trigger: Emergency rotation is initiated via keystone-manage storage rotate-dek --emergency, which connects to the cluster over gRPC with mTLS enforcement (see §1). Access requires RBAC authorization (storage-operator role) and dual-control confirmation via ConfirmRotateDek from a second operator within 5 minutes. Confirmation timeout: If the 5-minute window expires without confirmation, the pending emergency rotation is automatically aborted: the node commits a Raft proposal removing _meta:dek:pending. The abort is recorded in the audit log with the initiating operator identity, the expiry timestamp, and the fact that no confirmation was received. The partial-rotation recovery path (end of §6) checks for a rotation_id timestamp and ignores _meta:dek:pending entries older than 5 minutes that were never confirmed, preventing an aborted emergency rotation from being resumed on restart as a normal rotation.
2. Immediate containment: A fresh DEK is generated and committed via Raft following the standard flow (§6 step 1-2). The compromised DEK is marked revoked (not retired) in _meta:dek:revoked:<timestamp>, preventing its reuse for any decryption operation.
3. Impact assessment: The operator queries the per-record dek_version to identify all records encrypted under the compromised DEK epoch. This determines the scope of potentially exposed data.
4. Re-encryption: The background task re-encrypts all affected records under the new DEK following the standard CAS-on-version flow (§6 step 5).
5. Discard: The revoked DEK is discarded immediately — it is NOT stored in the retired DEK chain and is NOT available to any KMS role. If offline decryption of backups from the compromised epoch is required, the operator must use the backup manifest (§7) and the BDEK, not the compromised DEK.
6. Incident logging: The emergency rotation, affected record count, and operator identity are recorded in the audit log with a distinct event type (DEK_EMERGENCY_ROTATION).

Normal rotation cadence resumes after the emergency rotation completes; the dek_rotation_days timer is reset to account for the forced rotation age.

7. Backup and Restore

Backups in Keystone-NG are fundamentally Fjall snapshots. Because all values in Fjall are encrypted via the State DEK, disk snapshots contain exclusively AES-256-GCM ciphertext.

Backup Encryption Envelope

Each backup is wrapped in a backup-specific envelope to bind it to a point in time and a specific DEK epoch, preventing rollback or replay attacks:

AES-256-GCM(
  plaintext  = snapshot_bytes,
  key        = Backup DEK (BDEK),
  AD         = b"keystone-backup-v1" ++ snapshot_utc_epoch_be_u64 ++ dek_version_u32
)

The dek_version_u32 identifies which DEK epoch the snapshot was taken under (at time of snapshot, current DEK version). The backup bundle includes a DEK manifest listing all retired DEKs that may be required for offline decryption, handling cases where the snapshot spans a rotation boundary. The DEK manifest is a separate structure included in the backup bundle alongside the encrypted snapshot. It is encrypted with the BDEK using AES-256-GCM with AD bound to b"keystone-backup-manifest-v1" ++ snapshot_utc_epoch_be_u64 ++ dek_version_u32. Including dek_version_u32 in the manifest AD prevents swapping the manifest between two backups taken within the same UTC second under different DEK epochs. The manifest itself is not covered by the outer backup envelope but is independently encrypted and integrity-protected. BDEK incorporates dek_version_u32 in its derivation, binding each backup to its DEK epoch, and the AD provides independent replay protection.

Note: The timestamp is explicitly an 8-byte big-endian UTC epoch seconds value. String timestamps are prohibited as Associated Data due to ambiguity and fragility.

Restore Process

Restoring a snapshot to a new cluster strictly requires:

1. Access to the Backup DEK (backup_dek role) in the KMS.
2. Valid node identity credentials (SPIFFE SVIDs or Intermediate CA certs) for the new nodes before they join the cluster.
3. Unwrapping the backup envelope, loading it into Fjall, and immediately re-wrapping the restored DEK under the new cluster’s runtime KEK.

Retired DEKs must be retained in the KMS for the organization’s audit retention period (minimum 365 days) to allow offline decryption of archived backups. When a GDPR data erasure request arrives, the ability to decrypt that subject’s data from archived backups depends on the retained DEKs. The operator must re-encrypt the affected archived backups under a fresh DEK epoch, or shard the key material to reduce the DEK retention window. For GDPR Article 17 compliance, the worst- case re-encryption time for a full backup archive (measured at the organization’s maximum anticipated volume) must be documented and evaluated against the 30-day erasure timeframe. If re-encryption exceeds 30 days, the organization must deploy per-subject wrapping keys: each subject’s data envelope is encrypted under a subject-specific envelope key, which is in turn encrypted under the BDEK. Erasure of a subject’s data then requires only destroying the subject’s envelope key, not re-encrypting the entire archive. If re-encryption remains not feasible, the operator must document the inability to achieve full erasure and the residual risk, per GDPR Article 17.

Retired DEK Access Control: Retired DEKs are accessed through a distinct KMS role (backup_dek_offline) that is separate from the runtime backup_dek role. This limits the blast radius of a KMS breach or insider threat: compromising the runtime role does not grant access to retired keys and vice versa. Access to any retired DEK requires dual-control or break-glass approval. Operators should evaluate whether the 365-day retention mandate can be satisfied with a separate escrow mechanism instead of keeping keys live in the operational KMS.

8. gRPC Protocol & Code Definitions

Protocol Buffers:

message RaftEntry {
  uint64 term  = 1;
  uint64 index = 2;
  optional Membership membership = 3;

  // Encrypted app payload: [12b counter-nonce][ciphertext][16b GCM tag]
  // AD = big-endian(term) ++ big-endian(index)
  optional bytes app_data = 4;
}

message RaftResponse {
  // SENSITIVE: Ephemeral plaintext response, transmitted only over mTLS.
  // Zeroized by the sender immediately after gRPC send.
  // Rust type: ZeroizingResponse — must be zeroized after use.
  bytes payload = 1;
}

message ClearQuarantineRequest {}

message RotateDekRequest {
  bool emergency = 1;
}

message ConfirmRotateDekRequest {
  // The rotation_id of the pending emergency rotation that requires confirmation.
  string rotation_id = 1;
}

Implementation Note (Residual Risk): gRPC implementations (including tonic/prost) may internally buffer, clone, or copy message bytes before or during transmission (e.g., into the HTTP/2 frame buffer). Zeroizing the application-level struct does not guarantee the copies within the gRPC stack are also zeroized. A stream-based response that processes the minimum plaintext footprint should be preferred where feasible.

Compensating Controls: These are validated at startup pre-flight (§9):

1. Core dumps must be disabled (RLIMIT_CORE = 0). The startup pre-flight verifies this and refuses to start if the limit is nonzero.
2. Set PR_SET_DUMPABLE = 0 (Linux) to prevent ptrace and /proc/<pid>/mem access from co-located processes. The startup pre-flight verifies this and refuses to start if it fails.
3. If Tier 3 data (credential plaintext, TOTP seeds) ever flows through a RaftResponse, streaming must be evaluated to minimize the peak plaintext footprint within gRPC stack buffers.

OpenRaft Types (Rust):

#![allow(unused)]
fn main() {
openraft::declare_raft_types!(
    pub KeystoneConfig:
        D = EncryptedBlob,   // [u8] wrapper enforcing 16-byte tag check
        R = ZeroizingResponse,
        NodeId = u64,        // Manually configured; collision detection via
                              // (node_id, rpc_addr) comparison against existing
                              // membership at startup and on learner-add
        Node = SpiffeNode,
);

}

9. Zeroize & Memory Protection

Standard software zeroization (ZeroizeOnDrop) is insufficient on its own because operating system page swapping can silently write key material to disk (swap pages), bypassing in-process zeroization.

Memory Locking (mlock)

To guarantee key material is physically pinned to RAM and never written to swap storage, all keys (Dek, LogDek, StateDek) and their immediate working buffers must be allocated in memory-locked pages using the OS-level mlock(2) (Linux) or VirtualLock (Windows) APIs.

This is enforced via a memory-locking wrapper (e.g., memsec or secrecy with OS-level locking):

#![allow(unused)]
fn main() {
// Internally wrapped in a secrets-manager to guarantee locked heap allocations:
let buf: memsec::Malloc<[u8; 32]> = memsec::malloc().expect("mlock allocation");

}

Resource Limits: The Keystone process must request an RLIMIT_MEMLOCK sufficient for the key material pool at startup. If mlock allocation fails due to insufficient OS limits, the process logs a CRITICAL warning and refuses to start in production mode.

Startup Pre-Flight: At process startup (before KEK/DEK are loaded), the node verifies RLIMIT_CORE == 0 and PR_SET_DUMPABLE == 0. If either check fails, the node emits a CRITICAL log entry and refuses to start in production mode (same pattern used for RLIMIT_MEMLOCK). This ensures the compensating controls against gRPC stack plaintext exposure are actually active.

Enforcement Pipeline:

• A #[deny(clippy::mem_forget)] lint prevents accidental bypass of drop-based zeroization project-wide.
• cargo deny rules reject any storage dependency that transitively stores key material without implementing ZeroizeOnDrop.
• Key material types MUST NOT derive Debug or Display to prevent accidental formatting in logs, panics, or debugger output.
• Core dump configuration must exclude memory-locked pages from capture.
• Heap profiling tools are prohibited in production environments containing active key material. Enforcement: release builds strip profiling symbols (strip --strip-debug). A seccomp profile denies ptrace syscalls, and the PR_SET_DUMPABLE = 0 setting from the startup pre-flight prevents attach. Enforcement: (1) release builds strip profiling symbols (strip --strip-debug); (2) a seccomp profile denies ptrace syscalls (PR_SET_NO_NEW_PRIVS = 1 + SECCOMP_MODE_FILTER); (3) AppArmor deny rules block /proc/<pid>/mem access.

10. Security Invariants

Any code change violating the following is rejected at review:

1. No plaintext on disk: Every byte is encrypted with AES-256-GCM before the write call returns.
2. No DEK in plaintext outside mlock’d RAM: The DEK lives only in KMS-wrapped form on disk, and inside mlock’d Zeroizing buffers in memory.
3. Strict mTLS: Auto-join is permanently disabled. SVID or SAN URI patterns are mandatory.
4. No stale reads for sensitive data: Tier 2 and Tier 3 data must exclusively utilize the ReadIndex protocol.
5. No unauthenticated operations: GCM tag mismatch is fatal for the affected key; 3 distinct key failures within 60 seconds in the same Fjall partition trigger read-only quarantine (in-flight Raft proposals are drained, not interrupted). Quarantine state is durable: it is committed via a Raft proposal to _meta:quarantine:<partition>:<node_id> (partition first, so an operator’s ClearQuarantine can prefix-scan and remove every reporting node’s entry for a partition in one pass) so it persists across node restarts and is visible to all cluster members. GCM failures reflect node-local storage corruption, not a cluster-wide data problem, so blocking is node-scoped: only the reporting node re-enters quarantine (and refuses local reads) for that partition on restart; other nodes persist the record for audit visibility only and continue serving reads. Recovery requires the operator to run keystone-manage storage clear-quarantine on the affected node, which requires storage-operator RBAC authorization, is committed via Raft (so the flag is cleared cluster-wide), and is audit-logged with caller identity and timestamp. A single GCM tag failure emits a WARN log and increments a metric; two failures within 60 seconds emit an ERROR log and trigger an alert. Per-source-IP and per-identity rate limits apply (see §1): RotateDek limited to 2/hour, ClearQuarantine to 10/hour. Emergency rotation (RotateDekRequest{emergency: true}) requires dual-control approval from a second storage-operator via ConfirmRotateDek within 5 minutes.
6. No environment-variable KEK in production: The --dev-mode flag and KEYSTONE_ALLOW_ENV_KEK=1 are explicitly required to start with an environment-provided KEK.
7. NodeId collision detection: A node that detects a (node_id, rpc_addr) collision with an existing cluster member at startup must emit a fatal log entry and refuse to start. The leader additionally enforces this check when processing add_learner requests.
8. DEK bootstrap ordering: DEK generation MUST target an already-mlock’d allocation; it MUST NOT be generated into an unlocked buffer and subsequently copied.
9. Per-record write rate guard: When the per-record version counter exceeds a configurable threshold (default 2^30 updates per DEK epoch), the node emits a CRITICAL alert, blocks further updates to that key, and requires operator intervention. This prevents adversarial write flooding from exhausting the u32 nonce space within a single DEK epoch.
10. Nonce source exclusivity: Nonce sources for all AES-256-GCM contexts are defined in §2.2. Random nonces are prohibited. Any new encrypted context must define a deterministic, collision-resistant nonce strategy reviewed by the security team.
11. Deployment validation: A pre-flight CI/CD check or Kubernetes admission webhook must reject any production service definition containing --dev-mode or KEYSTONE_ALLOW_ENV_KEK. This prevents silent flag smuggling via systemd, container images, or manifests.
12. Startup pre-flight: At process startup (before KEK/DEK are loaded), the node must verify RLIMIT_CORE == 0 and PR_SET_DUMPABLE == 0. If either check fails, the node must refuse to start in production mode.
13. Non-extractable KEK key material (§2.5): PKCS#11 KEK key objects MUST be created with CKA_EXTRACTABLE = false / CKA_SENSITIVE = true; TPM KEK key objects MUST be created with fixedTPM | fixedParent and no duplication attribute. A KekProvider implementation that can export raw key bytes from the token/TPM is non-compliant regardless of how it is otherwise used.
14. No PKCS#11/TPM credential via environment variable: The PKCS#11 PIN and TPM auth value are supplied only via pkcs11_pin_file / tpm_auth_file (a file path in config). Neither may be supplied via an environment variable or inline config value — that channel is reserved exclusively for the dev-mode KEYSTONE_DEV_KEK path (invariant 6).
15. Authenticate-before-decrypt for Encrypt-then-MAC contexts: Any KekProvider that does not use an AEAD primitive natively (the TPM provider, §2.5.2) MUST verify the MAC over the full ciphertext before performing any decryption operation, and MUST use a constant-time comparison. Decrypting unauthenticated ciphertext, even transiently, is prohibited.

17. SecurityContext Design and Security Principles

Date: 2025-11-15 Updated: 2026-07-03

Status

Accepted; all production APIs documented with parameter descriptions, return values, and validation architecture. Clippy clean.

Validation Architecture

All auth types use a single validate() method returning AuthenticationError for both structural checks (field presence, length) and business rules (user enabled, domain enabled, trust chain). The validator crate is not used in auth.rs — all validation is manual, ensuring every check produces a dedicated, typed error (UserDisabled(id), DomainDisabled(id), AuthzPrincipalMismatch, etc.) rather than an opaque ValidationErrors bag.

Validation flow:

1. SecurityContext::validate() validates the principal (identity, domain, user data), then checks authentication-context-specific constraints (Trust/AppCred user_id match).
2. PrincipalInfo::validate() checks domain_id length, then delegates to IdentityInfo::validate().
3. IdentityInfo::validate() dispatches:
   • UserIdentityInfo::validate() — checks user_id length, user presence/match, user enabled, domain enabled.
   • PrincipalIdentityInfo::validate() — checks id and issuer non-empty; IdentityInfo then checks domain enabled.
4. ScopeInfo::validate() — checks domain/project/trust project enabled status.
5. SecurityContext::fully_resolved() — calls validate() + checks that scoped authorization carries non-empty roles.

The validator crate remains in use by other core-types modules (identity, assignment, token), but is not used by auth.rs.

Context

Keystone endpoints require a verified, untamperable security context to make authorization decisions. Previous Python implementations passed mutable context object that flattened the authentication and authorization information into a set of optional fields through the request lifecycle, creating a class of vulnerabilities where downstream code does not have unambiguous information.

The Rust Keystone must enforce:

• A security context cannot be used for policy enforcement before it is fully resolved
• Unscoped authentication is valid and must not be mistaken for unresolved context
• Authorization flows from authenticated parent tokens are propagated into the security context
• Test paths cannot bypass validation gates in production

Key types involved:

• SecurityContext (core-types) — holds principal, authentication context, authentication methods, authorization, audit IDs, token, token restriction. All fields are pub(crate) with explicit getter/setter accessors to prevent external mutation.
• ValidatedSecurityContext (core) — wrapper that gates the raw context behind a validation barrier. Internal field is private; only Deref is implemented (no DerefMut), so the wrapped context is externally immutable.
• AuthenticationResult — produced by a single auth method, may carry authorization from parent token
• AuthzInfo — scope + roles extracted at authentication time. scope is pub, roles is pub(crate) with setters (set_roles, roles, try_set_roles)
• ScopeInfo — enum capturing the authorization scope: Domain, Project, System, TrustProject, Unscoped. The TrustProject variant boxes its payload (TrustProjectInfo) to avoid inflating the enum size for the smaller variants (Domain, System, Unscoped).
• Credentials (policy) — subset of context projected for OPA policy evaluation. Uses read-only getters: principal(), authorization(), effective_roles()
• SecurityContextTestingBuilder (core-types, #[cfg]-guarded) — builder for constructing SecurityContext in test fixtures. Replaces positional for_testing() with named setters
• IntoAuthContext (core) — conversion trait that forces the caller to provide context information when converting a provider error into AuthenticationError::Provider

Decision

Two-Phase Validation: Construction Then Validation

SecurityContext is the raw, possibly incomplete structure. ValidatedSecurityContext wraps it and represents the validated, resolvable security context. The two-phase design ensures that no endpoint handler can observe a partially-authenticated context:

1. SecurityContext::try_from(AuthenticationResult) constructs the raw context from authentication results
2. ValidatedSecurityContext::new_for_scope(ctx, scope, state) is the only production path to obtain a validated context. The scope is passed as an explicit parameter (not derived from the context) so callers have unambiguous control over the target scope. It:
   • If ctx.authorization() already set and differs from scope, calls ctx.validate_scope_boundaries(&scope) to guard scope override
   • If ctx.authorization() is None, calls ctx.set_authorization_scope(scope)
   • Populates user_domain for IdentityInfo::User by querying the resource provider — required before xvalidate()
   • Calls ctx.validate() to check principal integrity (user enabled, appcred/ trust user_id match)
   • Checks token expiration: if ctx.expires_at() < Utc::now(), returns AuthTokenExpired
   • Runs auth-context-specific validation:
     • ApplicationCredential: verifies user_id match and AC expiration
     • Trust: validates trust delegation chain, trustor enabled, trustor domain enabled, trustor domain compatibility
   • Calls calculate_effective_roles(state, ctx, scope) (private, read-only) to resolve role assignments from the database
   • Calls ctx.set_effective_roles(roles) via setter
   • Returns ValidatedSecurityContext(ctx) on success

Production code can only obtain ValidatedSecurityContext through new_for_scope(). The ValidatedSecurityContext struct wraps the context in a private inner field. The Deref implementation provides read-only access (&SecurityContext); there is no DerefMut, into_inner(), or inner_mut(). All 8 fields on SecurityContext are pub(crate), so external crates cannot obtain &mut access to any field. Setter methods (set_token, set_authorization, set_effective_roles, set_token_restriction, expires_at_mut) are pub but require &mut self, which is unreachable once the context is wrapped in ValidatedSecurityContext.

#[cfg]-Guarded Test Constructors and Builder

During testing, two mechanisms are available under #[cfg(any(test, feature = "mock"))]:

1. ValidatedSecurityContext::test_new(ctx) — constructs a validated context without going through the validation pipeline. This allows unit tests and integration test mocks to inject a pre-built context.

2. SecurityContextTestingBuilder (accessed via SecurityContext::test_build()) — a named-setter builder that replaces the positional for_testing() approach. Required fields are authentication_context and principal; optional fields are token, authorization, expires_at, and token_restriction. The builder derives auth_methods from authentication_context.methods(). Both compile-time tests (#[cfg(test)]) and the optional mock feature gate these constructors, so production builds can never call them.

API Extractor Enforcement

The Auth extractor (core/src/api/auth.rs) is the Axum extractor that validates and resolves the context for every authenticated request. Two paths exist:

1. Extension injection (tests only): When ValidatedSecurityContext is present in request extensions, the extractor calls vsc.fully_resolved()? to verify the context is complete, then returns Auth(vsc). This path is #[cfg(any(test, feature = "mock"))]-guarded.

2. Token header flow (production): The extractor reads X-Auth-Token, calls state.provider.get_token_provider().authorize_by_token(), which builds the context, resolves roles via ValidatedSecurityContext::new_for_scope(), and returns the validated context. The extractor then calls vsc.fully_resolved()?, and returns Auth(vsc).

Both paths call fully_resolved() before returning, ensuring the validation gate cannot be bypassed.

fully_resolved() Semantics

The SecurityContext::fully_resolved() gate at core-types/src/auth.rs:581 enforces:

• authorization is None — returns Err(SecurityContextNotResolved): authorization has not been bound from the parent token or request scope
• authorization is Some(AuthzInfo { scope: Unscoped, roles: None }) — passes: unscoped is valid with no roles
• authorization is Some(AuthzInfo { scope: Scoped, roles: None }) — returns Err(SecurityContextNotResolved): scoped authorization must have roles
• authorization is Some(AuthzInfo { scope: _, roles: Some(_) }) — passes: roles are populated

The critical distinction is that unscoped authorization with roles: None is valid, while scoped authorization with roles: None indicates an incomplete resolution.

Authorization Propagation from Authentication

Authorization lives at the AuthenticationResult level, not nested inside TokenContext. This design allows any authentication method (token, SPIFFE, K8s, OIDC, etc.) to produce authorization context at authentication time rather than deferring all role computation to a later phase.

For token authentication, FernetToken::from_security_context(ctx, expires_at) constructs the appropriate token variant from the validated context, using the scope and role data from ctx.authorization(). The token provider’s build_authz_info_from_fernet_token() method maps each FernetToken variant to AuthzInfo { scope, roles } by fetching scope objects (project, domain, project_domain) from the database.

The TryFrom<AuthenticationResult> and TryFrom<Vec<AuthenticationResult>> both propagate the authorization into the resulting context:

• Single authentication result: the result’s authorization is set on the context
• MFA: the first result’s authorization is preferred; subsequent results fill in if the first is missing
• All MFA results must share the same principal or AuthnPrincipalMismatch is returned

Effective Role Calculation

calculate_effective_roles() in core/src/auth.rs is a private, read-only function that queries the assignment provider for effective role assignments based on the scope type. It takes &SecurityContext (immutable reference) and returns Vec<RoleRef>. The caller then sets roles via SecurityContext::set_effective_roles(), which uses the setter path through AuthzInfo::set_roles() (no &mut borrows escape the constructor).

Based on the scope type:

• Project scope: queries effective user+group role assignments on the project; for application credentials, takes the intersection of frozen AC roles with currently assigned user roles
• Domain scope: queries effective user+group role assignments on the domain
• System scope: queries effective role assignments on the system
• Trust scope: resolves trustor roles on the trust’s project scope; if the trust declares explicit roles, verifies the trustor still has those roles and applies implied role expansion
• Unscoped: no role query is performed; roles remain None

A trust (or restricted application credential) can also arrive on a plain Project scope rather than its native TrustProject/app-cred scope — most notably an EC2 credential minted under a trust and redeemed at /v3/ec2tokens, which reconstructs the delegated AuthenticationContext but presents a bare project scope. In that case role resolution must still be bounded by the delegation, not by the trustee’s own assignments on the project: calculate_effective_roles() detects AuthenticationContext::Trust under a ScopeInfo::Project and routes it through resolve_trust_roles() (the same path as TrustProject), and the application-credential path takes the intersection with the AC’s frozen roles. Without this bound a restricted delegation redeemed via EC2 could escape its role restriction (OSSA-2026-005 / CVE-2026-33551).

After assignment queries, if roles is empty and the scope is not unscoped, ActorHasNoRolesOnTarget is returned. Token restrictions are applied last to potentially narrow the role set.

Scope Boundary Validation

SecurityContext::validate_scope_boundaries(scope) at core-types/src/auth.rs validates whether the authentication context permits a requested scope type. It does NOT verify role ownership — only whether the auth method and any token restrictions allow the target scope. Returns ScopeNotAllowed on violation.

Key constraints:

• Application credentials cannot be scoped beyond their bound project
• Token restrictions block domain, system, trust, and unscoped scopes; project scope must match the restriction’s project ID
• Trust authentication cannot be re-scoped to a different trust
• K8s authentication is limited by its token restriction

This method is the mechanism that keeps a delegated token’s scope pinned to its delegation project, and it is what the policy-layer scope-drift tripwire (see Delegation Facts Sourced from the Authentication Chain) relies on. Note the pinning is enforced on scope change; the initial scope set at issuance is not re-checked here, so the tripwire in the policy layer remains the backstop.

Policy Credentials Projection

Credentials is the subset of ValidatedSecurityContext projected for OPA policy evaluation. The TryFrom<&ValidatedSecurityContext> implementation at core/src/policy.rs extracts user_id via sc.principal().get_user_id(), role_ids via sc.authorization().effective_roles(), and scope-specific identifiers (project_id, domain_id, system). The implementation uses only read-only getters — no &mut borrows are involved.

Unscoped tokens produce role_ids: [] with no scope ID set. The OPA policy must handle this unscoped case correctly. For scoped contexts, the implementation returns SecurityContextNotResolved if authorization.roles is unexpectedly None — this is a defense-in-depth check that catches the case where fully_resolved() was not properly gated.

Delegation Facts Sourced from the Authentication Chain

Delegation-related facts on Credentials — auth_type, is_delegated, unrestricted, trust, and delegated_project_id — are read from sc.authentication_context() (the immutable authentication chain), never inferred from the authorization scope. The token scope (project_id) is attacker-influenceable across a rescope, while the delegation object’s own project binding is fixed at delegation-creation time. Sourcing these facts from the chain is what lets a policy pin a delegated caller to the delegation’s own project rather than to whatever project the current token happens to be scoped to (OSSA-2026-015).

delegated_project_id: Option<String> carries this immutable binding:

• AuthenticationContext::ApplicationCredential → the application credential’s project_id
• AuthenticationContext::Trust → the trust’s project_id
• all non-delegated contexts → None

Policies anchor the delegation boundary on delegated_project_id and additionally assert credentials.project_id == credentials.delegated_project_id as a scope-drift tripwire: it fails closed if a delegated token’s scope is ever observed diverging from its delegation project. validate_scope_boundaries() keeps the two equal today, so the tripwire is defense-in-depth against a future regression in scope pinning, not the primary control.

Consequences

Security Improvements

• No mutable context exposure: ValidatedSecurityContext::inner() returns &SecurityContext. Only Deref is implemented (no DerefMut), so there is no path to obtain &mut SecurityContext from ValidatedSecurityContext. Clone produces an independent copy that does not share state.
• Private fields on SecurityContext: All 8 fields are pub(crate) in core-types. The ValidatedSecurityContext type in core is a different crate, so it also cannot mutate the fields directly — only through the getter/setter API. After wrapping, no &mut reference is reachable.
• Explicit getter/setter API: All field access goes through getters (returning &T or Option<&T>) and setters (taking &mut self). No interior mutability (RefCell, Cell, UnsafeCell, atomics) is used anywhere in the auth context types.
• Validation is mandatory: fully_resolved() is called both in the production Auth extractor path and the test extension-injection path, ensuring no endpoint receives an unresolved context.
• Scope boundary enforcement: validate_scope_boundaries() prevents auth methods with narrower scope permissions from being broadened by request-scoped scope specifications. The scope override guard in new_for_scope calls validate_scope_boundaries() when the requested scope differs from the context’s existing scope.
• Test/production separation: test_new() and SecurityContextTestingBuilder are compile-time excluded from production builds, preventing accidental bypass.

Performance Considerations

• Role computation at validation time: The new_for_scope() path performs 1-8 database queries depending on scope type and number of effective assignments. This cost is paid once per API request.
• Authorization propagation from token auth: The token provider builds AuthzInfo from FernetToken by fetching scope objects from the database (project, domain, project_domain). The role set is then re-queried by new_for_scope() for effective assignments (which may differ from token-frozen roles due to interim role removal).
• Revoked token expansion: The authorize_by_token path expands token role data from database before checking revocation. This is necessary since the token may be considered expired by project_id, role_id, user_id, or any combination of those. It is therefore necessary to have a fully expanded token before checking for the revocation.
• Trust validation overhead: The new_for_scope() path for trust contexts performs additional queries: trust delegation chain validation, trustor user lookup, and trustor domain enabled check. These are necessary for security but add 2-3 queries per trust-scoped authentication.

Maintenance Surface

• ValidatedSecurityContext adds one indirection layer to all handlers using Auth. The Deref implementation minimizes friction, but direct field access is not possible. Instead, use the getter API: ctx.principal(), ctx.authorization(), ctx.token(), ctx.token_restriction(), etc.
• Adding a new authentication method requires:
  1. Implementing the provider’s authentication logic
  2. Producing an AuthenticationResult (with authorization if applicable)
  3. Ensuring AuthenticationContext::methods() returns the correct method names
  4. Verifying validate_scope_boundaries() handles the new context variant
  5. Adding a match arm in new_for_scope() for auth-context-specific validation (even if empty, to trigger a compile error for future context additions)
• Adding a new ScopeInfo variant requires updating validate_scope_boundaries(), fully_resolved(), calculate_effective_roles(), ScopeInfo::validate(), FernetToken::from_security_context(), build_authz_info_from_fernet_token(), Credentials::try_from, and all downstream match arms that consume scope information.
• The ScopeInfo::TrustProject variant boxes its payload as Box<TrustProjectInfo> to keep the enum size reasonable. Adding fields to TrustProjectInfo only affects the trust variant, not the smaller variants (Domain, System, Unscoped).

18. Plugin Linking — Anchor Convention and build.rs

Date: 2026-05-28

Status

Accepted

Context

Keystone uses inventory for plugin registration. Driver crates (SQL, Raft, etc.) call inventory::submit! to register their SqlDriverRegistration items into a global table discovered at runtime by inventory::iter.

The problem is that the Rust compiler and linker treat inventory::submit! items as dead code when no strong symbol from the contributing crate is referenced by the binary. The linker skips the .rlib archive members entirely, causing inventory::iter to silently yield no results — drivers are present in the dependency tree but invisible at runtime.

Decision

Crate Naming Convention

All driver crates follow the naming scheme:

openstack-keystone-<PROVIDER>-driver-<DRIVER_TYPE>

Where <PROVIDER> identifies the subsystem (e.g. identity, role, federation) and <DRIVER_TYPE> identifies the backend technology (e.g. sql, raft).

Examples:

Infrastructure crates (api-types, config, core, core-types, distributed-storage, token-fernet) do not use this naming convention — they are not driver crates.

pub fn anchor() Convention

Every openstack-keystone-* crate exports a no-op function:

#![allow(unused)]
fn main() {
/// Linkage anchor — see ADR-0018. Referenced by `keystone` build.rs so the
/// linker extracts this crate's `.rlib` members, keeping `inventory::submit!`
/// sections visible at runtime.
#[allow(dead_code)]
pub fn anchor() {}
}

This is a compile-time convention enforced by the build script. Any crate in the dependency tree that does not provide pub fn anchor() will cause a compile error in the binary crate.

build.rs Auto-Discovery

crates/keystone/build.rs parses Cargo.toml at build time, discovers all dependencies whose name starts with openstack-keystone-, and generates a sidecar file (OUT_DIR/inventory_anchors.rs) containing a #[used] static that references each crate’s anchor() function:

#![allow(unused)]
fn main() {
#[used]
pub static _ANCHORS: &[fn()] = &[
    openstack_keystone_appcred_driver_sql::anchor,
    openstack_keystone_assignment_driver_sql::anchor,
    // ... (auto-discovered, sorted)
];
}

Only deps with driver in the name are collected (the build.rs filter), ensuring only actual driver crates are anchored.

The generated file is included into lib.rs via include!(), creating a strong reference chain:

keystone-manage (binary crate)
  └─ openstack_keystone (keystone crate)
       ├─ #[used] static _ANCHORS (generated by build.rs)
       │    ├─ appcred_driver_sql::anchor  ── forces .rlib extraction
       │    ├─ identity_driver_sql::anchor  ── forces .rlib extraction
       │    └─ ... (all *_driver_* crates)
       └─ (transitive deps carry inventory::submit! items)

Binary Crate Linkage

Binary crates (e.g. cli-manage) reference the generated static from the keystone crate to create the linkage chain:

#![allow(unused)]
fn main() {
#[used]
static _INVENTORY_LINK: &[fn()] = openstack_keystone::_ANCHORS;
}

This prevents the linker from stripping the keystone crate members, which in turn prevents stripping of all driver crate members.

Consequences

Positive

• Zero manual maintenance: Adding a new driver crate only requires adding it as a dependency of crates/keystone. build.rs discovers it automatically.
• Compile-time enforcement: The build fails if any openstack-keystone-* dependency is missing pub fn anchor(), preventing accidental breakage.
• Naming conveys intent: The openstack-keystone-<PROVIDER>-driver-<DRIVER_TYPE> suffix makes it immediately clear what crate provides.
• No special tooling: The solution uses standard Rust features (#[used] static, build.rs code generation, include!()).

Negative

• Every openstack-keystone-* crate has a one-line no-op function. This is negligible overhead.
• build.rs adds a trivial compilation step (~0.01s in practice, parses a single small text file).

Migration Notes

Existing driver crates were renamed in a single commit. All references (Cargo.toml workspace members, Cargo.toml dependencies, plugin_manager.rs, test code, Cargo.toml of test crates) were updated via the subagent.

See Also

• crates/keystone/build.rs — auto-discovery and anchor generation
• crates/keystone/src/lib.rs — include!() of generated anchors
• crates/cli-manage/src/db.rs — binary-side _INVENTORY_LINK reference

19. Credentials Provider Implementation

Date: 2026-06-09

Status

Proposed

Context

Keystone requires a secure mechanism to store and manage sensitive credentials (e.g., EC2 access keys, TOTP secrets) for users. These credentials must be encrypted at rest, support high-availability key distribution across clusters, and provide a safe path for key rotation without risking data loss.

Keystone-NG is deployed in parallel with the Python Keystone service and shares the same live database. This imposes hard constraints on this implementation:

• Keystone-NG never runs DDL against tables owned by the Python Keystone service (schema evolution remains exclusively under Python Keystone’s alembic control).
• All encryption and hashing behaviour must be byte-for-byte compatible with Python Keystone so that blobs written by either service can be decrypted by the other. Fernet compatibility is verified by cross-service tests.

Decision

1. Architecture & Data Model

The Credentials Provider serves as a secure, encrypted vault for storing sensitive authentication secrets used by various Keystone identity mechanisms. It implements a “blind storage” pattern where the core API manages the metadata and encryption, while the specific meaning of the secret is defined by the type field.

Supported Credential Types

The provider treats the type field as an open string to allow extensibility. Common types include:

• ec2: Stores AWS-compatible access and secret keys. Triggers deterministic ID generation via SHA-256 of the access key.
• totp: Stores Base32 encoded TOTP seeds for Multi-Factor Authentication (MFA).
• Custom Types: Any arbitrary string can be used to store secret blobs for third-party integrations.

Blob JSON Schemas Per Type

Each credential type carries a typed JSON blob. The following schemas define the expected structure:

EC2 Blob:

{
  "access": "AKIAIOSFODNN7EXAMPLE",
  "secret": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
  "trust_id": "optional, present if created via a trust-scoped token",
  "app_cred_id": "optional, present if created via an application credential",
  "access_token_id": "optional, present if created via an OAuth1 access token"
}

trust_id, app_cred_id, and access_token_id are mutually exclusive, optional delegation-context fields. They are populated from the scope of the token used to create the credential and must be passed through to the token provider on POST /v3/ec2tokens (see §3, “Credential metadata in the token”). They are part of the same encrypted JSON blob as access/secret — the field names above (not access_id) are the cross-service contract; a Rust and a Python node must serialize identical keys or the two services will silently fail to exchange delegation metadata.

Server-managed, never client-settable (OSSA-2026-005 / CVE-2026-33551): on create, the server discards any trust_id/app_cred_id/access_token_id supplied in the request’s blob and re-derives them from the actual authentication context of the creating request (trust or application credential; absent for direct authentication). Without this, an EC2 credential created while authenticated via a delegation would be indistinguishable from a directly-authenticated one at /v3/ec2tokens validation time, silently regaining the parent user’s full, unrestricted project role set on every subsequent use. On update, these fields are immutable and carried forward from the stored blob when the caller’s patch omits them (as any normal client would, since the fields are never meant to be client-supplied); a patch that explicitly supplies a different value, or supplies one where none was stored, is rejected.

TOTP Blob:

{
  "seed": "JBSWY3DPEHPK3PXP",
  "digits": 6,
  "period": 30
}

Custom types may use arbitrary JSON structures; the provider does not validate the blob contents beyond JSON-parseability.

Security Model

Because this provider stores critical secrets (including MFA seeds), it employs:

• Encryption-at-Rest: All secrets are stored as encrypted_blob using the Fernet (AES-128-CBC + HMAC-SHA256) scheme.
• Key Isolation: The encryption keys are stored in a separate filesystem-based repository, not in the database.
• Integrity Verification: The key_hash is stored alongside the blob to ensure the correct decryption key is used during rotation.

Persistence (Database Schema)

Credentials are persisted in the credential table, which is owned and schema-managed exclusively by the Python Keystone service via alembic. Keystone-NG treats this table as read/write but never issues DDL against it.

Schema Definition:

2. API Interface & Lifecycle

Create (POST /v3/credentials)

• Mandatory Fields:
  • type: The category of the credential.
  • blob: A JSON string representing the secret. For EC2, must contain an access key.
• Optional Fields:
  • project_id: Required if type is 'ec2'.
  • user_id: Defaults to the authenticated user. This default applies only when the request is user-scoped; it must not be applied when the caller holds a system-scoped token (to match Python Keystone behaviour). Under system scope there is no implicit “acting user” to fall back to, so if user_id is also omitted from the request body the server must reject the request with 400 Bad Request rather than defaulting it to anything (e.g. the system-scoped caller’s own user, which would silently create a credential owned by an operator account).
• ID Generation:
  • EC2: SHA-256(blob['access']) hex-encoded.
  • Others: Random UUID.

Read (GET /v3/credentials & GET /v3/credentials/{id})

• Returns the credential reference.
• Security: The encrypted_blob and key_hash are stripped from the response; the blob is decrypted to plaintext before serialisation.
• Wire format of blob: Python Keystone returns blob as a JSON-encoded string (the same string form that was originally submitted on create), not as a nested JSON object — clients are expected to json.loads() it themselves. Keystone-NG must serialise the blob field in its API response the same way (a string value), not as a parsed/nested object, or existing SDKs and clients that call json.loads(cred["blob"]) will break against Keystone-NG.
• List filtering — two-phase policy check (required to address CVE-2019-19687): The GET /v3/credentials endpoint first enforces the identity:list_credentials policy and applies driver-level hints (e.g. user_id, type query parameters). It then iterates the returned set and re-enforces identity:get_credential on each individual credential, dropping any record the caller is not permitted to read. This ensures that users with a project role cannot view credentials belonging to other users when enforce_scope is false. The performance implication is accepted and matches Python Keystone behaviour.
  • Policy target correctness: The per-item re-enforcement must build its policy target from that record’s own user_id/project_id, not from the requester’s identity or scope. Evaluating identity:get_credential against the wrong target (e.g. the caller’s own attributes, or a cached target from the first item) would make the re-check a no-op and reintroduce a variant of CVE-2019-19687 rather than closing it.

Delegation Project Boundary (OSSA-2026-015)

All CRUD operations on /v3/credentials must bind delegated authentication (trust-scoped tokens, application credentials) to the delegation’s own project_id, not just the credential’s user_id. Checking ownership via user_id alone is insufficient: a stolen or reused trust/application-credential token scoped to project A must not be able to read, modify, or delete a credential belonging to the same user but bound to project B, nor reach credentials with no project binding at all (e.g. TOTP/MFA seeds).

• The policy engine receives input.credentials.is_delegated (derived from [AuthenticationContext::is_delegated], true for trust/application-credential auth, including when carried forward through a re-scoped token) on every request.
• The boundary is anchored on input.credentials.delegated_project_id — the delegation’s own immutable project taken directly from the authentication chain held in [ValidatedSecurityContext] (trust.project_id / application_credential.project_id), not on input.credentials.project_id (the request’s token scope). Sourcing the boundary from the chain rather than the scope means a scope rebind can never move a delegated caller’s boundary. The two are pinned equal at token-issuance time ([SecurityContext::validate_scope_boundaries]), so policies additionally assert project_id == delegated_project_id for delegated callers as a scope-drift tripwire that fails closed.
• Show/Delete/Update: a delegated caller may only act on a credential whose project_id equals delegated_project_id; unscoped credentials (project_id == null) are unreachable via any delegated caller.
• Update: additionally, a delegated caller’s patch must not move the credential’s project_id outside the delegation’s own (delegated_project_id) project.
• Create: a delegated caller’s new credential must set project_id equal to delegated_project_id; delegated callers cannot create unscoped credentials.
• List: unaffected directly — the delegation boundary is enforced entirely by the per-item identity/credential/show re-check described above.
• Non-delegated authentication (password, token, TOTP, …) is unaffected; user_id-only ownership remains sufficient.
• Effective-role bounding on redemption: a trust presented on a plain project scope (an EC2 credential created under a trust, redeemed at POST /v3/ec2tokens, where the scope is rebuilt from the credential’s project rather than the trust’s own TrustProject scope) has its effective roles bounded by the trust’s delegated role set, never the trustee’s own project assignments — mirroring the application-credential role intersection so a delegated EC2 credential can never widen its role set beyond the delegation.

Restricted Application Credentials and EC2 (OSSA-2026-005)

A restricted application credential (unrestricted == false) must not be usable to create an ec2-type credential at all, via either POST /v3/credentials or POST /v3/users/{user_id}/credentials/OS-EC2. This is independent of the project-boundary check above and of the delegation role set: an EC2 credential, once created, authenticates via POST /v3/ec2tokens on its own terms (see §1, “Server-managed, never client-settable”) — restricting who may create one is the only point at which a restricted application credential’s intentionally narrow capability set can be enforced against this particular escape hatch.

• The policy engine receives input.credentials.auth_type (e.g. "application_credential") and, for application-credential authentication only, input.credentials.unrestricted (Some(bool); absent/null for every other auth method).
• identity/credential/create denies when auth_type == "application_credential", unrestricted is falsy, and the target credential’s type == "ec2"; every other credential type is unaffected.
• identity/os_ec2/create_credential applies the same restricted-app-cred denial unconditionally, since every credential created through that endpoint is ec2-typed.

Update (PATCH /v3/credentials/{id})

• Updatable: type, blob, project_id.
• Immutable: user_id and project_id may not be changed to point at a user or project the acting user has no access to (CVE-2020-12691). Within the blob, the following fields are additionally immutable: access (the EC2 access key — changing it would desynchronize the record from its SHA-256-derived id), trust_id, app_cred_id, and access_token_id.
• Process: Updating the blob triggers automatic re-encryption with the current Primary Key and updates key_hash.

Delete (DELETE /v3/credentials/{id})

• Supports deletion by ID, by User, or by Project.

Indirect User-Centric Endpoints (/v3/users/{user_id}/credentials/OS-EC2)

These endpoints provide legacy and user-scoped access to EC2 credentials:

• Listing: GET calls list_credentials_for_user filtered by type='ec2'. Results are flattened from the blob into explicit access and secret fields.
• Automatic Creation: POST can automatically generate access and secret keys via UUIDs if they are omitted from the request.
• Plaintext ID Lookup: For GET and DELETE operations, the credential_id provided in the URL is the plaintext access key. The server must hash this key (SHA-256) to locate the record in the database.

3. System Integration & Dependencies

The Credentials Provider is not only a standalone API but a critical component integrated into several core Keystone workflows.

Authentication Pipeline (TOTP/MFA)

The provider is a blocking dependency for the authentication flow when TOTP is enabled:

• Workflow: The TOTP auth plugin calls list_credentials_for_user(user_id, type='totp') during the token issuance process.
• Operation: The system decrypts all TOTP seeds for the user and generates current/previous window passcodes to verify the user’s input.
• Performance Requirement: Because this occurs during the login path, list_credentials_for_user must be highly performant to avoid increasing authentication latency. A single TOTP decryption (Fernet AES-128-CBC) is approximately 0.1ms. For a user with multiple TOTP credentials, the cost scales linearly. If latency becomes a bottleneck, consider caching decrypted TOTP seeds in memory with a short TTL (e.g., 60 seconds). The cache must store the plaintext seed (not the encrypted blob), and its TTL must be well within the key rotation window. In a multi-node deployment (Python nodes + Rust nodes sharing the same DB), each node maintains its own in-process cache; this is safe because TOTP seeds are not changed by key rotation — only the encrypted form changes — and the decrypted value remains stable across rotations.

Identity Lifecycle Management

The provider must support cascading deletions to prevent orphaned secrets:

• User Deletion: When a user is deleted, the system must call delete_credentials_for_user(user_id) to wipe all associated secrets.
• Project Deletion: When a project is deleted, the system must call delete_credentials_for_project(project_id) (primarily impacting EC2 credentials bound to projects).

API Transformation Layer

The provider supports the legacy /v3/users/{user_id}/credentials/OS-EC2 interface:

• Flattening: The provider’s blob output is flattened into explicit access and secret fields.
• Plaintext Mapping: The provider must support resolving credentials using the SHA-256 hash of a plaintext access key provided in the URL.

4. Encryption Architecture

The provider uses Fernet (symmetric encryption), which is built upon:

• AES-128 in CBC mode for encryption.
• HMAC-SHA256 for authentication.
• Base64url encoding for the final encrypted token.

Key Management (The Key Repository)

• Configuration:

  • [credential] provider: Defaults to fernet.
  • [credential] key_repository: Path to the keys directory (default: /etc/keystone/credential-keys/).
  • Important: This repository must be separate from the [fernet_tokens] repository.

• Storage: Keys are stored as individual files in a filesystem directory.

• Naming: Files use integer names. The highest number is the Primary Key.

• Cross-node synchronization (required precondition): Because the key repository is a local filesystem directory, not a table in the shared database, the whole byte-compatibility story in this section only holds if every Python node and every Rust node reads the same set of key files (e.g. a shared network filesystem, or a config-management job that distributes the directory to all nodes). credential_setup and credential_rotate are cluster-wide operations: a run is not complete until the resulting key files have been propagated to every node of both services. Keystone-NG must document this as a deployment requirement, not assume it happens implicitly from “sharing the same database”.

• Maximum active keys: The credential key repository is hard-capped at 3 active keys (MAX_ACTIVE_KEYS = 3). This matches the Python Keystone constant and is intentionally not configurable. Unlike Fernet token key rotation, credential key rotation is driven by key_hash tracking, not by a configurable window. The Rust implementation must enforce this same limit when loading the key repository.

• Rotation Logic (staged-key promotion, not primary renumbering):

  1. Setup (credential_setup) creates the first key as 0.tmp $\rightarrow$ 0 (staged; not yet used for encryption).
  2. On rotation (credential_rotate), the staged key 0 is renamed to (current_primary_index + 1) — e.g. if 1 is the current primary, 0 is renamed to 2. This renamed file is the new Primary. The old primary (1) is left in place, unchanged, and is still used for decryption.
  3. A fresh key is generated and written as the new staged 0.tmp $\rightarrow$ 0, ready for the next rotation cycle.
  4. If the number of key files now exceeds MAX_ACTIVE_KEYS (3), the oldest non-staged key file(s) are deleted to bring the count back down to 3.

  Note the staged key is never renumbered “in place” to become primary while simultaneously incrementing some other file — those are the same rename operation applied to the staged file, not the outgoing primary. An implementation that instead renames the outgoing primary upward while separately trying to promote 0 (as an earlier ambiguous phrasing of this section could be read) produces two files claiming to be primary and must be avoided.

• Security:

  • Directory must not be world-readable.
  • Files are created with umask 0o177 and a temporary-file-then-rename strategy to ensure atomicity.
  • A Null Key (base64.urlsafe_b64encode(b'\x00' * 32)) is provided as a fallback to facilitate upgrades. The Null Key must be removed immediately after initial setup. Any credential encrypted with the Null Key is effectively stored in plaintext with a well-known key. It exists solely as a transient migration aid and carries zero production tolerance.
  • Startup enforcement: Keystone-NG must check the key repository on startup. If any key file decodes to 32 null bytes (the Null Key), it must emit a hard warning log. Whether this is a hard-refuse-to-start condition must be controlled by an explicit, named configuration value (e.g. [credential] insecure_allow_null_key, defaulting to false) rather than an undefined “production mode” — refuse to start unless the operator has explicitly opted in. The Python service emits a warning on every encryption operation that uses the Null Key; Keystone-NG matches this behaviour and adds the startup gate.

key_hash Specification

This is a cross-service compatibility contract. Python Keystone and Keystone-NG share the same credential table; a key_hash written by one service must be interpretable by the other’s credential_migrate and credential_rotate commands.

The key_hash column is computed as follows (derived from keystone/credential/providers/fernet/core.py):

key_hash = SHA-1( keys[0] )   # hexdigest, lowercase

Where keys[0] is the raw bytes of the primary key file as read from disk — that is, the base64url-encoded key string, encoded as UTF-8, before base64url-decoding. This matches Python’s hashlib.sha1(keys[0]).hexdigest() where keys[0] is a bytes object obtained by reading the key file and stripping the trailing newline.

Important notes:

• The hash function is SHA-1, not SHA-256. Although SHA-1 is not recommended for security-sensitive uses, it is used here solely as a key-identifier (not for authentication), matching the Python implementation. Changing this to SHA-256 would silently break credential_migrate and credential_rotate against an existing database.
• The output is a lowercase hex string (40 characters), stored in the key_hash String(64) column.
• The input is the base64url-encoded key bytes as they appear in the file, not the raw 32-byte AES key they decode to.

Management Commands (keystone-manage / keystone-ng manage)

Administrative tasks are handled via three specific commands. These commands must not be run simultaneously from both the Python and Rust services against the same database. Because credential_rotate performs a safety check followed by a key promotion in two steps, concurrent execution from two nodes can race: one node’s migrate could change key_hash values between another node’s check and promote. Operational runbooks must treat these commands as mutually exclusive across services.

1. credential_setup: Populates the key_repository with initial keys. Must be run once during deployment.
2. credential_migrate: Identifies credentials encrypted with older keys (where key_hash $\neq$ SHA-1 hex of the current Primary Key) and re-encrypts them using the current Primary Key. Runs in batch chunks (default: 1000 credentials per transaction) with COMMIT between batches. Safe to run concurrently with active auth — reads are unaffected, writes are idempotent.
3. credential_rotate:
   • Safety Check: Verifies that all credentials are already encrypted with the current Primary Key (i.e. all key_hash values equal the SHA-1 hex of the current primary).
   • Action: Promotes a new key to Primary.
   • Failure: Aborts if any credential still uses an older key to prevent “over-rotation” (which would make those credentials indecipherable).

Encryption/Decryption Workflow

• Encryption: Use Primary Key $\rightarrow$ Encrypt $\rightarrow$ Store encrypted_blob and SHA-1(primary key file bytes as UTF-8) as key_hash.
• Decryption: Use MultiFernet (all active keys in repo, up to MAX_ACTIVE_KEYS = 3) to decrypt. The system attempts decryption with all available keys until one succeeds.

Re-encryption & Safety

To prevent data loss during rotation:

1. credential_migrate: Decrypts all credentials and re-encrypts them with the current Primary Key, updating key_hash to the SHA-1 of the new primary.
2. credential_rotate:
   • Check: Aborts if any credential’s key_hash $\neq$ SHA-1 of current Primary Key’s file bytes.
   • Action: Promotes a new primary key only after successful migration.

5. EC2 Credentials & Authentication

EC2 credentials enable AWS-style authentication, allowing clients to prove identity via request signing without transmitting the secret key.

Request Body Structure (POST /v3/ec2tokens)

The request body is a JSON object with a top-level "credentials" key:

{
  "credentials": {
    "access": "AKIAIOSFODNN7EXAMPLE",
    "signature": "<computed-signature>",
    "host": "identity.example.com:5000",
    "verb": "GET",
    "path": "/",
    "params": {
      "Action": "DescribeInstances",
      "SignatureVersion": "2",
      "SignatureMethod": "HmacSHA256",
      "Timestamp": "2026-06-11T12:00:00Z",
      "AWSAccessKeyId": "AKIAIOSFODNN7EXAMPLE"
    },
    "headers": {
      "Authorization": "AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20260611/RegionOne/ec2/aws4_request, SignedHeaders=host;x-amz-date, Signature=...",
      "X-Amz-Date": "20260611T120000Z"
    },
    "body_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
  }
}

The server extracts credentials["access"] to locate the credential record, and passes the full credentials dict to the signature verification logic. The body_hash is the SHA-256 hex digest of the request body (required for SigV4; use the empty-string hash for requests with no body).

Signature Version Detection and Dispatch

The server determines which signing algorithm to use via the following ordered decision procedure, sourced from Ec2Signer.generate() in keystoneclient.contrib.ec2.utils:

1. Read params["SignatureVersion"] from the credentials dict.

2. If SignatureVersion == "0" → use Version 0 algorithm.

3. If SignatureVersion == "1" → use Version 1 algorithm.

4. If SignatureVersion == "2" → use Version 2 algorithm.

5. If SignatureVersion is absent or does not match "0", "1", "2" → attempt SigV4 detection via _v4_creds():

   • Check credentials["headers"]["Authorization"] — if it starts with "AWS4-HMAC-SHA256", use Version 4.
   • Otherwise check credentials["params"]["X-Amz-Algorithm"] — if it equals "AWS4-HMAC-SHA256", use Version 4.
   • Important: AWS removed the SignatureVersion field from the SigV4 spec. SigV4 requests therefore never carry SignatureVersion in params; the _v4_creds detection path is not a fallback — it is the primary detection mechanism for SigV4.

6. If no version is identified → raise 400 Bad Request (unknown signature format).

This means the SignatureVersion query parameter is authoritative for v0/v1/v2, but absent for v4. The Rust implementation must replicate this exact precedence: do not assume SigV4 when SignatureVersion is simply missing — check the Authorization header or X-Amz-Algorithm param explicitly.

Per-Version Signature Algorithms

All versions use the decrypted secret from the credential blob as the key material.

Version 0 (Keystone-legacy, HMAC-SHA1)

Concatenate Action and Timestamp params, then HMAC-SHA1:

string_to_sign = params["Action"] + params["Timestamp"]   # UTF-8 bytes
signature      = Base64( HMAC-SHA1(secret_key, string_to_sign) )

Version 1 (Keystone-extended, HMAC-SHA1)

Iterate all params sorted case-insensitively by key, concatenate key+value, then HMAC-SHA1:

sorted_pairs   = sort params by key.lower()
string_to_sign = concat(key + value for key, value in sorted_pairs)  # UTF-8 bytes
signature      = Base64( HMAC-SHA1(secret_key, string_to_sign) )

Version 2 (AWS Query, HMAC-SHA256 preferred / HMAC-SHA1 fallback)

Build a canonical query string (keys and values percent-encoded with safe='-_~', sorted, joined with &), then HMAC:

canonical_qs   = "&".join( quote(k) + "=" + quote(v, safe='-_~')
                           for k, v in sorted(params.items()) )
string_to_sign = verb + "\n" + host + "\n" + path + "\n" + canonical_qs

Use HMAC-SHA256 if available (set params["SignatureMethod"] = "HmacSHA256"), otherwise fall back to HMAC-SHA1. The signature is Base64 of the HMAC digest.

Version 4 (SigV4, HMAC-SHA256 throughout)

SigV4 is a multi-stage process. All HMAC operations use SHA-256.

Step 1 — Canonical Request:

cr = "\n".join([
    verb.upper(),
    path,
    canonical_qs(verb, params),   # empty string for POST
    canonical_header_str(),        # lowercased key:stripped_value pairs + trailing \n
    auth_param("SignedHeaders"),   # from Authorization header or X-Amz-SignedHeaders param
    body_hash                      # SHA-256 hex of request body
])

canonical_header_str() iterates only the headers listed in SignedHeaders (from the Authorization header or the X-Amz-SignedHeaders query param). Header keys are lowercased and stripped; values are stripped. Each entry is key:value, lines joined by \n, with a trailing \n.

Boto compatibility quirk: Boto versions < 2.9.3 strip the port from the Host header when signing. The server detects this via the User-Agent header (regex Boto/2\.[0-9]\.[0-2]). When detected, the host entry in canonical_header_str uses only the hostname, dropping the port. This is separate from the port-stripping fallback in _check_signature and runs inside the canonical request construction itself.

Step 2 — String-to-Sign:

string_to_sign = "\n".join([
    "AWS4-HMAC-SHA256",
    param_date,                    # X-Amz-Date header (YYYYMMDDTHHMMSSZ), or X-Amz-Date param
    credential_scope,              # date/region/service/aws4_request from Credential field
    SHA256(cr.encode("utf-8")).hexdigest()
])

The date used in param_date must match the YYYYMMDD prefix in the Credential scope (credential_split[1]). If they do not match, the server raises an error immediately (no signature attempt).

Step 3 — Derived Signing Key:

k_date    = HMAC-SHA256( b"AWS4" + secret_key.encode(), date_str )
k_region  = HMAC-SHA256( k_date,    region )
k_service = HMAC-SHA256( k_region,  service )
k_signing = HMAC-SHA256( k_service, "aws4_request" )

region and service are extracted from credential_scope (positions 2 and 3 of the /-split). For Keystone EC2, service is typically ec2; for S3 tokens, service must be s3.

Step 4 — Final Signature:

signature = HMAC-SHA256( k_signing, string_to_sign.encode("utf-8") ).hexdigest()

Note: the final signature is a hex digest (lowercase), not Base64.

Signature Verification Flow

The full verification procedure in EC2TokensResource._check_signature():

1. Instantiate the signer with the decrypted secret key.
2. Call signer.generate(credentials) to produce the expected signature.
3. If credentials["signature"] is absent → raise 401 (“EC2 signature not supplied”).
4. Perform a constant-time string comparison between the client-supplied credentials["signature"] and the generated signature. Use hmac.compare_digest (or equivalent) to prevent timing attacks.
5. Port-stripping fallback: If comparison fails and credentials["host"] contains :, parse the host, strip the port (use hostname only), reinitialise the signer (a fresh HMAC instance is required to avoid state contamination), regenerate the signature, and repeat the constant-time comparison.
6. If either comparison succeeds → proceed to token issuance.
7. If both fail → raise 401 (“Invalid EC2 signature”).

The signer must be reinitialised between the original attempt and the port-stripping retry. The Python HMAC object is stateful (accumulated via update()); reusing it after a failed attempt produces incorrect results.

Timestamp Validation (Replay Attack Prevention)

Two timestamp locations depending on signature version (CVE-2020-12692 fix):

• v0 / v1 / v2: Timestamp is in credentials["params"]["Timestamp"]. Format: ISO 8601 (2026-06-11T12:00:00Z).
• v4: Timestamp is in credentials["headers"]["X-Amz-Date"] or credentials["params"]["X-Amz-Date"]. Format: YYYYMMDDTHHMMSSZ.

The server must check both locations and reject any request where the timestamp is outside the configured TTL window. The TTL is read from [ec2] auth_ttl in the shared keystone.conf, defaulting to 300 seconds (5 minutes) — this is the Python Keystone default and must not be confused with the 4-hour window that is only an AWS SigV2 recommendation, not what Keystone implements. Keystone-NG must read the [ec2] auth_ttl config value from keystone.conf and apply it identically.

Prior to the CVE-2020-12692 fix, SigV4 requests had no timestamp check because the timestamp appears in the Authorization header rather than a query parameter, and the original implementation only inspected query parameters. Keystone-NG must check both locations from the start.

The /v3/ec2tokens API Interface

Step-by-Step Authentication Algorithm:

1. Request Parsing: Parse the JSON body; extract the credentials object. Accept the body under either the top-level "credentials" key, or a legacy "ec2Credentials" key (Python Keystone accepts both for backwards compat).
2. Policy Enforcement: Enforce identity:ec2tokens_validate RBAC. Note: As of CVE-2025-65073, this endpoint now requires the caller to be authenticated (a user in the service group). Earlier versions treated /v3/ec2tokens as fully unauthenticated. Keystone-NG must enforce this policy and not mark the endpoint as @unenforced_api.
3. Credential Lookup: Query the credential table using SHA-256(access) as the record ID. Return 401 if not found. Type guard: reject (401) any record whose type != "ec2". The lookup keys on SHA-256(access) == id, an invariant only established for ec2-type credentials at creation (see §1, “Automatic Creation”); without this guard a credential mislabelled to a non-ec2 type — thereby dodging the ec2-only create-time guards (project binding, delegation stamping, the restricted-app-cred gate of OSSA-2026-005) — could still be redeemed here if its id ever collided with an access hash.
4. User/Project Validation: After locating the credential, verify:
   • The owning user is enabled (identity_api.assert_user_enabled).
   • The user’s domain is enabled.
   • The bound project is enabled (resource_api.assert_project_enabled). Return 401 for any disabled entity.
5. Secret Decryption: Decrypt encrypted_blob via the Fernet provider to recover the plaintext secret key.
6. Timestamp Validation: Extract the timestamp per the version-dependent rules above. Reject with 401 if outside the [ec2] auth_ttl window.
7. Signature Verification: Run the version-detection dispatch and _check_signature procedure described above.
8. Token Issuance: On success, issue a standard Keystone token scoped to the credential’s project_id and user_id. Return the token in the response body and in the X-Subject-Token header.
9. Failure: Return 401 Unauthorized for any verification failure.

Credential metadata in the token: If the EC2 credential was created via a trust (trust_id in the blob) or application credential (app_cred_id), this delegation metadata must be passed through to the token provider so it resolves the correct (bounded) role assignments — the trust/application-credential authentication context is rebuilt so the effective roles are bounded by the delegation’s role set, never the owner’s full project assignments. Omitting this was a historical bug fixed in Python Keystone. access_token_id (OAuth1) is rejected until OAuth1 delegation is implemented: redeeming such a credential would otherwise fall through to an unbounded EC2 authentication and silently drop the OAuth1 restriction.

Policy-input hygiene: the credential blob holds the decrypted secret (EC2 secret key, TOTP seed). No credential policy rule references it, so the API layer strips blob from every credential object before it is sent to the policy engine (identity/credential/{create,show,update} and the per-item show re-check on list). Shipping the plaintext secret to an external OPA would expose it to decision logging, turning the authorization channel into a secret exfiltration path.

Error Codes

6. Access Control & Permissions

RBAC Policies

• get_credential / list_credentials: ADMIN_OR_SYSTEM_READER_OR_CRED_OWNER.
• create / update / delete: ADMIN_OR_CRED_OWNER.

OS-EC2 Endpoint Policies

The legacy user-centric endpoints (/v3/users/{user_id}/credentials/OS-EC2) enforce the following access controls:

• GET (list): identity:os-ec2:read_credential. Requires the requester to be the credential owner or an administrator.
• POST (create): identity:os-ec2:create_credential. Requires owner or admin authorization. When called via application credential, the project ID must match the app credential’s bound project.
• GET/{credential_id} (read): identity:os-ec2:read_credential. The credential_id is resolved via SHA-256 hash of the plaintext access key.
• DELETE/{credential_id} (delete): identity:os-ec2:delete_credential. Same access rules as read.

Guardrails

• Application Credentials: Must be unrestricted to manage other credentials.
• EC2 Project Match: When creating EC2 credentials via an application credential, the project_id must match the application credential’s project.

20. Decoupled Multi-Tenant Identity Federation & Named ABAC Mapping Engine

Date: 2026-06-11

Status

Approved

Phase 5 Status: Mapping engine migration complete. Legacy federation mappings (/v4/federation/mappings) removed. allowed_redirect_uris migrated from mapping to IdentityProvider. All federation tests passing.

Context

High-performance, low-latency identity federation mapping for keystone-rs using a distributed Raft + FjallDB architecture.

1. Context & Motivation (Single vs. Dedicated Engine)

Identity federation platforms must map external cryptographic assertions into localized authorization contexts. Traditional identity systems treat authentication vectors as isolated, self-contained plugin mounts (e.g., separate auth/kubernetes, auth/oidc, and auth/cert backends).

In a high-throughput, multi-tenant distributed cloud operating system like keystone-rs built on a Raft + FjallDB consensus architecture, continuing down the path of dedicated plugin mapping engines creates severe architectural liabilities:

1. Massive Code Duplication: Re-implementing conditional expression evaluation (equals, any_of, regex) and macro-string parsing across multiple distinct protocol blocks widens the bug surface area.
2. Fragmented Security Boundaries: Ensuring strict multi-tenant isolation, data sanitization, and domain containment becomes exceptionally brittle when logic is spread across completely separate protocol codebases.
3. Raft Log Bloat & Invalidation Risks: Modifying multi-auth tenant parameters simultaneously requires executing separate, non-atomic API writes across distinct plugin endpoints, forcing independent entries through the Raft consensus log and risking partial, inconsistent authorization states.
4. Abuse of Token Restrictions: In legacy iterations, because external service accounts lacked a standard local user row, the system was forced to issue an unscoped token format and attach a heavy token_restriction payload to “clamp” the token into a project container. This abused a client-side narrowing tool as an administrative configuration table.

The Unified Engine Advantage

keystone-rs enforces a strict Split-Execution Model. Ingress adapters manage protocol-specific cryptographic validation (signature checking, CRL validation, remote TokenReview executions, and SPIFFE SVID bundle verification) and immediately flatten the output into a uniform text claims map (HashMap<String, Vec<String>>). Downstream authorization is then handled by a single, centralized, protocol-blind Unified Mapping Engine.

By combining this unified engine with a two-phase SecurityContext validation framework, unbacked service accounts (Kubernetes pods, automated mTLS agents, and SPIFFE control-plane daemons) become native, first-class citizens. They receive fully scoped, immutable tokens from birth without generating orphaned rows or database bloat in the local user tables.

Furthermore, to eliminate privilege escalation pathways, any ruleset that contains a control-plane bypass instruction (is_system: true) is structurally classified as an Immutable System Mapping. These maps are blocked from undergoing subsequent API modifications, updates, or incremental mutations of any kind.

Convergence of Local and Distributed Control Planes

To simplify verification logic and prevent security context bifurcation, the system merges its two system-level superuser authorization paths:

• The Mapped Route (is_system): Mapped via global, cluster-wide SPIFFE or infrastructure rulesets for service-to-service communication across separate physical nodes.
• The Local Bootstrap Route (is_admin): Established natively within the SecurityContext when an operator connects locally over a secure Unix Domain Socket (UDS) with a loopback SPIFFE identity matching the static application configuration file.

Both paths are verified through the same strict execution gates, resolving onto identical system-service shortcut variables to permit fast-path control-plane transactions.

Scope, Exclusions, and Trust Boundaries

This ADR defines the unified mapping model, validation rules, storage keyspace, and execution engine. It explicitly excludes:

• Session State Management (AuthState) — PKCE verifiers, OIDC state tokens, and nonce tracking are handled by the ingress layer.
• Ingress Trust Boundary — Ingress adapters are compiled in-tree or run as internal static libraries within the application binary’s native memory space, preventing side-channel data injection.
• Application Credentials for Virtual Users — Any principal initialized as IdentityInfo::Principal is strictly blocked from executing application credentials, regular credentials and trusts eliminating the risk of unbacked service accounts spawning persistent API keys.

2. Ingress Phase: Provider Configuration Resources (The Crypto Inputs)

The provider_id is a tenant-local functional slug binding an ingress protocol instance to its access rules. These configuration resource models explicitly contain their own domain_id and provider_id keys to enforce structural identifier symmetry across administrative lookup routines.

A. OIDC Identity Provider (IdP) Resource

#![allow(unused)]
fn main() {
pub struct OidcProviderResource {
    pub domain_id: Option<String>,      // Owning tenant domain boundary (None if global system mapping)
    pub provider_id: String,            // Functional configuration slug anchor
    pub issuer: String,                 // e.g., "https://auth.acme.com"
    pub client_id: String,              // Client ID registered at the external IdP
    pub client_secret: Option<String>,  // Secret used for authorization code exchanges
    pub jwks_uri: String,               // Cached public keys URI for signature verification
    pub allowed_redirect_uris: Vec<String>,
    pub oidc_scopes: Vec<String>,
    pub token_endpoint_auth_method: Option<String>,
}

}

B. Kubernetes Cluster Issuer Resource

#![allow(unused)]
fn main() {
pub struct K8sClusterResource {
    pub domain_id: String,
    pub provider_id: String,           // Functional configuration slug anchor
    pub kubernetes_host: String,       // e.g., "https://api.eks.amazonaws.com"
    pub kubernetes_ca_cert: String,    // Public cluster CA certificate
    pub token_reviewer_jwt: String,    // Service account token to execute TokenReviews
    pub disable_local_ca_jwt: bool,    // Force remote verification over local decoding
}

}

C. SPIFFE Trust Domain Resource

#![allow(unused)]
fn main() {
pub struct SpiffeTrustResource {
    pub domain_id: String,
    pub provider_id: String,           // Functional configuration slug anchor
    pub trust_domain: String,          // e.g., "prod.keystone.internal"
    pub trust_bundle_pem: String,      // Validating root keys for SVID validation
}

}

3. Downstream Phase: The Named Mapping Model

The rules engine evaluates claims maps using the MappingRuleSet. Rules are structured as an ordered vector where array position defines execution priority. However, each individual rule includes an immutable, alphanumeric name handle. This enables operators to execute fine-grained additions, deletions, and updates in the middle of the priority vector without relying on volatile integer indices.

Data Structural Spec (src/identity/mapping/model.rs)

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub enum DomainResolutionMode {
    Fixed,            // Locked to mapping.domain_id; claims templates in user_domain_id are rejected
    ClaimsOrMapping,  // System-Admin Only: Rules may override mapping.domain_id via claims templates
    ClaimsOnly,       // System-Admin Only: Neither mapping nor provider is bound to a domain
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum IdentitySource {
    Federation { idp_id: String },
    K8s { cluster_id: String },
    Spiffe { trust_domain: String },
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MappingRuleSet {
    pub mapping_id: String,
    pub domain_id: Option<String>,  // Forced to None ("global") for ClaimsOnly/ClaimsOrMapping modes
    pub provider_id: String,
    pub source: IdentitySource,
    pub domain_resolution_mode: DomainResolutionMode,
    pub allowed_domains: Vec<String>, // Whitelist of domain IDs that claims-based interpolation may resolve to. Mandatory and non-empty for ClaimsOnly/ClaimsOrMapping modes. For Fixed mode, must be empty (no claims-based interpolation possible)
    pub enabled: bool,
    pub rules: Vec<MappingRule>,
    pub ruleset_version: u128,    // Content-aware SHA-256 hash (first 16 bytes) of full ruleset — detects reordering, renaming, authorization swaps, not just addition/deletion
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MappingRule {
    pub name: String,
    pub description: Option<String>,
    pub r#match: MatchCriteria,
    pub identity: IdentityBinding,
    pub authorizations: Vec<Authorization>,
    pub groups: Vec<GroupAssignment>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct IdentityBinding {
    pub identity_mode: Option<IdentityMode>,   // Virtual shadow record (Ephemeral) or real user CRUD (Local)
    pub is_system: bool,                       // Nuclear control-plane shortcut bypass flag; defaults to false
    pub user_name: String,
    pub user_id:   Option<String>,
    pub user_domain_id: Option<String>,     // Template: resolves to domain UUID string at evaluation time
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum IdentityMode {
    /// Real user CRUD: create/find federated user row, sync group memberships.
    Local,
    /// Virtual shadow registry: HMAC-derived ID, no persistent user row.
    Ephemeral,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum Authorization {
    Project {
        project_id: String,
        project_domain_id: String,
        roles: Vec<RoleRef>,
    },
    Domain {
        domain_id: String,
        roles: Vec<RoleRef>,
    },
    System {
        system_id: String,
        roles: Vec<RoleRef>,
    },
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum ClaimCondition {
    Equals { claim: String, value: serde_json::Value },
    AnyOf { claim: String, values: Vec<serde_json::Value> },
MatchesRegex { claim: String, regex: String },
}

/* ClaimCondition helpers:
   - claim_name(): Extracts the claim key from any variant (Equals, AnyOf, MatchesRegex).
   - walk_all_claim_conditions(): Flattened iterator over all claim conditions nested
     within a MappingRule's match criteria. Walks recursively through nested groups
     to collect every leaf ClaimCondition. Used during write-time validation to verify
     all regex patterns and check template safety before persistence.
*/

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum MatchCriteria {
    AllOf(Vec<MatchCondition>),
    AnyOf(Vec<MatchCondition>),
    AllOfStrict {
        conditions: Vec<MatchCondition>,
        require_all_keys: bool,  // If true, match fails if any referenced claim key is absent from the claims map
    },
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub enum MatchCondition {
    Condition(ClaimCondition),
    Nested(Box<MatchCriteria>),
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
pub struct GroupAssignment {
    pub group_id: Option<String>,            // Optional group UUID — absent for Local identity mode (resolved by name at runtime)
    pub group_name: String,                 // Template for display/lookup; interpolated at runtime
    pub group_domain_id: Option<String>,
    pub strategy: Option<GroupStrategy>,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum GroupStrategy {
    CreateOrGet,
    Get,
}

}

4. State Persistence: The Shadow Virtual User Registry

Downstream OpenStack microservices (e.g., Nova, Neutron) or admin users may perform a GET /v3/users/{user_id} call to resolve user attributes. Furthermore, during token verification, the original HTTP claims map is completely gone - Keystone only receives the encrypted Fernet token byte string.

To fulfill the token roundtrip without passing bloated claims inside the token, the mapping engine derives a deterministic identifier for unbacked principals during authentication — computed as the first 16 bytes of HMAC-SHA256(cluster_salt, workload_id || provider_id), formatted as a UUIDv4-compatible string — and registers a stateful bridge record inside the Shadow Virtual User Registry within FjallDB.

cluster_salt is a 256-bit cryptographically random key generated at cluster bootstrap, stored in the static application configuration, and excluded from all API responses. HMAC-SHA256 replaces naive UUIDv5 (which relies on SHA-1) to provide a one-way, non-invertible derivation: even if an attacker knows the provider_id and workload_id (e.g., K8s service account names, SPIFFE URIs), they cannot reverse the salt or feasibly enumerate shadow registry keys without brute-forcing the full HMAC output space.

#![allow(unused)]
fn main() {
pub struct ResolvedGroupBinding {
    pub resolved_group_id: String,          // Immutable UUID anchor — prevents name-collision attacks
    pub group_domain_id: Option<String>,
    pub strategy: Option<GroupStrategy>,
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VirtualUserMetadata {
    pub user_id: String,            // Deterministic HMAC-SHA256-derived handle (formatted as UUIDv4-compatible string)
    pub unique_workload_id: String,
    pub mapping_id: String,         // Direct anchor to the MappingRuleSet that matched; deterministic verification lookup
    pub matched_rule_name: String,
    pub domain_id: Option<String>,
    pub resolved_user_name: String,
    pub is_system: bool,  // Immutably preserved from initial upsert — cannot be escalated or revoked by rule modification, preventing runtime privilege escalation
    pub resolved_group_bindings: Vec<ResolvedGroupBinding>,
    pub authorizations: Vec<Authorization>,        // Snapshot of authorizations at issuance — prevents live rule modification from altering cached tokens
    pub ruleset_version: u128,    // SHA-256 hash (first 16 bytes) captured at issuance — used to detect stale tokens against live ruleset
    pub enabled: bool,
    pub created_at: i64,
    pub last_authenticated_at: i64,       // PCI-DSS compliance tracking variable
}

}

A. Shadow Record Lifecycle & Virtual User Deactivation

Because the HMAC-SHA256-derived identifier is deterministic, repeated authentication by the same principal always resolves to the same user_id. On token issuance, the engine performs an upsert: if the shadow record already exists, it completely refreshes matched_rule_name, resolved_group_bindings, and resolved_user_name. Set enabled: true (a successful match indicates the principal is active; if previously deactivated, successful authentication reactivates). The created_at timestamp is immutably preserved from initial creation. The is_system flag is intentionally preserved from initial creation — on a subsequent upsert meta.is_system = meta.is_system prevents the flag from being modified. This is a deliberate security measure: once a principal is granted system-level service privileges, those privileges cannot be escalated nor revoked through ruleset modification alone. Revoking is_system requires setting enabled: false (deactivation) via the provider API, followed by a fresh authentication lifecycle against a corrected ruleset to re-evaluate privileges.

To maintain PCI-DSS compliance, the field last_authenticated_at tracks real-time usage. A dedicated background janitor task range-scans the registry keyspace nightly; any virtual profile that has failed to log an authentication event for more than 90 days is deactivation-set (enabled: false), and all corresponding live authorizations are dropped. This policy applies uniformly — including virtual users with is_system: true, which must re-attest within the 90-day window or be deactivated (SPIFFE control-plane daemons that authenticate periodically will naturally stay within this window).

Deactivation preferred over deletion. The janitor sets enabled: false instead of deleting records, preserving forensic evidence (identity bindings, authorization snapshots, activity timestamps) for incident response and compliance auditing. A separate archive cleanup task permanently deletes deactivated records after a configurable retention period (default: 365 days, configurable via [keystone] shadow_registry_archive_retention_days). The CADF maintenance event type captures these archive deletions with the record’s identity metadata in the attachment payload. The archive cleanup cadence is configurable (default: weekly, configurable via [keystone] shadow_registry_archive_cleanup_interval).

5. Execution Engine Logic (src/identity/mapping/engine.rs)

5.1. Claim Condition Evaluation Semantics

Each ClaimCondition variant is evaluated against the flattened claims map (HashMap<String, Vec<String>>). JSON primitive values from the claims are normalized to strings for comparison: Number and Bool are converted via their Display representation, String is used directly, and nested objects fall back to their JSON serialization. This ensures that a claim value of boolean true matches a rule condition specifying the string "true".

Regex caching. Precompiled regex patterns are cached in a thread-safe OnceLock-backed DashMap<String, Regex>. To prevent adversarial cache partition attacks, the map enforces a 1024-entry cap; once exceeded, the 100 least-recently-used entries are evicted (LRU policy) to retain frequently used patterns and minimize adversarial cache thrashing.

Runtime evaluation bounds. Two defenses protect against resource exhaustion during regex matching:

• Per-claim value limit. Each individual claim value is capped at 4096 bytes. Claims exceeding this limit are silently dropped from the flattened claims map before evaluation. This limits the input size against which any regex operates.
• Per-match timeout. Each MatchesRegex evaluation runs with a 2-second deadline. If a single regex match exceeds the timeout, it short-circuits to false and emits a CADF access event with RegexMatchTimeout outcome. This prevents adversarial claim values against legitimate regex patterns from causing CPU exhaustion.

Walker utility. MappingRule::walk_all_claim_conditions() provides a flat iterator over every ClaimCondition instance nested within a rule’s match criteria, used during write-time validation to verify all regex patterns pass ReDoS safety checks before persistence.

5.2. Match Criteria Resolution Semantics

A MatchCriteria node evaluates nested boolean structures recursively:

A MatchCondition dispatches to one of two branches:

This structure allows arbitrary nesting depth, enabling complex multi-claim combinations such as requiring an exact namespace match AND a regex-matched service account name within a claims_or_mapping scope.

5.3. Ruleset Entry Point: Match Evaluation

The ruleset evaluator iterates the rules vector top-to-bottom until the first matching rule. Evaluation is short-circuit: once a rule matches, subsequent rules are ignored (first-match-wins semantics).

Returned struct. A successful match populates MatchResult:

#![allow(unused)]
fn main() {
pub struct MatchResult {
    pub rule_name: String,
    pub user_name: String,
    pub user_id: Option<String>,
    pub user_domain_id: Option<String>,
    pub is_system: bool,
    pub identity_mode: Option<IdentityMode>,  // Propagated from matched rule's IdentityBinding
    pub authorizations: Vec<Authorization>,
    pub resolved_group_bindings: Vec<GroupRef>,   // D3: resolved by name at runtime for Local identity mode
    pub ruleset_version: u128,    // Content-aware SHA-256 hash (first 16 bytes) — anchors token validity to ruleset state
}
}

Evaluation algorithm. For each rule in priority order:

1. Enable gate. If ruleset.enabled == false, evaluation terminates immediately, returning None.
2. Match gate. Evaluate rule.match criteria against the claims map. If it does not match, proceed to the next rule.
3. Domain resolution. Determine user_domain_id according to the active DomainResolutionMode:

• If identity.user_domain_id contains a template, interpolate it via safe string interpolation.
  • If interpolation yields an empty string, fall back to ruleset.domain_id (enclosing domain).
  • If domain_resolution_mode is Fixed and the interpolated value does not match ruleset.domain_id exactly, abort the rule match and proceed to the next rule. This prevents silent fallback to an unexpected domain.
  • If the interpolated value matches the enclosing domain directly, accept it.
  • If the interpolated value is a valid UUID format, accept it pending existence check during upsert (the evaluator itself has no DB access; domain existence is validated later).
  • If the interpolated value is neither the enclosing domain nor a valid UUID, fall back to ruleset.domain_id to prevent domain escape.
  • Domain whitelist check. If ruleset.allowed_domains is present and the interpolated user_domain_id is not contained within it, fall back to ruleset.domain_id. This prevents a compromised IdP from injecting arbitrary domain identifiers to redirect principal resolution.
  • If no user_domain_id template exists, default to ruleset.domain_id.

4. user_name gate. Interpolate identity.user_name. If interpolation fails or produces an empty string, skip this rule and try the next. This prevents blank shadow registry records.
5. user_id resolution. If identity.user_id is present, interpolate it. Empty result falls back to the enclosing domain (permitted unlike user_name).
6. Group binding resolution. For each GroupAssignment, interpolate group_name using the truncating variant (overflow is acceptable for display fields). Emit a ResolvedGroupBinding containing only the resolved anchor UUID, domain, and strategy — the interpolated display name is discarded.
7. Result assembly. Return Some(MatchResult) populated with interpolated identity, snapshotted authorizations, resolved groups, and the content-aware ruleset version.

If no rule matches, return None.

Content-aware ruleset version. The ruleset_version is computed by serializing the structural payload — mapping_id, provider_id, domain_id, the JSON-serialized rules vector, domain_resolution_mode, allowed_domains, and enabled flag — into a canonical string, then hashing it with SHA-256 and extracting the first 16 bytes as a u128. This replaces a naive length-based counter, making the version resistant to rule reordering, renaming, authorization swaps, and cross-rule priority manipulation. The SHA-256 hash (128 bits yields a birthday collision window of ~2^64 attempts), preventing adversaries from crafting colliding rulesets to bypass TOCTOU detection during token verification.

5.4. String Interpolation & Template Safety

All template expansion is single-pass with no recursive substitution. Two macro patterns are recognized: ${claims.<key>} for claim values and ${enclosing_domain_id} for the ruleset’s enclosing domain. The interpolation regex is compiled once and cached via OnceLock to eliminate per-request compilation cost.

Non-truncating variant (strict). Used for user_name, user_id, and user_domain_id. The interpolation accumulates literal segments and substituted values in order. On any intermediate or final overflow past 256 characters, it returns Err(InterpolatedValueTooLong) — the caller is responsible for handling the error (e.g., skipping the rule for user_name).

Truncating variant (display). Used for group_name and other display-only fields. If interpolation exceeds 256 characters, the original template string is truncated to 253 characters and appended with .... This never extracts arbitrary claim values on overflow — only the static template is preserved for operator debugging.

Security properties:

• Single-pass expansion prevents nested template injection chains.
• Missing claims resolve to empty string (no error), but empty user_name causes the rule to be skipped during evaluation.
• The enclosing_domain_id macro is excluded from claim templates via write-time validation (§10.1) to prevent domain shadowing.

5.5. Engine Error Type

All execution-path failures funnel through a single typed enum:

#![allow(unused)]
fn main() {
#[derive(Debug, thiserror::Error)]
pub enum MappingEngineError {
    #[error("mapping ruleset is disabled")]
    MappingDisabled,
    #[error("mapping not found")]
    MappingNotFound,
    #[error("matched rule no longer exists in live ruleset")]
    MappingRuleNoLongerExists,
    #[error("database transaction error")]
    TransactionError,
    #[error("interpolation failed — claim key not available")]
    ClaimKeyNotFound,
    #[error("interpolated value exceeds length limit")]
    InterpolatedValueTooLong,
    #[error("ruleset version mismatch — token issued against stale ruleset")]
    RulesetVersionMismatch,
}

}

5.6. Real-Time Effective Role Calculation Core (core/src/auth.rs)

During token verification, calculate_effective_roles reconstructs the authorization context from the shadow registry. The algorithm:

1. System-service shortcut convergence. If ctx.is_admin(), set the is_system flag on the context. This unifies local loopback credentials with remote service accounts.

2. Principal dispatch. For IdentityInfo::Principal, proceed with shadow registry lookup. For IdentityInfo::User, use the existing role resolution path (unchanged).

3. Shadow registry bridge. Look up VirtualUserMetadata using the virtual user_id from the token. If missing or disabled, return an authentication error.

4. System flag propagation. If shadow_meta.is_system == true, set the system service flag on the context, enabling control-plane shortcut bypasses.

5. Live ruleset fetch. Use shadow_meta.mapping_id to resolve the index key index:mapping_id:<mapping_id>, which yields (domain_id, provider_id) coordinates, then fetch the live MappingRuleSet. If disabled, abort.

6. TOCTOU version check. Compute the content-aware SHA-256 version of the live ruleset. If it differs from shadow_meta.ruleset_version, reject the token with RulesetVersionMismatch containing both shadow and live versions for incident response audit trail.

7. Rule existence check. Verify that shadow_meta.matched_rule_name still exists in the live ruleset. If the rule was removed, abort with MappingRuleNoLongerExists.

8. Authorization from snapshot. Iterate shadow_meta.authorizations (the snapshotted version from issuance time, not the live ruleset). For each authorization variant, check scope match:

   • Project: If scope matches the target project and domain, extend roles.
   • Domain: If scope matches the target domain, extend roles.
   • System: If context is marked system service and scope is system-level, extend roles.

9. Group role resolution. For each resolved_group_binding in the shadow record, resolve group roles from the assignment provider:

   • On success, extend effective roles.
   • On failure with GroupStrategy::Get, abort with GroupNotFound.
   • On failure with GroupStrategy::CreateOrGet, create the group synchronously within the current authorization transaction, then extend effective roles. This ensures the group exists before the token is considered valid, eliminating a race window between async enqueuing and subsequent verification.

10. Token restriction application. If context has a token restriction, narrow the effective roles accordingly.

11. Empty role check. If no roles were accumulated and the scope is not unscoped and the context is not a system service, return ActorHasNoRolesOnTarget.

12. Deduplication. Sort and deduplicate the effective role list before returning.

6. Concrete Examples: How Mapping Rulesets Look

Use Case 1: SPIFFE Control-Plane Service Binding (is_system Enabled)

• Stored at: data:mapping:v1:domain_admin_infra:spiffe-local
• Context: Authorizes the core Nova service account to issue tokens and perform service-to-service background API transactions over the OpenStack control plane using an explicit system shortcut flag.

{
  "mapping_id": "7c8d9e0f-1a2b-3c4d-5e6f-7a8b9c0d1e2f",
  "domain_id": "domain_admin_infra",
  "provider_id": "spiffe-local",
  "source": {
    "type": "spiffe",
    "trust_domain": "prod.keystone.internal"
  },
  "domain_resolution_mode": "fixed",
  "allowed_domains": [],
  "enabled": true,
  "rules": [
    {
      "name": "nova-to-neutron-control-plane",
      "description": "Authorize Nova compute workload to bypass target constraints via system flag shortcut",
      "match": {
        "all_of": [
          {
            "type": "equals",
            "claim": "spiffe.id",
            "value": "spiffe://prod.keystone.internal/ns/openstack/sa/nova"
          }
        ]
      },
      "identity": {
        "user_name": "svc-nova-compute",
        "user_id": "spiffe-nova-compute",
        "is_system": true
      },
      "authorizations": [
        {
          "type": "system",
          "system_id": "all",
          "roles": [{ "type": "system_role", "name": "default-role" }]
        }
      ],
      "groups": []
    }
  ]
}

Use Case 2: OIDC Federation (Enterprise SSO Mapping)

• Stored at: data:mapping:v1:domain_hr:oidc-okta
• Context: Maps Okta enterprise SSO claims to internal project roles and groups. Demonstrates ClaimsOrMapping domain resolution, regex-based group parsing, and multi-role assignment per matched rule.

{
  "mapping_id": "a1b2c3d4-5678-90ab-cdef-123456789abc",
  "domain_id": "domain_hr",
  "provider_id": "oidc-okta",
  "source": {
    "type": "federation",
    "idp_id": "okta-enterprise-idp"
  },
  "domain_resolution_mode": "claims_or_mapping",
  "allowed_domains": ["domain_hr", "550e8400-e29b-41d4-a716-446655440001"],
  "enabled": true,
  "rules": [
    {
      "name": "hr-admin-role-binding",
      "description": "Grant HR admin team _member_ and hr_admin role on HR project",
      "match": {
        "all_of": [
          {
            "type": "matches_regex",
            "claim": "email",
            "regex": "^.*\\.hr@acme\\.com$"
          },
          {
            "type": "any_of",
            "claim": "groups",
            "values": ["HR-Admin", "HR-Super-Admin"]
          }
        ]
      },
      "identity": {
        "user_name": "${claims.preferred_username}",
        "user_id": "${claims.sub}",
        "user_domain_id": "${claims.domain_id}"
      },
      "authorizations": [
        {
          "type": "project",
          "project_id": "550e8400-e29b-41d4-a716-446655440001",
          "project_domain_id": "domain_hr",
          "roles": [
            { "type": "system_role", "name": "_member_" },
            { "type": "system_role", "name": "hr_admin" }
          ]
        },
        {
          "type": "domain",
          "domain_id": "domain_hr",
          "roles": [{ "type": "system_role", "name": "domain_admin" }]
        }
      ],
      "groups": [
        {
          "group_id": "550e8400-e29b-41d4-a716-446655440010",
          "group_name": "HR-Admins-Global",
          "group_domain_id": "domain_hr",
          "strategy": "create_or_get"
        }
      ]
    },
    {
      "name": "regional-team-scope",
      "description": "Match regional HR team members and assign to project using regex",
      "match": {
        "all_of": [
          {
            "type": "matches_regex",
            "claim": "groups",
            "regex": "^HR\\-Team\\-(NA|EU|APAC)$"
          }
        ]
      },
      "identity": {
        "user_name": "${claims.preferred_username}",
        "user_id": "${claims.sub}",
        "user_domain_id": null
      },
      "authorizations": [
        {
          "type": "project",
          "project_id": "550e8400-e29b-41d4-a716-446655440002",
          "project_domain_id": "domain_hr",
          "roles": [{ "type": "system_role", "name": "_member_" }]
        }
      ],
      "groups": [
        {
          "group_id": "550e8400-e29b-41d4-a716-446655440020",
          "group_name": "Regional-HR-${claims.groups}",
          "group_domain_id": "domain_hr",
          "strategy": "create_or_get"
        }
      ]
    },
    {
      "name": "default-reader",
      "description": "Catch-all fallback for unhandled Okta users",
      "match": {
        "all_of": [
          {
            "type": "matches_regex",
            "claim": "email",
            "regex": "^.*@acme\\.com$"
          }
        ]
      },
      "identity": {
        "user_name": "${claims.email}",
        "user_id": "${claims.sub}"
      },
      "authorizations": [
        {
          "type": "project",
          "project_id": "550e8400-e29b-41d4-a716-446655440003",
          "project_domain_id": "domain_hr",
          "roles": [{ "type": "system_role", "name": "reader" }]
        }
      ],
      "groups": []
    }
  ]
}

Use Case 3: Kubernetes Service Account Authorization

• Stored at: data:mapping:v1:domain_infra:k8s-eks-prod
• Context: Grants EKS-deployed workloads scoped access to OpenStack resources based on service account name and namespace. Demonstrates Fixed domain resolution with nested AnyOf match criteria.

{
  "mapping_id": "b2c3d4e5-6789-01bc-def0-23456789abcd",
  "domain_id": "domain_infra",
  "provider_id": "k8s-eks-prod",
  "source": {
    "type": "k8s",
    "cluster_id": "eks-prod-cluster-01"
  },
  "domain_resolution_mode": "fixed",
  "allowed_domains": [],
  "enabled": true,
  "rules": [
    {
      "name": "ci-pipeline-admin",
      "description": "Grant CI/CD pipeline service account admin access to infra projects",
      "match": {
        "all_of": [
          {
            "type": "equals",
            "claim": "k8s.serviceaccount.namespace",
            "value": "ci-pipeline"
          },
          {
            "type": "any_of",
            "claim": "k8s.serviceaccount.name",
            "values": ["build-runner", "deploy-agent"]
          }
        ]
      },
      "identity": {
        "user_name": "svc-k8s-${claims.k8s.serviceaccount.name}"
      },
      "authorizations": [
        {
          "type": "project",
          "project_id": "550e8400-e29b-41d4-a716-446655440010",
          "project_domain_id": "domain_infra",
          "roles": [{ "type": "system_role", "name": "admin" }]
        }
      ],
      "groups": []
    },
    {
      "name": "monitoring-reader",
      "description": "Read-only access for Prometheus/Grafana monitoring agents",
      "match": {
        "all_of": [
          {
            "type": "equals",
            "claim": "k8s.serviceaccount.namespace",
            "value": "monitoring"
          },
          {
            "type": "matches_regex",
            "claim": "k8s.serviceaccount.name",
            "regex": "^prometheus-.*$"
          }
        ]
      },
      "identity": {
        "user_name": "svc-k8s-${claims.k8s.serviceaccount.name}"
      },
      "authorizations": [
        {
          "type": "project",
          "project_id": "550e8400-e29b-41d4-a716-446655440010",
          "project_domain_id": "domain_infra",
          "roles": [{ "type": "system_role", "name": "reader" }]
        }
      ],
      "groups": [
        {
          "group_id": "550e8400-e29b-41d4-a716-446655440030",
          "group_name": "Monitoring-Agents",
          "group_domain_id": "domain_infra",
          "strategy": "get"
        }
      ]
    }
  ]
}

7. Runtime Mechanics & Token Lifecycle Roundtrips

Workflow: SPIFFE Workload Authentication & Verification

Phase A: Token Issuance (Login)

1. The Nova compute driver forwards its signed SPIFFE SVID certificate context to keystone-rs.
2. The SPIFFE ingress provider validates the cryptographic signature using the matching SpiffeTrustResource trust bundle PEM. It flattens the workload metrics into clean text claims (spiffe.id, spiffe.trust_domain) and hands them to the Mapping Engine.
3. The engine matches the rule "nova-to-neutron-control-plane" and reads its explicit identity properties, picking up the is_system: true instruction.
4. The system derives the persistent HMAC-SHA256 user_id for the workload. It executes an atomic transactional upsert against the shadow virtual user registry (detailed in Section 7.2 below), recording mapping_id (to anchor the live ruleset), matched_rule_name: "nova-to-neutron-control-plane", capturing resolved_user_name and resolved_group_bindings from the live claims map, and immutably recording is_system: true into the registry row. Once set, this flag is preserved across all subsequent upserts — the is_system privilege cannot be revoked by rule modification alone.
5. The token engine compiles a native SystemScope Fernet Token payload variant directly containing the virtual user_id. The token is returned to the Nova driver.

Phase B: Token Verification (Roundtrip)

1. The Nova driver calls Neutron to wire up a VM interface, attaching its token. Neutron hands the token back to Keystone’s authorize_by_token endpoint for verification.
2. The token provider decrypts the Fernet payload, matching the SystemScope layout and pulling the virtual user_id.
3. The engine builds an unverified, raw SecurityContext initializing the identity as an unbacked IdentityInfo::Principal.
4. The handler calls ValidatedSecurityContext::new_for_scope(). Inside calculate_effective_roles(), the engine reads the shadow user profile from FjallDB using the user_id. It uses mapping_id to look up the index key index:mapping_id:<mapping_id> which resolves the (domain_id, provider_id) coordinates, then fetches the live MappingRuleSet from data:mapping:v1:<domain_id>:<provider_id>. It encounters the active is_system: true flag.
5. The method calls ctx.set_system_service_flag(true), applying the control-plane shortcut bypass directly to the context memory segment. When projected into Credentials for policy analysis, the OPA engine registers is_system: true and cleanly validates the communication path.

7.2. Atomic Transactional Upsert Flow & Adaptive Rate Limiter

To shield the shadow registry from write-amplification DoS attacks, creation lookups pass through a sliding-window token bucket rate-limiter tracked per provider_id. The threshold is configurable via [keystone] shadow_registry_creation_rate_limit in keystone.conf (default: 50 operations per minute). When unique principal creation events spike past this threshold, the login path drops further entries with an HTTP 429 Too Many Requests status code.

A second rate-limit tier governs total authentications per provider_id (both new and existing principals), configurable via [keystone] shadow_registry_auth_rate_limit (default: 500 operations per minute). This prevents replay attacks against known valid principals from bypassing the creation-only limit. When this threshold is exceeded, all authentication attempts against the provider are rejected with HTTP 429 Too Many Requests until the sliding window expires.

Upsert algorithm. The virtual user record is persisted atomically within a single database transaction:

1. HMAC-SHA256 user_id derivation. Compute a deterministic identifier for the principal using a 256-bit per-cluster secret as the HMAC key: HMAC-SHA256(cluster_salt, workload_id || provider_id), taking the first 16 bytes and formatting as a UUIDv4-compatible string. The cluster_salt is generated at cluster bootstrap, stored in the static application configuration, and excluded from all API responses. HMAC-SHA256 (rather than UUIDv5/SHA-1) provides a one-way, non-invertible derivation: an attacker knowing provider_id and workload_id cannot reverse the salt or enumerate shadow registry keys. Identical principals always resolve to the same user_id within a cluster, while cross-cluster correlation is blocked by the per-cluster salt.

2. Domain existence validation. The effective_domain is determined from MatchResult.user_domain_id:

   • If user_domain_id is a valid UUID, check index:auth:domain_id:<uuid> for existence. If the domain exists, use it.
   • If user_domain_id is a human-readable slug, check index:auth:domain_slug:<slug> for existence. If found, resolve the mapped UUID and use it.
   • If the interpolated domain does not exist in either index, reject the upsert with ValidationResult::DomainNotFound, preventing non-existent UUIDs from persisting in shadow records (fixes the UUID-format domain gap).
   • If user_domain_id is absent or the above checks fail, fall back to ruleset.domain_id (the enclosing domain).

3. Read existing shadow record. Attempt to fetch from user:v1:virtual:<user_id>.

4. Merge or create.

   • Update path (existing record): Refresh mapping_id, matched_rule_name, resolved_user_name, resolved_group_bindings, and snapshotted authorizations. Update ruleset_version from the match result and last_authenticated_at to current timestamp. Set enabled: true — a successful match indicates the principal is active; if the record was previously deactivated, successful authentication reactivates it. The created_at timestamp is immutably preserved from initial creation. The is_system flag is immutably preserved from initial creation (meta.is_system = meta.is_system) — once a principal is granted system-level privileges, those privileges cannot be escalated nor revoked through ruleset modification alone. Revoking is_system requires setting enabled: false (deactivation) via the provider API.
   • Insert path (new record): Create a fresh VirtualUserMetadata populated with all fields from the match result, the validated effective_domain, enabled: true, and the current timestamp for created_at and last_authenticated_at.

5. Persist. Write the merged or new record atomically to the shadow keyspace.

7.3. Validation Error Code Reference

RoleNotFound closes a gap distinct from the other write-time checks above: a typo’d or invented role reference previously produced a rule whose authorization could never resolve to a real Role, and failed silently rather than at creation time (see ADR 0024 §8, where exactly this class of bug — invented SystemAdmin/DomainManager role literals with no backing Role — was found and fixed in the SCIM policies). Rule create/update now resolves every referenced RoleRef.id against the Role store (RoleApi::get_role) and rejects the ruleset with 422 if it does not exist. RoleRef.id is mandatory (unlike name, which is optional), so this is the field the check keys on.

8. Unified Keyspace Naming Scheme Summary

All indices, entries, structures, and metadata elements are maintained inside a single consolidated partition layer in FjallDB.

9. Administrative CRUD Management API Specification

A. Create Mapping Configuration

• HTTP Method / Path: POST /v4/mappings
• Validation Bounds — Two-Tier Enforcement:
  • Tier 1 (SystemAdmin Gate): If mapping.rules contains at least one target where identity.is_system == true, and ctx.is_admin() == false (no SystemAdmin credentials), the call is rejected with HTTP 403 Forbidden. Mappings with an active system-bypass flag are stamped as Immutable System Mappings.
  • Tier 2 (Authorization Bounds — Non-Admin Only): If ctx.is_admin() == false, the engine validates creator scope before persistence:
• Domain confinement — for Authorization::Project, project_domain_id must match the operator’s effective domain UUID; for Authorization::Domain, domain_id must match the operator’s domain. Cross-domain mappings are rejected with 403 Forbidden (CrossDomainMapping).
  • Role grant parity — for every Authorization, the operator must hold each role listed in roles on the target scope. Failure returns 403 Forbidden (RoleGrantUnauthorized).
  • System scope restriction — Authorization::System requires is_system: true on the mapping rule and is_admin() on the operator. Non-admin operators are rejected with 422 Unprocessable Entity (SystemScopeRequiresIsSystem).
  • Group assignment authority — for every GroupAssignment, the operator must hold admin on the target group. Failure returns 403 Forbidden (GroupAssignmentUnauthorized).
  • SystemAdmin bypass: When ctx.is_admin() == true, Tier 2 is bypassed entirely. The admin operator may map to any domain, grant any roles, and assign to any groups.

B. List Mapping Configurations (Tenant-Isolated)

• HTTP Method / Path: GET /v4/mappings?domain_id=domain_admin_infra
• Response: 200 OK

C. Get Mapping Profile

• HTTP Method / Path: GET /v4/mappings/{mapping_id}
• Response: 200 OK

D. Declarative Overwrite / Apply Update

• HTTP Method / Path: PUT /v4/mappings/{mapping_id}
• Immutability Protection: If the target configuration is flagged as an Immutable System Mapping, the engine will completely abort the operation, throwing an HTTP 422 Unprocessable Entity response. System mappings can never be updated; changes require an explicit DELETE statement followed by a fresh POST initialization block to preserve clean audit separation.
• Domain Immutability: The domain_id field is structurally immutable upon ruleset creation. Any PUT request attempting to modify domain_id to a different value is rejected with HTTP 422 Unprocessable Entity. The owning domain anchor cannot be migrated post-creation, as it forms the basis of the keyspace coordinate (data:mapping:v1:<domain_id>:<provider_id>) and all shadow registry lookups for principals issued under the ruleset.

E. Imperative Rule Mutation (Relative Anchoring Path)

• HTTP Method / Path: POST /v4/mappings/{mapping_id}/rules/mutate
• Immutability Protection: Throws an immediate HTTP 422 Unprocessable Entity if the target is an Immutable System Mapping, ensuring no mutation deltas can manipulate control-plane assets.

F. Virtual User Lifecycle Management

• Disable Virtual User:

  • HTTP Method / Path: PATCH /v4/virtual_users/{user_id}/disable
  • Response: 200 OK — returns the deactivated VirtualUser record
  • Effect: Sets enabled: false, triggers token revocation pipeline (revocation:v1:user:<user_id>), preserves forensic record for audit trail

• Enable (Reactivate) Virtual User:

  • HTTP Method / Path: PATCH /v4/virtual_users/{user_id}/enable
  • Response: 200 OK — returns the reactivated VirtualUser record
  • Effect: Sets enabled: true, re-activates the principal for future authentication

• Get Virtual User Profile:

  • HTTP Method / Path: GET /v4/virtual_users/{user_id}
  • Response: 200 OK — returns full VirtualUser metadata including snapshotted authorizations, resolved groups, and activity timestamps

10. Security Architecture, Invariant Protections & Auditing

10.1. Write-Time ReDoS and Immutability Validation Rules

The ValidationError structure explicitly manages our safety parameters:

#![allow(unused)]
fn main() {
#[derive(Debug, thiserror::Error)]
pub enum ValidationError {
    #[error("template references reserved key: {0}")]
    SystemTokenShadowing(String),
    #[error("ClaimsOnly mode requires user_domain_id template with ${claims.*}")]
    DomainClaimRequired,
    #[error("Fixed mode does not allow claims templates in user_domain_id")]
    DomainOverrideInFixedMode,
    #[error("rule name '{0}' is not a valid identifier")]
    InvalidRuleName(String),
    #[error("duplicate rule name '{0}' within ruleset")]
    DuplicateRuleName(String),
    #[error("system level bypass mappings are strictly immutable and can never be modified")]
    SystemMappingIsImmutable,
    #[error("regex pattern '{0}' is syntactically invalid")]
    InvalidRegexSyntax(String),
    #[error("regex pattern '{0}' exceeds complexity limit (AST size > 4096)")]
    RegexTooComplex(String),
    #[error("regex pattern '{0}' fails write-time ReDoS safety check: {1}")]
    RegexSafetyViolation(String, String),
    #[error("operator lacks role '{0}' on project '{1}' — cannot grant via mapping")]
    RoleGrantUnauthorized(RoleRef, String),
    #[error("mapping targets domain '{0}' which is outside the operator's domain")]
    CrossDomainMapping(String),
    #[error("operator cannot assign members to group '{0}'")]
    GroupAssignmentUnauthorized(String),
    #[error("system scope authorization requires is_system: true on the mapping rule")]
    SystemScopeRequiresIsSystem,
    #[error("non-admin operator cannot grant roles at domain scope for '{0}'")]
    DomainMappingUnauthorized(String),
    #[error("domain_resolution_mode '{0}' requires SystemAdmin privileges")]
    DomainResolutionModeRequiresAdmin(DomainResolutionMode),
    #[error("domain_resolution_mode '{0}' requires non-empty allowed_domains")]
    AllowedDomainsRequired(DomainResolutionMode),
    #[error("allowed_domains contains domain '{0}' outside operator scope")]
    AllowedDomainOutOfScope(String),
}

}

ReDoS Protection at Write-Time

Runtime caching mitigates repeated compilation of expensive regexes, but write-time validation is the primary defense. A regex condition is safe if it:

1. Passes regex_syntax::Parser AST validation — detects invalid syntax at parse time
2. Lacks nested quantifiers — (a+)+, (a*)*, (a{2,})* are rejected by recursive AST walk detecting Repetition nodes that directly contain another Repetition or alternation group as their child expression
3. Lacks unbounded alternation under quantifiers — (a|a)+, (a|b|c)* with overlapping branches are rejected
4. Stays within complexity bounds — AST string representation exceeds 4096 characters

The regex crate’s NFA engine guarantees linear backtracking at runtime, neutralizing many ReDoS vectors intrinsically. The write-time AST walk functions as a defense-in-depth layer, rejecting the remaining pathological patterns at ingestion time before they can enter the compiled cache.

Authorization Bound Validation

Every mapping rule is validated before persistence against the operator’s current privileges, enforcing two-tier authorization:

Tier 1 — SystemAdmin bypass. When ctx.is_admin() == true, all authorization bound checks are skipped. The admin operator may map to any domain, grant any roles, and assign to any groups.

Tier 2 — Regular operator constraints. When ctx.is_admin() == false, the engine validates:

• System scope requirement. For every Authorization::System variant, the parent rule must have identity.is_system == true. Otherwise, reject with SystemScopeRequiresIsSystem.
• Domain confinement. For Authorization::Project, project_domain_id must match the operator’s effective domain UUID. For Authorization::Domain, domain_id must match the operator’s domain. Cross-domain mappings are rejected with CrossDomainMapping or DomainMappingUnauthorized.
• Role grant parity. For every role listed in an authorization’s roles vector, the operator must hold that same role on the target scope. For project scope, this is checked against project-specific role assignments. For domain scope, against domain-level role assignments. Failure returns RoleGrantUnauthorized.
• Group assignment authority. For every GroupAssignment, the operator must hold admin on the target group_id. Failure returns GroupAssignmentUnauthorized.

Mapping Definition Validation

The validate_mapping_definition function orchestrates all write-time guards:

1. Template claim extraction. Extract every ${claims.<key>} reference from user_name, user_id, and user_domain_id templates. If any key equals enclosing_domain_id, reject with SystemTokenShadowing to prevent domain context shadowing.

2. Regex safety. For every MatchesRegex condition reachable via walk_all_claim_conditions(), invoke ReDoS validation. Any failure short-circuits with RegexSafetyViolation.

3. Authorization bounds. Invoke the two-tier authorization validation against the operator’s SecurityContext, project roles, domain roles, and group roles.

4. Domain resolution mode enforcement. Non-admin operators are strictly prohibited from creating ClaimsOrMapping or ClaimsOnly rulesets, as these modes enable domain escape and cross-domain virtual user creation. If a non-admin operator specifies these modes, reject with DomainResolutionModeRequiresAdmin.

5. Mode-internal consistency.

   • ClaimsOnly: The user_domain_id template must contain at least one ${claims.*} reference. Otherwise, reject with DomainClaimRequired.
   • Fixed: The user_domain_id must NOT contain any ${claims.*} reference. Otherwise, reject with DomainOverrideInFixedMode.
   • ClaimsOrMapping: Both claim templates and static values are permitted (admin only).

9. Claim Value Size Enforcement

Flattened claims maps are subject to size caps before evaluation:

• Per-claim limit. Each claim value must not exceed 4096 bytes. Values exceeding this limit are silently dropped from the claims map. This prevents a single oversized claim causing memory pressure or CPU exhaustion during regex evaluation.
• Total map limit. The total serialized size of the flattened claims map (HashMap<String, Vec<String>>) must not exceed 64 KiB. If exceeded, the ingress adapter rejects the authentication attempt with 413 Payload Too Large.

10. Domain Whitelist Enforcement

Mandatory for ClaimsOnly/ClaimsOrMapping. If domain_resolution_mode is ClaimsOnly or ClaimsOrMapping, allowed_domains must be present and non-empty. This prevents a compromised IdP from injecting arbitrary domain identifiers to redirect principal resolution. If allowed_domains is empty, reject with AllowedDomainsRequired. For Fixed mode, allowed_domains must be empty (no claims-based interpolation possible).

For every domain in allowed_domains:

• For non-admin operators, each domain must be within the operator’s effective domain or the operator’s own domain. Otherwise, reject with AllowedDomainOutOfScope.
• For admin operators, each domain must exist in the global domain index. Otherwise, reject with DomainNotFound.

11. Domain Whitelist Intersection Check

At evaluation time (§5.3, step 3), after interpolating user_domain_id from claims, the engine checks that the resolved domain falls within allowed_domains. If the interpolated value is not contained in the whitelist, fall back to ruleset.domain_id to prevent domain escape.

12. Real-Time Token Revocation Pipeline

Any Raft proposal that deactivates (enabled: false), deletes (archive cleanup), or alters a MappingRuleSet will automatically append explicit token validation revocation objects directly into the global validation engine (ADR 0009 keyspace):

• revocation:v1:mapping:<mapping_id> $\rightarrow$ Timestamp
• revocation:v1:user:<virtual_user_id> $\rightarrow$ Timestamp

All token lookup evaluation tasks cross-reference this keyspace prefix layout; matching tokens drop validation sessions instantly upon Raft log entry application on the local node. The mapping provider triggers the revocation pipeline by calling the revocation provider on virtual user deactivation (admin-initiated or janitor-triggered).

13. Normative CADF Auditing Trail Specifications

Every rule mutation, API declarative replacement, or administrative override emits a normative Cloud Auditing Data Federation (CADF) event format log into the system notifier bus. The following event types are emitted:

Example: Control Event (Mapping Mutation)

{
  "id": "cadf-uuid-v4-event-id",
  "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
  "eventType": "control",
  "eventTime": "2026-06-11T14:17:16Z",
  "action": "update/identity/mapping",
  "outcome": "success",
  "initiator": {
    "id": "usr_uuid_of_admin_initiator",
    "typeURI": "data/security/user",
    "name": "cloud-admin-operator",
    "domain_id": "default"
  },
  "target": {
    "id": "7c8d9e0f-1a2b-3c4d-5e6f-7a8b9c0d1e2f",
    "typeURI": "data/security/mapping",
    "name": "spiffe-internal"
  },
  "observer": {
    "id": "keystone-rs-raft-cluster-node-01",
    "typeURI": "service/compute/identity"
  },
  "attachments": [
    {
      "name": "mutation_delta",
      "contentType": "application/json",
      "content": {
        "operation": "insert",
        "rule_name": "nova-to-cinder",
        "is_system_applied": true
      }
    }
  ]
}

Example: Access Event (RulesetVersionMismatch)

{
  "id": "cadf-uuid-v4-event-id",
  "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
  "eventType": "access",
  "eventTime": "2026-06-11T14:20:00Z",
  "action": "read/identity/token/verify",
  "outcome": "failure",
  "initiator": {
    "id": "virtual-user-uuid",
    "typeURI": "data/security/virtual-user",
    "name": "svc-nova-compute"
  },
  "target": {
    "id": "7c8d9e0f-1a2b-3c4d-5e6f-7a8b9c0d1e2f",
    "typeURI": "data/security/mapping",
    "name": "spiffe-internal"
  },
  "attachments": [
    {
      "name": "reason",
      "contentType": "application/json",
      "content": {
        "error": "RulesetVersionMismatch",
        "shadow_version": 12345678901234567890,
        "live_version": 12345678901234567891
      }
    }
  ]
}

Example: Maintenance Event (Janitor Deactivation)

{
  "id": "cadf-uuid-v4-event-id",
  "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
  "eventType": "maintenance",
  "eventTime": "2026-06-11T03:00:00Z",
  "action": "disable/identity/virtual-user/janitor",
  "outcome": "success",
  "initiator": {
    "id": "janitor-task",
    "typeURI": "data/system/task"
  },
  "target": {
    "id": "virtual-user-uuid",
    "typeURI": "data/security/virtual-user",
    "name": "svc-decommissioned-daemon"
  },
  "attachments": [
    {
      "name": "reason",
      "contentType": "application/json",
      "content": {
        "last_authenticated_days_ago": 124,
        "is_system": false
      }
    }
  ]
}

Example: Maintenance Event (Archive Cleanup Deletion)

{
  "id": "cadf-uuid-v4-event-id",
  "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
  "eventType": "maintenance",
  "eventTime": "2026-06-11T03:00:00Z",
  "action": "delete/identity/virtual-user/archive-cleanup",
  "outcome": "success",
  "initiator": {
    "id": "archive-cleanup-task",
    "typeURI": "data/system/task"
  },
  "target": {
    "id": "virtual-user-uuid",
    "typeURI": "data/security/virtual-user",
    "name": "svc-decommissioned-daemon"
  },
  "attachments": [
    {
      "name": "reason",
      "contentType": "application/json",
      "content": {
        "deactivated_days_ago": 378,
        "last_authenticated_at": 1718000000,
        "is_system": false,
        "mapping_id": "7c8d9e0f-1a2b-3c4d-5e6f-7a8b9c0d1e2f",
        "resolved_user_name": "svc-decommissioned-daemon"
      }
    }
  ]
}

Example: Control Event (Virtual User Disable)

{
  "id": "cadf-uuid-v4-event-id",
  "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
  "eventType": "control",
  "eventTime": "2026-06-11T15:00:00Z",
  "action": "disable/identity/virtual-user",
  "outcome": "success",
  "initiator": {
    "id": "usr_uuid_of_admin_initiator",
    "typeURI": "data/security/user",
    "name": "cloud-admin-operator",
    "domain_id": "default"
  },
  "target": {
    "id": "virtual-user-uuid",
    "typeURI": "data/security/virtual-user",
    "name": "svc-compromised-agent"
  },
  "attachments": [
    {
      "name": "reason",
      "contentType": "application/json",
      "content": {
        "enabled": false,
        "is_system": false,
        "revocation_triggered": true
      }
    }
  ]
}

Example: Privileged Event (Admin Bypass)

{
  "id": "cadf-uuid-v4-event-id",
  "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
  "eventType": "privileged",
  "eventTime": "2026-06-11T15:00:00Z",
  "action": "create/identity/mapping",
  "outcome": "success",
  "initiator": {
    "id": "usr_uuid_of_admin_initiator",
    "typeURI": "data/security/user",
    "name": "cloud-admin-operator",
    "domain_id": "default"
  },
  "target": {
    "id": "new-mapping-uuid",
    "typeURI": "data/security/mapping",
    "name": "emergency-service-binding"
  },
  "attachments": [
    {
      "name": "privileged_details",
      "contentType": "application/json",
      "content": {
        "bypassed_tier": "authorization_bounds",
        "is_system_granted": true,
        "reason": "emergency_service_repair"
      }
    }
  ]
}

11. Migration Strategy

11.1. Federation Provider Field Translation

11.2. Claim Flattening Per Provider (Ingress Adapter Contract)

Size constraints. All ingress adapters must enforce the following limits:

• Per-claim value: max 4096 bytes (excess silently dropped)
• Total claims map: max 64 KiB (excess rejected with 413 Payload Too Large)

12. Implementation Plan

Implementation proceeds in five sequential phases, each deliverable, testable, and independently verifiable before advancing.

Phase 1: Mapping Provider & Raft Driver

Foundational layer to store, retrieve, and replicate MappingRuleSet objects and VirtualUser shadow records across the Raft cluster. The mapping provider owns the keyspace prefix data:mapping:v1: and the index index:mapping_id:.

• Implement MappingProvider trait exposing create, get, update, delete, enable, disable, and list operations against FjallDB.
• Implement the Raft driver for mapping mutations: serialize MappingRuleSet payloads, enforce Raft consensus ordering, and handle snapshot/compaction.
• Implement write-time validation pipeline: regex ReDoS safety, rule name uniqueness, template safety, allowed_domains intersection checks.
• Implement content-aware ruleset_version (SHA-256 first-16-bytes hasher).
• Implement virtual user enable/disable with CAS-based toggle.
• Deliverable: Cluster-internal API for mapping CRUD and virtual user lifecycle, verifiable via unit and integration tests.

Phase 2: Mapping CRUD & Evaluation API

HTTP API and engine integration layer. Operators create and manage rulesets; the evaluation engine exposes match testing utilities.

• Implement PUT, GET, DELETE, and POST endpoints under /v4/mappings/ (§9).
• Implement POST /v4/mappings/{mapping_id}/rules/mutate for imperative rule adjustments with relative anchoring.
• Wire MappingEngine into the ingestion pipeline: ingest flattened claims, iterate ruleset, produce MatchResult.
• Deliverable: Fully functional mapping API with engine evaluation, no upstream consumers yet.

Phase 3: SPIFFE Provider Migration

First upstream consumer migration. SPIFFE is lowest-risk: its rulesets map to Fixed domain resolution, deterministic SPIFFE ID claims, and static identity bindings.

• Rewrite SpiffeTrustResource authenticator to emit flattened claims, invoke the mapping engine, and consume MatchResult.
• Create SPIFFE rulesets via the mapping provider for existing trust domain configurations.
• Enable shadow registry upsert flow for SPIFFE principals.
• Deprecate SPIFFE bindings concept; route all SPIFFE logins through the unified engine.
• Deliverable: SPIFFE SVID authentication fully mediated by mapping engine; control-plane is_system principals issued via shadow registry.

Phase 4: Kubernetes Auth Provider Migration

Migrate the K8s TokenReview authenticator to the mapping engine.

• Rewrite K8sClusterResource authenticator to flatten TokenReview claims and invoke the mapping engine.
• Create K8s rulesets via the mapping provider, demonstrating nested match criteria and AllOfStrict guards.
• Enable shadow registry upsert for K8s service account principals.
• Deprecate legacy K8s_auth role.
• Deliverable: Kubernetes TokenReview authentication fully mediated by mapping engine.

Phase 5: Federation Provider Migration

Status: Complete. All steps from Phase 5 delivered.

Final and broadest migration. Existing federation providers (OIDC, JWT) carry the most complex claim profiles and legacy token_restriction patterns.

• Rewrite OidcProviderResource authenticator to invoke the mapping engine.
• DONE: flatten_federation_claims() helper with per-claim (4096 bytes) and total map (64 KiB) size caps in crates/keystone/src/federation/api/common.rs.
• DONE: OIDC callback handler rewritten to use flatten_federation_claims → MappingAuthRequest → authenticate_by_mapping pattern in crates/keystone/src/federation/api/oidc.rs (replaced 145+ lines of manual FederationBuilder/UserCreateBuilder/list_groups/create_group/set_user_groups_expiring CRUD).
• DONE: JWT login handler integrated with mapping engine: verifies JWT claims, flattens claims, delegates to authenticate_by_mapping (crates/keystone/src/federation/api/jwt.rs).
• DONE: Legacy Mapping migration utility (mapping_to_ruleset_create) in crates/core/src/mapping/migration.rs using field translation table (§11.1).
• DONE: ClaimsOrMapping and ClaimsOnly domain resolution modes enabled for Federation scenarios (engine handles all modes uniformly for all IdentitySource variants).
• DONE: IdentityMode::Local path in authenticate_local (crates/core/src/mapping/service.rs) with find_or_create_federated_user and sync_user_groups (group membership sync on every login).
• Remove token_restriction payload generation for federated principals; all scoping is handled natively by Authorization fields in MatchResult.
• DONE: token_restriction_id field removed from Mapping, MappingUpdate, and related SQL drivers (federation-driver-sql). The OIDC flow now uses Authorization::Project from the mapping engine for scope.
• Deprecate legacy federation mapping code path.
• DONE: Legacy federation mapping CRUD removed. The /v4/federation/mappings/ API endpoints, controller, service methods, and policy files have been eliminated. Mapping, MappingListParameters, MappingUpdate types removed from core-types and api-types. mapping_id removed from AuthState core type and FederatedAuthState DB entity. MappingNotFound, MappingTokenProjectDomainUnset, MappingTokenUserDomainUnset error variants removed from FederationProviderError. oidc_scopes moved from legacy Mapping into IdentityProvider core and API types. Auth flow now requires default_mapping_name to be set on the IDP; no longer accepts mapping_id or mapping_name override in IdentityProviderAuthRequest. Database tables federated_mapping and mapping are retained for backward compatibility but no longer managed by active CRUD code.
• DONE: Allowed redirect URIs migrated to IdentityProvider. The allowed_redirect_uris field was moved from the legacy Mapping to the IdentityProvider core/API types, SQL driver, and migration. Redirect URI validation at auth-init rejects URIs not in the allowlist when set, providing OIDC redirect_uri authorization code interception protection. Empty or unset list means no restriction (backward compatible). Replaces the previous mapping.allowed_redirect_uris path.
• Deliverable: All federation authentication fully mediated by the unified mapping engine; legacy token_restriction pattern eliminated.

13. Implementation Deviations from ADR Spec

This section documents decisions made during implementation that deviate from the original specification.

D1. MappingRuleSet — provider_id field removed

The MappingRuleSet struct does not carry a separate provider_id field. The ingress provider instance is identified through source: IdentitySource, which contains the relevant anchor (idp_id, cluster_id, or trust_domain) as its enum variant payload. The provider_id slug used in keyspace coordinates is derived from the source field at storage time.

D2. DomainResolutionMode — allowed_domains consolidated into enum variants

The allowed_domains whitelist was moved from a separate field on MappingRuleSet into the ClaimsOrMapping and ClaimsOnly enum variants of DomainResolutionMode. This encodes the constraint “must be non-empty for ClaimsOnly/ClaimsOrMapping, must be empty for Fixed” into Rust’s type system, eliminating cross-field runtime validation.

D3. ResolvedGroupBinding replaced with GroupRef

The custom ResolvedGroupBinding struct was replaced with GroupRef (defined in crate::identity::group), mirroring the existing RoleRef pattern. The strategy field from the original ResolvedGroupBinding was dropped — group resolution strategy (CreateOrGet/Get) is encoded in GroupAssignment within the live ruleset, which the engine fetches during verification. The persisted shadow record only needs the group anchor (id + domain_id + name).

D4. MappingRuleSetUpdate — mode variant is immutable

The MappingRuleSetUpdate type carries allowed_domains as a separate Option<Vec<String>> field rather than replacing the entire DomainResolutionMode. The service layer merges the new allowed_domains into the existing variant, preventing an operator from changing Fixed → ClaimsOrMapping (or vice versa) via update. The resolution mode variant itself is immutable after creation.

D5. is_system: bool — defaults to false

The is_system field on IdentityBinding is typed as bool (not Option<bool>) with a serde(default) attribute that resolves missing JSON to false. This removes ambiguity — an omitted field means the operator did not grant system privileges.

D6. GroupStrategy::CreateOrGet — default for GroupAssignment

The strategy field on GroupAssignment defaults to CreateOrGet rather than requiring explicit specification, as it is the more permissive operator-friendly default (fewer failures when groups are not pre-provisioned).

D7. MappingRule — provider_id not present

MappingRule does not carry provider_id. It is nested within MappingRuleSet, which identifies the provider through source: IdentitySource. All rule-level context is inherited from the parent ruleset.

D8. Virtual user lifecycle — deactivation preferred over deletion

The janitor task sets enabled: false instead of deleting records. A separate archive cleanup task permanently removes deactivated records after a configurable retention period (default: 365 days, configurable via [keystone] shadow_registry_archive_retention_days). This preserves forensic evidence (identity bindings, authorization snapshots, activity timestamps) for incident response and compliance auditing. The original spec specified immediate deletion. The provider interface is extended with explicit enable_virtual_user and disable_virtual_user methods. The mapping provider calls the revocation provider upon virtual user deactivation to trigger the token revocation pipeline.

D9. GroupAssignment.group_id — String changed to Option<String>

The group_id field on GroupAssignment is Option<String> rather than String. For Local identity mode, groups are resolved by name at runtime via group_name interpolation, and no pre-assigned group_id is required. Operations like find_or_create_federated_user and sync_user_groups resolve groups by GroupRef.name at authentication time. For Ephemeral identity mode, GroupAssignment.group_id should be specified to prevent group naming collisions.

D10. Legacy Mapping migration utility — mapping_to_ruleset_create

The migration utility (mapping_to_ruleset_create in crates/core/src/mapping/migration.rs) performs the field translation described in §11.1, with the following deviations:

• bound_audiences is converted to a single ClaimCondition::AnyOf on the "aud" claim, rather than individual equality checks per audience.
• groups_claim generates a single GroupAssignment with group_id: None and strategy: CreateOrGet, deferring group resolution to authentication time.
• token_project_id generates an Authorization::Project with an empty roles vector — the operator must populate roles post-migration.
• bound_claims entries are added as individual ClaimCondition::Equals conditions within an AllOf criteria, preserving their original semantics.
• r#type (Oidc/Jwt) is dropped since IdentitySource::Federation covers both flows.
• token_restriction_id is dropped as obsolete; scoping is now handled by Authorization fields.
• allowed_redirect_uris and oidc_scopes are dropped as provider-level configuration (not part of ruleset semantics).
• The generated rule name is "legacy-mapping-rule" with a description referencing the original mapping name and type.

D11. MatchResult.identity_mode — new field

The MatchResult struct carries identity_mode: Option<IdentityMode> (default None), which propagates IdentityBinding.identity_mode from the matched rule. When None, defaults to Ephemeral for all sources to maintain backward compatibility with existing mapping rules. This field determines whether the authentication flow should use real user CRUD (Local) or the virtual shadow registry (Ephemeral).

21. Stateless API-Key Ingress & Ephemeral Security Contexts for SCIM

Date: 2026-06-12

Last-revised: 2026-07-02 (implementation-status review)

Status

Accepted

Implementation-status review 2026-07-02: implementation confirmed complete against every §7 Security Invariant and §5/§6 requirement (see §8). Status moved from Proposed to Accepted. §6.B’s subtle-crate wording corrected to match what’s actually implemented (Argon2’s own constant-time comparison); this was a doc/code drift, not a security gap. Added janitor and full-pipeline integration test coverage (previously mock/unit-test only) per §8.

Security review 2026-06-24:

• F1 MEDIUM: empty authorization list now fails authentication instead of producing an unscoped context (§4 + Invariant 1);
• F2 MEDIUM: §3 Step 2 XFF algorithm aligned with §6.E to prevent IP-allowlist bypass via leftmost-take (Invariant 4);
• F3 LOW: allowed_ips: None semantics specified as “no restriction” (Invariant 5);
• F4 LOW: compute_deterministic_user_id input contract specified as client_id-derived (Invariant 6). New §7 Security Invariants section added.

Context

Machine-to-machine SCIM provisioning integrations utilizing static bearer tokens, bypassing standard Fernet token lifecycle requirements.

Reference

Extends ADR 0017 (Security Context) and ADR 0020 (Unified Mapping Engine).

1. Context & Motivation

Enterprise Identity Providers (IdPs) utilizing the System for Cross-domain Identity Management (SCIM) protocol generally require a static, long-lived “Secret Token” passed directly to target API endpoints via the Authorization: Bearer <Token> HTTP header.

To support SCIM seamlessly without requiring a prior credential exchange at /v3/auth/tokens, keystone-rs utilizes a specialized ingress adapter capable of intercepting API keys, verifying them, and constructing a fully valid SecurityContext in a zero-roundtrip, stateless execution path, compliant with the Unified Mapping Engine (ADR 0020).

2. Ownership & Storage Model

API Keys are Domain-Owned Machine Identities, strictly decoupled from human user accounts.

A. Keyspace Configuration

B. Resource Data Structure

To prevent secret leakage in application error logs, the struct implements a custom Debug trait that explicitly replaces secret_hash with [REDACTED]. All timestamps are stored as UTC Epoch seconds.

#![allow(unused)]
fn main() {
#[derive(Clone, Serialize, Deserialize)]
pub struct ApiClientResource {
    pub domain_id: String,
    pub provider_id: String,
    pub client_id: String,             // Public UUID for management API references
    pub lookup_hash: String,           // Fast SHA-256 hash of the token for O(1) DB index lookups
    pub secret_hash: String,           // PHC format Argon2id hash (e.g., $argon2id$v=19$m=65536$...)
    pub allowed_ips: Option<Vec<String>>,  // None = no IP restriction (any source IP accepted)
    pub description: Option<String>,
    pub enabled: bool,
    pub created_at: i64,               // UTC Epoch seconds
    pub expires_at: i64,               // Mandatory TTL (UTC Epoch seconds)
    pub last_used_at: Option<i64>,     // UTC Epoch seconds
    pub revoked_at: Option<i64>,       // Tombstone for audit retention (UTC Epoch seconds)
    pub revoked_by: Option<String>,    // User ID of the revoking operator
}

}

C. Token Generation & Opaque Formatting

The token consists of a prefix, high-entropy random data, and a CRC32 checksum for fast format validation. Format: kscim_{32_bytes_base62_entropy}_{crc32}

When the token is generated:

1. lookup_hash is computed as SHA-256(entropy) to serve as the database index.
2. secret_hash is computed as Argon2id(entropy) for cryptographic verification.
3. The client_id (a standard UUID) is returned to the administrator for CRUD operations but is never embedded in the token or HTTP headers.

3. Execution Flow: The Ingress Pipeline

The middleware processes incoming SCIM requests through a strict, short-circuiting pipeline.

Step 1: Format Check & Hash-Based Rate Limiting

1. The middleware computes the CRC32 of the entropy. If it does not match the appended checksum, the request is dropped immediately. (Note: This is strictly a cheap format validity check to reject malformed data, not a cryptographic security boundary).
2. The middleware computes the fast SHA-256(entropy) to derive the lookup_hash.
3. Rate Limiting: A sliding-window token bucket enforces a strict rate limit keyed on the lookup_hash. If the request fails the CRC32 check (meaning no valid entropy exists), the rate limiter falls back to keying on the source IP to absorb brute-force garbage traffic. This ensures legitimate SCIM traffic originating from shared enterprise egress IPs (like Entra ID NAT gateways) is not inadvertently blocked by other tenants.

Step 2: Database Lookup & IP Whitelisting

1. The middleware queries FjallDB for data:api_client:v1:<domain_id>:<lookup_hash>.
2. It verifies enabled: true and current_utc_seconds < expires_at.
3. It determines the effective client IP using the rightmost non-trusted-proxy IP algorithm: append the raw TCP peer address to the right of the configured trusted_header chain (X-Forwarded-For by default, or RFC 7239 Forwarded by explicit opt-in), then walk right-to-left, returning the first address that is not in the statically configured trusted_proxies CIDR array. Exactly one header is trusted because each proxy must be configured to strip client-supplied values for that header. If the raw TCP peer is not in trusted_proxies, it is used directly (the headers are not consulted). This prevents leftmost-entry spoofing through untrusted intermediate hops. The resulting effective IP is then validated against allowed_ips CIDR blocks. If allowed_ips is None, the IP check is skipped (no restriction applies).

Step 3: Cryptographic Verification & Lazy Re-Hash

The entropy is verified against the PHC-formatted secret_hash using tokio::task::spawn_blocking.

• Lazy Re-Hash: If the hash verifies successfully but the PHC string parameters (e.g., memory cost) are lower than the currently configured global minimums, the engine enqueues an asynchronous task to re-hash and update the database record.
• last_used_at is updated asynchronously.

Step 4: Ephemeral Context Hydration (Anti-Bleed Scoping)

To prevent cross-domain privilege bleeding, an Ephemeral Security Context must operate under exactly one scope. API Keys are domain-owned machine identities (§2); by design only a domain-scoped authorization is accepted. This is an allowlist – Authorization::Domain is the sole accepted variant – rather than a denylist naming each forbidden authorization type, so it also covers any authorization type added in the future.

#![allow(unused)]
fn main() {
pub async fn hydrate_ephemeral_context(...) -> Result<ValidatedSecurityContext, AuthenticationError> {
    // Derived exclusively from client_id (the unique key UUID), never from
    // provider_id, so each API key produces a distinct audit identity even
    // during N:1 rotation periods (§5.D).
    let user_id = compute_deterministic_user_id(resource.client_id);

    // Initialize strictly as Unscoped.
    let mut ctx = SecurityContext::new_ephemeral(IdentityInfo::Principal(PrincipalInfo { user_id }), ScopeInfo::Unscoped);

    // Invariant: a key whose UME mapping resolves to zero authorizations MUST
    // fail authentication. Returning an unscoped/role-less context would push
    // the access decision entirely onto downstream OPA policy coverage.
    if match_result.authorizations.is_empty() {
        return Err(AuthenticationError::NoAuthorizationsFound);
    }

    // Enforce Single-Scope Constraint
    if match_result.authorizations.len() > 1 {
        return Err(AuthenticationError::MultipleScopesForbidden);
    }

    validate_target_entities_are_active(state, &match_result.authorizations)?;

    let authorization = &match_result.authorizations[0];

    // System scopes are strictly forbidden for API-Key ingress.
    if matches!(authorization, Authorization::System { .. }) {
        return Err(AuthenticationError::SystemScopeForbiddenForApiKey);
    }

    // Allowlist: only a domain-scoped authorization is accepted. API Keys
    // are domain-owned machine identities (§2), so anything else --
    // `Authorization::Project` included -- is rejected here rather than
    // being enumerated as its own forbidden case.
    let Authorization::Domain { domain_id, roles } = authorization else {
        return Err(AuthenticationError::NonDomainScopeForbiddenForApiKey);
    };
    ctx.set_scope(ScopeInfo::Domain(domain_id.clone()));

    let mut effective_roles = roles.clone();
    effective_roles.sort();
    effective_roles.dedup();
    Ok(ValidatedSecurityContext::finalize(ctx, effective_roles))
}

}

4. Routing & Boundary Enforcement

TLS 1.3 Minimum Floor: Long-lived bearer tokens are functionally equivalent to passwords. Enforcement is delegated to the infrastructure Reverse Proxy (Nginx/HAProxy), which is strictly configured to require TLS 1.3 for all traffic terminating at the /SCIM/v2 paths.

Sub-Router Isolation: The API-Key middleware is mounted exclusively on the SCIM sub-router. Core OpenStack infrastructure endpoints utilize the standard Fernet middleware and will reject API keys outright.

5. Administrative CRUD, Auditing & OPA Policies

A. Privilege Matrix & OPA Integration

Management of API keys relies strictly on defined OPA policies per ADR-0002.

• identity:api_key:create
• identity:api_key:list
• identity:api_key:update
• identity:api_key:revoke
• identity:api_key:simulate_access

The DomainManager Role: These policies require the DomainManager role (or SystemAdmin). DomainManager represents an explicit administrative capability scoped strictly to managing identities and integrations within a domain. DomainAdmin must not be used, as it provides overarching infrastructure privileges. The DomainManager role and its associated policy mappings must be formally ratified in an upcoming revision to ADR 0002 (OpenStack Policy Engine Integration) to ensure central governance of the RBAC hierarchy.

Implementation status: The five policies above are implemented under policy/identity/api_key/ (create.rego, list.rego, update.rego, revoke.rego, simulate_access.rego; a sixth, show.rego, gates a GET /v4/api-keys/{client_id} endpoint not explicitly enumerated by this section but added for consistency with every other v4 resource), each requiring the pre-existing manager role scoped to the key’s own domain (this codebase’s realization of DomainManager, used identically by identity.user.* and identity.mapping.ruleset.*), or admin/is_admin (SystemAdmin). Unlike identity.user.list, there is no reader carve-out on identity:api_key:list — all actions sit at the same privilege bar.

ADR 0024’s SCIM resource-CRUD policies (identity/scim/user/*) reuse this same manager/admin string convention, but resolve it through a materially different path: those requests are authenticated by an API key, which carries no Role/RoleAssignment at all, so the manager/admin/scim_provisioner strings they check are produced entirely by the realm’s own MappingRuleSet output (§3 Step 4 above), never by a RoleAssignment row. The admin CRUD surface for API keys/realms documented in this section is the opposite case — invoked by a Fernet-authenticated human operator, where manager/admin can be a real RBAC grant.

This does not by itself constitute the formal ADR 0002 ratification called for above (see §8).

B. CRUD Endpoints

• POST /v4/api-keys: Generates a new key.
• GET /v4/api-keys: Lists metadata.
• PUT /v4/api-keys/{client_id}: Updates configurations.

C. Revocation & Incident Response

• POST /v4/api-keys/{client_id}/revoke: Emergency Revocation Path. Sets enabled: false, stamps revoked_at and revoked_by, and emits a CADF event (action: revoke). It does not perform a hard delete. This preserves the cryptographic footprint (lookup_hash) and metadata for incident response audits. Physical storage reclamation is deferred to the janitor after the organization’s audit retention period.
• Revocation is irreversible via PUT. ApiKeyApi::update MUST reject (409 Conflict) any patch that sets enabled: true on a key whose revoked_at is set. Without this, an emergency revocation could be undone by an ordinary configuration update, defeating its purpose as an incident-response control (see Invariant 9, §7).

D. Zero-Downtime Key Rotation (N:1 Provider Mapping)

Because the ApiClientResource is indexed by its cryptographic lookup_hash rather than the provider slug, the architecture natively supports an N:1 relationship between API keys and a provider_id. Multiple distinct keys can safely declare the same provider_id without database collision.

To execute a zero-downtime rotation, operators generate a new Key B bound to the existing provider_id. Both Key A and Key B will independently resolve against the exact same ADR 0020 mapping ruleset (data:mapping:v1:<domain_id>:<provider_id>). The IdP is updated with Key B, traffic migrates seamlessly, and Key A is subsequently revoked.

E. The Dry-Run Auditing Endpoint

• Endpoint: POST /v4/api-keys/simulate-access
• Payload: {"client_id": "<uuid>"} (Shifted to the body to prevent client_id leakage in proxy access logs).
• Authentication Required: Strictly requires DomainManager or SystemAdmin credentials via a valid Fernet token.
• Behavior: Performs a mock authentication pass, returning a fully resolved JSON matrix detailing the API key’s current authorization topology.

6. Threat Model & Required Mitigations

A. Targeted Credential Stuffing & DoS

• Measures: Rate limiting is enforced via a token bucket keyed primarily on lookup_hash to protect legitimate shared IdP egress IPs. Argon2id verification is constrained to a bounded spawn_blocking pool that sheds load (503 Service Unavailable) if saturated.

B. Argon2id Parameters & Timing Side-Channels

• Measures: Argon2id parameters are globally defined in keystone.conf with OWASP-compliant strict minimums (e.g., $m=65536, t=3, p=4$). Dummy hashes for invalid tokens utilize these exact parameters. Comparisons against the PHC strings are constant-time, performed internally by the argon2 crate’s verify_password (not a separate manual subtle::ConstantTimeEq step – verify_password already provides this property, so a second comparison would be redundant).

C. Write-Time is_system Prohibition

• Measures: Allowing an API Key to hold ScopeInfo::System is highly dangerous. To prevent silent failures during auth-time, the prohibition is shifted to rule creation. If a DomainManager attempts to create or update a mapping rule where the provider_id belongs to an IdentitySource::ApiClient, and any authorization grants is_system: true or Authorization::System, the Mapping Engine CRUD API rejects it immediately with 422 Unprocessable Entity. The same write-time guard also enforces a domain-scope-only allowlist for IdentitySource::ApiClient rulesets: once is_system/Authorization::System is excluded, every remaining authorization MUST be Authorization::Domain – API Keys are domain-owned machine identities (§2) – so Authorization::Project (or any other non-domain authorization) is rejected the same way.

D. OPSEC Leakage & Log Injection

• Measures: 1. The API-key routing prefix allows explicit integration with DLP secret scanners (e.g., GitHub Advanced Security).

2. The client_id is excluded from the token format, neutralizing capability oracle attacks.
3. The Axum middleware actively scrubs the Authorization header from all internal application traces.

E. Forwarding-Header Spoofing

• Measures: IP allowlisting uses the rightmost non-trusted-proxy IP algorithm (§3 Step 2): the raw TCP peer is appended to the right of the configured trusted_header chain (X-Forwarded-For by default; RFC 7239 Forwarded requires explicit opt-in), then the chain is walked right-to-left and the first address not in trusted_proxies is used as the effective client IP. If the TCP peer is untrusted, the header is not consulted at all. Trusting one explicitly selected header prevents a client-forged alternative header from overriding the chain actually sanitized by the proxy. This also prevents an attacker from bypassing allowed_ips by routing through an untrusted intermediate proxy that prepends a spoofed originating IP to the forwarding chain before a trusted proxy. allowed_ips: None means no IP restriction is applied.

F. Janitor Disablement, Asynchronous Drift & Physical Reclamation

• Finding: Asynchronous last_used_at writes may occasionally drop under heavy system pressure, causing active integrations to drift toward the 90-day PCI-DSS janitor threshold. Furthermore, tombstoned records from revocations accumulate indefinitely.
• Measures: 1. Drift Absorption: The system documents a maximum acceptable async write staleness of 24 hours. The janitor operates with a 7-day grace period beyond the 90-day threshold, mathematically absorbing this write-failure window. Before executing a disablement, the janitor emits a CADF event (action: disable_inactive) and pushes an administrative alert payload to the system notification bus.

2. Physical Reclamation: To prevent unbounded keyspace bloat, the janitor executes a secondary garbage-collection phase. Any ApiClientResource containing a revoked_at timestamp older than 365 days is permanently purged from FjallDB.

3. Per-key fault isolation: a single key failing its disablement or purge (e.g. a storage CAS conflict with a concurrent admin update) MUST NOT prevent the rest of the sweep pass from running. Failures are counted and logged, and retried on the next pass.

7. Security Invariants

The following invariants MUST hold at all times. Any implementation deviation is a security defect.

1. No-authorizations → authentication failure. hydrate_ephemeral_context MUST return Err(AuthenticationError::NoAuthorizationsFound) when the UME resolves zero authorizations for a key. It MUST NOT produce a ValidatedSecurityContext with ScopeInfo::Unscoped and an empty role set.

2. Single-scope enforcement. A key MUST NOT authenticate if its UME mapping resolves to more than one authorization entry (MultipleScopesForbidden).

3. System scope prohibited at ingress. An Authorization::System match in hydrate_ephemeral_context MUST return SystemScopeForbiddenForApiKey. The companion write-time prohibition (§6.C) is defense-in-depth, not a substitute for this runtime check.

3a. Domain scope only, by allowlist. API Keys are domain-owned machine identities (§2). Once the Invariant 3 system-scope check passes, hydrate_ephemeral_context MUST accept only Authorization::Domain; any other authorization (Authorization::Project included) MUST return NonDomainScopeForbiddenForApiKey rather than resolving a non-domain ScopeInfo. This is an allowlist keyed on the accepted variant, not a denylist enumerating each forbidden one, so it also covers any authorization type added in the future. The companion write-time prohibition (§6.C) is defense-in-depth, not a substitute for this runtime check.

4. XFF rightmost-non-trusted algorithm. Effective client IP MUST be the rightmost address in the XFF chain (with TCP peer appended) that is not in trusted_proxies. Implementations MUST NOT use XFF[0] (leftmost) as the effective IP under any trusted-proxy configuration.

5. allowed_ips: None means unrestricted, not deny-all. When allowed_ips is absent from an ApiClientResource, the IP check MUST be skipped entirely. Implementations MUST treat a missing field and Some([]) identically (no restriction).

6. compute_deterministic_user_id MUST be derived from client_id. The ephemeral user_id is computed from the key’s unique client_id UUID, not from provider_id. Two distinct keys sharing a provider_id MUST produce different user_ids so their audit records are not conflated.

7. Dummy-hash timing parity. When no ApiClientResource is found for a given lookup_hash, a full Argon2id dummy computation using current global parameters MUST be performed before returning a failure response, preventing timing-based enumeration of valid lookup hashes.

8. Argon2id minimum parameters enforced. The parameters embedded in any stored PHC string MUST be validated against configured minimums before accepting a verification as sufficient. Parameters below the floor trigger a lazy re-hash regardless of verification outcome.

9. Revocation is irreversible via the update surface. ApiKeyApi::update MUST reject with a conflict error any patch that would set enabled: true on an ApiClientResource whose revoked_at is Some. A revoked key MUST NOT become authenticatable again through PUT /v4/api-keys/{client_id}; the only way back into service is administratively creating a new key (§5.D covers zero-downtime rotation for exactly this case).

8. Implementation Status

• Done:

  • The SCIM ingress authentication pipeline (§3), including all security invariants (§7).

  • The write-time is_system prohibition (§6.C), plus the domain-scope-only allowlist (Invariant 3a): both hydrate_ephemeral_context (crates/core/src/api/api_key_auth.rs) and the write-time mapping validation (crates/core/src/mapping/validation.rs) accept only Authorization::Domain for IdentitySource::ApiClient rulesets, since API Keys are domain-owned machine identities (§2). Authorization::Project is rejected as a consequence of the allowlist, not as a named special case. The simulate-access dry-run endpoint (§5.E) mirrors this and reports matched: false for any non-domain match.

  • The storage layer and internal ApiKeyApi/ApiKeyBackend traits (§2, §5.D) with a Raft-backed implementation, including the janitor’s cross-domain list_all and hard-delete purge operations (§6.F).

  • Rate limiting (§6.A).

  • The OPA policies for §5.A (policy/identity/api_key/), plus a show policy for the GET /v4/api-keys/{client_id} endpoint this section does not explicitly enumerate.

  • The /v4/api-keys* HTTP admin surface (§5.B): create, list, show, update. update rejects (409 Conflict) re-enabling a revoked key (Invariant 9, §5.C), enforced in ApiKeyService::update (crates/core/src/api_key/service.rs) so it holds for every caller, not just the HTTP layer.

  • The revoke endpoint (§5.C), including a CADF audit event (action: revoke).

  • The dry-run simulate-access endpoint (§5.E). Deviates from the literal request shape in one way: the payload also carries domain_id alongside client_id, because this implementation’s storage partitions ApiClientResource by domain (§2.A), making a client_id-only lookup impossible without it. The same constraint applies to show/update/revoke, which take domain_id as a query parameter rather than encoding it in the (flat, ADR-specified) URL path. It also does not call MappingApi::authenticate_by_mapping – that path may provision a real user row for IdentityMode::Local rules, an unacceptable side effect for a dry-run endpoint – and instead evaluates the ruleset and reads the matched Authorization’s roles directly.

  • The janitor (§6.F): an in-process, leader-gated tokio::time::interval sweep (mirroring the storage crate’s existing emergency-rotation confirmation-timeout sweeper in crates/storage/src/app.rs) that disables keys inactive beyond janitor_inactive_days + janitor_grace_days, purges tombstones older than janitor_tombstone_retention_days, and emits a CADF event (action: disable_inactive) per disablement. Per-key failures are isolated (§6.F.3): one key’s disablement/purge error is logged and counted in JanitorReport::errors, not propagated, so it cannot stall the rest of the pass.

    Action strings are intentionally more specific than this ADR’s earlier control/maintenance category wording (revoke/disable_inactive rather than a repeated generic label) – more useful for audit filtering; the wording above has been reconciled to match the code rather than the other way around.

  • Integration test coverage exercising the real Raft-backed provider and a live HTTP router, not just mocks: a janitor sweep suite (tests/integration/src/api_key/janitor.rs) covering disablement, tombstone purge, and cross-domain sweeping against the real storage/CAS layer; and a full-pipeline suite (tests/integration/src/api_key/ingress.rs) driving real requests through openstack_keystone::scim::router() – successful end-to-end authentication, wrong-secret rejection, an XFF-spoof-through-a-trusted- proxy regression case (Invariant 4), and rate-limit tripping (§6.A).

• Known gap: the ADR’s “pushes an administrative alert payload to the system notification bus” (§6.F) is not implemented – no pub/sub or webhook dispatch infrastructure exists in this codebase yet. The janitor emits a structured warn! log and its CADF event (action: disable_inactive) as the closest existing substitutes; a real notification channel is unbuilt follow-up work, not something to improvise here.
• Not yet done: the DomainManager role’s formal ratification in ADR 0002. In the interim, the OPA policies enforce the equivalent scoped privilege using this codebase’s existing manager role (§5.A).

ADR 0022: Handler-Level Rate Limiting via Governor

Date: 2026-06-15

Status

Proposed

Context

The keystone-rs project (Keystone-NG) requires robust rate limiting to protect against brute-force attacks, credential stuffing, and general API resource exhaustion. The project currently utilizes the axum web framework for its HTTP routing and request handling.

While the Rust ecosystem offers several middleware-based rate limiting solutions (e.g., tower layers or axum-specific wrapper crates), evaluating these reveals significant drawbacks:

• Maintenance Decay: Many wrapper crates are poorly maintained, tightly coupling our core security infrastructure to abandoned dependencies.
• Inflexible Abstractions: Middleware layers operate before the request reaches the handler. They struggle to apply limits based on complex, request-specific business logic.
• Static Configuration Limitations: Middleware often expects statically compiled limits, whereas operational environments demand dynamically configurable and toggleable rate limits via external configuration files.
• Legacy Compatibility: Configuration must be parsed from the existing keystone.conf INI file to ensure seamless co-existence with the legacy Python Keystone service (oslo.config).

We need a rate-limiting solution that:

• Uses heavily vetted, actively maintained dependencies (governor directly).
• Allows operators to define quotas or selectively disable limits entirely via the standard INI application configuration file.
• Is explicitly invoked within the axum handlers rather than globally.
• Emits standardized, RFC-compliant HTTP 429 error responses with Retry-After without leaking identifying information in headers.

This ADR complements ADR 0010 (account lockout). ADR 0010 protects against brute-force by locking the user account after N failed attempts with a fixed lockout duration governed by conf.security_compliance. Rate limiting per this ADR operates independently: it throttles requests before they reach password verification, protecting CPU resources and database load regardless of authentication outcome. The two mechanisms are additive — rate limiting fires first, then ADR 0010 lockout fires on repeated failures.

Decision

We will implement rate limiting by utilizing the governor crate directly within our axum handlers. Rate limiter constraints will be parsed from the standard keystone.conf INI configuration file at startup. Handlers will manually evaluate limits, and if a limit is exceeded, they will construct a standardized HTTP 429 Too Many Requests response containing a Retry-After header.

1. Configuration-Driven Limits

Rate limit thresholds (burst capacity and replenishment rates) MUST be defined in the application’s INI configuration file (e.g., keystone.conf). Hardcoding limits in the application binary is strictly prohibited.

Operators must be able to disable specific governors independently to support varying deployment topologies (e.g., disabling the application-level IP limit if an upstream WAF or Load Balancer already enforces it).

Configuration Schema (INI)

To remain compatible with OpenStack’s oslo.config patterns, rate limits will be grouped into distinct sections within the keystone.conf file:

[rate_limit_global_ip]
enabled = true
burst_size = 100
replenish_rate_per_second = 10

[rate_limit_user_auth]
# When false, the governor is not instantiated, and handlers bypass this check
enabled = false
burst_size = 5
replenish_rate_per_second = 1

[rate_limit_trusted_proxies]
# Exactly one header trusted proxies sanitize. The default is x_forwarded_for;
# use forwarded only when every trusted proxy strips client-supplied RFC 7239
# Forwarded values before writing its own.
trusted_header = x_forwarded_for
# Comma-separated list of trusted proxy CIDR ranges.
# Leave empty to use the direct peer address (suitable for non-proxied
# deployments). Example: 10.0.0.0/8,192.168.1.0/24
trusted_proxies =

Configuration Bounds

To prevent operators from silently misconfiguring the limiters into an ineffective state, the config parser MUST enforce the following bounds:

Values outside these ranges MUST cause startup failure identically to the fail-hard rule for replenish_rate_per_second = 0.

State Representation

To support disablement, the injected AppState will wrap the limiters in an Option. If enabled = false is parsed from the INI config, the application state stores None for that specific governor, and the handler safely ignores the check.

If initialization fails (e.g., invalid parameter like replenish_rate_per_second = 0) and the governor is marked enabled = true, the application MUST fail to start. Silent fallback to no-limiting on misconfiguration is a security regression.

#![allow(unused)]
fn main() {
pub type DefaultKeyedRateLimiter<K> = RateLimiter<K, DefaultKeyedStateStore<K>, governor::clock::MonotonicClock>;

pub struct RateLimitState {
    pub global_ip_limiter: Option<Arc<DefaultKeyedRateLimiter<String>>>,
    pub user_auth_limiter: Option<Arc<DefaultKeyedRateLimiter<String>>>,
}
}

2. Error Response Construction

When a rate limit is exceeded, the application must immediately halt request processing and return an HTTP 429 Too Many Requests status code. To allow clients to back off gracefully, the response MUST include specific HTTP headers calculated from governor’s internal state.

Required Headers

• Retry-After: Indicates how many seconds the client must wait before making a new request. This is extracted from governor’s NotUntil error payload, which calculates the difference between the next allowed cell and the current clock time. This is the only rate-limit-specific header required. No additional headers that might leak identifying information (e.g., the key that triggered the limit) should be included, to avoid user enumeration or PII exposure.

Standardized Evaluation Method

To ensure uniform error formatting across all handlers, limit evaluation is abstracted into a generic helper function.

#![allow(unused)]
fn main() {
use axum::{
    http::{HeaderMap, HeaderValue, StatusCode},
    response::{IntoResponse, Response},
};
use governor::{clock::Clock, state::keyed::DefaultKeyedStateStore, RateLimiter};
use std::sync::Arc;

pub type DefaultKeyedRateLimiter<K> =
    RateLimiter<K, DefaultKeyedStateStore<K>, governor::clock::MonotonicClock>;

/// Evaluates a key against a given governor.
/// Constructs a standardized HTTP 429 Response if the limit is exceeded.
pub fn check_rate_limit<K: Clone>(
    limiter: &Arc<DefaultKeyedRateLimiter<K>>,
    key: &K,
) -> Result<(), Response> {
    limiter.check_key(key).map_err(|not_until| {
        // .as_secs() truncates sub-second waits to 0; Retry-After: 0 means
        // "retry immediately" per RFC 7231 and causes a thundering herd.
        let wait_secs = not_until
            .wait_time_from(governor::clock::MonotonicClock::default().now())
            .as_secs()
            .max(1);

        let mut headers = HeaderMap::new();
        headers.insert(
            "Retry-After",
            HeaderValue::from_str(&wait_secs.to_string())
                .unwrap_or_else(|_| HeaderValue::from_static("60")),
        );

        (
            StatusCode::TOO_MANY_REQUESTS,
            headers,
            format!("Rate limit exceeded. Retry in {wait_secs}s"),
        )
            .into_response()
    })
}
}

3. Handler Execution Flow

Handlers execute limit checks based on their specific business logic, gracefully handling cases where a governor has been disabled via the INI configuration.

For unauthenticated endpoints, per-IP rate limiting is the only throttle applied before user lookup. Per-username rate limiting is applied only after the user is confirmed to exist in the database, to prevent key-exhaustion bypass attacks where the attacker crafts an infinite supply of novel usernames, each with its own quota.

The client IP used for per-IP limiting MUST be the originating client address, not the reverse proxy’s address. resolve_client_ip (see Invariant 9) extracts the correct IP by consulting exactly the configured trusted_header only when the direct peer appears in the configured trusted_proxies list.

#![allow(unused)]
fn main() {
pub async fn create_token(
    State(state): State<Arc<AppState>>,
    ConnectInfo(peer): ConnectInfo<SocketAddr>,
    headers: HeaderMap,
    Json(payload): Json<AuthPayload>,
) -> Result<Response, StatusCode> {
    // 1. IP Check (if enabled in keystone.conf)
    // resolve_client_ip checks trusted_proxies config; falls back to peer addr.
    // IPv6 addresses are mapped to their /64 prefix before lookup.
    if let Some(ip_limiter) = &state.rate_limits.global_ip_limiter {
        let client_ip = resolve_client_ip(peer.ip(), &headers, &state.trusted_proxies);
        let key = rate_limit_key_for_ip(client_ip);
        if let Err(rejection_response) = check_rate_limit(ip_limiter, &key) {
            return Ok(rejection_response);
        }
    }

    // 2. Look up the user to verify existence BEFORE applying per-user limits.
    // Using the raw username as a rate-limit key before confirming existence
    // allows an attacker to craft novel usernames, each with independent quota.
    let user = state
        .identity_provider
        .get_user_by_name(&payload.username, &payload.domain)
        .await
        .map_err(|_| StatusCode::UNAUTHORIZED)?;
    // NOTE: In production this maps only to NotFound; other errors propagate as 500.
    // The sample simplifies for brevity.

    // 3. User Account Check (if enabled in keystone.conf)
    // Applied only after user existence is confirmed.
    if let Some(user_limiter) = &state.rate_limits.user_auth_limiter {
        if let Err(rejection_response) = check_rate_limit(user_limiter, &user.id) {
            return Ok(rejection_response);
        }
    }

    // ... Proceed with heavy bcrypt/argon2 hashing and database verification ...
}
}

4. Security Invariants

Any code change violating the following is rejected at review:

1. No Hardcoded Limits: burst_size and replenish_rate MUST originate from the keystone.conf INI configuration state.
2. Fail-Hard Initialization: If a governor is configured with enabled = true but cannot be initialized (e.g., invalid parameters such as replenish_rate_per_second = 0), the application MUST fail to start. Silent fallback to no-limiting is a security regression.
3. Response Uniformity: The HTTP 429 response body and headers MUST be identical regardless of which governor tripped, to prevent additional disclosure channels beyond the status code itself. The 429 response MUST include a correctly calculated Retry-After header. No additional headers that expose the rate-limit key or any identifying information should be present, to prevent user enumeration via rate-limit probing.
4. Pre-Hash Enforcement: Rate limit checks for authentication endpoints MUST be executed before any CPU-intensive operations (e.g., Argon2id/Bcrypt password verification) are triggered.
5. Distinct Buckets: Different operational contexts MUST use physically separate RateLimiter instances in the application state. Multiplexing different entity types into the same keyed store is prohibited.
6. Monotonic Clock: Rate limiters MUST use governor::clock::MonotonicClock to prevent NTP backward shifts from resetting quota windows.
7. Key Normalization: Any key derived from user input (e.g., username) MUST be normalized before being used as a rate-limit key. Normalization MUST apply, in order: NFKC Unicode normalization, case-folding to lowercase, and any additional canonicalization applied by the authentication logic (e.g., stripping a trailing @). Omitting NFKC allows Unicode homoglyph variants (e.g., Cyrillic а vs. Latin a) to create distinct buckets for the same logical identity. The normalization function MUST be a single shared utility used by both the rate limiter and the authentication pipeline.
8. Post-Lookup User Throttle: For unauthenticated endpoints, per-IP rate limiting is the primary throttle applied before any database lookup. Per-user rate limiting MUST only be applied after the user is confirmed to exist. This prevents key-exhaustion bypass attacks where an attacker crafts novel usernames, each with independent quota.
9. Trusted Proxy Source IP: The IP address used as a rate-limit key MUST be the originating client IP, not the address of any intermediate reverse proxy. When trusted_proxies is non-empty, the client IP MUST be extracted from the first untrusted hop in exactly the configured trusted_header (X-Forwarded-For by default; RFC 7239 Forwarded by explicit opt-in), and only when the direct peer IP falls within a configured trusted CIDR range. When trusted_proxies is empty, the direct peer address MUST be used. Accepting both headers implicitly, or accepting a header from an untrusted peer, allows a client to spoof its apparent IP and defeat per-IP limiting.
10. Minimum Retry-After: The Retry-After header value MUST be at least 1 second. Sub-second wait durations MUST be rounded up to 1. A value of 0 is valid per RFC 7231 (“retry immediately”) and causes a thundering herd when many clients receive it simultaneously.

Consequences

• Per-node limits in scaled deployments: governor uses in-memory DefaultKeyedStateStore. Each keystone-rs pod maintains its own counter. In a deployment with N instances behind a load balancer, the effective rate limit is N times the configured value. Operators should either configure limits per-node or migrate to a shared-state store (e.g., Redis-backed) for cluster-wide enforcement in the future.
• IPv6 rate limiting via prefix aggregation: IPv6 privacy extensions randomize source addresses per connection, making raw per-address limiting ineffective. Per-IP rate limiting defaults to aggregating IPv6 addresses by their /64 network prefix, which corresponds to the subnet allocated to a single host’s privacy extensions. Per-/128 precision can be enabled by the operator but is not the default. IPv4 addresses are rate-limited per-/32.
• Memory overhead and store eviction: The keyed state store retains an entry per unique key. Under adversarial conditions (unique-key flooding), entries must be aggressively pruned. The store MUST:
  • Cap at a configurable maximum entry count (default: 10,000).
  • Be trimmed every 60 seconds via a background task using LRU eviction: when the cap is reached, the least-recently-seen entry is evicted to make room for the new key. LRU is preferred over fail-open (dropping new keys) because fail-open degrades rate limiting for all users simultaneously once the store is full — including legitimate users who are not flooding — whereas LRU preserves quotas for actively-authenticating sessions.
  • The global IP limiter remains effective as a backstop regardless of per-user store state.
• Clock source: DefaultClock uses SystemTime, which can be adjusted backward by NTP drift, resetting quota windows. Using MonotonicClock (required by Invariant 6) prevents this. The tradeoff is that MonotonicClock has no notion of wall-clock time, so quota replenishment cannot be aligned to calendar boundaries (e.g., daily resets). For security rate limiting this is desirable — an operator cannot accidentally reset attacker quotas by changing the system clock.
• Trusted proxy configuration gap: Operators deploying behind a reverse proxy who leave trusted_proxies empty will rate-limit by the proxy’s IP instead of the client’s, meaning all traffic shares a single bucket and the global IP limiter trips almost immediately under normal load. Operators MUST configure trusted_proxies to match their load balancer’s CIDR(s) before enabling the global IP limiter in proxied deployments. The startup log SHOULD emit a warning when global_ip_limiter is enabled and trusted_proxies is empty, to surface the misconfiguration early.
• Key length: Extremely long input values (e.g., a 64 KB username) would be inserted into the rate-limit HashMap as-is, consuming memory before any limit is applied. Input length MUST be validated at the API boundary (e.g., reject usernames longer than 255 bytes with HTTP 400) before the rate-limit key is derived. This check belongs to the authentication handler, not to the rate limiter itself.
• Key normalization: Per-user rate-limit keys MUST be normalized before lookup. This means NFKC Unicode normalization, case-folding, stripping trailing @, and any other canonicalization applied by the authentication logic. Without NFKC, Unicode homoglyph variants produce distinct keys for the same logical identity, each with independent quotas. The normalization function MUST be a single shared utility to ensure consistency between the rate limiter and the authentication pipeline.
• User enumeration trade-off: Per-user rate limiting inherently enables user enumeration: if an attacker sends enough requests for alice to trip the limit (429), but requests for alice_nonexistent return 401, the attacker learns that alice is a valid account. This is mitigated by this ADR’s decision to apply per-user limiting only after user lookup (Invariant 8). Since the lookup already occurs, the enumeration risk is acceptable; the IP limit remains the primary throttle for unknown accounts.
• HashMap timing side channel: The DefaultKeyedStateStore performs hash-map lookups per key, which vary in execution time depending on whether the key exists (insert vs. update path). This timing difference can enable user enumeration independently of the 429 status code. This is inherent to the data structure and reinforces the decision to use IP-keyed limiting as the primary throttle for unauthenticated flows, where the timing channel is less actionable (IPs are not secrets).
• Federation and application credential coverage: Rate limiting on POST /v3/auth/tokens does not automatically cover federation endpoints (OS-FEDERATION/identity_providers/{id}/protocols/{protocol}/authenticate, OS-FEDERATION/protocols/{protocol}/authenticate), application credential flows, or token-based operations. These endpoints perform cryptographic verification and are CPU-intensive, making them denial-of-service targets. A follow-up ADR (tracking issue: TBD) will extend handler-level rate limiting to these endpoints with appropriate IP-based governance.
• New dependency: The governor crate must be added to the workspace Cargo.toml. It is actively maintained with no known security advisories.
• Relationship with ADR 0010 (account lockout): Rate limiting provides a first-layer defense by throttling all requests. ADR 0010 provides a second layer by locking the account after sustained failed authentication. Both mechanisms are independent and additive. A rate-limited request never reaches the lockout logic, reducing database writes for failed-auth counters.

23. CADF-Compliant Phased Auditing Architecture

Date: 2026-06-16

Status

Accepted

Security review 2026-06-24: Seven findings applied — HMAC canonicalization (RFC 8785/JCS), boot_session_id CSPRNG requirement, initiator.host sanitization rules, refresh_hmac_key version-collision fix, HMAC key retention policy, cross-node spool tamper detection, and map_event_to_action dangling-reference correction.

Context

For feature parity with OpenStack’s pycadf, our Axum/Tonic application requires CADF-compliant auditing - capturing both perimeter ingress and business-layer mutations. Our Rust architecture enforces that a security context cannot be used for policy enforcement before it is fully resolved, using an externally immutable ValidatedSecurityContext with read-only getters. This design prevents PII leaks while maintaining zero-trust guarantees.

Decision

We implement a hybrid auditing architecture producing standardized CADF events, dispatched asynchronously to configurable sinks across three phases: Framework, Perimeter (Extractor + Middleware), and Provider (Hooks).

Phase 1: General Audit Framework & CADF Types

Strict Rust representation of the CADF standard and async dispatch machinery.

The CADF Payload: To ensure unsigned events cannot serialize, CadfEvent wraps a private CadfEventPayload and signature via serde(flatten).

#![allow(unused)]
fn main() {
#[derive(Serialize, Deserialize, Clone)]
pub struct CadfEventPayload {
    id: String,
    seq: u64,
    boot_session_id: String,
    hmac_key_version: u64,
    version: String,
    domain: String,
    correlation_id: String,
    event_time: String,
    action: String, outcome: String,
    outcome_reason: Option<String>,
    initiator: Initiator,
    target: Target, observer: Observer,
}

impl CadfEventPayload {
    fn sign(self, dispatcher: &AuditDispatcher) -> CadfEvent {
        dispatcher.finalize_event(self)
    }
    /// Internal test-tooling only — NOT the SIEM verification path.
    /// External SIEMs MUST implement verification by: (1) parse received JSON,
    /// (2) remove the `signature` key, (3) serialize the remainder in JCS
    /// canonical form (RFC 8785), (4) compute HMAC-SHA256 with the key
    /// identified by `hmac_key_version`. Cross-language test vectors
    /// (tests/audit/hmac_vectors.jsonl) cover this exact path.
    fn from_cadf(evt: &CadfEvent) -> Self {
        let e = evt.payload();
        Self {
            id: e.id.clone(), seq: e.seq, boot_session_id: e.boot_session_id.clone(),
            hmac_key_version: e.hmac_key_version, version: e.version.clone(),
            domain: e.domain.clone(), correlation_id: e.correlation_id.clone(),
            event_time: e.event_time.clone(), action: e.action.clone(),
            outcome: e.outcome.clone(), outcome_reason: e.outcome_reason.clone(),
            initiator: e.initiator.clone(), target: e.target.clone(),
            observer: e.observer.clone(), }
    }
}

#[derive(Serialize, Deserialize, Clone)]
pub struct CadfEvent {
    #[serde(flatten)]
    event: CadfEventPayload,
    signature: String,
}

impl CadfEvent {
    pub fn payload(&self) -> &CadfEventPayload { &self.event }
    pub fn signature(&self) -> &str { &self.signature }
    pub fn correlation_id(&self) -> &str { &self.event.correlation_id }
    pub fn id(&self) -> &str { &self.event.id }
    pub fn seq(&self) -> u64 { self.event.seq }
    pub fn boot_session_id(&self) -> &str { &self.event.boot_session_id }
}

#[derive(Serialize, Clone)]
pub struct Initiator {
    id: String, project_id: Option<String>, domain_id: Option<String>,
    /// Pre-auth signal (EC2 access key, federation idp_id). No PII.
    /// Content arrives before authentication and is fully attacker-controlled.
    /// Sanitization rules (enforced at construction, not by Initiator itself):
    ///   EC2 access key  — must match /^AKIA[A-Z0-9]{16}$/; rejected otherwise.
    ///   Federation idp_id (UUID) — passed through sanitize_audit_id().
    ///   Federation idp_id (non-UUID) — filtered to [a-zA-Z0-9._-], max 64 chars.
    ///   Any other value — filtered to printable ASCII (0x20–0x7E), max 128 chars.
    ///   Field is omitted (None) if empty after filtering.
    host: Option<String>,
}
impl Initiator {
    fn new(id: String, project_id: Option<String>, domain_id: Option<String>,
        host: Option<String>) -> Self
    { Self { id, project_id, domain_id, host } }
    pub fn id(&self) -> &str { &self.id }
    pub fn project_id(&self) -> Option<&str> { self.project_id.as_deref() }
    pub fn domain_id(&self) -> Option<&str> { self.domain_id.as_deref() }
}
#[derive(Serialize, Clone)]
pub struct Target { pub id: String, pub type_uri: String }
#[derive(Serialize, Clone)]
pub struct Observer { pub node_id: String, pub id: String }
}

VerifiedFernetToken - Opaque wrapper only constructible post-verification. The _score: NonZeroU32 constructor guard proves crypto validation passed.

#![allow(unused)]
fn main() {
pub struct VerifiedFernetToken(FernetToken);

impl VerifiedFernetToken {
    pub(crate) fn from_verified(token: FernetToken,
        _score: std::num::NonZeroU32) -> Self { Self(token) }
    pub fn user_id(&self) -> &str { self.0.user_id() }
    pub fn domain_id(&self) -> Option<&str> { self.0.domain_id() }
}
}

ID sanitization - Strips non-ASCII, caps at 64 chars:

#![allow(unused)]
fn main() {
fn sanitize_audit_id(id: &str) -> String {
    if id.trim().is_empty() { return "unknown".to_string(); }
    let cleaned: String = id.chars()
        .filter(|c| c.is_ascii_hexdigit() || *c == '-')
        .take(64).collect();
    if cleaned.is_empty() { return "unknown".to_string(); }
    // Strict UUID check: len 36, 4 hyphens at canonical positions, 32 hex digits.
    // Reject any non-UUID format — prevents crafted ID bypass.
    if cleaned.len() == 36
        && cleaned.chars().filter(|c| *c == '-').count() == 4
        && cleaned.chars().filter(|c| c.is_ascii_hexdigit()).count() == 32
        && cleaned.get(8..9) == Some("-")
        && cleaned.get(13..14) == Some("-")
        && cleaned.get(18..19) == Some("-")
        && cleaned.get(23..24) == Some("-") {
        cleaned
    } else { "unknown".to_string() }
}
}

The Audit Dispatcher - Dual-channel QoS with atomic HMAC rotation:

#![allow(unused)]
fn main() {
pub struct AuditDispatcher {
    perimeter_sender: mpsc::Sender<CadfEvent>,  // 4096, best-effort
    critical_sender: mpsc::Sender<CadfEvent>,   // 256, fail-closed
    node_id: Arc<str>,
    hmac_key_and_version: ArcSwap<(Arc<[u8]>, u64)>,
    boot_session_id: String,
    seq_counter: AtomicU64,
    dropped_count: Arc<AtomicU64>,
    last_drop_log_time: AtomicU64,
    log_baseline: std::time::Instant,
    postaudit_dropped_count: Arc<AtomicU64>,
    events_total: Arc<AtomicU64>, // total events dispatched (perimeter + critical)
}

impl AuditDispatcher {
    // Signs over unsigned payload. HMAC input is the JCS-canonical (RFC 8785)
    // UTF-8 JSON of all payload fields with keys in lexicographic order, no
    // extra whitespace. Most Option-typed fields serialize as `null` when
    // absent. Exception: `Initiator.host` uses skip_serializing_if and is
    // **omitted entirely** (not set to `null`) when absent. SIEMs MUST
    // re-serialize the received JSON (minus `signature`) without inserting
    // absent keys. This is the sole canonical form for HMAC verification.
    fn finalize_event(&self, partial: CadfEventPayload) -> CadfEvent {
        let (key, version) = self.hmac_key_and_version.load_full().as_ref();
        let completed = CadfEventPayload {
            seq: self.seq_counter.fetch_add(1, Ordering::SeqCst),
            boot_session_id: self.boot_session_id.clone(),
            hmac_key_version: **version, ..partial };
        let sig = compute_hmac_sha256(&completed, &**key);
        CadfEvent { event: completed, signature: sig }
    }

    /// Best-effort: drops if full. Floor-rate log: at least once/sec.
    pub fn dispatch(&self, event: CadfEvent) {
        self.events_total.fetch_add(1, Ordering::Relaxed);
        let cid = event.correlation_id().to_string();
        if self.perimeter_sender.try_send(event).is_err() {
            let count = self.dropped_count.fetch_add(1, Ordering::Relaxed);
            let now_us = self.log_baseline.elapsed().as_micros() as u64;
            let should_log = (count % 1024) == 0
                || (self.last_drop_log_time.load(Ordering::Relaxed) + 1_000_000)
                    <= now_us;
            if should_log {
                self.last_drop_log_time.store(now_us, Ordering::Relaxed);
                error!(dropped_count = count, correlation_id = %cid,
                    "audit channel full, event dropped (best-effort)");
            }
        }
    }

    /// Fail-closed: blocks until sent.
    pub async fn dispatch_critical(&self, event: CadfEvent)
        -> Result<(), AuditChannelDead>
    {
        self.events_total.fetch_add(1, Ordering::Relaxed);
        self.critical_sender.send(event).await.map_err(|_| AuditChannelDead)
    }

    /// MUST be called from a single serialized context (dedicated key-rotation
    /// task). Concurrent invocations produce version collisions (two different
    /// keys share the same version), breaking SIEM verification.
    pub(crate) fn refresh_hmac_key(&self, new_key: Arc<[u8]>, new_version: u64) {
        self.hmac_key_and_version.store(Arc::new((new_key, new_version)));
    }
}
}

Spooling & Replay: Workers drain channels to sinks. On shutdown, unsent critical events spool. On startup, HMAC-verified and replayed. Corrupted lines are skipped to recover adjacent valid events; file quarantined at end.

boot_session_id: MUST be a UUIDv4 generated from the OS CSPRNG at process startup, before any request handling. MUST NOT be derived from a wall-clock timestamp, PID, or any predictable value. It is never persisted; its sole purpose is to namespace the seq counter within a single process lifetime. SIEMs MUST partition seq-gap detection by (node_id, boot_session_id) to avoid false gap alerts across restarts.

Phase 2: Perimeter Auditing (Ingress & Completion)

Captures access attempts at the boundary.

1. Ingress (Auth Extractor): The Axum middleware already injects a request-id header (UUIDv4). If the client already sent a request-id header, the middleware must strictly ignore its value and overwrite it with a fresh UUIDv4 (prevent client-controlled correlation spoofing). correlation_id is derived from this header value, not from the request token. Extracts Initiator from fully resolved ValidatedSecurityContext, token parse failure: outcome: "failure", Initiator is all "unknown" (no partial data from untrusted payload). On partial validation failure (token parsed but policy/scope failed): uses VerifiedFernetToken from vsc.verified_token() to extract sanitized initiator. For endpoints with pre-auth identity signals (EC2 access key, federation idp_id), include non-PII identifiers as initiator.host or a custom attachment — these don’t require a validated context and don’t risk PII leakage.

2. Completion (Middleware): Post-handler extracts CorrelationId and ReadOnlyInitiator, emits event with HTTP status mapped to outcome.

Error sanitization - Exhaustive match prevents silent PII leakage:

#![allow(unused)]
fn main() {
fn error_variant_name(error: &KeystoneApiError) -> String {
    match error {
        KeystoneApiError::Unauthorized { source, .. }
        | KeystoneApiError::Forbidden { source, .. } => {
            source.downcast_ref::<AuthenticationError>()
                .map(|e| sanitize_authentication_error(e).to_string())
                .unwrap_or_else(|| "Unauthorized".to_string())
        }
        KeystoneApiError::NotFound { .. } => "NotFound".to_string(),
        KeystoneApiError::Conflict { .. } => "Conflict".to_string(),
        KeystoneApiError::BadRequest { .. } => "BadRequest".to_string(),
        KeystoneApiError::RateLimited { .. } => "RateLimited".to_string(),
        KeystoneApiError::Gone { .. } => "Gone".to_string(),
        KeystoneApiError::InternalServerError => "InternalServerError".to_string(),
        KeystoneApiError::ServiceUnavailable => "ServiceUnavailable".to_string(),
        KeystoneApiError::GatewayTimeout => "GatewayTimeout".to_string(),
        e => e.type_name().unwrap_or("UnknownError").to_string(),
    }
}

fn sanitize_authentication_error(e: &AuthenticationError) -> &'static str {
    match e {
        AuthenticationError::DomainDisabled(_) => "DomainDisabled",
        AuthenticationError::ProjectDisabled(_) => "ProjectDisabled",
        AuthenticationError::TrustorUserDisabled(_) => "TrustorUserDisabled",
        AuthenticationError::UserDisabled(_) => "UserDisabled",
        AuthenticationError::UserLocked(_) => "UserLocked",
        AuthenticationError::UserPasswordExpired(_) => "UserPasswordExpired",
        AuthenticationError::Provider { source, .. } => {
            extract_provider_name(source).unwrap_or("ProviderError")
        }
        AuthenticationError::Validation(_) => "ValidationError",
        AuthenticationError::StructBuilder { .. } => "StructBuilderError",
        AuthenticationError::TokenExpired(_) => "TokenExpired",
        AuthenticationError::TokenRevoked(_) => "TokenRevoked",
        AuthenticationError::AuthCredentialNotFound(_) => "AuthCredentialNotFound",
        AuthenticationError::AuthCredentialExpired(_) => "AuthCredentialExpired",
        AuthenticationError::AuthCredentialMalformed(_) => "AuthCredentialMalformed",
        AuthenticationError::PrincipalNotUnique(_) => "PrincipalNotUnique",
        AuthenticationError::InvalidAuthMethod(_) => "InvalidAuthMethod",
    }
}

/// Type-only dispatch: no provider error string content is used. Guarantees
/// PII in third-party provider errors (emails, tokens) never reaches audit.
fn extract_provider_name(source: &Box<dyn std::error::Error>)
    -> Option<&'static str>
{
    if source.is::<identity::IdentityProviderError>() { Some("Identity") }
    else if source.is::<catalog::CatalogProviderError>() { Some("Catalog") }
    else if source.is::<role::RoleProviderError>() { Some("Role") }
    else if source.is::<assignment::AssignmentProviderError>() { Some("Assignment") }
    else { None }
}
}

Semantic action mapping - Hardcodes v3/v4 paths, sanitizes Operation::Other:

#![allow(unused)]
fn main() {
fn map_event_to_action(event: &Event) -> String {
    match &event.operation {
        Operation::Create => "create".to_string(),
        Operation::Update => "update".to_string(),
        Operation::Delete => "delete".to_string(),
        Operation::Disable => "disable".to_string(),
        Operation::Enable => "enable".to_string(),
        Operation::Authenticate => "authenticate".to_string(),
        Operation::Revoke => "revoke".to_string(),
        Operation::Other(action) => {
            // Sanitize: ASCII alphanumeric + /, -, _; cap 64 chars; reject empty.
            let s: String = action.chars()
                .filter(|c| c.is_ascii_alphanumeric()
                    || *c == '-' || *c == '_' || *c == '/')
                .take(64)
                .collect();
            if s.is_empty() { "unknown".to_string() } else { s }
        }
    }
}
}

Phase 3: Provider Auditing via Context-Aware Hooks

ProviderHooks (on_event) is fire-and-forget without context. Instead, AuditHook receives context and outcome, dispatched inline with fail-closed semantics. Reentrancy prevented via tokio::task_local!.

#![allow(unused)]
fn main() {
pub trait AuditHook: Send + Sync {
    async fn on_auditable_event(&self, ctx: &ValidatedSecurityContext,
        event: &Event, outcome: &AuditOutcome) -> Result<(), AuditDispatchError>;
}

pub enum AuditOutcome { Attempt, Success, Failure { reason: String } }
/// Sanitize hook error to stable literal. Prevents {:?} debug formatting
/// from leaking type names or internal diagnostics into CADF outcomes.
/// Hook errors only abort the provider op; they never flow into outcome_reason.
pub enum AuditDispatchError {
    DispatcherDead,
    HookFailed { description: &'static str },
    Reentered,
}

impl EventDispatcher {
    /// Fail-closed pre-audit: any hook error aborts the provider operation.
    /// Collects hook errors (except DispatcherDead short-circuits).
    pub async fn emit_critical(&self, ctx: &ValidatedSecurityContext,
        event: &Event, outcome: &AuditOutcome) -> Result<(), AuditDispatchError>
    {
        let is_reentered = EMIT_CRITICAL_RECURSION.try_with(|v| *v).unwrap_or(false);
        if is_reentered { return Err(AuditDispatchError::Reentered); }
        EMIT_CRITICAL_RECURSION.scope(true, async move {
            let audit = self.audit_hooks.lock().await.values().cloned().collect::<Vec<_>>();
            let mut error_count = 0u64;
            for hook in &audit {
                match hook.on_auditable_event(ctx, event, outcome).await {
                    Err(AuditDispatchError::DispatcherDead) =>
                        return Err(AuditDispatchError::DispatcherDead),
                    Err(AuditDispatchError::HookFailed { .. }) => error_count += 1,
                    Err(AuditDispatchError::Reentered) => error_count += 1,
                    Ok(()) => {}
                }
            }
            if error_count > 0 {
                return Err(AuditDispatchError::HookFailed {
                    description: "hook execution failed",
                });
            }
            // ... fire-and-forget regular hooks ...
            Ok(())
        }).await
    }
}
}

Audit-Before-Commit (Fail-Closed Transaction Safety):

#![allow(unused)]
fn main() {
macro_rules! audited_op {
    ( dispatcher: $dispatcher:expr, ctx: $ctx:expr, event: $event:expr,
      operation: $op:expr, error_variant: $err_variant:path ) => {{
        let event = $event;
        // Pre-audit (Attempt): fails if dispatcher dead
        $dispatcher.emit_critical($ctx, &event, &AuditOutcome::Attempt).await
            .map_err(|e| $err_variant { source: e })?;
        let result = $op.await;
        // Post-audit: use emit_critical. If channel full, write compensating
        // local JSONL log to guarantee dual-delivery path to SIEM.
        let outcome = match &result {
            Ok(_) => AuditOutcome::Success,
            Err(e) => AuditOutcome::Failure {
                reason: error_variant_name(e).to_string() }
        };
        if $dispatcher.emit_critical($ctx, &event, &outcome)
            .await.is_err()
        {
            // Fallback: local compensating log (structured JSONL, independent ship)
            // Includes operation and resource ID for forensic SIEM lookup.
            error!(
                correlation_id = %$ctx.correlation_id().to_string(),
                outcome = ?outcome,
                event_operation = ?$event.operation,
                event_resource = ?$event.payload,
                "post-audit channel full — compensating local log written"
            );
            $dispatcher.postaudit_dropped_count.fetch_add(1, Ordering::Relaxed);
        }
        result
    }};
}
}

Provider usage: audited_op! { dispatcher: ..., ctx: ..., event: ..., operation: ..., error_variant: ProviderError::AuditDispatchFailed }

CADF Hook: Single CadfAuditHook translates events to CADF, signs via CadfEventPayload::sign(), dispatches via dispatch_critical(). Wired at startup: state.event_dispatcher.subscribe_audit(CadfAuditHook).await.

Security Compliance vs. PII Requirements

• Data Minimization: Initiator has only UUIDs. Human-readable fields (usernames, emails) are excluded by design.
• PII Redaction: username, display_name, email_address, project_name, domain_name excluded. Future fields require opaque wrappers.
• Outcome Isolation: outcome_reason limited to sanitized variant name.
• HMAC Signing: CadfEvent wraps (CadfEventPayload, signature) via serde(flatten). Private fields prevent unsigned construction. Per-node signing key derived via:

  HKDF-Expand(KEK, info="keystone-audit-hmac-v1:{node_id_utf8}", L=32)
  

  The node_id suffix ensures each node holds a distinct signing key; a compromised node cannot forge audit records attributed to other nodes. This aligns with ADR 0016-v2 §3.1 (which uses node_id_u64_be for Raft nodes; here we use the UTF-8 encoding of the string node ID). HKDF-Expand-only is used because the KEK is already uniformly random (Extract is a no-op security-wise). Key+version as ArcSwap<(Arc<[u8]>, u64)> for atomic rotation (ADR 0016-v2 §6.2). HMAC input is the JCS-canonical (RFC 8785) serialization of the payload (all fields, lexicographically sorted keys, compact, null fields included).
• HMAC Key Retention: The KEK store MUST retain all HMAC key versions for at least max(spool_drain_timeout + SIEM_lag_budget, 24h). Key versions are monotonically increasing and permanent — never reused. The version number is supplied by the key-rotation task (not derived by refresh_hmac_key itself) to prevent version collisions under concurrent rotation attempts. SIEMs MUST cache all key versions seen and MUST NOT delete them without operator confirmation.
• Spool Integrity: Corrupted/tampered lines skipped (recovery-first). Per-node spool path (audit-spool-{node_id}.jsonl) eliminates shared-file races; advisory lock as secondary guard. Because the HMAC key is per-node (node_id is bound into the key derivation), the SIEM can reject events whose observer.node_id does not match the key used to verify their signature; mismatches MUST be quarantined as tamper indicators, not silently accepted.
• Delivery Guarantee: At-least-once delivery. SIEMs must deduplicate on CadfEvent.id (unique node_id:uuid).
• Attempt Reconciliation: SIEM treats an Attempt with no corresponding Success/Failure within 300s as outcome: unknown and triggers a warning alert. Loki query: sum_over_time({app="keystone"} | cadf_outcome="attempt" [10m]) - sum_over_time({app="keystone"} | cadf_outcome=~"success|failure" [10m]) > 0
• Verified Token Boundary: build_initiator_from_error() accepts only VerifiedFernetToken (constructible post-verification via _score). Partial context failure = authorization issue, not crypto issue.
• Provider Error Sanitization: extract_provider_name uses type-only dispatch (is::<T>()). No error string content used.

Observability

dropped_count / postaudit_dropped_count exported as Prometheus gauges:

groups:
  - name: keystone_audit
    rules:
      - alert: KeystoneAuditDropsVolumetric
        expr: |
          rate(keystone_audit_dropped_total[5m]) > 100 and
          rate(keystone_audit_dropped_total[5m]) /
          rate(keystone_audit_events_total[5m]) > 0.05
        for: 2m
        labels: { severity: critical }
        annotations:
          summary: "Audit drops >100/s (>5% of perimeter events)"
          description:
            "Possible volumetric attack. Check rate limiting (ADR 0022)."

      - alert: KeystoneAuditPostauditDrops
        expr: |
          increase(keystone_audit_postaudit_dropped_total[5m]) > 0 and
          rate(keystone_audit_events_total[5m]) > 0
        for: 1m
        labels: { severity: critical }
        annotations:
          summary: "Post-audit outcome record lost after DB commit"
          description:
            "The outcome record (Success/Failure) for a high-criticality op
            (disable_user, delete_credential) was dropped. The pre-audit Attempt
            exists, but the final outcome is lost. Compensating local log
            entries (structured JSONL) should be independently shipped to the
            SIEM for dual-delivery."

Related ADRs

• 0016-v2: HMAC key from KEK. KEK rotation calls refresh_hmac_key().
• 0017: ValidatedSecurityContext in hooks. correlation_id(), verified_token() for partial context.
• 0020: Mapping engine errors sanitized via error_variant_name().
• 0022: Rate-limiting. 429 produces outcome: "client_error". Audit drop alerts correlated with rate limiter health.

Alternatives

1. Provider wrapper traits: Rejected — 30+ enum expansions. AuditHook is a single subscription point.
2. Mutable context propagation: Rejected — Rust ownership enforces integrity.
3. Single-channel dispatch: Rejected — dual channels provide QoS isolation.
4. Post-serialization signing: Rejected — two-phase builder prevents unsigned-in-channel window.

Accepted Risk: Millisecond Durability Gap

dispatch_critical() returns Ok(()) after placing the signed event into the in-memory mpsc channel (256 depth). If the node hard-crashes (power loss, kernel panic) before the background worker drains the event to the spool file, that signed event is lost from RAM. The DB transaction that triggered the event has already committed, so the audit trail has a gap.

Why synchronous fsync is rejected: Adds ~10-50ms latency per critical provider operation. At Keystone scale (thousands of ops/sec), this causes SLO violation. A per-event WAL is equally expensive. The in-memory channel provides ordering without blocking the provider call path.

Mitigations: Graceful shutdown (SIGTERM) drains the full channel (10s budget) before exit — zero events lost. Spool replay covers process restarts via graceful shutdown handlers. Compensating local logs for post-audit drops provide dual delivery for high-criticality ops.

Risk acceptance: The millisecond crash window trades an extremely rare single-event loss for guaranteed sub-1ms provider latency. This aligns with OpenStack’s design philosophy: audit is advisory for SIEM compliance, not a hard transactional requirement.

Consequences

• Security: Two-event perimeter + fail-closed provider audit ensures complete coverage. Post-audit uses emit_critical with compensating local log fallback for dual-delivery. KeystoneAuditPostauditDrops alert is critical.
• Performance: Dual channels isolate perimeter (4096, best-effort) from critical (256, fail-closed). Not a substitute for rate limiting (ADR 0022).
• Correctness: Correlation IDs link perimeter through provider events.
• Integrity: Two-phase builder, sanitized error names, monotonic seq.
• Shutdown: 10s drain timeout, disk spool. Up to 4096 perimeter events may be lost. Corrupted spools quarantined for forensic triage.

24. SCIM v2 Resource Provisioning: Multi-Realm Identity Lifecycle per Domain

Date: 2026-07-01

Last-revised: 2026-07-08 (PR1+PR2+PR3+PR4+PR5 implementation note)

Status

Proposed

Implementation note (2026-07-06): PR1 (Realm Foundation), PR2 (Users vertical slice), PR3 (Groups + membership isolation), and PR4 (Protocol Surface Completion — filter grammar, PATCH, ETags, discovery endpoints) have landed. During implementation, the realm/user identity design was refined beyond what this ADR originally specified: ScimRealmResource gained a mandatory idp_id link to a federation IdentityProvider, and a SCIM-provisioned User’s id is derived deterministically rather than server-assigned, so it converges with a later federated JIT login for the same person instead of producing a duplicate account. §2.A and §4 below have been updated to match the as-built behavior; the “Rejected alternative” callout in §4 is revised accordingly. Group, by contrast, keeps a normal server-assigned id and an optional externalId — nothing federates in as a Group, so there is no convergence hazard a deterministic id would solve. §5’s filter/PATCH/ETag design matches the as-built PR4 behavior as written, with one addition worth noting here: closing the ETag CAS guarantee (§5.E) required fixing a latent bug in the storage driver’s compare-and-swap path that predates PR4 (a concurrent-write violation was detected by the store but never surfaced to the caller) — ScimResourceIndex.version now bumps on every PUT/PATCH, not only ones that change externalId, so the ETag is meaningful on every write.

Implementation note (2026-07-08): PR5 (Janitor Purge Phase, §6.C) has landed, matching the as-written design with one naming deviation: the retention config lives at [scim_resource] janitor_deprovisioned_retention_days (default 365) rather than a top-level [keystone] scim_deprovisioned_retention_days — this codebase nests janitor-tunable config on the owning provider’s own config struct throughout (see [api_key] janitor_tombstone_retention_days, ADR 0021 §6.F), and PR5 follows that existing convention rather than introducing a new top-level config table. The sweep itself is a leader-gated hourly background task (crates/core/src/scim_resource/janitor.rs), mirroring the API Key janitor’s structure exactly: for every tombstoned (deprovisioned_at set) ScimResourceIndex older than the retention window, it hard-deletes the underlying User/Group row via the existing IdentityApi::delete_user/ delete_group, purges the ScimResourceIndex anchor and its externalId claim in one storage transaction, and emits a CADF delete event. Per-item failures are isolated and retried on the next pass, exactly like the API Key janitor. The operator-triggered purge-now erasure-request path (§6.C last paragraph) is a new authenticated endpoint, DELETE /v4/scim-realms/{domain_id}/{provider_id}/purge/{resource_type}/{keystone_id}, gated by a new identity/scim_realm/purge OPA policy (admin, or manager scoped to the realm’s own domain — the same authorization boundary as identity/scim_realm/disable, since purging a realm’s resource is at least as sensitive as disabling the realm). It refuses to purge a resource that is not already deprovisioned, since that would silently skip the role- stripping and session-revocation steps DELETE /Users|Groups/{id} performs — an operator must soft-delete first, then purge. This is the final phase in this ADR’s implementation plan besides CLI parity (§12); this ADR stays Proposed until that lands.

Reference

Extends ADR 0002 (OPA), ADR 0017 (Security Context), ADR 0020 (Unified Mapping Engine), ADR 0021 (Stateless API-Key Ingress), ADR 0023 (Audit). Amends ADR 0021 §3 Step 4 (see §2.C below).

1. Context & Motivation

Enterprise IdPs (Okta, Entra ID, Workday) push identity lifecycle events — create, update, deactivate — for users and groups via SCIM, independently of however those same users later authenticate (OIDC, SAML, or direct Keystone credentials). Provisioning is therefore a distinct concern from authentication: it manipulates persistent User/Group rows, not the ephemeral shadow principals of ADR 0020’s Unified Mapping Engine (UME).

A single Keystone domain frequently represents an organization boundary that receives feeds from more than one authoritative source at once — for example, an Okta tenant provisioning full-time employees and a Workday-driven system provisioning contractors into the same domain. Both feeds must coexist without either one able to see, rename, or delete the other’s records, and without either clobbering a human administrator’s manually created accounts.

2. The Realm Model: Many Realms per Domain

A SCIM realm is the same tenant-local coordinate already used throughout ADR 0020/0021: the pair (domain_id, provider_id). A domain MAY register any number of independent, concurrently active realms. Each realm owns its own externalId and userName namespace, its own API keys (per ADR 0021 §5.D, N keys may still rotate under one provider_id), and its own provisioned resources. Realms within the same domain are fully isolated from one another (§7).

A. ScimRealmResource

Registering a realm is an explicit administrative act — creating an ApiClientResource (ADR 0021) alone does not enable SCIM resource provisioning for that provider_id. This separates “an API key that authenticates” from “an API key permitted to provision identities,” so API keys minted for unrelated ABAC/system-integration mapping rulesets can never accidentally provision Users/Groups.

#![allow(unused)]
fn main() {
pub struct ScimRealmResource {
    pub domain_id: String,
    pub provider_id: String,       // shared coordinate with ApiClientResource / MappingRuleSet
    pub idp_id: String,            // federation IdentityProvider this realm's users belong to
    pub display_name: String,
    pub enabled: bool,
    pub created_at: i64,
    pub updated_at: i64,
}
}

Keyspace: data:scim_realm:v1:<domain_id>:<provider_id>.

idp_id is mandatory and must resolve to an existing IdentityProvider (checked at both realm create and update; an unresolvable idp_id is 404). This exists because of the identity-convergence scheme in §4: a SCIM-provisioned User’s id is derived from (domain_id, externalId), the same formula used for a federation JIT shadow user’s id — so the realm has to know, up front, which IdentityProvider’s sub claims its externalIds are expected to equal for that convergence to actually line up. A realm not bound to a real IdP would still create syntactically valid User rows, but they’d never converge with anything, silently defeating the point of §4’s scheme.

Groups provisioned under a realm always inherit the realm’s own domain_id — no separate target-domain override is offered, keeping “one realm, one domain” the literal ownership boundary even though a domain may host many realms.

B. Realm Activation Gate

Every SCIM Users/Groups request first resolves the authenticated API key’s provider_id (already known to the ingress layer — it is a field on ApiClientResource, ADR 0021 §2.B) and looks up data:scim_realm:v1:<domain_id>:<provider_id>. If absent or enabled: false, the request is rejected with 403 Forbidden before touching any User/Group storage.

C. Amendment to ADR 0021: Realm-Aware Context Hydration

hydrate_ephemeral_context (ADR 0021 §3 Step 4) currently discards provider_id once the UME match resolves authorizations — it is never carried onto the ValidatedSecurityContext. This ADR amends that step: the resolved provider_id MUST be threaded through into a ScimRealmContext { domain_id, provider_id } available to a new ScimRealmAuth extractor (parallel to ApiKeyAuth), used exclusively by the /SCIM/v2 resource handlers introduced here. This is additive to ADR 0021’s payload and does not change its authentication semantics.

Scope restriction. ScimRealmAuth requires ScopeInfo::Domain matching the path’s {domain_id}. Project-scoped API keys receive 403 Forbidden on all /SCIM/v2/{domain_id}/Users and /Groups routes — identity lifecycle provisioning is a domain-level operation, never project-scoped. (The existing diagnostic whoami route is unaffected and keeps accepting any scope.)

Write-time ruleset constraint. Because a realm’s provider_id shares its MappingRuleSet coordinate (ADR 0020 §3) with the general UME, nothing inherently stops an operator from adding a rule to that same ruleset that resolves Authorization::Project for some other, unrelated claim match — which would make ScopeInfo::Domain above pass or fail per-request unpredictably for what is nominally “the same realm.” To close this structurally rather than leave it as a runtime surprise, the Mapping Engine CRUD API (ADR 0020 §9.A) MUST reject, with 422 Unprocessable Entity, any attempt to write a rule containing Authorization::Project into a MappingRuleSet whose provider_id has an active ScimRealmResource. This mirrors the existing write-time is_system prohibition for ApiClient sources (ADR 0021 §6.C): a SCIM realm’s ruleset may only ever resolve Authorization::Domain (matching its own domain_id) — System is already forbidden for all API-key ingress.

3. Resource Ownership & the SCIM Index

A. ScimResourceIndex

Every SCIM-provisioned User or Group is anchored by an ownership record kept separate from the mutable resource itself (mirroring how ADR 0020 keeps VirtualUserMetadata distinct from live claims) so that provenance cannot be altered by the SCIM PATCH surface itself.

#![allow(unused)]
fn main() {
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[serde(rename_all = "snake_case")]
pub enum ScimResourceType { User, Group }

pub struct ScimResourceIndex {
    pub domain_id: String,
    pub provider_id: String,        // owning realm — the sole authority for §3.C
    pub resource_type: ScimResourceType,
    pub keystone_id: String,        // User.id or Group.id; also the SCIM "id"
    pub external_id: Option<String>,
    pub version: u64,               // monotonic; source of the SCIM ETag (§5.E)
    pub deprovisioned_at: Option<i64>,
    pub created_at: i64,
    pub updated_at: i64,
}
}

B. Keyspace Summary

The realm-scoped index in the table above exists for fast, realm-owned lookup and version resolution (§3.C, §5.E) — it is deliberately not the sole uniqueness check. See §3.D: POST (create) additionally performs a domain-wide collision check against core Identity, independent of realm ownership.

C. Ownership Fencing Algorithm

GET/PUT/PATCH/DELETE /Users/{id} (and /Groups/{id}) execute:

1. Fetch data:scim_resource:v1:<domain_id>:<provider_id>:<type>:<id> using the caller’s own provider_id from ScimRealmContext.
2. If absent, return 404 Not Found — indistinguishable from “the resource does not exist,” even when a same-ID or same-userName resource exists under a different realm or was created manually. This prevents realm confusion / IDOR without leaking existence information across realms.
3. If deprovisioned_at is set, GET/list treat it as absent (404); repeat DELETE is idempotent (404 per RFC 7644 guidance for re-delete).

POST /Users and POST /Groups (create) check the realm-scoped externalId index and the domain-wide userName/displayName uniqueness check defined in §3.D; either collision returns 409 Conflict with scimType: "uniqueness" (§10). Note the asymmetry with step 2 above: existence is hidden cross-realm for ID-addressed reads/writes (IDOR protection), but is deliberately not hidden for name collisions at create time (§3.D) — the two serve different goals.

D. Domain-Wide Create-Time Uniqueness

§1 states that concurrent realms — or a realm and a human administrator — must never silently produce two identities that collide on userName within one domain. A check scoped only to the calling realm’s own index (as in an earlier draft of this ADR) cannot detect that: research into the existing schema (crates/identity-driver-sql) found no pre-existing global unique constraint on (name, domain_id) for users, so a second realm — or a manual POST /v3/users — creating the same userName would otherwise succeed unnoticed, leaving two User rows with identical name+domain_id and no way for a non-SCIM-aware lookup (openstack user show, a UME rule matching on user_name) to disambiguate them.

To close this, POST /Users and POST /Groups perform a domain-wide existence check — a live query against core Identity for any existing User/Group in domain_id whose name matches (case-insensitive), regardless of which realm, or no realm, created it — in addition to the realm-scoped externalId check. Any match, cross-realm or manual, rejects the create with 409 Conflict (scimType: "uniqueness", §10). This uniqueness check is deliberately not folded into the realm-scoped ScimResourceIndex lookup used by §3.C for read/update/delete ownership fencing: the two checks answer different questions — “does this name already exist anywhere in the domain” (create-time, domain-wide) versus “do I own the resource at this ID” (read/write-time, realm-scoped) — and conflating them would either leak cross-realm existence on reads (weakening §3.C’s IDOR protection) or fail to catch cross-realm collisions on create (the gap being closed here).

Race condition (TOCTOU). A read-then-write existence check by itself is not sufficient: two POSTs for the same (domain_id, name) issued concurrently (realistic under IdP-driven bulk onboarding, or two realms syncing the same person independently) can both pass the check before either commits, producing the exact duplicate this section exists to prevent. Since no unique constraint on (name, domain_id) exists today (that’s the gap this section opened with), this ADR requires the check-and-insert to be closed by one of the two mechanisms below, not by the live query alone:

• Preferred: a UNIQUE(domain_id, LOWER(name)) constraint added to the core User/Group tables in identity-driver-sql, with the SCIM create path treating the resulting constraint-violation error as the 409 Conflict trigger instead of (or in addition to) the pre-flight query. This makes uniqueness correct under concurrency by construction, at the cost of a schema migration shared with non-SCIM user/group creation.
• Fallback, if a schema-wide constraint is out of scope for this ADR’s first cut: the existence check and the row insert execute inside a single serializable database transaction scoped to (domain_id, name), so a second concurrent transaction targeting the same name blocks or fails at commit rather than at the earlier read.

Either way, the pre-flight query described above remains as a fast-path rejection for the common (non-racing) case; it is the commit-time guarantee that actually closes the race, and this ADR is not satisfied by the pre-flight check alone.

4. Resource Schemas & Attribute Mapping

SCIM provisioning targets real, persistent Keystone User/Group rows — never the ADR 0020 ephemeral shadow-registry path. A SCIM-provisioned user is expected to authenticate later through an entirely separate channel (OIDC, password, passkey); SCIM only manages the account’s existence and attributes.

Identity convergence with federation JIT (as-implemented). externalId is mandatory on POST .../Users (400 if empty/absent), and the created User.id is not server-assigned: it’s derived deterministically as generate_public_id(domain_id, externalId, "user") — the identical sha256-based formula this codebase’s ADR 0020 UME path already uses to derive a federation JIT shadow user’s id. The user row is created as UserType::NonLocal (no password, no local_user row). The practical effect: a person provisioned ahead of time via SCIM (externalId == the IdP’s sub claim), who later authenticates for the first time via that same realm’s idp_id (§2.A), converges onto the same User row a JIT login would otherwise have created from scratch — rather than ending up with two accounts for one person, one SCIM-managed and one federation-managed. POST .../Users additionally probes for a user already occupying that deterministic id (e.g. one a federated JIT login already created before SCIM provisioning caught up) and returns 409 Conflict (scimType: "uniqueness") rather than surfacing a raw primary-key-collision error from the Identity driver.

Rejected alternative: reusing User.federated: Option<Vec<Federation>> (on UserResponse/UserCreate/UserUpdate; Federation { idp_id, protocols, unique_id }) to carry the SCIM externalId directly. Federation is scoped to authentication-protocol linkage (idp_id + protocol_id) and is not realm-fenced or version-tracked; overloading it would conflate two different provenance concepts and bypass the ownership fencing in §3.C. ScimResourceIndex is kept as a dedicated, parallel structure for provenance/ownership instead — the convergence above is achieved purely through the shared id-derivation formula, not by writing into User.federated.

Attributes without a first-class Keystone field are namespaced under extra["scim_*"] rather than added as new top-level User columns — avoiding a core-identity schema migration for display-only SCIM metadata.

5. Protocol Surface (Pragmatic Subset)

Full RFC 7644 compliance (arbitrary filter expressions, arbitrary PATCH path expressions, /Bulk) is explicitly not targeted for v1. This mirrors the DoS-hardening posture already established elsewhere in the codebase (regex ReDoS bounds and per-claim size caps in ADR 0020 §5.1, token-bucket rate limiting in ADR 0021 §6.A).

A. Endpoints

• POST /SCIM/v2/{domain_id}/Users, GET .../Users, GET .../Users/{id}, PUT .../Users/{id}, PATCH .../Users/{id}, DELETE .../Users/{id}
• POST /SCIM/v2/{domain_id}/Groups, GET .../Groups, GET .../Groups/{id}, PUT .../Groups/{id}, PATCH .../Groups/{id}, DELETE .../Groups/{id}
• GET /SCIM/v2/{domain_id}/ServiceProviderConfig, GET .../Schemas, GET .../ResourceTypes — static discovery documents, honestly advertising bulk.supported: false, sort.supported: false, and describing the restricted filter grammar below (Okta/Entra ID both tolerate a filter.supported: true with a narrower attribute set than the spec’s maximum).

B. Filter Grammar

filter     := term (LOGICAL_OP term)*      // homogeneous chain only — "and" and "or" MUST NOT be mixed in one filter string
term       := ATTR OP value
LOGICAL_OP := "and" | "or"
OP         := "eq" | "ne" | "co" | "sw" | "pr"

Any attribute or operator outside these tables, a mixed and/or chain, nested/parenthesized expressions, or a filter string exceeding 512 bytes / 8 terms is rejected with 400 Bad Request (scimType: "invalidFilter"). co/sw are only evaluated against attributes that already carry a realm index (§3.B), bounding worst-case cost to that index’s range, never a full table scan.

C. PATCH Operation Support

Operations: [{op, path, value}] is accepted only for these top-level, scalar path targets: active, userName/displayName, externalId, name.givenName, name.familyName, plus members (Group, add/remove only — the common “push group” pattern). Any other path (complex filter expressions like emails[type eq "work"].value, array-index paths) returns 400 Bad Request (scimType: "invalidPath"). PUT performs a full declarative replace of all mapped attributes, including a full membership resync for Groups (remove-then-add against the target member set).

D. Pagination

startIndex (1-based, default 1), count (default 20, max 200). Listing and totalResults are computed via a bounded prefix range-scan over the realm’s own data:scim_resource:v1:<domain_id>:<provider_id>:<type>:* keyspace, excluding deprovisioned_at-set entries. This is a linear scan bounded to one realm’s resource count (consistent with existing janitor range-scans elsewhere); a maintained counter is deferred as a future optimization if realm sizes prove large enough to matter.

Group listing cost. The bound above covers the scim_resource index scan itself, not members hydration. A GET /Groups page fans out into one user_group_membership lookup per group on the page — bounded per-group by the same 1000-member cap as §11, so a full page’s worst case is count × 1000 membership rows (e.g. 200 × 1000 for a max-size page), not unbounded, but materially larger than the resource-index scan alone. Clients needing cheaper listing should page with a smaller count when membership detail isn’t required, or use GET /Groups/{id} for individual membership detail.

E. ETags / Concurrency

ScimResourceIndex.version is a monotonic counter incremented on every SCIM-driven PUT/PATCH, serialized as a weak ETag: W/"<version>". PUT and PATCH requests carrying If-Match are rejected with 412 Precondition Failed if the header value doesn’t match the current version — closing the lost-update race inherent to concurrent push-group syncs from a single IdP.

This guarantee only holds if the If-Match compare, the field write, and the version increment happen as one atomic operation against the backing store — a read-compare-then-write done as three separate calls reintroduces the same race it’s meant to close, just moved into the version counter itself. The handler MUST perform this as a single compare-and-swap against the data:scim_resource:v1:... row (or an equivalent single transaction against the underlying User/Group table when the write also touches core Identity fields), rejecting with 412 if the stored version has moved between the initial read and the write. A high-frequency IdP sync burst is exactly the case this is meant to survive, not merely the common case.

F. Explicitly Out of Scope for v1

/Bulk, arbitrary filter path expressions, sortBy/sortOrder, and multi-valued complex attribute PATCH addressing (emails[type eq "work"]). Extending any of these later requires a ratifying revision to this ADR given their DoS/complexity surface.

6. Deprovisioning Semantics

A. DELETE /Users/{id} → Soft-Disable Only

Consistent with the retention pattern already used for API keys (ADR 0021 §5.C) and the UME shadow registry (ADR 0020 §4.A), DELETE on a User never hard-deletes. It:

1. Sets User.enabled = false.
2. Stamps ScimResourceIndex.deprovisioned_at.
3. Triggers the existing token revocation pipeline (revocation:v1:user:<user_id>, ADR 0020 §9.F) so live sessions die immediately.
4. Emits a CADF disable event (§9).

Subsequent GET/PATCH/PUT against the same id from the owning realm return 404 Not Found (tombstoned), matching RFC 7644’s expectation that a deleted resource is inaccessible, while the underlying row and its audit trail survive for incident response.

B. DELETE /Groups/{id} → Neutralize + Tombstone

Group has no enabled field in the current schema (unlike User), and RFC 7644’s Group schema defines no active attribute either. Rather than adding a new field to core Group for this single caller, or hard-deleting (which would silently leave any inherited role grants dangling and does not match the “preserve audit trail” rationale used everywhere else in this codebase), Group deletion:

1. Immediately clears the group’s role assignments (closing the live authorization surface — this is the security-relevant action, since a “deleted-looking” group that still grants roles would be a silent escalation path). A role-stripped group grants nothing regardless of who remains listed as a member, so this alone is sufficient to neutralize the group.
2. Stamps ScimResourceIndex.deprovisioned_at and hides the group from all SCIM GET/List responses (404), identically to a User tombstone. Membership is deliberately left intact at this point — clearing it would destroy exactly the forensic snapshot (who belonged to the group at the moment of deletion) that the “preserve audit trail” rationale for not hard-deleting is meant to protect, and retaining it poses no live authorization risk once step 1 has stripped the group’s roles.
3. Retains the Group shell and its membership snapshot for the same retention window as (C) below, then the janitor hard-deletes the row (and its membership records) together.

C. Janitor Purge

A background janitor (extending the pattern of ADR 0020 §4.A’s archive cleanup and ADR 0021 §6.F’s physical reclamation) permanently deletes User and Group rows whose ScimResourceIndex.deprovisioned_at is older than [keystone] scim_deprovisioned_retention_days (default: 365 days), removing the ScimResourceIndex anchor and its external_id/name index entries in the same transaction.

Regulatory retention risk. A fixed 365-day default of PII (extra["scim_*"] fields, external_id) held in a soft-deleted-but-readable-by-operators state purely to preserve an audit snapshot is a data-minimization / right-to-erasure tension for deployments under GDPR or comparable regimes — a deployer cannot justify a full year of retention against an erasure request just because this ADR’s default says so. This ADR treats scim_deprovisioned_retention_days as deployer-controlled specifically so regulated deployments can set it far below 365 days (including near-zero, trading away most of the forensic window for compliance), and additionally requires an operator-triggered purge-now path — a janitor invocation scoped to a single keystone_id that ignores the retention window — so a verified erasure request does not have to wait for the configured period to elapse. Choosing the right default retention for a given jurisdiction is a deployment/compliance decision this ADR deliberately leaves to the operator rather than prescribing centrally.

7. Cross-Realm & Membership Isolation

Beyond the per-resource ownership fencing in §3.C, group membership writes are fenced transitively: a Group members entry (add, via PUT or PATCH) MUST reference a User owned by the same realm (same provider_id) as the group itself. A membership reference to a user owned by a different realm, or to a manually-created user with no ScimResourceIndex entry at all, is rejected with 400 Bad Request (scimType: "invalidValue"). This prevents one IdP integration from reaching across realm boundaries — or into human-managed accounts — merely by guessing or enumerating a Keystone user ID.

8. Authorization & OPA Policies

Realm CRUD (POST/GET/PATCH /v4/scim-realms) is invoked by a Fernet-authenticated human operator, not a SCIM API key, so its authorization reuses the actual, pre-existing manager role (this codebase’s realization of “DomainManager” — see ADR 0021 §5.A) or admin/is_admin (never DomainAdmin), under new policies named per the slash-separated convention actually used by every implemented policy call site in crates/keystone/src/api/v4/** and the corresponding .rego packages (e.g. identity/user/create) — not the colon-separated form identity:api_key:create that ADR 0021 §5.A used only in prose and was never implemented:

• identity/scim_realm/create / identity/scim_realm/list / identity/scim_realm/show / identity/scim_realm/disable

SCIM resource CRUD authorization is enforced exactly like any other v4 endpoint per ADR 0002, but with one important distinction from realm CRUD above: SCIM resource requests are authenticated exclusively via API-key ingress (ADR 0021), and ApiClientResource carries no Role field and no RoleAssignment at all. The roles evaluated by these policies are therefore never RBAC-assigned — they are entirely the Authorization::Domain{roles} value produced by evaluating the realm’s own MappingRuleSet (ADR 0020 UME / ADR 0021 §3 Step 4 hydrate_ephemeral_context) at request time. An operator grants access by authoring a mapping rule whose output includes the role string manager, admin, or scim_provisioner onto the realm’s provider_id — not by assigning a Keystone Role to anything, since no such assignment surface exists for API keys. These policies are evaluated against:

• identity/scim/user/create / identity/scim/user/list / identity/scim/user/show / identity/scim/user/update / identity/scim/user/delete
• identity/scim/group/create / identity/scim/group/list / identity/scim/group/show / identity/scim/group/update / identity/scim/group/delete

Note for ADR 0021: its §5.A policy names should be corrected to the same slash convention in a future revision of that ADR; this ADR does not attempt to fix 0021 retroactively, only avoids repeating its naming inconsistency.

Role-existence enforcement (ADR 0020 §7.3): the manager/admin/ scim_provisioner role strings above are only meaningful if a Role with that exact name actually exists — the naming-drift bug this ADR originally shipped with (invented SystemAdmin/DomainManager literals with no backing Role, silently producing an unreachable authorization) is now caught structurally: mapping rule create/update rejects any RoleRef whose id doesn’t resolve against the Role store with 422 Unprocessable Entity (MappingProviderError::RoleNotFound), rather than relying on this ADR’s prose staying in sync with the rego by hand.

The §3.C ownership-fencing check happens before OPA evaluation and is not a substitute for it — a realm’s own credential may still lack a role authorizing a given operation even against its own resources.

9. Auditing

Every SCIM write emits a CADF event per ADR 0023’s actually-implemented CadfEventPayload, which carries a single action: String (no separate category field exists in crates/audit/src/types.rs — ADR 0021 §5.C’s mention of a control category is unimplemented prose, not a real field, and this ADR does not repeat it). The action is drawn from the existing Operation enum (crates/core-types/src/events.rs): Create/Update for writes, Disable for the deprovisioning paths in §6.

target.type_uri is data/security/account (User) or data/security/group (Group). realm_provider_id and external_id are captured in the event attachment for cross-referencing against the IdP’s own provisioning logs.

initiator.id caveat. Per ADR 0021 §3 Step 4 / §5.D, initiator.id is derived from the authenticating API key’s client_id, not from provider_id — this is per-key, not per-realm, identity, chosen precisely so distinct keys sharing a provider_id produce distinct audit identities. Consequently initiator.id on SCIM CADF events changes across a zero-downtime key rotation (ADR 0021 §5.D) even though the realm performing the action has not changed. Consumers correlating “who acted on behalf of this realm” across a rotation window MUST group by the realm_provider_id attachment field, not by initiator.id.

This is an operational gap, not just a documentation note: a SIEM/SOC pipeline built against initiator.id alone (a reasonable default, since that’s the field ADR 0021 already calls the actor identity) will silently split one realm’s activity into two apparently-unrelated actors across a rotation, which is precisely the window where a leaked pre-rotation key is most likely to be abused. This ADR does not itself ship a fix for downstream SIEM configuration, but requires that operational runbooks for SCIM ingress (a deliverable of the rollout, not of this ADR) explicitly call out realm_provider_id as the correlation key, and that alerting rules keyed purely on initiator.id stability are flagged as insufficient for SCIM traffic during review.

10. Error Mapping (RFC 7644 §3.12)

SCIM error responses use the standard envelope:

{
  "schemas": ["urn:ietf:params:scim:api:messages:2.0:Error"],
  "status": "409",
  "scimType": "uniqueness",
  "detail": "userName already exists within this domain"
}

11. Threat Model

• Realm confusion / IDOR: mitigated structurally by §3.C — ownership is checked before any read/write, independent of role authorization.
• Cross-realm membership injection: mitigated by §7.
• Provisioning DoS: SCIM resource writes ride the same per-lookup_hash token-bucket rate limiter as authentication (ADR 0021 §6.A); a second, write-specific limiter keyed on provider_id ([keystone] scim_realm_write_rate_limit, default 500/min, mirroring ADR 0020 §7.2’s shadow-registry limiter) bounds bulk provisioning bursts from a single compromised or misconfigured realm.
• Membership-graph bombs: a single PATCH/PUT is capped at 1000 members entries; larger syncs must paginate across multiple requests.
• Filter/PATCH complexity: bounded to the tables in §5.B/§5.C — eliminates the arbitrary-expression parsing surface that would otherwise require its own ReDoS-style hardening.
• Silent group privilege retention on delete: addressed by §6.B’s immediate role-assignment clearing, rather than treating “deleted-looking” as sufficient.
• Name-collision race (TOCTOU): a naive read-then-write uniqueness check (§3.D) is racy under concurrent creates for the same name; closed by requiring a DB-level unique constraint or an equivalent serializable check-and-insert transaction, not the pre-flight query alone.
• Audit-trail fragmentation across key rotation: initiator.id changes across a zero-downtime SCIM key rotation (§9); mitigated by requiring downstream correlation on realm_provider_id, and flagged as an operational requirement for SIEM/SOC configuration, not something this ADR can enforce in-band.

12. Consequences

• New keyspaces: scim_realm, scim_resource, plus two lookup indices (§3.B). No migration of existing User/Group schemas for the fallback path in §3.D; the preferred path (a UNIQUE(domain_id, LOWER(name)) constraint) does require a schema migration shared with non-SCIM create paths.
• ADR 0021’s ingress hydration gains an additive provider_id field on the ephemeral context (§2.C) — existing non-SCIM consumers are unaffected.
• ADR 0020’s Mapping Engine CRUD API gains a new write-time validation rule (§2.C): rulesets whose provider_id is bound to an active ScimRealmResource may not contain Authorization::Project entries.
• POST /Users/Groups now performs a domain-wide userName/displayName existence check (§3.D) against core Identity, in addition to the realm-scoped externalId check — the first piece of cross-realm uniqueness enforcement introduced for user/group names, though still not a general schema-level constraint outside the SCIM create path.
• New CRUD API family /v4/scim-realms and OPA policies (§8, slash-separated naming) require CLI support, per the standing convention from ADR 0006 (“New APIs must be implemented in the CLI”).
• /Bulk, full filter grammar, and full PATCH path expressions are deliberately deferred; broader RFC 7644 compliance requires a follow-up ADR revision once real-world IdP integration experience justifies the added complexity.
• The janitor gains a new purge phase (§6.C) alongside its existing API-key and shadow-registry archive phases, plus an operator-triggered purge-now path for erasure requests (§6.C) that bypasses the configured retention window.
• ScimResourceIndex.version writes (§5.E) and the §3.D uniqueness check-and-insert both require compare-and-swap / transactional semantics from the backing store rather than separate read-then-write calls — a correctness requirement on the implementation, not just documentation.

25. Dynamic Auth Plugins via WebAssembly

Date: 2026-07-02

Status

Proposed

Reference

Contrasts with ADR 0018 (Plugin Linking - static, compile-time linking). Extends ADR 0017 (Security Context), ADR 0023 (Audit). Reuses the resource-bound patterns established in ADR 0020 §5.1 (regex evaluation timeout/size caps). Amends ADR 0020 §2/§3 (IdentitySource gains a WasmPlugin variant; MappingContext gains an optional plugin-hash field - §4 “mapping Mode”). Reuses ADR 0024 §3.B (externalId lookup index) for SCIM-aware identity linking

• §4 “Admin-Authorized External Identity Linking.”

1. Context & Motivation

All extensibility in keystone-rs today is resolved at compile time:

• Backend drivers (SQL, Raft, etc.) are separate crates registered through inventory::submit! and forced into the link graph by the anchor() / build.rs convention (ADR 0018). Adding or changing a driver requires a new crate, a Cargo dependency edit, and a full rebuild + redeploy of the binary across the cluster.
• Authentication methods are a closed Rust enum, AuthenticationContext (crates/core-types/src/auth.rs:1169), with variants Password, Admin, Token, ApplicationCredential, Oidc, K8s, Trust, WebauthN, Mapping. crates/core/src/api/auth.rs matches this enum exhaustively to build a SecurityContext. The [auth] methods config value (crates/config/src/auth.rs:22-27) is a Vec<String> allowlist, but every string in it must already correspond to a variant compiled into the binary.

This is the correct model for first-party backends (ADR 0018’s stated goal: zero manual maintenance, compile-time enforcement). It is the wrong model for operator- or customer-specific authentication logic: a proprietary SSO quirk, a legacy directory bridge, a step-up/MFA policy tied to an internal risk service, or a one-off migration bridge that a specific deployment needs but that has no place in the upstream keystone-rs tree. Today, satisfying any of these requires either forking keystone-rs or waiting on a release that adds a bespoke enum variant - in both cases a full recompile and coordinated cluster-wide redeploy.

This ADR introduces a second, orthogonal extensibility mechanism: dynamic auth plugins, compiled to WebAssembly, loaded from disk at process startup, and invoked as first-class authentication methods - without touching keystone-rs source or its build.

Requirements

A plugin must be able to:

1. Act as a full authentication method: receive the raw login request and either accept it (producing an identity + claims) or reject it - not merely observe a decision made elsewhere.
2. Perform outbound HTTP calls (e.g. call a third-party risk API or legacy directory as part of the decision).
3. Call a curated set of internal Keystone operations when necessary - most importantly, provision a local user on first login.
4. Authenticate a user who already exists through some other channel (most notably SCIM provisioning, ADR 0024) - not only identities the plugin provisions itself. See §4 “Three Operating Modes” for how this is satisfied without weakening requirement 1’s guarantees.
5. Redirect a request to a different, already-registered auth method based on inspecting the raw credential, for clients that cannot be made to request a custom method name themselves (application_credential-shaped auth for Terraform is the motivating case) - without that routing decision itself being able to authenticate anyone. See §4 “Guest Contract - route Mode.”

Non-goals

• Replacing the ADR 0018 static-driver model for first-party backends.
• Per-domain/multi-tenant plugin scoping (a domain admin installing their own plugin). See §8 (Open Questions) - this ADR restricts plugins to cluster-global, system-admin-installed only.
• Hot reload / upload-via-API. Plugins are loaded once at process startup from local disk (§5). A future ADR may revisit this (§8).
• General-purpose scripting for non-auth extension points (policy, mapping rules, etc.). Those already have dedicated mechanisms (OPA - ADR 0002; the Unified Mapping Engine - ADR 0020).

Threat Model

Installing a plugin already requires filesystem and keystone.conf write access on every node - operationally equivalent to root on the Keystone host. This ADR therefore does not attempt to defend against an operator who deliberately installs a plugin they wrote to be malicious; that is out of scope, the same way a malicious [database] connection value is out of scope for any other ADR.

What it does defend against, because both are realistic even for a plugin the operator trusts and reviewed:

1. A buggy or exploited plugin - third-party code, however well-reviewed, can have logic errors or be compromised via a supply-chain issue in its own dependencies (the plugin author’s build, not keystone-rs’s). The blast radius of such a bug or compromise must be bounded to “this one auth method behaves incorrectly,” not “arbitrary account takeover” or “arbitrary internal API access.”
2. An anonymous network attacker - every plugin invocation is reachable pre-authentication (it is the authentication step), so the design must assume a remote, unauthenticated party can trigger plugin execution and http_fetch calls at will, and bound the damage that’s possible from that alone (resource exhaustion, SSRF, credential exposure).
3. Widened credential exposure from route mode specifically - a full_auth/mapping plugin is only invoked for requests already addressed to it by name, so a buggy or exploited one is exposed to at most the traffic an operator explicitly opted into routing there. A route-mode plugin breaks that property: it must run on every request whose identity.methods matches its inspect_methods list (§4 “Guest Contract - route Mode”), including ones it ultimately passes through unmodified, so it sees raw credential material (headers, method payloads) for a strictly larger slice of login traffic than any other plugin in this design - for example, every application_credential attempt in the cluster, not just ones actually meant for a custom handler. A bug or compromise here doesn’t grant broader authentication power (§4’s structural constraints - target-method allowlist, no identity resolution, no scope access - still bound that), but it does mean the plugin is a wider observation surface: it is positioned to see, and potentially exfiltrate via a compromised http_fetch call, credential material belonging to logins it has no legitimate reason to act on. inspect_methods scoping (§4) and the headers/payloads allowlists are the load-bearing controls for this actor - narrowing what triggers invocation and what a triggered invocation can see is the only mitigation available, since the plugin must structurally be able to inspect something about every matching request to decide whether to route it.

Every control in §4–§7 is sized against these three actors, not against a deliberately hostile plugin author.

2. Decision Summary

3. Runtime: Extism on wasmtime

No WASM runtime exists anywhere in the current dependency tree (Cargo.lock only contains wasm-bindgen/wasm-streams, transitive browser-target deps of an unrelated crate) - this is greenfield.

Extism is chosen over raw wasmtime or wasmer because it already solves the three hardest parts of this problem as a framework, rather than requiring keystone-rs to invent them:

• A stable, versioned Plugin-Development-Kit (PDK) ABI with official SDKs for the languages a third-party plugin author is likely to use (Rust, Go, JS/TS, Python, C, Zig, …) - we do not have to design or document our own guest-side calling convention.
• A built-in host-function registration model (extism::Function) that maps directly onto the “curated allowlist” capability model this ADR requires (§6)
  • each host function is registered per-Plugin instance, so a plugin simply cannot see a host function it wasn’t given.
• A built-in HTTP allow-list (allowed_hosts on extism::Manifest) for the guest-initiated HTTP use case (§6.A), instead of keystone-rs having to hand-roll a WASI-sockets bridge.
• It already wraps wasmtime for fuel metering, wall-clock timeouts (Plugin::new builder timeout), and linear-memory limits (§7) - the resource-bounding primitives this ADR needs are present, not something to build from scratch.

The trade-off is an additional framework dependency with its own release cadence, on top of wasmtime. Given no existing runtime is present to reconcile with, and the PDK ergonomics directly serve requirement 3 (guest-language diversity for third-party plugin authors), this is judged worth it.

4. Hook Point: Plugins as a Full Auth Method

A plugin is registered under a plugin name that is used verbatim as an entry in [auth] methods (crates/config/src/auth.rs:22-27 already accepts arbitrary strings - no config-schema change needed there). At authentication time, if the requested method name does not match a builtin (password, token, openid, …), it is looked up in a new WasmPluginRegistry, following the same “resolve backend by string name” pattern already used by PluginManagerApi::get_x_backend (crates/core/src/plugin_manager.rs:56-120).

[auth]
methods = password,token,openid,application_credential,acme_risk_sso

Three Operating Modes: full_auth vs mapping vs route

The namespace-scoped identity binding in this ADR (below) is necessary to stop a plugin from asserting an arbitrary pre-existing identity, but it has a direct consequence: a plugin can, by construction, only ever authenticate users it itself provisioned. That’s correct for genuinely new external identities, but it leaves no path for a plugin to serve as an auth method for users who already exist through some other channel - most importantly, users provisioned via SCIM (ADR 0024), which is an explicit, realistic requirement, not an edge case.

A separate, unrelated gap: some clients never let the operator choose which auth method name is requested at all. Terraform’s OpenStack provider, for example, always use built-in auth method names - it has no concept of a custom method name. A real-world pattern built around this constraint (and the direct inspiration for route mode below) is keystonemiddleware-style request rewriting: a component ahead of the auth logic inspects the incoming application_credential_id and, based on its shape, rewrites the request so it is dispatched to a different handler entirely - the routing decision and the authentication decision are two separate concerns, made by two different pieces of code, neither of which is the client. Neither full_auth nor mapping mode can express this: both require the client to already know which method name to ask for.

Rather than loosening the identity-binding namespace to cover the first gap (which would reopen the account-takeover class this ADR exists to prevent - see “Identity Binding” below), or collapsing the routing concern into an authentication concern (which would let a request-shaping decision double as a credential-verification decision - see “route” below), each plugin declares one of three operating modes at config time (mode = full_auth | mapping | route, §5; defaults to full_auth for everything already described in this ADR):

• full_auth (default, as designed above) - the plugin is the terminal identity authority for its method name. It calls provision_user/ find_user and returns Allow/Deny itself. Can reach pre-existing users only via an admin-authorized link (see “Admin-Authorized External Identity Linking” below)
  • never by unscoped lookup.
• mapping - the plugin has no authority to terminate authentication at all. It only transforms/normalizes the incoming request into a claims map, which is handed to the existing, already-reviewed Unified Mapping Engine (ADR 0020) to make the actual identity and authorization decision - exactly the same engine already trusted to resolve OIDC/K8s/SPIFFE claims to real or ephemeral users, including real, pre-existing local users via its IdentityMode::Local path. Because the plugin never asserts an identity itself, no ResolvedIdentityHandle, no namespace, no binding of any kind is needed for this mode - it’s the direct, safe way to get the “simple auth request rewrite” behavior for users the plugin didn’t provision, including SCIM-provisioned ones, without touching the account-takeover defense at all.
• route - the plugin has no authority to terminate authentication and no authority to decide identity at all; it only sees the raw, pre-dispatch request and decides which already-registered auth method should actually handle it, optionally relabeling that method’s payload. This is a request-routing decision, not a credential-verification decision - the target method (builtin or another plugin) still performs its own full, unweakened verification. See “Guest Contract - route Mode” below.

mapping mode is the recommended default for plugins whose job is fundamentally “translate a proprietary/legacy assertion into claims” (the SCIM-adjacent case). full_auth mode remains available for plugins that must make a genuinely custom, non-claims-matching judgment call - a real-time risk score, for instance

• that can’t be expressed as a static mapping rule. route mode is for the narrower case where the client cannot be made to ask for a different method by name, but the credential it always sends is self-describing enough for a router to redirect it to the handler that actually knows how to verify it.

Guest Contract - full_auth Mode

Each plugin exports a single Extism entry point:

authenticate(request: AuthPluginRequest) -> AuthPluginResponse

#![allow(unused)]
fn main() {
// Host <-> guest wire types (JSON over the Extism call boundary)
struct AuthPluginRequest {
    /// Raw credential payload from the identity.<method> block of the
    /// v3/v4 auth request, exactly as received.
    payload: serde_json::Value,
    /// Allowlisted subset of inbound HTTP headers - only headers the
    /// plugin's config explicitly opts into (`exposed_headers = ...`,
    /// §5) are forwarded. Everything else is never handed to the guest.
    /// This is an allowlist, not a denylist: a header added to Keystone in
    /// the future is excluded by default rather than silently exposed. A
    /// fixed set - `Authorization`, `Cookie`, `X-Auth-Token`,
    /// `X-Service-Token`, `X-Subject-Token`, `Proxy-Authorization` -
    /// additionally can **never** appear in
    /// `exposed_headers` regardless of what an operator configures; the
    /// config loader rejects any plugin config that lists one of them,
    /// the same fail-loud posture as §5. This is deliberate
    /// defense-in-depth: `exposed_headers` being operator-controlled makes
    /// it easy for a copy-pasted example config to silently re-expose
    /// exactly the headers this mechanism exists to protect.
    headers: HashMap<String, String>,
    /// The trusted transport peer address only. This is the socket peer
    /// Keystone actually accepted the connection from, **not** a value parsed
    /// from a forwarding header unless the public TCP peer is an explicitly
    /// configured trusted proxy. `[auth_plugins] trusted_header` selects
    /// exactly one header that those proxies must sanitize (`x_forwarded_for`
    /// by default, or `forwarded` by explicit opt-in). A plugin will predictably
    /// build IP
    /// allowlisting, geo, or step-up logic on this field; handing it a
    /// client-spoofable `X-Forwarded-For` value would let an anonymous caller
    /// (§1 Threat Model, actor 2) forge whatever source address defeats that
    /// logic. `None` when no trusted address can be established, rather than an
    /// untrusted guess.
    ///
    /// **Implementation note on a degenerate configuration.** The resolver
    /// (`crate::net::resolve_client_ip`) walks the configured header chain
    /// right-to-left looking for the first entry *not* in `trusted_proxies`,
    /// which is the actual, spoof-resistant client address. If every entry in
    /// the chain - including the raw TCP peer - is itself a configured
    /// trusted proxy (an operator misconfiguration: a trusted-proxy CIDR that
    /// also matches real client addresses, or a chain with no genuine client
    /// hop at all), the resolver falls back to the trusted peer's own
    /// address rather than `None`. This is still a real, non-spoofable,
    /// trusted address (never an unverified guess from the request itself),
    /// just not necessarily the true originating client - a narrower
    /// deviation from "`None` when no trusted address can be established"
    /// than it may first appear, but one worth operators being aware of if
    /// `trusted_proxies` is scoped too broadly.
    remote_addr: Option<String>,
}

/// Opaque, single-invocation, host-issued handle. Returned by the
/// `provision_user`/`find_user` host functions (§6.B, §6.C) and the only
/// thing a plugin can present back to the host to claim an identity - see
/// "Identity Binding" below. Not a `user_id`; has no meaning outside the
/// `Store` instance that issued it and expires with that invocation.
struct ResolvedIdentityHandle(String);

enum AuthPluginResponse {
    Allow {
        /// Must be a handle this exact invocation received from a
        /// `provision_user` or `find_user` call - never a plugin-supplied
        /// user_id. See "Identity Binding" below for why.
        resolved_identity: ResolvedIdentityHandle,
        /// Extra claims to attach to AuthenticationContext for downstream
        /// policy (OPA) visibility - analogous to OidcContext claims.
        /// Bounded and reserved-key-checked by the host; see §6.F.
        claims: HashMap<String, serde_json::Value>,
    },
    Deny {
        /// Operator-facing reason, CADF-audited; never shown to the client.
        reason: String,
    },
}
}

This makes the plugin a peer of the existing Oidc/K8s variants rather than a request-mutation filter in front of them: it owns the full credential-verification decision for its method name. AuthenticationContext gains one variant:

#![allow(unused)]
fn main() {
WasmPlugin {
    plugin_name: String,
    claims: HashMap<String, serde_json::Value>,
    token: Option<FernetToken>,
},
}

The variant carries no plugin_sha256: the FernetToken payload is a fixed enum with no plugin-bearing variant (a WasmPlugin login mints an ordinary scoped token), so there is nowhere to embed and later re-compare a module hash. Version binding is instead keyed on plugin_name at verification time against the plugin’s configured valid_since cutoff - see “Plugin Version Binding” below.

which flows through the existing ValidatedSecurityContext::new_for_scope() pipeline (crates/core/src/auth.rs:79-181) unchanged - a plugin-authenticated principal is validated (enabled checks, expiry, effective-role calculation) exactly like any other IdentityInfo::User.

Guest Contract - mapping Mode

A mapping-mode plugin exports a different entry point, with no power to name an identity:

#![allow(unused)]
fn main() {
mapping(request: AuthPluginRequest) -> MappingResponse
}

#![allow(unused)]
fn main() {
enum MappingResponse {
    /// Flattened claims map, handed verbatim to the Mapping Engine's rule
    /// evaluator (ADR 0020 §5) as if it came from an OIDC/K8s/SPIFFE
    /// ingress adapter. There is no Allow variant here - the plugin cannot
    /// terminate authentication, only feed the engine that does.
    Claims(HashMap<String, serde_json::Value>),
    Deny {
        reason: String,
    },
}
}

AuthenticationContext is not extended for this mode - a successful mapping-mode login produces the existing Mapping(MappingContext) variant (crates/core-types/src/auth.rs:1200), because the Mapping Engine, not the plugin, made the decision. Mechanically:

1. ADR 0020’s IdentitySource enum (crates/core-types/src/mapping/resolution.rs, per ADR 0020 §3) gains a variant: WasmPlugin { plugin_name: String }, alongside the existing Federation, K8s, Spiffe.
2. Each mapping-mode plugin is automatically assigned provider_id = "wasm:<plugin_name>" in the Mapping Engine’s coordinate space. An operator writes ordinary MappingRuleSet rules under that provider_id (POST /v4/mappings, ADR 0020 §9.A) exactly as they would for a real OIDC provider - including IdentityMode::Local rules that resolve to real, pre-existing users (SCIM-provisioned or otherwise) by matching claims the plugin produced against whatever attributes identify that user.
3. The plugin’s Claims(...) response is passed to the Mapping Engine’s existing evaluator unmodified - the same engine, the same domain whitelist (0020 §3 allowed_domains), the same regex/size bounds (0020 §5.1), the same TOCTOU ruleset_version check (0020 §5.5–§5.6). Nothing new is built; the plugin is just a new kind of ingress adapter feeding into infrastructure this codebase already trusts.
4. If no MappingRuleSet exists under wasm:<plugin_name> for the target domain, evaluation returns MappingNotFound (0020 §5.5) and the login is rejected - fail-closed by construction: a plugin can authenticate no one until an admin has explicitly authored rules for it.

Why this needs no identity binding at all. Two properties of the Mapping Engine, both pre-existing and unmodified, do the work ResolvedIdentityHandle does for full_auth mode:

• Claims only ever drive rule matching, never become privilege directly. ClaimCondition/MatchCriteria (0020 §5.1–§5.2) read claim values to decide which admin-authored rule fires; the rule’s own hardcoded identity, authorizations, and is_system fields - not the raw claims - determine the outcome. A plugin cannot inject a claim that becomes is_system: true the way an unbounded claims map could in full_auth mode (§7 “Response Payload Bounds” reserved-key denylist exists precisely because that mode lacks this property) - there is no rule-independent path from “plugin said X” to “principal has privilege Y.”
• Ruleset lookup is provider_id-isolated. A mapping-mode plugin’s claims are only ever evaluated against rules filed under its own wasm:<plugin_name> coordinate (0020 §8 keyspace: data:mapping:v1:<domain_id>:wasm:<plugin_name>) - never against the ruleset backing a real SPIFFE trust domain or OIDC IdP. A compromised or buggy plugin cannot forge claims that get matched against a different source’s rules; the worst it can do is cause a mismatch/no-match against rules an admin wrote specifically expecting its own output.

Plugin-version binding for mapping mode - not enforceable at verification today (implementation deviation, recorded here rather than left implicit). The original intent was the same valid_since cutoff as full_auth, recovering the plugin name at verification time from the matched ruleset’s IdentitySource::WasmPlugin { plugin_name } via the token’s mapping_id. That requires the minted token to actually carry a mapping_id (or equivalent plugin-recoverable linkage) - it does not: a successful mapping-mode login mints an ordinary DomainScope/ProjectScope/… token (FernetToken::from_security_context’s AuthenticationContext::Mapping(_) arm, crates/core-types/src/token.rs) whose payload carries only methods = ["mapped"], the same as any OIDC/K8s/SPIFFE-sourced mapped login. There is no mapping_id field anywhere in a FernetToken payload, so verification (TokenService::validate_to_context_impl, crates/core/src/token/service.rs) has nothing to recover a plugin name from

• widening the payload to carry one is exactly the kind of per-record bookkeeping this ADR otherwise avoids, and was judged not worth it for this first iteration.

Consequence: a token minted through a mapping-mode plugin is not invalidated by bumping that plugin’s valid_since, unlike a full_auth token (whose methods does carry the plugin name and is checked at verification, see “Plugin Version Binding” below). An operator responding to a compromised mapping-mode plugin must fall back to the mechanisms every other token-holder relies on: issue revocation events for the affected users, or rely on a short token TTL. Closing this gap - by adding a plugin-recoverable field to the relevant token payloads - is left as future work (§8), not required for this ADR’s threat model, since mapping mode’s own structural constraint (the plugin never asserts an identity - §4 “Why this needs no identity binding at all”) already bounds what a compromised mapping-mode plugin can do to “produce claims the Mapping Engine’s already-authored rules for it will match or reject,” not “assert an identity directly.”

Capability restriction. provision_user, find_user, and assign_role (§6.B–D) are meaningless in mapping mode - the Mapping Engine owns provisioning and role resolution instead. Granting any of them to a mode = mapping plugin is a config-load-time error, the same fail-loud posture used throughout §5, rather than a silent no-op. Only http_fetch (§6.A) applies to both modes.

Guest Contract - route Mode

route mode answers a narrower question than either mode above: not “who is this?” but “which already-registered method should even attempt to answer that?” It runs before method dispatch, on the raw v3/v4 auth request, and its only power is to relabel which method handles the request and what payload that method sees - it never resolves an identity, never returns Allow, and is not itself subject to the namespace-scoped identity binding below (there is no identity for it to bind).

#![allow(unused)]
fn main() {
route(request: RouteRequest) -> RouteResponse
}

#![allow(unused)]
fn main() {
struct RouteRequest {
    /// The `identity.methods` list exactly as the client sent it, before
    /// any method resolution has happened.
    requested_methods: Vec<String>,
    /// Allowlisted subset of inbound HTTP headers - same `exposed_headers`
    /// mechanism and same hard denylist (`Authorization`, `Cookie`, ...) as
    /// `full_auth`/`mapping` (§4 "Guest Contract - `full_auth` Mode").
    headers: HashMap<String, String>,
    /// Raw JSON payload for each `identity.<method>` block the plugin's
    /// config has declared it needs to inspect (`inspect_methods`, §5).
    /// Blocks for methods not in `inspect_methods` are never included -
    /// a router configured to look at `application_credential` never sees
    /// the body of an unrelated `password` block, even on a request that
    /// carries both.
    payloads: HashMap<String, serde_json::Value>,
    /// Trusted transport peer only - identical provenance rule and
    /// spoofing rationale as `AuthPluginRequest.remote_addr` above.
    remote_addr: Option<String>,
}

enum RouteResponse {
    /// Leave the request exactly as received; ordinary method resolution
    /// proceeds as if no router plugin were installed.
    Passthrough,
    /// Reroute to `target_method`, replacing the `identity.<target_method>`
    /// block with `payload` verbatim. `target_method` MUST be a member of
    /// this plugin's configured `route_targets` allowlist (§5) - a
    /// response naming any other method is rejected by the host as a
    /// malformed response (§7), not corrected or silently dropped.
    Route {
        target_method: String,
        payload: serde_json::Value,
    },
    Deny {
        reason: String,
    },
}
}

Structural constraints, enforced host-side, not by convention:

1. Target-method allowlist. A route_targets list (§5) is required at config load; a Route response naming a method outside it is treated as a malformed response under §7’s failure semantics - the login is rejected, not redirected to an unintended handler. route_targets itself is subject to the same reserved-name check as plugin registration (§4 “Reserved Auth-Method Names”) - a router can never be configured to target admin, trust, or any other method capable of reaching system scope.
2. scope is immutable to the router. The host constructs the re-dispatched request by replacing only the named identity.<target_method> block; RouteResponse carries no scope field at all, so there is no code path by which a router can widen or redirect the requested project/domain/system scope. Scope resolution remains entirely owned by whichever method (or the Mapping Engine, for a mapping-mode target) ultimately handles the rerouted request.
3. Single-shot. A request that has already been through one route-mode dispatch is not eligible for another - the host does not re-invoke any router (the same one or a different one) on a request it has already rewritten. This is enforced structurally (the rerouted request carries an internal flag no guest can set or clear), not left as an operator convention, and is what keeps this mechanism from degenerating into unbounded recursive rewriting.
4. No identity, no claims, no credential synthesis. The target method still performs its own complete, unweakened verification against whatever payload it receives - a router deciding “this looks like a credential for user X” is never treated as proof of that. payload may only carry values the plugin was actually able to read out of payloads/headers in this same invocation (the host does not prevent a plugin from fabricating an arbitrary JSON payload, since it cannot distinguish “reshaped” from “synthesized” at the type level - but the target method’s own secret/ signature verification is exactly the backstop that makes fabrication harmless: a router can relabel which handler sees a credential, and can restructure the bytes it was given, but cannot manufacture a passing verification result for a credential it never actually possessed).
5. Trigger scoping. inspect_methods (§5) bounds not just what the plugin sees but whether it is invoked at all: a request whose identity.methods contains none of the configured inspect_methods entries never reaches this plugin. This is the main lever for containing the router’s blast radius - without it, a router would need to run on every login attempt for every method, including ones (password, token) it has no legitimate reason to ever see.
6. Fail-closed, independent budget. A route-mode failure (trap, timeout, fuel exhaustion, malformed response, off-allowlist target) rejects the login exactly like any other plugin failure (§7) - it never falls through to dispatching the original, un-routed request. Because a router is reachable by a strictly larger slice of traffic than a full_auth/mapping plugin (every request matching inspect_methods, not just requests already addressed to this plugin by name), its invocation_rate_limit_per_minute/ max_concurrent_invocations budget (§7) is tracked independently of the target method’s own budget - a saturated router degrades only routing decisions for its inspect_methods, never the target method’s headroom for requests that reach it directly.

Capability restriction. provision_user, find_user, and assign_role (§6.B–D) are config-load-time errors for mode = route, identically to mapping - a router does not resolve or grant anything, only redirects. http_fetch (§6.A) is permitted (a router may need to consult an external service to decide a route) but its cost is paid on every matching request, including ones that end up as Passthrough; operators should budget timeout_ms accordingly given route mode sits ahead of, not instead of, the target method’s own processing time.

Audit. The mandatory audit wrapping (§6.E) records, for every route-mode invocation, the client’s originally-requested method list, the plugin’s decision (Passthrough/Route/Deny), and - for Route - the resulting target_method. This is deliberate: without it, a CADF event for the eventually-authenticated request would show only the routed-to method, making it look as though the client had requested that method directly, which is exactly the wrong picture for an operator investigating a routing plugin gone wrong.

No router version binding on the issued token - by design. A route-mode plugin does not mint a token; the target method it routes to does, and that token is subject to the target’s version binding (the target plugin_name’s valid_since cutoff, for a full_auth or mapping target), not the router’s. This is intentional, not an oversight: the token-version-binding defense (§4 “Plugin Version Binding”) exists to invalidate credentials a vulnerable authenticator minted, and a router never authenticates - it cannot mint a credential for anyone the target method didn’t independently verify (constraint 4 above). Patching a buggy router therefore has nothing to retroactively invalidate: any token that exists was authorized by a target method’s own still-bound verification, and the router’s fix takes effect on the next request the instant its new module loads at startup (§5). The one thing a router bug does get bound into is the audit trail, which is where a routing bug actually needs to be reconstructable.

Identity Binding (full_auth Mode): Handles Into a Plugin-Owned Namespace, Not Raw user_id

A plugin is never given a channel to assert “bind this token to user_id <arbitrary-uuid>.” If it were, a single logic bug or malicious response from a plugin would be a full authentication bypass - it could name any existing account, including a cloud-admin’s, without ever checking a credential for it.

It is not enough, however, to merely require the plugin to obtain a handle before asserting an identity - a handle-issuing find_user that performs a general, unscoped lookup (“does any user named <x> exist?”) reopens exactly the same bypass one function call later: a plugin (buggy, or simply never implementing the credential check it’s supposed to) could call find_user("admin"), receive a valid handle for the real admin account, and return Allow - never having verified anything. The host would then correctly-but-uselessly confirm “yes, this handle was legitimately issued,” because the lookup itself was the hole.

The actual control is namespace scoping, not merely indirection through a handle. provision_user and find_user do not perform a general Keystone user search at all - they operate exclusively against a federated-identity mapping private to that plugin, keyed by (plugin_name, external_id), where external_id is a string the plugin’s own guest logic derives (e.g. from a verified external SSO subject or signed assertion) and has no relationship to a Keystone username. This is the same pattern this codebase already uses for OIDC/K8s/SPIFFE federation - find_federated_user(ctx, idp_id, unique_workload_id) (crates/core/src/mapping/service.rs:374-433) never searches local accounts by name either; it only ever resolves (idp_id, external_id) pairs a prior provisioning step created under that same source.

#![allow(unused)]
fn main() {
// Host functions §6.B / §6.C - signatures pinned here specifically so an
// implementer cannot accidentally build the unscoped version above.
fn provision_user(external_id: String, user: UserCreate) -> ResolvedIdentityHandle;
fn find_user(external_id: String) -> Option<ResolvedIdentityHandle>;
}

Mechanically:

1. provision_user(external_id, user) creates (or, on a repeat call with the same external_id, returns the existing) real User row via IdentityBackend::create_user, and records a mapping (plugin_name, external_id) -> user_id - separate storage from, and invisible to, any other plugin or auth method. find_user(external_id) looks up only within that same (plugin_name, external_id) mapping. Neither function will ever resolve a user_id that this plugin did not itself provision - a password-authenticated admin account, a user created via the API, or an identity provisioned by a different plugin are all structurally unreachable, not merely policy-forbidden.

2. Both return an opaque ResolvedIdentityHandle unrelated to the real user_id, from which the host can recover (user_id, domain_id) on the Allow.resolved_identity echo below.

   Implementation deviation: a signed token, not a per-Store map. The original intent was an in-memory handle -> (user_id, domain_id) map scoped to the single Store created for that invocation - random, unguessable, and structurally impossible to reuse across invocations because the map itself doesn’t outlive one. The actual implementation (CoreHostFunctions in crates/core/src/auth_plugin.rs) instead HMAC-signs {plugin_name, user_id, domain_id, expires_at} with a process-lifetime random key and hands the signed bytes back as the “handle.” This is a consequence of how host functions are registered in this implementation (§6 deviation, above): the extism::Function closures are shared across every concurrent invocation of a plugin’s compiled module, not instantiated fresh per Store, so there is no per-invocation-scoped place left to keep an in-memory map safely isolated between concurrent requests. A signed, expiring token gives an equivalent security property - unforgeable without the key, and resolves only to exactly what a prior provision_user/find_user call in this plugin’s namespace actually returned - without relying on state scoped to a single invocation. The expires_at is set generously past a plugin’s own timeout_ms budget (default 60 seconds) rather than truly one-shot: the trade-off is a small window in which a captured handle from one invocation could in principle be replayed into a later one, versus the ADR’s original “expires with that invocation” property. This is bounded, not open-ended - find_user’s live-domain_id re-check (step 3, “Admin-Authorized External Identity Linking”) still applies to a replayed handle exactly as it would to a fresh one - but it is a real, if narrow, deviation from a strictly per-invocation-scoped handle.

3. If the guest’s Allow.resolved_identity verifies against the signing key (or, in the originally-intended design, matches an entry in that map), the host substitutes the real (user_id, domain_id) when constructing the token. If it does not verify - a fabricated handle, a tampered handle, one issued for a different plugin, an expired one, or no provision_user/find_user call was ever made - the request is rejected exactly like any other malformed response (§7), and the mismatch itself is audited as a suspicious event.

4. A plugin granted neither provision_user nor find_user (§6) has no way to produce a valid handle at all, and can therefore only ever Deny - this is enforced at config-load time: registering a plugin as an [auth] methods entry without at least one of those two capabilities is a startup configuration error, not a silent no-op.

This makes “which real accounts can this plugin authenticate as” a direct, auditable, structurally bounded function of what that specific plugin has itself provisioned - never a lookup against the wider pool of existing Keystone accounts, and never a free-form claim. What the host still cannot do - because it’s inherent to letting arbitrary code define an auth method - is verify that the plugin performed a correct credential check before calling provision_user/find_user with a given external_id. Namespace scoping bounds the failure mode to “this plugin’s own provisioned identities might be mis-authenticated,” never “any account in the system might be.”

Admin-Authorized External Identity Linking (full_auth Mode)

For the case mapping mode doesn’t cover - a full_auth plugin that must remain the terminal identity authority, but needs to authenticate a pre-existing user it did not itself provision (a SCIM-provisioned user, for instance, where the login decision genuinely can’t be expressed as a static mapping rule) - the (plugin_name, external_id) -> user_id table find_user reads from (above) can also be populated by an administrator, out of band, instead of only by the plugin’s own provision_user calls. The plugin side is unchanged: find_user(external_id) still just resolves whatever is in that table today, whether the entry came from the plugin provisioning it or from an admin linking it.

Domain restriction is re-checked at resolve time, not only at link time. The provision_domain_id/allowed_provision_domains bound (§6.B) is enforced both when the link is created (below) and every time find_user resolves a handle: the host re-reads the target user’s current domain_id and rejects the resolution (auditing it as a mismatch, §6.E) if that user has since moved outside the plugin’s configured domain set. A link-time-only check would leave a stale window - an identity linked while in-domain, then administratively moved to another domain, would otherwise remain authenticatable by a plugin that was never granted reach into the user’s new domain. Because the check rides on the user’s live domain_id, a domain move closes that reach immediately without needing a separate link-cleanup step.

Why admin-authorized rather than plugin-self-service. This is the one mechanism in this ADR that intentionally lets a full_auth plugin reach an identity it didn’t create - so it has to be gated by something the plugin itself cannot trigger. An admin action, taken once, out of band, requiring ordinary Keystone RBAC (not the plugin’s own runtime logic), is that gate. A coarser alternative - letting a plugin resolve any user within its configured domain without a per-identity link - was considered and rejected: it would trade the “only explicitly-authorized identities are reachable” guarantee for zero admin overhead, and a buggy or exploited plugin (§1 Threat Model, actor 1) could then authenticate as anyone in that domain, not just identities someone deliberately opted in. The per-identity (or per-realm, below) authorization step is the load-bearing control; nothing here is meant to be bypassable for convenience.

API. POST /v4/auth_plugins/{plugin_name}/identity_links with body {external_id, user_id}. RBAC-tiered the same way ADR 0020 §9.A gates mapping writes: system-admin authorization is required to link a user who holds any system-scope role assignment; domain-admin authorization, scoped to the target user’s own domain, suffices otherwise. The endpoint additionally enforces the plugin’s provision_domain_id/allowed_provision_domains restriction (§6.B) against the target user’s domain - a link can never place a user outside the domain(s) the plugin was already configured to reach, keeping that invariant uniform regardless of whether an identity arrived via self-provisioning or admin-linking. Re-linking an external_id that already has an entry is rejected (409 Conflict); an admin must explicitly DELETE .../identity_links/{external_id} first - no silent overwrite, so an external_id can’t be quietly reassigned to a different user by mistake or by a compromised admin session skating past a diff review. Both create and delete are CADF-audited via the same mandatory infrastructure as §6.E, and DELETE additionally triggers the existing token-revocation pipeline for the unlinked user_id (the same mechanism ADR 0020 §9.F uses when a virtual user is disabled) - an unlinked identity can’t keep using tokens issued while the link was live.

SCIM convenience. Rather than requiring an admin to hand-copy internal user_id UUIDs, the same endpoint accepts {scim_provider_id, scim_external_id} in place of {external_id, user_id}: the host resolves user_id via the existing index:scim:external_id:<domain_id>:<provider_id>:<type>:<external_id> index (ADR 0024 §3.B) instead of introducing parallel storage, and sets the plugin’s external_id to the SCIM externalId the IdP already assigned that user. This is the direct answer to “users provisioned by SCIM need to be authenticatable via a custom plugin”: an admin pairs the plugin with the SCIM realm’s already-tracked identities, one link (or a small bulk batch) at a time, rather than the plugin ever being able to decide that for itself.

Plugin Version Binding & Token Invalidation (full_auth Mode)

Mirroring the ruleset_version TOCTOU defense in ADR 0020 §5.5–§5.6 (a token issued under one mapping ruleset is rejected if the live ruleset has since changed), a token minted via WasmPlugin is invalidated when the plugin behind its plugin_name is patched. The mechanism is a timestamp cutoff, not an embedded hash: the FernetToken payload is a fixed variant set with no plugin-bearing case (a WasmPlugin login mints an ordinary scoped token that already records its own issued_at), so there is no room to embed a plugin_sha256 in the token and re-compare it later. Instead, each plugin’s config carries an optional valid_since timestamp (§5). On every verification of a WasmPlugin-authenticated token, the host compares the token’s issued_at against the valid_since configured for that plugin_name: if issued_at predates valid_since, the token is rejected with a dedicated PluginVersionMismatch error, forcing re-authentication against the current plugin logic. An operator patching a plugin to fix a security bug bumps valid_since (normally alongside the pinned sha256) to the cutover instant, which invalidates every token the vulnerable version minted while leaving the rest of the process running. This is verification-time only: a brand-new login has no token yet (issued_at is set as the token is minted), so a past valid_since never blocks fresh authentication - it only invalidates already-outstanding tokens.

The trade-off relative to an automatic hash-drift check is that invalidation is driven by an operator action (bumping valid_since) rather than falling out of the sha256 change itself: an operator who swaps the .wasm and updates sha256 but forgets to advance valid_since leaves the old version’s tokens valid until they expire naturally. Treating valid_since as a mandatory companion to any sha256 change - the same “plugin config change is a staged rollout, not an ordinary edit” discipline §5 already calls for - is the load-bearing operator convention here.

Bulk Revocation on Plugin Compromise (full_auth Mode)

Version binding above stops a token minted under a since-patched plugin from being used, but does nothing about the persistent state a plugin already wrote while it was live and trusted - accounts it provisioned via provision_user, role assignments it granted via assign_role, and identity links an admin created for it (“Admin-Authorized External Identity Linking”, above). If an operator’s response to a compromise is “patch the plugin and move on,” that state is exactly what an attacker who exploited it would want left behind. Cleaning it up one DELETE .../identity_links/{external_id} at a time, or by hand-querying the CADF audit trail (§6.E, which records plugin_name on every such write) for everything to walk back, is only manual, error-prone rollback under incident-response time pressure.

API. POST /v4/auth_plugins/{plugin_name}/revoke_all. System-admin only

• this is a cross-domain action by construction, since a plugin’s provision_domain_id/allowed_provision_domains (§6.B) can span multiple domains and its role grants (§6.D) can land on any project within them, so no narrower RBAC tier is meaningful. Scoped entirely to the named plugin, in one call it:

1. Disables (does not delete) every user_id the plugin provisioned via provision_user, and every user_id reachable only through an admin-authorized identity link to it (above) - reusing the same disable path ADR 0020 §9.F already uses for a disabled virtual user, not a new deletion code path.
2. Deletes every remaining identity_links entry for that plugin - the batched equivalent of the existing per-external_id DELETE, above.
3. Triggers the existing token-revocation pipeline for every affected user_id, so a token minted before the operation ran cannot keep working on a since-disabled account - the same window the per-identity DELETE already closes for one identity at a time, now closed for all of them at once.

It deliberately does not revoke the role assignments the plugin granted via assign_role. Attributing a stored assignment to the plugin that created it would require every grant to carry a per-record origin marker - exactly the kind of per-write bookkeeping this ADR rejects for version scoping (below), and which the assignment store does not otherwise need. Because disabling the account already denies all access, a leftover grant is inert unless an operator later re-enables that user; at that point it is the operator’s responsibility to review the re-enabled user’s assignments against the CADF audit trail (§6.E records plugin_name on every assign_role) and revoke any they deem compromised via the existing per-grant revocation API. This keeps “get everything shut off fast” free of schema additions, and leaves selective assignment cleanup - like the selective account reinstatement discussed below - a deliberate manual step rather than an automatic one.

Each of the above is individually CADF-audited exactly as its single-record equivalent already is (§6.E) - this endpoint is a bulk driver of existing, already-reviewed disable/revoke/unlink operations, not a new privileged code path with its own semantics. It responds with a per-category count (users disabled, links deleted) so the operator gets confirmation of blast radius covered without a separate audit-trail query, and re-running it against a plugin with no remaining state is a no-op (200, all-zero counts) - safe to include in a standard incident-response runbook without first checking whether a prior run already covered it.

Why plugin-name-scoped, not version-scoped. The action targets everything attributable to plugin_name, not only writes made while one specific plugin binary (sha256) was loaded. Scoping to a single vulnerable version would require every provisioning/grant/link record to carry the sha256 active at write time - bookkeeping this ADR does not otherwise need (version binding, above, is a single per-plugin valid_since timestamp compared to a token’s issued_at, and needs no per-record hash at all) - and would leave state from any other version of the same plugin untouched, the wrong default the moment an operator’s trust in a plugin binary has been broken. An operator confident only one version is implicated can still hand-verify individual accounts against the audit trail’s plugin_name + timestamp before re-enabling them; this endpoint optimizes for “get everything shut off fast,” not selective reinstatement.

Reserved Auth-Method Names

At config load, a plugin’s name is checked against the fixed set of builtin method names (password, token, openid, application_credential, trust, webauthn, mapped, k8s, admin, totp). A collision is a startup configuration error - a plugin can never be registered under a name that would shadow or be confused with a compiled-in auth method.

A route-mode plugin’s route_targets (§5) list is checked against the same set, minus the strictly reachable subset: admin and trust may never appear in a route_targets list regardless of plugin, since neither is a method a router’s blast radius should ever be able to reach - this is a startup configuration error, identical in posture to the name-collision check above.

5. Distribution & Loading

Plugin bytecode lives on the local filesystem. A new config section, one sub-entry per plugin, follows the existing per-subsystem [section] key = value convention (e.g. crates/config/src/k8s_auth.rs):

[auth_plugins]
plugins = acme_risk_sso,tf_appcred_router
# Header every trusted proxy sanitizes. Defaults to x_forwarded_for.
trusted_header = x_forwarded_for
# Comma-separated proxy CIDRs; empty trusts no forwarding header.
trusted_proxies =

[auth_plugin.acme_risk_sso]
path = /etc/keystone/plugins/acme_risk_sso.wasm
sha256 = 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
# Plugin version binding (§4 "Plugin Version Binding"): any token whose
# issued_at predates this instant is rejected with PluginVersionMismatch,
# forcing re-auth against the current module. Bump this to "now" whenever
# `sha256` changes so tokens minted by the previous binary stop verifying.
# Optional - omit it and no token is ever rejected on version grounds.
valid_since = 2026-07-02T00:00:00Z
# full_auth (default): plugin is the terminal identity authority, may call
# provision_user/find_user/assign_role, can reach pre-existing users only
# via an admin-created identity_link (§4). mapping: plugin only produces
# claims for the Mapping Engine (ADR 0020) to evaluate; provision_user/
# find_user/assign_role are config-load errors in this mode (§4). route:
# plugin runs pre-dispatch and may only relabel identity.methods + hand a
# payload to one allowlisted target method - provision_user/find_user/
# assign_role are also config-load errors in this mode (§4).
mode = full_auth
# Capabilities are host functions this plugin may call (§6 A-D only -
# auditing is mandatory host-side infrastructure, not a capability; see §6.E).
capabilities = http_fetch,provision_user,find_user
exposed_headers = X-Acme-Session-Id
allowed_hosts = risk.acme.example.com
# Host injects this header on every http_fetch call; the value itself is
# read from the referenced environment variable and never enters guest
# memory (§6.A).
http_fetch_auth_header = Authorization
http_fetch_auth_secret_env = ACME_RISK_API_KEY
# provision_user (§6.B) may only create users in this domain; a UserCreate
# targeting any other domain_id is rejected before reaching IdentityBackend.
provision_domain_id = domain_acme_sso
assign_role_allowed = reader,member
timeout_ms = 750
fuel_limit = 50000000
memory_limit_mb = 32
# Per-(plugin, source-IP) bucket, checked before the plugin-wide bucket below
# (§7 "Invocation Rate Limiting & Concurrency") - keeps one anonymous caller
# from exhausting this method's shared budget for everyone else.
invocation_rate_limit_per_source_per_minute = 20
invocation_rate_limit_per_minute = 300
max_concurrent_invocations = 16

# route-mode example: Terraform's OpenStack provider always sends
# identity.methods = [application_credential]. This plugin inspects the
# application_credential_id shape and, for IDs matching its own convention,
# reroutes the request to a separately-registered full_auth plugin
# (hacked_appcred_handler, not shown) that performs the real verification;
# every other application_credential request passes through unmodified to
# the builtin handler.
[auth_plugin.tf_appcred_router]
path = /etc/keystone/plugins/tf_appcred_router.wasm
sha256 = 3b5d5c3712955042212316173ccf37be9de53d6c84a5c7c8e6e0e5e7f5f8a1b
mode = route
# Only requests whose identity.methods includes one of these entries ever
# invoke this plugin at all - a plain `password` login never reaches it.
inspect_methods = application_credential
# Methods this plugin is permitted to reroute a request to. A Route
# response naming anything else is rejected as malformed (§4 "Guest
# Contract - route Mode"), never dispatched.
route_targets = hacked_appcred_handler
capabilities =
timeout_ms = 200
fuel_limit = 5000000
memory_limit_mb = 8
invocation_rate_limit_per_minute = 6000
max_concurrent_invocations = 64

Loading. All configured plugins are loaded and their checksums verified independently, once, at process startup. If a given plugin’s file is missing or its SHA-256 does not match the pinned value, that plugin is disabled - not registered as an auth method, not reachable by any identity.methods entry - and the host emits a CRITICAL-level structured log line plus a dedicated metric/counter (e.g. keystone_auth_plugin_load_failure{plugin_name}) naming the plugin and the mismatch, wired to whatever alerting an operator already has on Keystone process health. The node itself, every other correctly verified plugin, and every builtin auth method (password, openid, k8s, …) start and serve normally. This is a deliberate change from an earlier draft of this ADR, which made a checksum mismatch a hard process-startup failure for the whole node - rejected here because a plugin failing to load degrades availability of exactly one auth method (fail-closed at the request level, identical to any other plugin runtime failure, §7), and blocking every builtin method cluster-wide over one misconfigured plugin was a disproportionate availability cost for what is very often a copy-paste typo in keystone.conf rather than a genuine tamper/corruption event.

Cross-node divergence is the trade-off this creates, and it is accepted explicitly rather than left implicit. Because each node verifies its own copy of the .wasm file independently, a node with a correct hash keeps a plugin live while a sibling node with a typo’d or stale hash disables it - the cluster can run with that auth method inconsistently available across nodes until an operator acts on the alert. This is judged preferable to the previous whole-cluster-down failure mode precisely because it is loud rather than silent: the CRITICAL alert exists specifically to convert what would otherwise be an unnoticed divergence into a paged one. Operators who need the stronger all-or-nothing guarantee this ADR previously provided can still get it operationally - verify the pinned hash against every node’s .wasm file as a pre-deploy gate in their rollout pipeline - and should continue to treat a plugin config change with the same care as a schema migration, not an ordinary config edit.

Why filesystem, not Raft/FjallDB. Given the cluster-global-only trust scope (§8), there is no per-domain data to replicate, so the main advantage of storing bytecode in the replicated log - automatic cross-node consistency for tenant-authored content - does not apply. Filesystem distribution defers that machinery until a per-tenant plugin model is actually needed.

6. Host Functions: Curated Capability Allowlist

A plugin can only call the host functions listed in its capabilities config entry (§5). The security property is that an unlisted call can never be successfully exercised, structurally or otherwise - not merely a permission check that could be bypassed by a bug in the check itself.

Implementation deviation from “not registered at all.” The original intent was that an ungranted function is absent from the guest’s import table entirely. The actual implementation registers all four host functions (§6 A-D) into every plugin’s compiled extism::Plugin module whenever any HostFunctions provider is configured, and each function’s closure independently rejects a call its plugin’s capabilities didn’t grant, before doing anything else. This is a deliberate, documented substitution (crates/auth-plugin-runtime/src/host_functions.rs), not an oversight: wasmtime requires every guest-declared import to resolve at instantiation time regardless of whether the compiled module actually calls it - an unresolved import fails every invocation of that module, not just the specific call that would have used it - so selectively omitting a function’s registration per plugin is not viable for a single compiled module shared across differently-configured invocations. From the guest’s perspective the result is identical either way: an ungranted capability structurally cannot be exercised, since the closure’s gate check runs before any host-side effect (DB write, outbound HTTP call, etc.) and is not something plugin-supplied input can influence or bypass.

A. http_fetch

Backed by Extism’s built-in HTTP bridge, restricted to the plugin’s allowed_hosts list (§5) - the same allow-list-of-hosts posture already implicit in how keystone-rs talks to fixed external services (OPA base URL, K8s API server). Unlike the existing reqwest::Client in crates/keystone/src/k8s_auth_client.rs:73-75, which sets no per-request timeout, the host-side bridge for http_fetch must enforce both a connect and a total-request timeout derived from the plugin’s timeout_ms budget - closing a gap noted during this ADR’s research rather than propagating it into a new, more exposed (third-party-triggered) call path.

Because every http_fetch call is ultimately triggered by an anonymous, pre-authentication caller (§1 Threat Model, actor 2), the bridge is an SSRF surface and is hardened accordingly, as a hard requirement of this ADR rather than an implementation detail:

• Connect-time IP validation, not just hostname matching. allowed_hosts is a hostname allowlist, but DNS is attacker-adjacent (a plugin’s configured third-party host is not attacker-controlled, but its DNS resolution path is outside Keystone’s control). The bridge re-resolves the hostname at connect time on every call (no long-lived resolution cache) and rejects the connection if the resolved address falls in a private, loopback, link-local, multicast, or cloud-metadata range (RFC 1918, RFC 4193, 127.0.0.0/8, 169.254.0.0/16 including 169.254.169.254, etc.) - regardless of what the configured hostname was. The socket then connects to the exact IpAddr that was validated - the host does not hand the hostname back to the HTTP client to resolve a second time. This distinction is load-bearing, not pedantic: a validate-then-re-resolve implementation reopens the exact DNS-rebinding TOCTOU this control exists to close, because the second lookup can return a private address the first one didn’t. Validation and connection must observe the same resolved address. This closes the standard DNS-rebinding bypass of a hostname-only allowlist.
• No automatic redirect following. The bridge does not follow HTTP redirects by default; a 3xx response is returned to the guest as-is. An operator who needs redirect support opts in per-plugin (http_fetch_follow_redirects = true), and even then each redirect hop is re-validated against both allowed_hosts and the IP-range check above before being followed - a redirect is not permitted to silently escape the allowlist. The whole chain shares a single timeout_ms wall-clock budget, not one budget per hop: each hop’s own request timeout is the remaining time before that shared deadline, so a redirect chain cannot cost up to MAX_REDIRECTS + 1 times the plugin’s configured budget - the total wall-clock cost of a (possibly multi-hop) http_fetch call stays bounded to the same timeout_ms §7’s per-invocation deadline is sized against.
• Outbound secrets are host-injected, never guest-visible. If a plugin needs to authenticate to its external service, the secret value (API key, bearer token) is not embedded in the .wasm binary and is not passed into guest memory. Instead, http_fetch_auth_header + http_fetch_auth_secret_env (§5) tell the host which header to attach and which host-side environment variable (or, in a future iteration, secret store reference) to read the value from; the host attaches it to the outbound request after the guest has already specified the rest of the request, so the plugin’s WASM code and the .wasm file distributed to every node never contain the credential in any form.

This is a stricter posture than the existing reqwest-based clients elsewhere in keystone-rs (OPA, K8s TokenReview) because those call fixed, operator-configured single endpoints that are never influenced by an anonymous caller’s request; http_fetch is triggered by exactly such a caller on every invocation.

B. provision_user

provision_user(external_id: String, user: UserCreate) -> ResolvedIdentityHandle (§4 “Identity Binding”). Wraps IdentityBackend::create_user(state: &ServiceState, user: UserCreate) -> Result<UserResponse, IdentityProviderError> (crates/core/src/identity/backend.rs:119-123) - the same call the Unified Mapping Engine’s Local identity mode uses today (find_or_create_federated_user, crates/core/src/mapping/service.rs:374-433) - but, critically, is namespaced by (plugin_name, external_id) rather than performing a general create-or-lookup by username: a repeat call with the same external_id returns the same provisioned user rather than erroring or creating a duplicate, and no other plugin or auth method can resolve a user_id through this mapping. This idempotency must be atomic on the (plugin_name, external_id) mapping (upsert / unique-constraint-backed insert-or-get), not a check-then-create: with max_concurrent_invocations > 1 two first-login requests for the same external_id run in parallel Stores, and a non-atomic implementation races into either a duplicate User row or a unique-constraint error surfaced as a spurious auth failure. The mapping’s own uniqueness on (plugin_name, external_id) is the serialization point. This directly satisfies requirement 3 (“provision user on first login”): the guest calls provision_user with the external identifier and fields it has already verified (federated attributes, username), and the host performs the actual DB-backed creation under the existing identity backend - the plugin never gets raw DB or ServiceState access, only this single narrow, namespace-scoped entry point. The host does not return the real user_id to the guest - it returns a ResolvedIdentityHandle that the guest can only use by echoing it back in Allow.resolved_identity.

Field sanitization. The UserCreate handed to the host is not written through verbatim. Identity-critical fields are host-controlled, not guest-controlled:

• id - always host-generated. A guest-supplied id is rejected (not ignored), because a plugin that could name the new user’s UUID could target an existing account’s id - colliding with, or attempting to overwrite/alias, a password-authenticated admin - which is exactly the arbitrary-user_id claim the identity-namespace design (§4) exists to prevent, smuggled in one layer down through the create path.
• domain_id - see domain restriction below.
• Any privilege- or auth-relevant option field (e.g. federated-vs-local flags, password material, admin-ish user options) is either host-fixed or rejected; the guest may only supply the narrow set of descriptive attributes it has actually verified (external username, display attributes). The accepted-field set is an allowlist, so a field added to UserCreate in the future is excluded by default rather than silently passed through from the guest.

Domain restriction. user.domain_id is not accepted verbatim from the guest. Each plugin’s config declares provision_domain_id (§5) - a single fixed domain (or, if genuinely needed, allowed_provision_domains, a small explicit list) - and any UserCreate targeting a domain outside that set is rejected before reaching IdentityBackend. This mirrors ADR 0020’s allowed_domains whitelist for federated provisioning (0020 §3, §7.3 AllowedDomainsRequired) and bounds a buggy plugin (§1 Threat Model actor 1) to the domain(s) an operator explicitly intended to hand it write access to, rather than “any domain in the cluster.”

C. find_user

find_user(external_id: String) -> Option<ResolvedIdentityHandle> (§4 “Identity Binding”). Read-only lookup within the same (plugin_name, external_id) namespace provision_user writes to - used for idempotent “does this identity already exist” checks before provisioning. This is not a general username/attribute search over the Keystone user table: find_user structurally cannot resolve a handle for any account it (or a prior invocation of the same plugin) did not itself create via provision_user. This is the load-bearing restriction that makes handle-based identity binding (§4) an actual authentication-bypass defense rather than an extra function call in front of the same bypass. For entries populated by admin-authorized linking rather than the plugin’s own provisioning, find_user additionally re-validates the resolved user’s live domain_id against the plugin’s provision_domain_id / allowed_provision_domains on every call (§4 “Admin-Authorized External Identity Linking”), so a post-link domain move revokes reach immediately.

D. assign_role

Grants a role assignment for the newly resolved principal (identified by a ResolvedIdentityHandle from B/C - the same anti-impersonation constraint applies here: a plugin cannot assign a role to an arbitrary user_id it merely names). Kept as a separate, individually-grantable capability from provision_user so a plugin can be scoped to “create users but never touch role assignments” if an operator wants that split.

Scope restriction. A plugin’s assign_role grant is bounded on three axes, all enforced host-side:

• Which role - assign_role_allowed (§5): an explicit, per-plugin allowlist of role names it may assign.
• Which target project/domain - the assignment may only land on a project or domain within the plugin’s own provision_domain_id / allowed_provision_domains set (§6.B). The role name allowlist alone does not bound where the grant applies; without this, a plugin holding member in assign_role_allowed could grant its self-provisioned user member on an arbitrary project - including a sensitive one in another domain (e.g. the admin project). A grant whose target project/domain falls outside the configured provisioning domain(s) is rejected before reaching the assignment backend, the same fail-loud posture as the provision_user domain check.
• Which scope type - the load-bearing control against privilege escalation: assign_role only ever targets project/domain scope on the identity the same invocation resolved, and there is no code path by which a WASM plugin invocation can reach system scope - mirroring ADR 0020 §9.A’s rule that Authorization::System requires is_system: true plus admin-level authorization, neither of which a plugin invocation ever carries. (An earlier draft of this ADR additionally proposed rejecting assign_role_allowed entries for roles that “carry admin privileges” - cut here because Keystone roles have no intrinsic, statically-inspectable privilege flag at config-load time; role semantics are policy-interpreted, e.g. role:admin in Rego, not a DB property queryable before OPA/the database are even necessarily reachable. Claiming that check existed would have been a control on paper only. The scope-type restriction above is the real, implementable defense and is sufficient given plugins never carry system scope regardless of which role name is assigned.)

E. Mandatory Audit Wrapping (not a capability)

Unlike A–D, auditing is not something a plugin opts into or calls - it is host-side infrastructure that unconditionally wraps every invocation of A–D and the top-level authenticate outcome, regardless of what a plugin’s capabilities list contains. Each wrapped call emits a CADF-compatible event via the existing AuditHook (crates/core/src/events.rs:107-115) infrastructure (ADR 0023) - a dedicated EventPayload variant recording plugin_name, the host function called (if any), and outcome. It is registered as an inline, fail-closed hook (matching AuditHook’s existing semantics: if the audit hook itself fails, the triggering call fails closed too), not the fire-and-forget ProviderHooks pattern. Because this is infrastructure rather than a capability, there is no capabilities entry that enables or disables it, and no way for a plugin - buggy or otherwise - to provision a user or assign a role without that action being recorded.

F. Sandbox Baseline: No WASI

No WASI (preview1 or preview2) imports are registered into any plugin’s extism::Plugin instance - no ambient filesystem, clock, random, or environment access beyond what A–D above explicitly provide. This is a hard requirement of the host-function registration step (§3), not a follow-up item: a plugin’s only way to reach outside its own linear memory is through the capabilities in A–D that its config explicitly grants.

Capabilities not listed above (arbitrary internal method invocation by name, raw storage access, token minting) are intentionally not exposed. If a future use case needs one, it is added to this fixed list explicitly, in a follow-up ADR amendment - not opened up generically.

7. Resource Limits & Failure Semantics

Every authenticate invocation runs under three independent bounds, mirroring the existing precedent of the mapping engine’s regex evaluator (2-second deadline, 4 KiB per-value cap - ADR 0020 §5.1), each configurable per-plugin with cluster-wide defaults:

1. Fuel metering (fuel_limit) - bounds total instruction count, independent of wall-clock (protects against a plugin that spins without making forward progress on I/O, where a timer alone might not fire predictably under load).
2. Wall-clock deadline (timeout_ms) - bounds total invocation time including any http_fetch calls the guest makes.
3. Linear memory cap (memory_limit_mb) - bounds guest heap growth.

On any failure - a WASM trap, fuel exhaustion, timeout, an attempted call to an undeclared host function, or a response that fails to deserialize as AuthPluginResponse - the login attempt is rejected (generic 401 Unauthorized; no internal detail leaked to the client) and a CADF Failure audit event is emitted with a sanitized reason (§6.E). There is no automatic fallback to another [auth] methods entry: exactly as today, the client selects which method(s) it is attempting via the identity.methods field of the auth request, and a plugin failing its own method is not silently retried as password. This fail-closed posture was chosen because an auth method is exactly the kind of fail-closed-only inline hook AuditHook already models (crates/core/src/events.rs:102-106) - a partially-failed authentication decision must never be treated as an implicit “try something weaker instead.”

Isolation between requests. One wasmtime::Engine and one compiled extism::Plugin module are shared per process (compilation is expensive; correctness of the compiled module is immutable), but each authenticate call gets a fresh Store/plugin instance state - no mutable state persists across invocations. This prevents one request’s execution from leaking data into or influencing another’s, and means a fuel/memory exhaustion in one invocation cannot degrade a concurrent one.

Invocation Rate Limiting & Concurrency

Per-invocation resource bounds (fuel, wall-clock, memory) limit the cost of a single call, but every plugin invocation is reachable by an anonymous, pre-authentication caller (§1 Threat Model, actor 2) - nothing above bounds how many invocations happen concurrently or per unit time. Left unbounded, this is both a direct resource-exhaustion DoS against Keystone (unbounded parallel Store/wasm instances) and an SSRF/DDoS amplification vector against whatever host is in allowed_hosts (§6.A), since each anonymous login attempt can trigger an outbound HTTP call.

Three bounds apply per plugin, mirroring the two-tier rate limiter the Unified Mapping Engine already uses for its own externally-triggered shadow-registry writes (shadow_registry_creation_rate_limit, shadow_registry_auth_rate_limit

• ADR 0020 §7.2), reusing the governor crate already a workspace dependency (Cargo.toml:82):

1. Per-source rate limit (invocation_rate_limit_per_source_per_minute, §5) - a sliding-window token bucket keyed on (plugin_name, remote_addr), using the same trusted, non-spoofable remote_addr AuthPluginRequest carries (§4) - never a raw X-Forwarded-For value unless it arrived over a configured trusted-proxy hop. Checked first, in front of bound 2: a caller from a single source exceeding it is rejected with 429 Too Many Requests before it can touch the plugin-wide budget at all, audited as a RateLimited outcome scoped to that source. This is what actually stops one anonymous attacker (§1 Threat Model, actor 2) from being the party who burns bound 2’s shared budget for everyone else on that method.
2. Invocation rate limit (invocation_rate_limit_per_minute, §5) - a sliding-window token bucket per plugin, shared across all sources. Exceeding it rejects further authenticate calls for that plugin with 429 Too Many Requests until the window clears, audited as a RateLimited outcome.
3. Concurrency cap (max_concurrent_invocations, §5) - a semaphore bounding how many Store instances (and therefore how many in-flight http_fetch calls) may execute simultaneously for that plugin. A request arriving when the cap is saturated is rejected the same way as (1)/(2) rather than queued, to avoid building an unbounded backlog of pending authentications under load.

Defaults are conservative and layered (§5 example: 20/min per source, 300/min per plugin, 16 concurrent) and all three are per-plugin, not global, so one plugin’s traffic cannot starve another’s budget.

Bound 1’s keyed store is shrunk on a periodic tick, not per-request. Every distinct source address bound 1 sees - necessarily including anonymous, pre-authentication callers (§1 Threat Model, actor 2) - allocates an entry in the underlying keyed rate-limit store, which is never freed on its own; left unaddressed, a long-running process accumulates one entry per distinct source address it has ever seen. The process’s existing minute-scale background maintenance tick evicts entries whose bucket has fully recovered (i.e. is indistinguishable from a source never seen before) for every loaded plugin’s bound-1 store - a straightforward memory-bookkeeping fix, not a rate-limiting behavior change: an evicted, truly-idle source’s next request is treated exactly as a first-ever request would be, which is already correct.

Internal/admin interface behavior. Bound 1 applies only to public ingress. An unproxied public request is keyed by its raw TCP peer; a proxied public request is keyed by the client resolved through trusted_header. Internal mTLS and admin requests deliberately pass remote_addr = None, even though the mTLS listener records its shared mesh peer for audit logging, so unrelated internal services cannot exhaust one another’s per-source bucket. Those requests retain bounds 2 and 3.

Response Payload Bounds

Fuel/memory/timeout bound what a plugin can do internally; they do not bound the size or shape of the AuthPluginResponse JSON the host must deserialize from the guest’s output. Mirroring the caps ADR 0020 already applies to its own claim-derived data (4 KiB per claim value, 256-char interpolation limit - 0020 §5.1, §5.4), the host enforces, before attempting to deserialize the response:

• A hard cap on total response size (default 64 KiB) - an oversized response is rejected without being parsed.
• A cap on the claims map: at most 64 entries, keys at most 256 bytes, values at most 4 KiB each (matching ADR 0020’s existing claim-value limit).
• Structural namespacing, not a denylist. Every claim a plugin emits is carried under a single reserved envelope key and surfaced to downstream OPA / SecurityContext construction only as plugin_claims.<plugin_name>.<key> - never merged into the top-level claim namespace. A plugin therefore cannot set, shadow, or collide with any privilege-relevant top-level key regardless of what it names its own claims, because its keys structurally live in a sub-object no policy reads as authoritative identity/authorization input. This is deliberately a structural containment (like the (plugin_name, external_id) identity namespace in §4) rather than a blocklist: a denylist of forbidden keys (is_system, is_admin, roles, effective_roles, …) can only ever enumerate the privilege-relevant keys known today - the moment an operator writes a Rego policy or a future SecurityContext field that trusts a top-level claim not on the list (e.g. groups driving group→role mapping, project_id, trust_id), the denylist is silently incomplete and the plugin can inject it. Namespacing closes that entire class instead of chasing it key by key. As defense-in-depth the host additionally rejects (fails the whole Allow closed) any attempt by a plugin to emit the reserved envelope key itself or a claim key prefixed __keystone, the same fail-loud posture as ADR 0020’s SystemTokenShadowing write-time check (0020 §7.3).

Any violation is treated as a malformed response under the failure semantics above: the login is rejected, and the specific bound that was exceeded is recorded in the audit event (§6.E) for operator diagnosis, without echoing attacker-influenced content back into logs.

8. Open Questions / Future Work

• mapping-mode version binding at verification - not implemented. See §4 “Plugin-version binding for mapping mode.” A mapping-mode token has no plugin-recoverable field in its FernetToken payload, so bumping a mapping-mode plugin’s valid_since does not invalidate outstanding tokens the way it does for full_auth. Closing this requires either widening a token payload to carry a plugin-recoverable linkage (the per-record bookkeeping this ADR otherwise avoids) or a different invalidation mechanism entirely. Until then, incident response for a compromised mapping-mode plugin relies on revocation events or short token TTLs, not valid_since.
• Per-domain plugin scoping. This ADR deliberately restricts plugins to cluster-global, system-admin-installed, to keep the trust model simple for a first iteration. Extending this to a (domain_id, provider_id)-scoped model - parallel to how OIDC/K8s/SPIFFE providers work today (ADR 0020 §2) - is plausible future work, but raises open questions this ADR does not answer: could a domain admin install a plugin that calls provision_user outside their own domain? Should allowed_hosts be domain-restricted too? A follow-up ADR should address this once there is a concrete multi-tenant use case.
• Hot reload / upload API. Startup-only loading (§5) is the simplest correct starting point. An admin API to upload new plugin versions without a process restart - likely backed by Raft/FjallDB replication once plugins are no longer purely cluster-global-static - is deferred.
• Signing beyond a pinned checksum. SHA-256 pinning (§5) catches corruption and accidental drift but is not a substitute for a real code-signing chain if plugins are ever sourced from outside the operator’s own build pipeline. Not needed while distribution is “operator places the file themselves,” but worth revisiting if plugins become installable from a registry.
• Secret rotation. http_fetch_auth_secret_env (§6.A) reads a secret from a host-side environment variable at call time, which is enough to keep the value out of guest memory and out of the distributed .wasm file, but rotating it still requires a process restart today, same as any other env-var-sourced Keystone secret. A dedicated secret-store integration (rotation without restart) is future work, not required for this ADR’s threat model (§1).
• Remediation after a plugin is pulled for a security bug - addressed. Resolved by “Bulk Revocation on Plugin Compromise” (§4): a single POST /v4/auth_plugins/{plugin_name}/revoke_all disables everything the plugin provisioned/granted/was linked to and revokes affected tokens, on top of the token-level protection version binding (§4 “Plugin Version Binding”) already provides. What remains genuinely open: that endpoint is deliberately plugin_name-scoped, not scoped to a single plugin binary version (see its “Why plugin-name-scoped” rationale) - an operator who wants to reinstate only the state attributable to a different, non-vulnerable version of the same plugin must still identify and re-enable that subset by hand against the audit trail.
• Coarser domain-scoped resolution (considered, rejected). An earlier draft of this ADR considered letting a full_auth plugin resolve any user within its provision_domain_id without a per-identity admin link, purely to reduce admin overhead for operators with many pre-existing users to onboard. Rejected: it would mean a buggy or exploited plugin (§1 Threat Model, actor 1) could authenticate as anyone in that domain, not just identities someone deliberately opted in - trading away the one guarantee (“only explicitly-authorized identities are reachable”) this design is built around, for convenience. The SCIM bulk-linking convenience above (resolving via scim_provider_id/scim_external_id) covers the “many users to onboard” case without that trade-off - it’s still one explicit link per identity, just without requiring the admin to already know the internal user_id.

9. Consequences

Positive

• Operators can add custom authentication logic without forking keystone-rs or waiting on an upstream release - the stated requirement.
• The curated host-function allowlist (§6) means the security review surface for “what can a plugin actually do” is a fixed, auditable list per plugin, not “whatever Rust code can reach.”
• Namespace-scoped identity binding (§4) structurally prevents a plugin from authenticating as an arbitrary existing account - not just by blocking a raw user_id claim, but by making find_user/provision_user incapable of resolving any identity outside that plugin’s own (plugin_name, external_id) mappings. This is the strongest guarantee in this design: it bounds the blast radius of a buggy or exploited plugin (§1 Threat Model, actor 1) to “can authenticate as identities it itself provisioned,” never “can authenticate as anyone already in the system.”
• Fail-closed failure handling (§7) and mandatory, non-optional CADF auditing (§6.E) mean a misbehaving or exploited plugin cannot silently degrade into weaker-than-expected authentication or provision/grant without a trace.
• Per-source, per-plugin rate limiting and concurrency caps (§7) bound the damage an anonymous caller (§1 Threat Model, actor 2) can do simply by hitting the login endpoint, and specifically prevent a single bad actor from exhausting one auth method’s shared budget for every legitimate user of it - extending the same two-tier pattern ADR 0020 already established for its own externally-triggered writes with a source-scoped front tier.
• A bulk revoke_all admin endpoint (§4 “Bulk Revocation on Plugin Compromise”) turns “a plugin was compromised” from a manual, per-record cleanup exercise under incident-response pressure into a single audited call that disables everything the plugin ever provisioned, granted, or was linked to.
• http_fetch’s connect-time IP re-validation and host-injected secrets (§6.A) close the standard SSRF and credential-exposure pitfalls of an “allowed-hosts HTTP proxy” feature up front, rather than as a follow-up hardening pass.
• mapping mode (§4) lets a plugin serve pre-existing users - including SCIM-provisioned ones (ADR 0024) - without any identity-binding machinery at all, by delegating the actual decision to the already-reviewed Mapping Engine (ADR 0020). This is a strictly additive safety property: the plugin structurally cannot terminate authentication in this mode, so it inherits the Mapping Engine’s existing guarantees rather than needing new ones.
• Admin-authorized external identity linking (§4) gives full_auth plugins a path to pre-existing users too, for cases mapping mode can’t express, without ever letting the plugin itself decide who it can authenticate as - the gate is an ordinary RBAC-checked, audited, revocable admin action, not new plugin-facing trust.
• Reuses Extism’s existing HTTP allow-list and resource-limiting primitives rather than hand-rolling a WASI-sockets bridge and a custom fuel/timeout system.
• Closes an existing gap (missing per-request HTTP timeout in k8s_auth_client.rs) for the new, more exposed call path, without needing to touch the existing K8s client itself.
• route mode (§4) lets a client that can only ever send a fixed method name - Terraform’s application_credential-shaped auth is the motivating case - be transparently redirected to the handler that actually knows how to verify its credential, without collapsing the routing decision and the authentication decision into one piece of code. Because the target method still performs its own full verification and the router itself can never resolve or grant anything, this reuses the same “narrow, structurally-bounded capability” posture as the rest of this ADR rather than introducing a new trust model.

Negative

• A new runtime dependency (extism + wasmtime) with its own release cadence, security-patch surface, and binary-size cost, orthogonal to the existing inventory-based plugin model (ADR 0018) - the codebase now has two distinct extensibility mechanisms, which must be kept clearly documented as serving different purposes (first-party static vs. third-party dynamic).
• Filesystem-based distribution (§5) means the operator, not keystone-rs, is responsible for keeping the .wasm file and its pinned hash consistent across every node; a mismatch on a given node - including a plain typo in the pinned hash, not just a genuine tamper/corruption case - disables only that plugin on that node (§5), which can leave one auth method inconsistently available across the cluster until an operator acts on the accompanying CRITICAL alert. This trades the previous design’s stronger (whole-cluster-blocking) consistency guarantee for availability, and still raises the operational cost of a plugin update to that of a coordinated, carefully-staged rollout if an operator wants to avoid the divergence window entirely.
• Cluster-global-only scoping (§8) means this does not yet serve a multi-tenant “let domain admins bring their own auth plugin” use case - only system admins can install plugins.
• Guest-language plugin authors must target Extism’s PDK ABI, which is a new toolchain requirement distinct from ordinary keystone-rs Rust development.
• The namespace-scoped identity model (§4) and mandatory audit wrapping (§6.E) add host-side bookkeeping (per-Store handle maps, a dedicated (plugin_name, external_id) mapping table, non-bypassable hook dispatch) beyond what a naive “trust the plugin’s JSON” implementation would need - a deliberate complexity/safety trade-off given this is an authentication surface.
• Persistent state a plugin creates or is linked to (provisioned users, granted roles, admin-created identity links) is not automatically undone when the plugin is later patched or removed for a security issue - only future tokens are blocked (§4 “Plugin Version Binding”). Cleanup today is a manual, audit-log-driven operator task (§8).
• Three operating modes, a new admin API (identity_links), and an amendment to ADR 0020’s IdentitySource enum and MappingContext payload (§4) widen this ADR’s surface area beyond a single, self-contained mechanism - an operator now has to understand which mode a given plugin runs in to reason about what it can reach, and mapping-mode plugins depend on an admin having separately authored Mapping Engine rules for them (§4 step 4) or they authenticate no one.
• route mode is reachable by a strictly larger slice of traffic than full_auth/mapping plugins - every request whose identity.methods matches its inspect_methods list, not just requests already addressed to it by name
  • so it sees raw credential material (headers, payload fields) for logins it may ultimately have no involvement in beyond Passthrough. This is a real increase in what third-party WASM code is exposed to compared to the rest of this ADR’s “opt-in by name” model, contained only by inspect_methods scoping and the payload/header allowlists already used elsewhere (§4).

See Also

• doc/src/adr/0018-plugin-linking.md - the static/compile-time counterpart this ADR deliberately does not replace.
• doc/src/adr/0017-security-context.md - the validation pipeline a plugin-authenticated principal flows through unchanged.
• doc/src/adr/0020-mapping-engine.md §5.1 - precedent for bounded, timeout/size-capped evaluation of untrusted-shaped input.
• doc/src/adr/0020-mapping-engine.md §2, §3, §5.3, §9.A - IdentitySource, MappingRuleSet/allowed_domains, MappingContext, and admin-write RBAC tiering, all extended or reused by mapping mode and identity linking (§4).
• [openstack_keystone_core_types::mapping::resolution::IdentitySource] enum, gains the WasmPlugin variant for mapping mode.
• [MappingContext] - mapping-mode token invalidation reuses its existing mapping_id to recover the plugin’s valid_since from the matched ruleset’s IdentitySource::WasmPlugin; no new field is added.
• doc/src/adr/0023-audit.md - CADF audit event model reused for plugin invocation auditing.
• doc/src/adr/0024-scim-v2-provisioning.md §3.A–B - ScimResourceIndex and the externalId lookup index reused by the SCIM identity-linking convenience (§4).
• [AuthenticationContext] enum.
• [AuthenticationContext::Mapping], the variant a successful mapping-mode login produces.
• [openstack_keystone_core::identity::backend::IdentityBackend::create_user], the target of the provision_user host function.