This document is also available in these non-normative format:
This document is licensed under
Creative Commons Attribution 4.0 International Public License
This document defines a standardized method for logging decisions to allow or deny API requests to enable reconstruction and analysis of historical decisions.
This is a draft that could be altered, removed or replaced by other documents. It is not a recommendation approved by TO.
The Authorization Decision Log standard defines a standard method for logging authorization decisions. The standard builds on the information models defined by [AuthZen] to log decisions.
It augments this with a structure for recording environmental factors that affected the evaluation decision. These are separated into active policies, additional information sources and configuration of the evaluation engine as per the architectural components introduced by [NIST.SP.800-162], also known as the "PxP" architecture.
Additionally, it includes a non-normative outline introducing the concerns, principles and requirements for developing and maintaining such a log in concordance with legislation.
This standard defines a uniform approach for logging authorization decisions, enabling organizations to provide effective accountability for historical decisions.
The standard provides a structured format for all contextual and environmental parameters that affect decisions. A full implementation of the standard allows historical decisions to be replayed for analysis.
Functionally, the standard is applicable to any API request. Organizationally, it applies to Dutch governmental bodies — such as the central government, provinces, municipalities, and water boards — as well as institutions in the public or semi-public sector.
The target audience for the standard includes organizations involved in making and accounting for access decisions, both within their own organization and in collaborations with other parties. This may include the Dutch government and other relevant authorities.
The following list defines terminology used throughout this document.
Authorization
Authorization is the process of deciding whether to, fully or partially, allow or deny requests for processing (API requests) and enforcing these decisions.
Externalized Authorization Management
Externalized Authorization Management (EAM) is an architectural pattern in which authorization decisions are made outside of the applications that enforce the decision.
Log
A structured record of events, called log records, generated by a service. Logs are intended for operational monitoring, auditing, and troubleshooting.
Log record
A single unit of information within a log, representing one recorded event. Each log record is immutable once written and collectively forms the log.
Policies
A set of one or more rules that determine whether a request should be allowed or denied.
Source
A source of one or more pieces of information, such as attributes or policies, which affected an authorization decision and which come from a single source.
When a source is later queried for historical information, a single identifier, such as a version, hash, or timestamp, should be sufficient to retrieve all historical information from that source.
Replay
The recreation of the environment in which an authorization decision took place, allowing for analysis of historical decisions.
This section describes the common EAM architecture within which the authorization decisions are logged.
The Authorization Decision Log defines a standard format for recording decisions made by Externalized Authorization Management (EAM) systems.
The goal of the standard is to enable the recreation of the environment in which historical authorization decisions were made to enable analysis of decisions while preventing unnecessary data duplication.
The inputs for records in the Authorization Decision Log come from the following standard EAM or PxP components as introduced in [NIST.SP.800-162] and adopt the information model introduced in [AuthZEN].
In a federated context, such as introduced by [FSC - Core], both the consumer outway and provider inway function as a PEP for incoming and outgoing requests. Both the consumer and the provider ask an internal PDP to decide on allowing the request. Both of these decisions can be logged using this standard.
When combined with tracing headers such as [trace-context] introduced by [Logboek dataverwerkingen] and the FSC Transaction ID used in [FSC - Logging], this enables full traceability across complex multi-organizational processing chains.
See the sequence diagram below for an example of such a flow.
The standard EAM architecture has the following conceptual components. These can be deployed as standalone applications, combined in various configurations, or even implemented within a single monolithic application.
A Policy Enforcement Point (PEP) intercepts a user's request, sends it to the Policy Decision Point (PDP) for evaluation, and then enforces the resulting "permit" or "deny" decision. A PEP is typically implemented as an API gateway or as a component within the application itself.
A Policy Administration Point (PAP) is where access policies are authored, managed, and stored. It is responsible for distributing current policies to the PDP and archiving previous versions for traceability. This can be a dedicated commercial or open-source tool or a version control system like a Git repository.
A Policy Information Point (PIP) enriches access requests with additional attributes needed to make a decision. For example, it might retrieve a user's role or the sensitivity level of a data record from an external source. PIPs are often integrated directly into the PDP.
A Policy Decision Point (PDP) evaluates incoming requests from the PEP against the relevant policies (from the PAP) and contextual data (from the PIP) to make a "permit" or "deny" decision. The PDP is often a separate application or sidecar container.
It is important to keep in mind that these are architectural components. They can also be physically implemented in a single application. In such a case the authorization interceptor can be considered the PEP; the authorization handler, the PDP; the services it invokes, the PIP; and the Git repository of the application itself, the PAP.
This section delineates the scope of the standard.
The specification defines an interface for persisting log entries. This is the component that MUST be consistent across organizations to ensure interoperability.
The management of a log, however, is left to the discretion of individual implementations. Consequently, the specification does NOT define behavior or interfaces for:
See 5. Information Management and Compliance for an overview of various aspects which MAY be required for legal and regulatory compliance.
To provide accountability for historical authorization decisions it must be possible to recreate the information and environment that affected the decision. The PDP provides the information required for this to the Authorization Decision Log in the form of a Log Record, as defined in the specification below.
The PDP SHOULD ensure that a Log Record has been persisted to durable storage before providing the Policy Enforcement Point with the decision. The PDP then flushes this durable storage to the Authorization Decision Log, ensuring that the log record has been persisted and can be used to provide accountability when needed.
This section provides the specification for the protocols and interfaces to be used and the expected behavior of the components.
The protocols used between the engine and the log are not prescribed in this standard.
It is RECOMMENDED to use the OpenTelemetry Protocol (OTLP) for the interaction between the Application and the log.
Each decision log entry SHOULD be persisted to durable storage before the PDP provides a decision response to the PEP. If persistence of log entries is not confirmed, historical decisions may end up not being logged.
The log MUST enforce TLS on connections, in accordance with the standard practice established within the organization.
See 5. Information Management and Compliance for an overview of additional behavior that MAY be required for legal and regulatory compliance.
The interface MUST have fields that can identify the request.
It is RECOMMENDED to use [trace-context] to identify requests by implementing the following fields:
| Field | Type | Description |
|---|---|---|
trace_id |
16 byte | Unique identifier of trace that follows data processing |
span_id |
8 byte | Unique identifier of span within the data processing |
When requests incorporate [FSC - Logging], the following field SHOULD be implemented:
| Field | Type | Description |
|---|---|---|
transaction_id |
string | Unique identifier of FSC transaction id of this request |
The interface MUST implement the following fields describing the request and its response:
| Field | Type | Description |
|---|---|---|
timestamp |
timestamp | The exact point in time when the authorization decision was made |
type |
string | The type of request as defined in the [AuthZen] standard |
request |
object | Input for the decision in [AuthZen] format |
response |
object | Output of the decision in [AuthZen] format |
The interface MAY implement the following fields:
| Field | Type | Description |
|---|---|---|
policies |
object | Policy sources that affected the decision |
information |
object | Information sources used in the decision |
configuration |
object | Policy Decision Point configuration and metadata |
Implementations MAY include additional fields. Such fields MUST NOT alter the semantics of the defined fields and SHOULD be ignored by recipients that do not recognize them.
Each log record should be uniquely identified. Using these identifiers the log entry can be related to other logs and vice-versa.
The specification supports a number of different identifiers: a W3C Trace Context to integrate with [Logboek dataverwerkingen], a transaction id to integrate with [FSC - Logging], and a generic ID for when neither of the other identifiers are available.
The trace_id field is a 16-byte unique identifier that represents the trace context for the data processing operation. This field SHOULD follow the W3C Trace Context specification. The trace ID enables correlation of the authorization decision with related operations.
The span_id field is an 8-byte unique identifier that represents the specific span within the trace context. This field SHOULD follow the W3C Trace Context specification. The span ID identifies the particular operation or step within the data processing flow where the authorization decision was made.
The transaction_id field is a string value that represents the FSC transaction to which this request belongs. If a W3C trace context is available it should also be included.
If none of the other identifiers can be supported, a generic fallback identifier can be included in the id field.
This can be any value, simple or complex, which can contain any kind of request identifier.
The minimal information required for an entry in the log consists of a request for a decision and a response with the evaluated decision.
This section describes the key fields required for logging this request/response cycle.
The timestamp field represents the exact point in time when the authorization decision was made. The timestamp SHOULD be in [RFC3339] format to ensure consistent interpretation across different systems and regions.
The type field represents the type of request that was made. This value identifies the AuthZEN endpoint that was invoked.
Its value MUST be a string containing the key value of the relevant endpoint as defined in "Endpoint Parameters" of the "Policy Decision Point Metadata" as defined in [AuthZEN] with the _endpoint suffix omitted.
For example, a request to the URL defined by the search_subject_endpoint in the PDP metadata would have the type of search_subject.
The request field is an object that represents the input to the decision. This field MUST be in [AuthZen] format as defined for the given request type.
Portions of the request MAY be omitted for privacy reasons. If information is omitted, this omission SHOULD be documented or indicated in the log record. If the omitted information was used by the Policy Decision Point, then full accountability can no longer be provided.
The response field is an object that represents the output of the decision. This field MUST be in [AuthZen] format as defined for the given request type.
Portions of the response MAY be omitted for privacy reasons. If information is omitted, this omission SHOULD be documented or indicated in the log record. If information that was used by the Policy Enforcement Point is omitted then full accountability can no longer be provided.
In the following example a manager called Alice attempts to approve a holiday request for a team member called Bob, but the request is denied because she does not have signing authority.
Her browser submits the following request to the API:
The application, acting as the PEP, then submits the following HTTP request to the PDP:
The PDP then determines that Alice can't sign on behalf of the company and thus cannot approve the holiday request. It returns the following response:
The following JSON object contains a non-normative example of a log record describing this example:
The policies field represents a versioned reference to the policies that the PDP used to evaluate the request. In a PxP architecture, this represents the information that would come from the Policy Administration Point (PAP).
A PDP can have one or more sources of policies which can be individually versioned. To accommodate that the policies field is an object in which each key identifies a specific, versioned, policy source.
All policy sources that have affected the decision MUST be included. The value associated with each key refers to a unique version of the policy source. The information in this field MUST be sufficient to retrieve all policies from the policy sources that were used in the authorization decision.
Non-normative examples include:
We can extend the example of the holiday-approval request by adding a reference to a Git repository in which current HR approval policies are documented. In the example below the git hash of the version currently deployed together with the PDP is 6266d07750c44b4c9b05d0801b752c0ef884e4f6.
More complex references can be achieved by using an object as the version identifier. If, for example, the HR application takes part in a federation with predefined policies for different maturity levels. The following non-normative example shows how those policies can be referenced using a semantic version combined with a filter for policies relevant to the current maturity level.
The following JSON object contains a non-normative example of a log record describing this example:
The information field represents all the supporting information used in the evaluation of the access decision. In a PxP architecture, this field represents the information that would come from Policy Information Points (PIPs).
It is an object in which each key identifies an information source. All information sources that have affected the decision SHOULD be included. The value of this field SHOULD either contain the information that was used in the access decision or be sufficient to retrieve the information.
In the example of the holiday approval, the ability to sign is accessed through the can_sign field of the user. The Policy Information Point (PIP) called can-sign-api requests this via an API from the HR application using the request below:
And the API returns the following response.
The following JSON object contains a non-normative example of a log record describing this example.
In this example the entire response is stored in the log record as it is a small response without sensitive data. In most cases it is recommended to use a reference to the data instead. See 3.4 Sources and referencing for more information.
The configuration field represents the information required to recreate the software configuration that evaluated the original decision. In a PxP architecture, this primarily represents the configuration of the Policy Decision Point (PDP), but MAY also include configuration of Policy Information Points (PIPs) and Policy Administration Points (PAPs).
It is an object in which each key identifies a configuration source. All configuration sources that have affected the decision SHOULD be included. The value of this field SHOULD either contain the configuration that was used in the access decision or be sufficient to retrieve the configuration.
Non-normative examples include:
In the example below we extend the holiday approval request example by describing the version of the language used by the PDP and the configuration of the can-sign-api PIP.
In this example the configuration is stored in the log record itself. To reduce data duplication it is generally recommended to use a reference to the configuration instead. See 3.4 Sources and referencing for more information.
Policy, information and configuration sources MAY be included in the log directly.
This is generally undesirable however as it introduces duplication, increases the size of the log and increase security requirements for the log by including sensitive data.
To address this we describe several methods of referencing sources from the log below.
Some information sources offer the ability to 'time-travel' by providing a version at which to query. In such cases, the data itself may be omitted and the version can be stored instead.
The version identifier can be a simple value, such as a string or number, or a complex object, such as an array or object containing multiple version identifiers.
The following non-normative example shows a reference to a specific semantic version of a policy source.
In a complex case, such as limiting requests for open data per IP per minute across a large number of servers, the version could also consist of an array of partition offsets in a Kafka stream of HTTP request.
In case the source of information offers the ability to 'time-travel' by providing a timestamp at which to query, then the data itself may be omitted.
It is RECOMMENDED to use the timestamp defined in the time field in the context of the request as the base time. In that case the information source MAY be omitted fully.
If a different timestamp is used, then it SHOULD be included in [RFC3339] format.
When system clocks are not aligned properly, a system may be asked to provide information for a timestamp that lies in the future. This can be mitigated by requesting the policies of a few seconds or minutes ago at the expense of reducing the speed with which policy changes can be deployed.
In the context of REST APIs developed by the Dutch government the Temporal extension of the [REST API Design Rules] can be used for this purpose.
For information sources that are logged in an external log, a request identifier is needed to look up the corresponding request in the external log.
It is RECOMMENDED to use the W3C Trace Context standard as the request identifier. Such a request SHOULD have the same trace_id as the request to the PDP, in which case the source reference can consist of only the value of the span_id.
It is RECOMMENDED to log requests in the [WARC] format as it includes all request and response headers that may be used in the authorization decision.
The following non-normative example shows a log record for a request to find all subjects capable of approving a holiday request:
Which would result in the following WARC entries logging the REST API call to the HR system:
The ability to provide accountability depends on the log's level of detail. A balance must be struck between capturing enough information to accurately replay historical decisions and practical challenges like data duplication and scalability. The appropriate level of detail depends on the organization's specific context and legal requirements, as the highest level is not always necessary.
To ensure historical accuracy while minimizing data storage, referencing external information (e.g., via a timestamp or version number) is preferred over storing copies. This approach keeps logs lean but is contingent on the ability of source systems to provide versioned historical data.
For full replayability, the log must also identify the exact version and configuration of the policy engine that evaluated the decision; however, providing reliable versioning for the engine may not always be feasible, depending on the infrastructure.
We have identified four levels of detail, in order from least to most detail. Each level builds on the information from the previous level.
At the most basic level only the decision request and the decision response are logged.
The decision request and response MAY contain all information required for an audit log, as described by [ISO/IEC 27002:2022] and [BIO2]. If that is the case, and all auditable actions are decided on by the PDP, the Authorization Decision Log MAY be used as an audit log.
At this level of detail log requests contain the following keys, as defined in 3. Specifications:
| Field | Required | Reference |
|---|---|---|
trace_id |
optional | 3.3.1.1 W3C Trace Context |
span_id |
optional | 3.3.1.1 W3C Trace Context |
timestamp |
required | 3.3.2.1 Timestamp |
type |
required | 3.3.2.2 Type |
request |
required | 3.3.2.3 Request |
response |
required | 3.3.2.4 Response |
In addition to the request and response, the exact version of the policies that were used to evaluate the request can be programmatically retrieved.
At this level of detail log requests contain the following keys, as defined in 3. Specifications:
| Field | Required | Reference |
|---|---|---|
trace_id |
optional | 3.3.1.1 W3C Trace Context |
span_id |
optional | 3.3.1.1 W3C Trace Context |
timestamp |
required | 3.3.2.1 Timestamp |
type |
required | 3.3.2.2 Type |
request |
required | 3.3.2.3 Request |
response |
required | 3.3.2.4 Response |
policies |
required | 3.3.3 Policy Sources |
All information used in the evaluation can be programmatically retrieved. This allows full replayability, assuming the engine (PDP) behaves identically or can be manually recreated in the correct state, which is generally achievable.
At this level of detail log requests contain the following keys, as defined in 3. Specifications:
| Field | Required | Reference |
|---|---|---|
trace_id |
optional | 3.3.1.1 W3C Trace Context |
span_id |
optional | 3.3.1.1 W3C Trace Context |
timestamp |
required | 3.3.2.1 Timestamp |
type |
required | 3.3.2.2 Type |
request |
required | 3.3.2.3 Request |
response |
required | 3.3.2.4 Response |
policies |
required | 3.3.3 Policy Sources |
information |
required | 3.3.4 Information Sources |
In addition to all information used in the evaluation, the environment and configuration of the system that evaluates the decision can also be accurately recreated. This provides full, guaranteed replayability and maximum accountability.
The configuration of all components that influence the decision should be included. While this may be limited to the configuration of the PDP, it often also requires configuration of the PIP and sometimes the PAP as well.
At this level of detail log requests contain the following keys, as defined in 3. Specifications:
| Field | Required | Reference |
|---|---|---|
trace_id |
optional | 3.3.1.1 W3C Trace Context |
span_id |
optional | 3.3.1.1 W3C Trace Context |
timestamp |
required | 3.3.2.1 Timestamp |
type |
required | 3.3.2.2 Type |
request |
required | 3.3.2.3 Request |
response |
required | 3.3.2.4 Response |
policies |
required | 3.3.3 Policy Sources |
information |
required | 3.3.4 Information Sources |
configuration |
required | 3.3.5 Configuration Sources |
The higher the level of detail, the more useful the log is for determining the context of an authorization decision. On the other hand, higher levels of detail also introduce challenges around scalability, technical feasibility, and security.
Conversely, the lowest level of detail may not be sufficient to provide effective accountability. This depends on the data processing which is being authorized and legal requirements for it.
For that reason it's important to decide for different use cases which level of detail is required and appropriate. Aiming for the highest level of detail for all authorization decisions is thus not necessary.
This, non-authoritative, section addresses the governance, security, and privacy-compliance aspects of managing an Authorization Decision Log (ADL).
Conformance to this standard does not, by itself, guarantee legal or regulatory compliance. The implementing organization is solely responsible for ensuring its implementation adheres to all applicable frameworks, such as the General Data Protection Regulation (GDPR / AVG) and relevant security baselines, such as [ISO/IEC 27001:2022], [ISO/IEC 27002:2022], and [BIO2]. Each organization is responsible for its own Authorization Decision Log. There is no central log, although logs of several organizations can be aggregated if desired.
The following sections list aspects that should be taken into consideration when creating a compliant and secure logging solution.
Logging authorization decisions creates a new processing of data, which must be compliant with privacy regulations if personal data is involved.
When collecting personal data, the specific purposes for logging (e.g., operational auditing, forensic analysis, citizen accountability) must be defined and documented in a formal policy before implementation. Data collection must be limited to these defined purposes.
A core principle is to avoid storing unnecessary information, especially sensitive data. The goal is to make the log precise, compact, and manageable. Instead of duplicating large amounts of (personal) data, prefer storing only the data used in the decision or even just a reference that allows the state at the time of the decision to be recreated.
A logging policy must be implemented to manage what is logged based on risk. When logging sensitive data for high-risk use cases, this should be explicitly documented and approved.
Implementers should consider pseudonymization and anonymization for personal data. For example, personal identifiers can be hashed or aliased and location data can be randomized.
When aggregate statistics, such as decisions per type per time period, are sufficient, implementers should consider using aggregation as a method of data minimization and anonymization.
A data retention policy must be defined and enforced. Retention periods must be based on the defined purposes and legal obligations. These may differ for different types of log entries.
As a general guideline, operational logs (for debugging, support) should be retained for a short period (months), while forensic/audit logs may be retained for longer periods (years).
Logs that have exceeded their defined retention period must be automatically and securely purged.
If logs contain personal data, they must be able to comply with data subject access requests.
The log's structure, purpose, and retention must be documented in the organization's Register of Processing Activities.
A Data Protection Impact Assessment (DPIA) must be conducted for any implementation that involves large-scale or high-risk processing of personal data.
Access to log data should be restricted based on the principle of least privilege.
It is essential to define clear policies for authorizing access to the Authorization Decision Log or parts thereof. These decisions to provide or deny access to the log should also be included in the Authorization Decision Log.
Common and important usage policies include:
The log must be protected against unauthorized access, modification, and deletion.
Key concerns for ensuring security and integrity include:
As well as sections marked as non-normative, all authoring guidelines, diagrams, examples, and notes in this specification are non-normative. Everything else in this specification is normative.
The key words MAY, MUST, MUST NOT, RECOMMENDED, and SHOULD in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.