Policy Tracing and Explainability¶
Privacera PolicySync connectors emit structured audit events that explain how policies are loaded, applied, and enforced on target data services. These events give administrators and security teams a granular, auditable view of grant and revoke operations, changelog execution, loader statistics, and long-running work progress.
PolicySync writes events to the policysync_audit stream. Audit Server indexes them into Apache Solr by default and can forward the same JSON payloads to Amazon SQS, Amazon SNS, or object storage (Amazon S3, Azure Blob, Google Cloud Storage) when those destinations are configured.
Every ACCESS, POLICY_RESPONSE, and ON_DEMAND event lists its outcomes, on success as well as on failure, as numeric response codes in extraInfo.response[]. A LOADER event lists one when its load failed, and a TEMPLATE_UPDATE_RESPONSE event lists the failures of a template update.
What you can trace¶
| Area | Event types | What it shows |
|---|---|---|
| Access decisions | ACCESS | Each grant or revoke (or identity lifecycle step) after the connector executes it on the target service |
| Policy batches | CHANGELOG, POLICY_RESPONSE | Changelog lifecycle and per-policy ACL rollups for Ranger policy updates |
| Template updates | TEMPLATE_UPDATE_RESPONSE | Databricks Unity Catalog: one summary per changed row-filter template, with the tables re-created, unchanged, and failed |
| Background sync | LOADER | Resource, principal, permission, tag, and audit loader statistics at fixed intervals or on demand |
| Long-running work | PROGRESS | Incremental progress while a large changelog or loader processes many items |
| On-demand sync | ON_DEMAND | The outcome of each resource requested in an on-demand sync |
| Reliability / containment | RESOURCE_PROCESS_TASK_TIMEOUT | A resource-process task abandoned by the changelog time-box after exhausting retries (opt-in diagnostics) |
Event destinations¶
| Destination | Configuration |
|---|---|
| Apache Solr (default) | Audit Server; query from Privacera Portal or Solr directly |
| Amazon SQS | Send audits to Amazon SQS |
| Amazon SNS | Send audits to Amazon SNS |
| Object storage | Send audits to S3, Send audits to GCS |
| Multiple sinks | Connector audit output destinations |
Handler requirement for external sinks
PolicySync audit handler V2 must be enabled when routing connector events to SQS or object storage. See Connector audit output destinations.
For Ranger plugin access audits (distinct from PolicySync connector tracing), see Audit log JSON format.
Event JSON format¶
The same JSON payload shape is used when Audit Server indexes events into Solr and when events are forwarded to Amazon SQS, Amazon SNS, or object storage.
Wire contract¶
| Topic | Contract |
|---|---|
| Wire format | One JSON object per event; UTF-8 |
| Naming | Property names use camelCase |
| Enum-like values | String literals such as eventType, status, and changeLogType use a fixed vocabulary: UPPERCASE with underscores (for example RANGER_POLICIES, COMPLETED). If a transport lowercases or aliases them, normalize before comparison |
| Extensibility | Unknown root keys and extraInfo keys may appear. Ignore or store them unless you explicitly handle them |
Event types¶
eventType | Description |
|---|---|
ACCESS | Grant or revoke on data resources, or per-step identity events (user or group lifecycle, membership) |
CHANGELOG | Changelog-driven policy or ACL batch execution; extraInfo.policies and/or completion stats |
LOADER | Statistics about resource, principal, permission, tag, or audit loading from the target service or Ranger |
PROGRESS | Progress updates for long-running changes (resource or permission processing) |
POLICY_RESPONSE | Per-policy summary after a Ranger policies changelog completes |
CONFIG_CHANGE | Connector configuration change notification (when enabled) |
ON_DEMAND | Outcome of one requested resource in an on-demand sync. See Send On-Demand Sync Audits to S3, SQS, or SNS |
RESOURCE_PROCESS_TASK_TIMEOUT | A resource-process task abandoned by the changelog time-box after exhausting retries; status=FAILED, with time-box context (stuck phase, budgets, abandonedWhileQueued) in extraInfo. Emitted only when the opt-in time-box is enabled. See Changelog slow-task diagnostics |
TEMPLATE_UPDATE_RESPONSE | Databricks Unity Catalog: summary of one changed row-filter template after its tables were re-applied. See Audits of a template update |
The ACCESS, POLICY_RESPONSE, and ON_DEMAND events list their outcome codes in extraInfo.response[]. A failed LOADER event can list one, and a TEMPLATE_UPDATE_RESPONSE event lists its failures. For the codes by event type, see Response codes; to find the ACCESS events behind a failure, see Link a failure to its ACCESS audits.
Common fields¶
All event types include these fields.
| Field | Type | Description |
|---|---|---|
eventType | String | ACCESS, CHANGELOG, LOADER, PROGRESS, POLICY_RESPONSE, CONFIG_CHANGE, ON_DEMAND, RESOURCE_PROCESS_TASK_TIMEOUT, or TEMPLATE_UPDATE_RESPONSE. Always uppercase |
evtTime | String | Event timestamp. ISO 8601 (for example 2025-11-14T01:31:00.170Z) or yyyy-MM-dd HH:mm:ss depending on the emitter |
timeTakenMs | Integer | Wall-clock duration in milliseconds for the work unit this event represents |
status | String | Lifecycle state; uppercase. See Status |
msg | String | Optional human-readable detail (for example failure or skip reason) |
connectorType | String | Connector family (for example lakeformation, databricks_unity_catalog, databricks_sql_analytics) |
connector | String | Connector instance name or identifier |
extraInfo | Object | Optional metadata; shape depends on eventType |
uuid | String | Optional document identifier when assigned by the sink |
Status¶
| Value | Typical meaning |
|---|---|
QUEUED | Accepted or scheduled; execution not yet started |
STARTED | Execution has begun |
RUNNING | Execution in progress |
PROCESSING | Progress pipeline processing state |
COMPLETED | Finished successfully |
FAILED | Finished with error |
GRANT_PARTIALLY_APPLIED | Finished with part of the work applied and part failed: some ACL operations of a POLICY_RESPONSE, or some tables of a TEMPLATE_UPDATE_RESPONSE |
SKIPPED | No change applied (nothing to do, or intentionally bypassed) |
Changelogs and changeLogType¶
PolicySync records work in changelogs. For each stream of updates it maintains a changelog and applies deltas to the target service.
Common changeLogType values¶
Canonical changeLogType strings use UPPERCASE with underscores. Privacera may add new values in future releases.
changeLogType | Role |
|---|---|
RANGER_POLICIES | Policy changes from the portal; delta vs store; update ACLs in the service |
SERVICE_RESOURCES | Service resource loader detected a delta; update ACLs |
RANGER_PRINCIPALS | Delta while loading users, groups, or roles from Ranger; reconcile principals in the service |
SERVICE_PRINCIPALS | Delta while loading principals from the service; reconcile back and refresh ACLs |
SERVICE_PERMISSIONS | Service permission loader detected a delta vs the service; update ACLs |
RANGER_TAGS | Ranger tag metadata pipeline; tag-related ACL refresh |
RANGER_ROLES | Ranger roles and related Ranger changelog stream |
TEMPLATE_UDF_REAPPLY | Databricks Unity Catalog: a row-filter template changed; re-apply the tables that use it. Ends with one TEMPLATE_UPDATE_RESPONSE per changed template |
Correlating changeLogId¶
When an event includes changeLogId, use these rules together with ACCESS and CHANGELOG events:
| Rule | Detail |
|---|---|
| Scope | A changeLogId applies to one PolicySync connector instance only |
| Global uniqueness | Not guaranteed across deployments; two instances may independently emit the same id |
| Safe correlation | Use connector (instance identifier) together with changeLogId |
Type-specific fields¶
POLICY_RESPONSE¶
Emitted once per policy when a RANGER_POLICIES changelog completes. Per-policy ACL rollups live in extraInfo.aclsExecuted; the CHANGELOG completion event carries id and version summaries only.
| Field | Type | Description |
|---|---|---|
changeLogId | String or Integer | Changelog run id on this connector instance |
changeLogType | String | Changelog type (typically RANGER_POLICIES) |
policyId | String or Integer | Policy identifier |
policyVersion | String or Integer | Policy version |
policyLabel | String | Correlation or trace label |
policyName | String | Policy name |
extraInfo.aclsExecuted¶
| Field | Type | Description |
|---|---|---|
total | Integer | ACL operations considered |
applied | Integer | Successfully applied |
failed | Integer | Failed |
skipped | Integer | Skipped |
timeTakenMS | Integer | Time spent on ACL execution for this policy, in milliseconds. Some emitters may use timeTakenMs; treat as equivalent after normalizing keys |
Other extraInfo fields¶
| Field | Type | Description |
|---|---|---|
response | Array | One entry per distinct outcome of the policy's ACL operations and owner skips. See Response codes |
ownerIgnore | Object | Grants skipped because the grantee owns the resource: count and reason |
failedReasons | Array | Failure reasons, each as {reason, count}: at most 10, highest count first. Absent when nothing failed |
skippedReasons | Array | Skip reasons, in the same shape as failedReasons. Absent when nothing was skipped |
policy | Object | For an added or updated policy in a RANGER_POLICIES changelog: the policy's createTime and updateTime, written as yyyy-MM-dd HH:mm:ss, the format evtTime has in the event payload. Compare updateTime with evtTime to see how long the change took to apply; Apache Solr and Privacera Portal show evtTime as ISO 8601 instead |
Failure text can name objects
failedReasons, skippedReasons, and the msg of POLICY_RESPONSE and ACCESS events carry the connector's failure text, which can include table, principal, template, and condition names. For example, a template-based row filter's PRINCIPAL_OVERLAP message names the principal. Only AWS Lake Formation normalizes this text, and it keeps single-word names, so treat these fields as sensitive when you route events to Amazon SQS, Amazon SNS, or Amazon S3.
ACCESS¶
Emitted after a grant or revoke (or identity operation) is executed on the target service.
| Field | Type | Description |
|---|---|---|
changeLogId | String or Integer | Required. Changelog run id on this connector instance |
changeLogType | String | Required. One of the common changeLogType values |
policyId | Integer or Array | Policy identifier, or list when several Ranger policies merge into one operation |
policyVersion | String or Array | Policy version(s), aligned with policyId |
policyName | String | Policy name |
policyType | String | Policy type (typically access) |
policyLabel | String | Correlation or trace label |
resourceType | String | Data resource (table, schema, …) or identity object (user, group, …) |
resource | String | Resource path or principal identifier |
permission | String | Data permission (select, insert, …) or connector-defined operation |
action | String | grant or revoke for ACLs; identity verbs such as create, update, add_member, remove_member where applicable |
cmd | String | Optional command executed |
cmdType | String | Command channel: sql or api |
actor | String | User, group, or role name |
actorType | String | USER, ROLE, or GROUP |
ondemandTaskId | String or Integer | Optional. Present when the event originates from an event-driven on-demand task |
Typical field mapping (connector-specific)¶
| Connector | permission | resource | actor / actorType | cmd |
|---|---|---|---|---|
| Lake Formation | e.g. select | Table path (<account>/<region>/<database>/<table>) | IAM role ARN (ROLE) | Grant line as executed |
| Databricks Unity Catalog | e.g. execute | catalog/schema/table | e.g. account users (GROUP) | GRANT … SQL statement |
extraInfo for ACCESS¶
| Field | Type | Description |
|---|---|---|
response | Array | One entry: the outcome of this grant or revoke, for success, skip, and failure alike. See Response codes |
CHANGELOG¶
| Field | Type | Description |
|---|---|---|
changeLogId | String or Integer | Changelog run id on this connector instance |
changeLogType | String | Changelog type |
extraInfo for CHANGELOG¶
| Field | Type | Description |
|---|---|---|
policies | Array | Optional. One entry per policy in the changelog execution summary |
principals | Object | Optional. User, group, or role update counts |
resources | Object | Optional. Resource sync counts (aggregate or by type) |
policiesChunkIndex | Integer | Optional. 1-based index when completion is split across multiple events |
policiesChunkCount | Integer | Optional. Total number of completion chunks |
policiesTotalCount | Integer | Optional. Total policies in the full completion |
Each element of extraInfo.policies[]:
| Field | Type | Description |
|---|---|---|
id | String | Policy identifier |
version | String | Policy version |
label | String | Correlation or trace label |
name | String | Policy name |
aclsExecuted | Object | ACL rollup (same shape as extraInfo.aclsExecuted; omitted on Solr completion summaries that only carry id and version) |
Each principal kind under extraInfo.principals (users, groups, roles):
| Field | Type | Description |
|---|---|---|
found | Integer | Principals found |
added | Integer | Principals added |
updated | Integer | Principals updated |
deleted | Integer | Principals deleted |
LOADER¶
| Field | Type | Description |
|---|---|---|
changeLogId | String or Integer | Loader run id on this connector instance |
changeLogType | String | Loader pipeline type (for example SERVICE_RESOURCES) |
loaderType | String | Data being loaded. See loaderType values |
ondemandTaskId | Integer | Optional. Present when the loader starts from an event-driven task |
triggerFrom | String | How the loader started: CONFIG_CHANGE, SCHEDULED, or EVENT |
loaderType values¶
| Value | Meaning |
|---|---|
LOAD_SERVICE_RESOURCES | Service resource loader |
LOAD_SERVICE_PERMISSIONS | Service permission loader |
LOAD_SERVICE_PRINCIPALS | Service principal loader |
LOAD_RANGER_PRINCIPALS | Ranger principal loader |
LOAD_SERVICE_AUDITS | Service audit loader |
LOAD_SERVICE_TAGS | Service tag loader |
LOAD_SERVICE_TAG_RECONCILE | Service tag reconcile loader |
LOAD_SERVICE_TAGS_PERMISSIONS | Service tag permissions loader |
LOAD_RANGER_TAGS_PERMISSIONS | Ranger tag permissions loader |
RECOVERY_STATE | Internal RocksDB recovery lifecycle |
extraInfo for LOADER¶
| Field | Type | Description |
|---|---|---|
resources | Object | Resource sync summary, keyed by resource type |
tags | Object | Tag sync summary, keyed by tag type |
resourcesNotInSync | Integer | Resources not in sync with Ranger policies (permission loader) |
totalTimeTakenMs | Integer | Total loader duration |
users, groups, roles | Object | Principal scan counts (found, updated, ignored) for service principal loader |
response | Array | Only when the load failed and the connector classified the failure: one entry. See Response codes |
principals | Object | Ranger principal loader (LOAD_RANGER_PRINCIPALS): counts for users, groups and roles. See Ranger principal loader counts |
startTime | String | When the run started, as yyyy-MM-dd HH:mm:ss, in the same format and time zone as evtTime in the event payload. On STARTED and on the run's COMPLETED or FAILED audit. Not on the per-resource LOAD_SERVICE_TAG_RECONCILE audits |
endTime | String | When the run ended, in the same format. On COMPLETED and FAILED only |
Each resource or tag type under extraInfo.resources or extraInfo.tags:
| Field | Type | Description |
|---|---|---|
found | Integer | Items found |
added | Integer | Items added |
updated | Integer | Items updated |
deleted | Integer | Items deleted |
Ranger principal loader counts¶
Each principal kind under extraInfo.principals (users, groups, roles) of a LOAD_RANGER_PRINCIPALS audit:
| Field | Type | Description |
|---|---|---|
found | Integer | Principals found in Ranger |
added | Integer | Principals added. Counts only changes the service applied |
updated | Integer | Principals updated. Counts only changes the service applied |
deleted | Integer | Principals deleted. Counts only changes the service applied |
failed | Object | Changes the service did not apply, by action: add, update and remove |
A principal change that fails is tried again on the next sync. When some changes fail, the audit is still COMPLETED and msg lists the failed changes. It is FAILED when every change failed or the sync stopped with an error.
PROGRESS¶
| Field | Type | Description |
|---|---|---|
changeLogId | String or Integer | Progress context changelog id |
changeLogType | String | Progress context changelog type |
extraInfo.progress¶
| Field | Type | Description |
|---|---|---|
total | Integer | Cumulative items processed so far |
sinceLastReport | Integer | Items processed since the previous progress report |
Example events¶
Values such as IDs, names, and times are examples. The examples of failures show only the fields that carry the outcome and the fields that link events together.
POLICY_RESPONSE¶
POLICY_RESPONSE — grants partially applied (Databricks Unity Catalog)¶
A policy where four ACL operations applied, four failed in three different ways, and one grant was skipped because the grantee owns the resource. The counts add up to 8 ACL operations plus 1 owner skip. Only the 5001 entry is worth retrying.
POLICY_RESPONSE — no change¶
A Ranger policy change that applied no grants carries one entry, with count 0:
ACCESS — Lake Formation (table SELECT grant)¶
ACCESS — Databricks Unity Catalog (schema EXECUTE grant)¶
ACCESS — Databricks Unity Catalog (failed grant)¶
A failed grant on Databricks Unity Catalog. The failure is transient (5001), so the same grant may succeed if it is run again.
A successful grant from the same policy carries 2001:
CHANGELOG — Lake Formation¶
Large policy updates
For very large RANGER_POLICIES completions, PolicySync may split extraInfo.policies across multiple CHANGELOG events and add chunk metadata. See CHANGELOG completion audit size limits.
LOADER — service resource loader¶
A successful loader run has no response[].
LOADER — failed resource load¶
A resource load that failed because the backend was unavailable. The code is 5001, so the next load may succeed.
ON_DEMAND — grants partially applied¶
One requested resource whose grants partly failed. response sits beside output, not inside it.
ON_DEMAND — resource not found¶
A request for a table that does not exist in the source. The status is FAILED, but the code is in the EXPECTED range: there is nothing to fix or retry.
PROGRESS¶
Response codes¶
extraInfo.response[] lists an event's outcomes. Each entry carries a stable numeric code, so you can act on an event without parsing its msg. ACCESS, POLICY_RESPONSE, and ON_DEMAND events always carry it; a LOADER event carries it only when its load failed, and a TEMPLATE_UPDATE_RESPONSE event only when a table failed.
Breaking change in 9.2.48.1
The single error code is removed. Read extraInfo.response[] instead.
| Event | Removed field |
|---|---|
ON_DEMAND | extraInfo.output.errorCode |
POLICY_RESPONSE | extraInfo.errorCode |
ACCESS of a failed template-based row filter | extraInfo.errorCode |
For the new code of each old value, see Moving from the single error code.
For a while you receive both shapes: events from before the upgrade stay in Apache Solr and in archives such as Amazon S3, connectors not yet upgraded keep sending the old shape, and a rollback brings it back. Tell them apart by event type, not by release:
ACCESS,POLICY_RESPONSE,ON_DEMAND: an event withoutresponse[]is an old event. Read itserrorCode, or itsstatuswhen it has none.LOADER,TEMPLATE_UPDATE_RESPONSE: an event withoutresponse[]succeeded.
Other changes in 9.2.48.1¶
No top-level field was added, removed, or renamed, and PolicySync applies grants as before. The other changes:
- New event type
TEMPLATE_UPDATE_RESPONSE(Databricks Unity Catalog): one summary per changed row-filter template, after its tables are re-applied. A consumer or an Amazon SNS filter policy that lists event types must add it to receive it. See Audits of a template update. - Template on
ACCESSevents: theACCESSevents of a template-based row filter carryextraInfo.udf_template(name,version,updateTime). extraInfo.response[]is new onACCESS,POLICY_RESPONSE,ON_DEMAND, and failedLOADERevents. Each entry hascode,msg,origin,serviceCode, andcount. The code's name and category are not in the event; look them up in the tables on this page.- Policy times: for an added or updated policy in a
RANGER_POLICIESchangelog,POLICY_RESPONSEcarriesextraInfo.policy = {createTime, updateTime}. CompareupdateTimewithevtTimeto see how long the change took to apply. Both areyyyy-MM-dd HH:mm:ssin the payload; Apache Solr and Privacera Portal showevtTimeas ISO 8601. - Code in the failure text: the failure text in the
ACCESS,POLICY_RESPONSE, andLOADERmsg, and infailedReasons, can start with a bracketed code, for example[SYSTEM_ERROR | retryable=true]. OnLOADERit is the backend's code when there is one, for example[HTTP_503 | retryable=true]. It is for people reading the audit; code should readresponse[]. -
Normalized failure text: ARNs, email addresses, IPv4 addresses, request IDs, and quoted or dotted names such as
database.tableare replaced with?, and the text is cut to 512 characters. This applies to:- AWS Lake Formation: the
ACCESSmsgand the error message in itsextraInfo, and thePOLICY_RESPONSEmsgandfailedReasons. The AWS error code is kept, and is also inresponse[].serviceCode. - Every connector: the
msgof a failedLOADERevent, andoutput.errorDetails.serviceErroronON_DEMAND(see Why the outcome failed).
A name with none of these shapes, such as a single-word table name, is kept, so treat the text as sensitive.
- AWS Lake Formation: the
Moving from the single error code¶
Each old value becomes one or more entries in extraInfo.response[]. Branch on each entry's code.
ON_DEMAND output.errorCode:
| Old value | New response[] entries |
|---|---|
SYNCED_GRANTS_APPLIED | 2001 for the applied ACL operations, plus 1003 or 1004 for skipped ones |
NO_CHANGE_DETECTED | 2002, count 1 |
NO_POLICY_FOUND | 2003, count 1 |
RESOURCE_DELETED | 2004, count 1, plus the task's ACL entries, for example 2001 |
RESOURCE_NOT_FOUND | 1006, count 1. status is still FAILED |
RESOURCE_LOAD_FAILED | The load failure's code: 4505; 5001 when transient; 1006 or 4503 when the backend reports a missing resource or principal |
GRANT_FAILED | One entry per failure code of the ACL operations, for example 4503 or 5001. 4502 when grant application failed with no failed ACL operation, or its classification, such as 5001, when it could not start |
GRANT_PARTIALLY_APPLIED | 2001 for the applied ACL operations, plus one entry per failure code. status is FAILED |
Template-based row filter (Databricks Unity Catalog):
| Old field | New response[] entries |
|---|---|
POLICY_RESPONSE extraInfo.errorCode, for example DDL_FAILED | The code's number, 4001 to 4014 (DDL_FAILED is 4002), with count the ACL operations it covers. See Template-based row filter |
ACCESS extraInfo.errorCode | The code's number, count 1 |
The response entry¶
| JSON | |
|---|---|
| Field | Type | Description |
|---|---|---|
code | Integer | Stable number for the outcome. Its range gives its category and whether a retry can help. A published number is never changed or reused |
msg | String | The code's fixed text, listed in Code messages. The same on every event with that code, so it never contains backend error text. The wording can change between releases; branch on code instead |
origin | String | Who decided the outcome: PRIVACERA, or the backend service. See Origin |
serviceCode | String | The backend's own code, such as a Databricks error condition, a SQLState, an AWS error code, or HTTP_<status>. Absent when the backend gave no code. See Backend error codes |
count | Integer | How many of the event's outcomes this entry covers |
Every field except serviceCode is always present.
Code ranges¶
The range of code gives its category, the way an HTTP status class does. The category names in this table are the ones the code tables on this page use; they are not fields of the event.
code range | Category | Retry can help | Meaning |
|---|---|---|---|
1000–1999 | EXPECTED | No | Intended, nothing to do: a skip, a permission that was already gone, a resource that no longer exists at the source |
2000–2999 | SUCCESS | No | The intended outcome: grants applied, nothing to change, a resource deleted at the source |
3000–3999 | PARTIAL | No | Some of a policy's grants applied and some failed. Never a response[] entry; see below |
4000–4499 | VALIDATION | No | The policy or template cannot be applied as written. Fix the policy or template |
4500–4999 | PERMANENT | No | The backend refused for a reason a retry does not clear, such as a missing principal or resource, or a denied grant |
5000–5999 | TRANSIENT | Yes | The backend failed for an infrastructure reason. The same request may succeed unchanged |
Only a code from 5000 to 5999 is worth retrying. It means that running the same operation again later may succeed; it does not mean that PolicySync retries it.
PARTIAL describes an event's overall outcome, never one entry. A policy where some grants applied and some failed is a POLICY_RESPONSE with status GRANT_PARTIALLY_APPLIED; an ON_DEMAND event reports it as FAILED with msg Grants partially applied; some grants failed. In both cases response[] lists each outcome as its own entry.
RESOURCE_NOT_FOUND has two numbers. It is 4501 (PERMANENT) when a grant or revoke targets a missing resource, and 1006 (EXPECTED) when a resource is missing while resources are loaded, for example because the table was dropped; nothing was applied. Each number has its own msg, but branch on code, not on the name or the text.
Origin¶
origin | Meaning |
|---|---|
PRIVACERA | Privacera decided the outcome: a skip, a template-based row-filter failure, a GRANT_FAILED for a grant or revoke the connector generated no usable statement for, or an on-demand resource load interrupted by a connector restart |
DBX | Databricks Unity Catalog |
DBX_SQL | Databricks SQL |
AWS_LAKEFORMATION | AWS Lake Formation, including the AWS Glue calls the Lake Formation connector makes |
Every other outcome, success included, names the backend service. Other connectors report their connector type in uppercase, with each run of characters other than letters and digits replaced by _.
Code messages¶
The msg of each code that can appear in response[]. The wording can change between releases; branch on code.
code | msg |
|---|---|
1003 | Grant or revoke not applied by the connector; skipped |
1004 | Grant skipped: the resource owner is not mapped to a managed Ranger user |
1005 | Revoke rejected because the permission was already absent |
1006 | Resource not found in source; nothing to apply |
2001 | Resource synced; access grants applied |
2002 | No change detected since last sync, grant not reapplied |
2003 | Resource synced; no matching policy found, so no grants were applied |
2004 | Resource deleted from source |
4001 | Referenced row-filter template not found, or not a Template UDF |
4002 | Databricks rejected the row-filter function or SET ROW FILTER statement |
4003 | Row-filter condition is not defined in the template |
4004 | The same principal is assigned to more than one template condition |
4005 | Template row-filter items are mixed with SQL items or another template on the same table |
4006 | Row-filter column binding is invalid |
4007 | A bound column is not a column of the table |
4008 | A bound column's type differs from the template parameter type |
4009 | A row-filter template placeholder was left unfilled |
4010 | Row-filter template could not be rendered |
4011 | Row-filter template body references an undeclared column |
4012 | Template row-filter policy found but template-based row filtering is disabled on the connector |
4013 | Template row-filter function name is taken by a function not managed by PolicySync |
4014 | Column metadata is not loaded for the table, so the template row filter cannot be checked |
4501 | Resource not found in target service |
4502 | Failed to grant |
4503 | Grantee principal not found in the target service |
4504 | Resource or grantee principal not found in the target service |
4505 | Resource load failed |
5001 | Transient backend or infrastructure failure |
Backend error codes (serviceCode)¶
When a grant, revoke, or resource load fails, serviceCode keeps the backend's own error code unchanged, and PolicySync maps it to a code.
- Branch on
code, not onserviceCode. The lists below are common values, not a complete set. - The same
serviceCodecan map to a differentcodeon a grant or revoke than on a load; the tables say which. - A
serviceCodePolicySync does not recognize maps to4502on a grant or revoke, and4505on a load. - A resource that an on-demand request names but the source does not have is
1006, with noserviceCode.
Databricks Unity Catalog and Databricks SQL use the same rules; origin is DBX or DBX_SQL. serviceCode is one of:
- A Databricks error condition, such as
TABLE_OR_VIEW_NOT_FOUND, or a SQLState: grants, revokes, and loads that run SQL. Unity Catalog loads columns and foreign tables with SQL; Databricks SQL loads everything with SQL. HTTP_<status>: Unity Catalog loads of catalogs, schemas, and tables, which use the REST API, and SQL statements whose HTTP request Databricks rejected.
A compound condition such as SCHEMA_DOES_NOT_EXIST.RESOURCE_DOES_NOT_EXIST is mapped on whichever part PolicySync recognizes.
serviceCode (examples) | What it means | Privacera Code |
|---|---|---|
TABLE_OR_VIEW_NOT_FOUND, TABLE_NOT_FOUND, VIEW_NOT_FOUND, SCHEMA_NOT_FOUND, CATALOG_NOT_FOUND, DATABASE_NOT_FOUND, RESOURCE_DOES_NOT_EXIST, PATH_NOT_FOUND | The table, view, schema, or catalog does not exist | 4501 on a grant or revoke, 1006 on a load |
PRINCIPAL_DOES_NOT_EXIST, PRINCIPAL_NOT_FOUND, USER_NOT_FOUND, GROUP_NOT_FOUND, SERVICE_PRINCIPAL_NOT_FOUND, NO_SUCH_USER, NO_SUCH_GROUP | The grantee user, group, or service principal does not exist | 4503 |
INSUFFICIENT_PERMISSIONS, PERMISSION_DENIED, UNAUTHORIZED_ACCESS, ACCESS_DENIED, FORBIDDEN | The connector's identity is not allowed to run the operation | 4502 on a grant or revoke, 4505 on a load |
PARSE_SYNTAX_ERROR, SYNTAX_ERROR, INVALID_PARAMETER_VALUE, UNSUPPORTED_FEATURE, FEATURE_NOT_SUPPORTED, INVALID_STATE | The grant or revoke is not valid for the target, for example a privilege that does not apply to the object | 4502 on a grant or revoke, 4505 on a load |
HTTP_400, HTTP_401, HTTP_403, HTTP_404, and any other status not listed as transient below | Databricks rejected the request, for example because the connector's token expired or was revoked. On a Unity Catalog load, HTTP_404 means the catalog, schema, or table does not exist. Running the operation again does not help | 4502 on a grant or revoke, 4505 on a load |
INTERNAL_ERROR, SERVICE_UNAVAILABLE, TEMPORARILY_UNAVAILABLE, UNAVAILABLE, REQUEST_LIMIT_EXCEEDED, TOO_MANY_REQUESTS, RESOURCE_EXHAUSTED, DEADLINE_EXCEEDED, IO_ERROR, QUERY_TIMEOUT, STATEMENT_TIMEOUT, OPERATION_TIMEOUT, ABORTED, CONCURRENT_APPEND, CONCURRENT_DELETE_READ, CONCURRENT_DELETE_DELETE, CONCURRENT_UPDATE, CONCURRENT_WRITE, CONCURRENT_TRANSACTION, DELTA_CONCURRENT_APPEND, DELTA_CONCURRENT_DELETE_READ, DELTA_CONCURRENT_DELETE_DELETE, DELTA_CONCURRENT_WRITE, HTTP_429, HTTP_500, HTTP_502, HTTP_503, HTTP_504, SQLState class 08, 40, 53, 57, 58 | A transient backend or infrastructure failure; running the same operation again later may succeed | 5001 |
Some conditions look transient but need a fix, so they map to 4502 on a grant or revoke and 4505 on a load: DELTA_CONCURRENT_TRANSACTION, and any condition ending in .CHANGE_TYPE_COLUMN. An 08S01 or 40001 that the JDBC driver derived from the message text, rather than received from Databricks, is not transient either.
AWS Lake Formation — serviceCode is the AWS error code (for example EntityNotFoundException) or HTTP_<status>.
serviceCode (examples) | What it means | Privacera Code |
|---|---|---|
InvalidInputException | On a grant or revoke, the table or resource no longer exists in Lake Formation, or the request is not valid for it. On a load, the request was rejected | 4501 on a grant or revoke, 4505 on a load |
InvalidInputException whose message starts with No permissions revoked, or EntityNotFoundException, on a revoke | The permission was already absent, so the revoke had nothing to remove | 1005 |
EntityNotFoundException, except on a revoke | The resource or the grantee does not exist; AWS reports one code for both and does not say which. On a load, the resource does not exist | 4504 on a grant, 4502 on a row-level security change, 1006 on a load |
AccessDeniedException | The connector's IAM role is not allowed to run the operation | 4502 on a grant or revoke, 4505 on a load |
AlreadyExistsException (row-level security) | A data cell filter with the same name already exists for the database and table | 4502 |
InternalServiceException, InternalFailure, ServiceUnavailableException, ThrottlingException, TooManyRequestsException, ConcurrentModificationException, OperationTimeoutException, HTTP_429, HTTP_500, HTTP_502, HTTP_503, HTTP_504 | A transient AWS failure; PolicySync retries throttling and conflict errors, and reports this code if the operation still does not succeed | 5001 |
Response codes by event type¶
| Event | One event covers | extraInfo.response[] |
|---|---|---|
ACCESS | One grant or revoke | One entry, count 1, for success, skip, and failure alike |
POLICY_RESPONSE | One policy | One entry per distinct outcome of the policy's ACL operations and owner skips |
ON_DEMAND | One requested resource | The outcomes of its ACL operations when it reached grant application; otherwise one entry, count 1, for its own outcome. A failed resource load adds the load failure's code. See Limitations |
LOADER | One loader run | Only when the load failed and the connector classified the failure: one entry, count 1. A successful loader run has no response[] |
TEMPLATE_UPDATE_RESPONSE | One changed row-filter template (Databricks Unity Catalog) | Failures only, counted in tables: one entry per distinct failure. The tables re-created and unchanged are in extraInfo.tableCount, not entries. Absent when no table failed. See Audits of a template update |
Each table below lists every code its event type can carry.
- Entries are sorted by
count, highest first. Do not rely on the order of entries with the samecount. - An event lists at most ten entries. When there are more, the retryable (
5xxx) entries are kept first, then the highest counts. - A
FAILEDevent can carry onlyEXPECTEDentries, for example an AWS Lake Formation revoke of a permission that was already gone (1005), or an on-demand request for a resource that does not exist (1006). An alert built onstatusshould also check the code's range.
ACCESS codes¶
code | strCode | Category | Event status | Description |
|---|---|---|---|---|
2001 | SYNCED_GRANTS_APPLIED | SUCCESS | COMPLETED | The grant or revoke was applied |
1003 | GRANT_SKIPPED | EXPECTED | SKIPPED | The connector chose not to apply the grant or revoke, for example because grant updates are turned off, the resource type is not one it grants on, or the same grant was already applied in the batch. msg gives the reason |
1005 | PERMISSION_ALREADY_REVOKED | EXPECTED | FAILED | The backend refused a revoke because the permission was already absent. There is nothing to fix or retry. Reported by AWS Lake Formation |
4001–4014 | Template codes | VALIDATION | FAILED | A template-based row filter could not be applied. Reported by Databricks Unity Catalog. See Template-based row filter |
4501 | RESOURCE_NOT_FOUND | PERMANENT | FAILED | The grant or revoke targeted a resource that does not exist in the target service |
4502 | GRANT_FAILED | PERMANENT | FAILED | The backend rejected the grant or revoke, for example for missing privileges, or the failure could not be classified more precisely. serviceCode keeps the backend's code when there is one |
4503 | PRINCIPAL_NOT_FOUND | PERMANENT | FAILED | The grantee (user, group, or service principal) does not exist in the target service. Reported by Databricks Unity Catalog and Databricks SQL |
4504 | RESOURCE_OR_PRINCIPAL_NOT_FOUND | PERMANENT | FAILED | The backend reported that the resource or the grantee does not exist, without saying which. Reported by AWS Lake Formation |
5001 | SYSTEM_ERROR | TRANSIENT | FAILED | The backend failed for an infrastructure reason, such as throttling, a timeout, a dropped connection, or an internal service error. The same grant may succeed if run again |
A grant skipped because the grantee owns the resource (1004) has no ACCESS event; it is counted on the policy's POLICY_RESPONSE.
POLICY_RESPONSE codes¶
Each entry counts the policy's ACL operations, or owner skips, that had that outcome. The failure codes are the same as on the policy's failed ACCESS events.
code | strCode | Category | Description |
|---|---|---|---|
2001 | SYNCED_GRANTS_APPLIED | SUCCESS | ACL operations that were applied |
2002 | NO_CHANGE_DETECTED | SUCCESS | The policy ran no ACL operations. Listed alone, with count 0 |
1003 | GRANT_SKIPPED | EXPECTED | ACL operations the connector chose not to apply. The ACCESS events give each reason |
1004 | RESOURCE_OWNER_SKIPPED | EXPECTED | Grants skipped because the grantee owns the resource and is not mapped to a managed Ranger user; ownership already gives access. count equals ownerIgnore.count. These have no ACCESS event |
1005 | PERMISSION_ALREADY_REVOKED | EXPECTED | Revokes the backend refused because the permission was already absent. They count as failed in aclsExecuted, so they affect the event status, but there is nothing to fix or retry. Reported by AWS Lake Formation |
4001–4014 | Template codes | VALIDATION | Template-based row filters that could not be applied. Reported by Databricks Unity Catalog. See Template-based row filter |
4501 | RESOURCE_NOT_FOUND | PERMANENT | ACL operations on a resource that does not exist in the target service |
4502 | GRANT_FAILED | PERMANENT | ACL operations the backend rejected, or that could not be classified more precisely. Also listed with count 0 when the changelog failed and none of the policy's ACL operations did |
4503 | PRINCIPAL_NOT_FOUND | PERMANENT | Grants to a principal that does not exist in the target service. Reported by Databricks Unity Catalog and Databricks SQL |
4504 | RESOURCE_OR_PRINCIPAL_NOT_FOUND | PERMANENT | ACL operations on a resource or principal that does not exist. Reported by AWS Lake Formation |
5001 | SYSTEM_ERROR | TRANSIENT | ACL operations that failed for an infrastructure reason. Only these are worth retrying |
The event status comes from the policy's ACL counts, aclsExecuted.applied and aclsExecuted.failed:
| Case | status | response[] |
|---|---|---|
| Every ACL operation applied | COMPLETED | 2001 with count = ACL operations applied, plus 1003 or 1004 for any skips |
| Some applied, some failed | GRANT_PARTIALLY_APPLIED | 2001, plus one entry per failure code, for example 4503 x2 and 5001 x1 |
| Every ACL operation failed | FAILED | One entry per failure code, for example 4503 x5 |
| Only skips, or only owner skips | COMPLETED | 1003 or 1004, with their counts |
| A policy change applied no ACL operations | COMPLETED | 2002 x0 |
| The changelog itself failed | FAILED | The ACL outcomes, plus 4502 x0 when none of them failed; 4502 x0 alone when no ACL operation ran |
How the entries add up:
- While a policy has ten distinct outcomes or fewer, the counts add up to
aclsExecuted.totalplusownerIgnore.count. Beyond ten, entries are left out as described in Response codes by event type, so the counts add up to less. - There is no policy-level retry flag. To find what to retry, follow Link a failure to its
ACCESSaudits.
ON_DEMAND codes¶
An ON_DEMAND event's entries come from three sources (the Source column):
- ACL operations: a resource that reached grant application lists the outcomes of the on-demand task's ACL operations, grouped as on
POLICY_RESPONSE. They cover the whole task, so every changed resource in the request lists the same entries. A resource whose outcome isRESOURCE_DELETEDalso lists2004. - Resource: any other resource lists its own outcome, one entry with
count1. - Resource load: when the load failed, every resource in the request also lists the load failure's code.
code | strCode | Category | Source | Description |
|---|---|---|---|---|
2001 | SYNCED_GRANTS_APPLIED | SUCCESS | ACL operations | The resource changed and a matching policy applied its grants |
2002 | NO_CHANGE_DETECTED | SUCCESS | Resource | The resource is unchanged since the last sync, so grants were not re-applied |
2003 | NO_POLICY_FOUND | SUCCESS | Resource | The resource changed, but no policy matched it, so no grants were applied |
2004 | RESOURCE_DELETED | SUCCESS | Resource | The resource no longer exists in the source; it was removed and its grants were revoked |
1003 | GRANT_SKIPPED | EXPECTED | ACL operations | ACL operations the connector chose not to apply |
1004 | RESOURCE_OWNER_SKIPPED | EXPECTED | ACL operations | Grants skipped because the grantee owns the resource and is not mapped to a managed Ranger user |
1005 | PERMISSION_ALREADY_REVOKED | EXPECTED | ACL operations | Revokes refused because the permission was already absent. Reported by AWS Lake Formation |
1006 | RESOURCE_NOT_FOUND | EXPECTED | Resource, resource load | The resource does not exist in the source, for example a dropped table, so nothing was applied. The event status is FAILED. See Limitations for a load that stops on a missing catalog or schema |
4001–4014 | Template codes | VALIDATION | ACL operations | Template-based row filters that could not be applied. Reported by Databricks Unity Catalog. See Template-based row filter |
4501 | RESOURCE_NOT_FOUND | PERMANENT | ACL operations | ACL operations on a resource that does not exist in the target service |
4502 | GRANT_FAILED | PERMANENT | ACL operations, resource | ACL operations the backend rejected or that could not be classified more precisely; or grant application failed with no failed ACL operation, for example before any ran or because the changelog failed after the grants applied. When grant application could not start and the failure was classified, the entry is that classification instead, such as 5001 |
4503 | PRINCIPAL_NOT_FOUND | PERMANENT | ACL operations, resource load | Grants to a principal that does not exist, or a resource load that failed on a missing principal. Reported by Databricks Unity Catalog and Databricks SQL |
4504 | RESOURCE_OR_PRINCIPAL_NOT_FOUND | PERMANENT | ACL operations | ACL operations on a resource or principal that does not exist. Reported by AWS Lake Formation |
4505 | RESOURCE_LOAD_FAILED | PERMANENT | Resource load | The resource load did not complete, for a reason that is neither transient nor a not-found error, or that could not be classified |
5001 | SYSTEM_ERROR | TRANSIENT | ACL operations, resource, resource load | An ACL operation, the resource load, or the start of grant application failed for an infrastructure reason, such as throttling or a dropped connection. A load interrupted by a connector restart is also reported this way, with origin PRIVACERA, on every resource in the request. A new request may succeed |
The event status:
COMPLETEDwhen the resource's outcome is a success (2001–2004) and no ACL operation of the on-demand task failed.FAILEDotherwise. A partial grant hasmsgGrants partially applied; some grants failed, and a resource not in the source has1006.- Failed ACL operations count for the whole task: when one fails, every resource in the request that reached grant application is
FAILED, even if its own grants applied. A resource that did not change or was not found keeps its own outcome.
For how each outcome is decided, see Outcome codes.
output.lfForward is unchanged. AWS Table-Level Federation adds it when it forwards an on-demand sync to AWS Lake Formation, and names the reason LF_SYNC_FORWARD_FAILED when the forwarded task was not created. That reason has the number 4506 in the code registry, but it is never a response[] entry and does not change the resource's outcome.
LOADER codes¶
A LOADER event lists a code only when the load failed (status FAILED) and the connector classified the failure. The connectors that classify failures are Databricks Unity Catalog, Databricks SQL, and AWS Lake Formation, for the loaders that read the target service: resources, permissions, principals, and tags.
code | strCode | Category | Description |
|---|---|---|---|
1006 | RESOURCE_NOT_FOUND | EXPECTED | The load stopped because something it reads, such as a catalog or schema, does not exist. See Limitations |
4503 | PRINCIPAL_NOT_FOUND | PERMANENT | The load failed on a principal that does not exist in the target service. Reported by Databricks Unity Catalog and Databricks SQL |
4505 | RESOURCE_LOAD_FAILED | PERMANENT | The load failed for a reason that is neither transient nor a not-found error, or that could not be classified |
5001 | SYSTEM_ERROR | TRANSIENT | The load failed for an infrastructure reason, such as throttling or a dropped connection. The next load may succeed |
Template-based row filter (Databricks Unity Catalog)¶
When a template-based row filter cannot be applied, the failed ACCESS event, the policy's POLICY_RESPONSE, and an ON_DEMAND event for the table carry a code from 4001 to 4014, with origin PRIVACERA.
DDL_FAILED(4002) andUNDECLARED_COLUMN(4011) come from Databricks rejecting a statement. They carry the Databricks error condition asserviceCode.- A transient Databricks failure is
5001SYSTEM_ERRORinstead, with originDBX.
For the list of codes, see Codes in the PolicySync audits. For causes and fixes, see Errors from the connector.
Limitations of response codes¶
- On
ON_DEMAND, a resource that reached grant application lists the ACL outcomes of the whole on-demand task, not only its own. When one request changes several resources, a transient failure on one of them gives each of them a5001entry and makes each of themFAILED. To find the resource that failed, read theACCESSevents with the sameondemandTaskId. - When a whole resource load stops because a catalog or schema is not found, the failed
LOADERevent, or everyON_DEMANDevent of an on-demand request, can report1006RESOURCE_NOT_FOUND, which is in theEXPECTEDrange. Checkstatusas well as the code. - When an on-demand task fails before it reaches the target service, the entry can name the target service as its
origin.
Link a failure to its ACCESS audits¶
POLICY_RESPONSE and ON_DEMAND events summarize many ACL operations. Each ACL operation also has its own ACCESS event, which names the resource, the permission, the principal, and the statement. To see which grants failed and which applied, find the ACCESS events behind the summary.
Fields that link the events¶
| Field | ACCESS | POLICY_RESPONSE | ON_DEMAND | LOADER |
|---|---|---|---|---|
connector | Yes | Yes | Yes | Yes |
changeLogId | Yes | Yes | Yes | Yes |
changeLogType | Yes | Yes | Yes | Yes |
policyId | List of IDs | One ID | No | No |
policyVersion | List, aligned with policyId | One version | No | No |
ondemandTaskId | When the grant came from an on-demand request | No | Yes | When the load came from an on-demand request |
extraInfo.response[].code | One entry | One entry per outcome | One entry per outcome | One entry when the load failed |
A changeLogId is unique only within one connector instance, so always match connector together with changeLogId.
A TEMPLATE_UPDATE_RESPONSE carries connector, changeLogId, and changeLogType (TEMPLATE_UDF_REAPPLY). Its failed tables are the FAILED ACCESS events with the same connector and changeLogId whose extraInfo.udf_template.name is the summary's template (the same run can apply other templates too); see Find the failed tables.
From a POLICY_RESPONSE¶
- Note the event's
connector,changeLogId,policyId, andpolicyVersion. - Find the
ACCESSevents with the sameconnectorandchangeLogIdwhosepolicyIdlist contains thatpolicyId. When several Ranger policies give the same grant, theACCESSevent lists all of them; the version at the same position inpolicyVersionis the version that was applied. - Split them by outcome:
- Failures:
statusFAILED. Each one'sresponse[0].codeandserviceCodematch one of the failure entries in thePOLICY_RESPONSE. - Successes:
statusCOMPLETEDwith code2001. - Skips:
statusSKIPPEDwith code1003.
- Failures:
- Retry only the
ACCESSevents whose code is from5000to5999. Fix the policy, the principal, or the resource for the others.
An owner skip (1004) has no ACCESS event; it is counted in the POLICY_RESPONSE ownerIgnore.
In the example events, the POLICY_RESPONSE for policy 812 version 7 on changelog 48213 has a 5001 entry with count 1. The ACCESS event on finance/ledger/transactions with the same connector, changeLogId, and policyId carries that 5001, and is the grant to retry. The ACCESS event on finance/ledger/accounts carries 2001 and is one of the four that applied.
From an ON_DEMAND event¶
- Note the event's
connectorandondemandTaskId. - Find the
ACCESSevents with the sameconnectorandondemandTaskId. Each one names its resource and carries its own code. - Split and act on them as for a
POLICY_RESPONSE.
The ACL outcomes on an ON_DEMAND event cover the whole on-demand task; see Limitations. The ACCESS events show which resource each outcome belongs to.
An ON_DEMAND event whose resource did not reach grant application, for example 1006, 2002, 2003, or 4505, has no ACCESS events.
From a LOADER event¶
A failed load applies no grants, so it has no ACCESS events. Its code tells you whether the next load may succeed (5001) or the configuration needs a fix. When the resource load of an on-demand request fails, every ON_DEMAND event of the request, with the same ondemandTaskId, lists the load failure's code.
- Prev topic: Overview of Connectors
- Related: Audit Server
- Related: PolicySync configuration