pipeline_events

Creates, updates, deletes, gets or lists a pipeline_events resource.

Overview

Name	pipeline_events
Type	Resource
Id	databricks_workspace.pipelines.pipeline_events

Fields

The following fields are returned by SELECT queries:

list

Name	Datatype	Description
id	string	A time-based, globally unique id.
▶error	object
event_type	string	The event type. Should always correspond to the details
level	string	The severity level of the event. (ERROR, INFO, METRICS, WARN)
maturity_level	string	Maturity level for event_type. (DEPRECATED, EVOLVING, STABLE)
message	string	The display message associated with the event.
▶origin	object	Describes where the event originates from.
▶sequence	object	A sequencing object to identify and order events.
timestamp	string	The time of the event.
▶truncation	object	Information about which fields were truncated from this event due to size constraints. If empty or absent, no truncation occurred. See https://docs.databricks.com/en/ldp/monitor-event-logs for information on retrieving complete event data.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
list	select	pipeline_id, deployment_name	filter, max_results, order_by, page_token	Retrieves events for a pipeline.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
deployment_name	string	The Databricks Workspace Deployment Name (default: dbc-abcd0123-a1bc)
pipeline_id	string	The pipeline to return events for.
filter	string	Criteria to select a subset of results, expressed using a SQL-like syntax. The supported filters are: 1. level='INFO' (or WARN or ERROR) 2. level in ('INFO', 'WARN') 3. id='[event-id]' 4. timestamp > 'TIMESTAMP' (or >=,<,<=,=) Composite expressions are supported, for example: level in ('ERROR', 'WARN') AND timestamp> '2021-07-22T06:37:33.083Z'
max_results	integer	Max number of entries to return in a single page. The system may return fewer than max_results events in a response, even if there are more events available.
order_by	array	A string indicating a sort order by timestamp for the results, for example, ["timestamp asc"]. The sort order can be ascending or descending. By default, events are returned in descending order by timestamp.
page_token	string	Page token returned by previous call. This field is mutually exclusive with all fields in this request except max_results. An error is returned if any fields other than max_results are set when this field is set.

SELECT examples

list

Retrieves events for a pipeline.

SELECT
id,
error,
event_type,
level,
maturity_level,
message,
origin,
sequence,
timestamp,
truncation
FROM databricks_workspace.pipelines.pipeline_events
WHERE pipeline_id = '{{ pipeline_id }}' -- required
AND deployment_name = '{{ deployment_name }}' -- required
AND filter = '{{ filter }}'
AND max_results = '{{ max_results }}'
AND order_by = '{{ order_by }}'
AND page_token = '{{ page_token }}'
;