pipeline_updates

Creates, updates, deletes, gets or lists a pipeline_updates resource.

Overview

Name	pipeline_updates
Type	Resource
Id	databricks_workspace.deltalivetables.pipeline_updates

Fields

The following fields are returned by SELECT queries:

listupdates

Name	Datatype	Description
cluster_id	string
pipeline_id	string
update_id	string
cause	string
config	object
creation_time	integer
full_refresh	boolean
full_refresh_selection	array
refresh_selection	array
state	string
validate_only	boolean

getupdate

Name	Datatype	Description
pipeline_id	string
request_id	string
update_id	string
cause	string
config	object
creation_time	integer
full_refresh	boolean
state	string

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
listupdates	select	deployment_name		List updates for an active pipeline.
getupdate	select	deployment_name		Gets an update from an active pipeline.
startupdate	exec	deployment_name		Starts a new update for the pipeline. If there is already an active update for the pipeline, the request will fail and the active update will remain running.

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
deployment_name	string	The Databricks Workspace Deployment Name (default: dbc-abcd0123-a1bc)

SELECT examples

listupdates

List updates for an active pipeline.

SELECT
cluster_id,
pipeline_id,
update_id,
cause,
config,
creation_time,
full_refresh,
full_refresh_selection,
refresh_selection,
state,
validate_only
FROM databricks_workspace.deltalivetables.pipeline_updates
WHERE deployment_name = '{{ deployment_name }}' -- required;

getupdate

Gets an update from an active pipeline.

SELECT
pipeline_id,
request_id,
update_id,
cause,
config,
creation_time,
full_refresh,
state
FROM databricks_workspace.deltalivetables.pipeline_updates
WHERE deployment_name = '{{ deployment_name }}' -- required;

Lifecycle Methods

startupdate

Starts a new update for the pipeline. If there is already an active update for the pipeline, the request will fail and the active update will remain running.

EXEC databricks_workspace.deltalivetables.pipeline_updates.startupdate 
@deployment_name='{{ deployment_name }}' --required 
@@json=
'{
"full_refresh": {{ full_refresh }}, 
"validate_only": {{ validate_only }}, 
"cause": "{{ cause }}", 
"refresh_selection": "{{ refresh_selection }}", 
"full_refresh_selection": "{{ full_refresh_selection }}"
}';