dbfs_streams

Creates, updates, deletes, gets or lists a dbfs_streams resource.

Overview

Name	dbfs_streams
Type	Resource
Id	databricks_workspace.filemanagement.dbfs_streams

Fields

The following fields are returned by SELECT queries:

SELECT not supported for this resource, use SHOW METHODS to view available operations for the resource.

Methods

The following methods are available for this resource:

Name	Accessible by	Required Params	Optional Params	Description
create	insert	deployment_name		Opens a stream to write to a file and returns a handle to this stream. There is a 10 minute idle timeout on this handle. If a file or directory already exists on the given path and
close	delete	deployment_name		Closes the stream specified by the input handle. If the handle does not exist, this call throws an exception with
addblock	exec	deployment_name		Appends a block of data to the stream specified by the input handle. If the handle does not exist, this call will throw an exception with

Parameters

Parameters can be passed in the WHERE clause of a query. Check the Methods section to see which parameters are required or optional for each operation.

Name	Datatype	Description
deployment_name	string	The Databricks Workspace Deployment Name (default: dbc-abcd0123-a1bc)

INSERT examples

create

Opens a stream to write to a file and returns a handle to this stream. There is a 10 minute idle timeout on this handle. If a file or directory already exists on the given path and

INSERT INTO databricks_workspace.filemanagement.dbfs_streams (
data__path,
data__overwrite,
deployment_name
)
SELECT 
'{{ path }}',
'{{ overwrite }}',
'{{ deployment_name }}'
RETURNING
handle
;

Manifest

# Description fields are for documentation purposes
- name: dbfs_streams
  props:
    - name: deployment_name
      value: string
      description: Required parameter for the dbfs_streams resource.
    - name: path
      value: required
    - name: overwrite
      value: string

DELETE examples

close

Closes the stream specified by the input handle. If the handle does not exist, this call throws an exception with

DELETE FROM databricks_workspace.filemanagement.dbfs_streams
WHERE deployment_name = '{{ deployment_name }}' --required;

Lifecycle Methods

addblock

Appends a block of data to the stream specified by the input handle. If the handle does not exist, this call will throw an exception with

EXEC databricks_workspace.filemanagement.dbfs_streams.addblock 
@deployment_name='{{ deployment_name }}' --required 
@@json=
'{
"handle": "{{ handle }}", 
"data": {{ data }}
}';