Skip to main content


Allows you to read or write plain Text files


Reads data from Text files at the given Location.

Source Parameters

LocationFile path where the Text files are locatedTrueNone
SchemaSchema to be applied on the loaded data. Can be defined/edited as JSON or inferred using Infer Schema buttonTrueNone
Recursive File LookupThis is used to recursively load files from the given Location. Disables partition discovery. An exception will be thrown if this option and a partitionSpec are specified.FalseFalse
Line SeparatorDefines the line separator that should be used for reading or writing.False\r, \r\n, \n
Read as a single rowIf true, read each file from input path(s) as a single row.FalseFalse


Generated Code

def read_avro(spark: SparkSession) -> DataFrame:
.text("dbfs:/FileStore/customers.txt", wholetext = False, lineSep = "\n")


Target Parameters

Write data as text files at the specified path.

LocationFile path where text files will be written toTrueNone
CompressionCompression codec to use when saving to file. This can be one of the known case-insensitive shorten names (none, bzip2, gzip, lz4, snappy and deflate).FalseNone
Write ModeHow to handle existing data. See this table for a list of available options.Trueerror
Partition ColumnsList of columns to partition the Text files byFalseNone
Line SeparatorDefines the line separator that should be used for writing.False\n

The Text data source supports only a single column apart from the partition columns. An AnalysisException will be thrown if the DataFrame has more than 1 column apart from parition columns as the input DataFrame to the Target Gem.

Supported Write Modes

Write ModeDescription
overwriteIf data already exists, overwrite with the contents of the DataFrame
appendIf data already exists, append the contents of the DataFrame
ignoreIf data already exists, do nothing with the contents of the DataFrame. This is similar to a CREATE TABLE IF NOT EXISTS in SQL.
errorIf data already exists, throw an exception.


Generated Code

def write_text(spark: SparkSession, in0: DataFrame):
.text("dbfs:/FileStore/customers.txt", compression = "gzip", lineSep = "\n")

To know more about tweaking Text file related properties in Spark config click here.