configuration#

Configuration properties for Tumult Core.

Functions#

get_java11_config()

Return a Spark config suitable for use with Java 11.

check_java11()

Check for running on Java11+, and make sure the correct options are set.

get_java11_config()#

Return a Spark config suitable for use with Java 11.

You can build a session with this config by running code like: SparkSession.builder.config(conf=get_java11_config()).getOrCreate().

Return type

pyspark.conf.SparkConf

check_java11()#

Check for running on Java11+, and make sure the correct options are set.

Raises

SparkConfigError – If Spark is running on Java 11 or higher, but is not configured with the options required for Java 11 or higher.

Classes#

Config

Global configuration for programs using Core.

class Config#

Global configuration for programs using Core.

classmethod temp_db_name()#

Get the name of the temporary database that Tumult Core uses.

Return type

str

Exceptions#

exception SparkConfigError(java_version, spark_conf, message)#

Bases: RuntimeError

Exception raised when a misconfigured Spark session is running.

Parameters
  • java_version (Optional[int]) –

  • spark_conf (pyspark.sql.conf.RuntimeConfig) –

  • message (str) –