configuration#
Configuration properties for Tumult Core.
Functions#
Return a Spark config suitable for use with Java 11. |
|
Check for running on Java11+, and make sure the correct options are set. |
- get_java11_config()#
Return a Spark config suitable for use with Java 11.
You can build a session with this config by running code like:
SparkSession.builder.config(conf=get_java11_config()).getOrCreate()
.- Return type
pyspark.conf.SparkConf
- check_java11()#
Check for running on Java11+, and make sure the correct options are set.
- Raises
SparkConfigError – If Spark is running on Java 11 or higher, but is not configured with the options required for Java 11 or higher.
- Return type
None
Classes#
Global configuration for programs using Core. |
Exceptions#
- exception SparkConfigError(java_version, spark_conf, message)#
Bases:
RuntimeError
Exception raised when a misconfigured Spark session is running.