sparknlp.util#

Contains various utilities.

Module Contents#

Classes#

CoNLLGenerator

EmbeddingsDataFrameUtils

Utility for creating DataFrames compatible with multimodal embedding models (e.g., E5VEmbeddings) for text-only scenarios.

Functions#

get_config_path()

get_config_path()[source]#
class CoNLLGenerator[source]#
static exportConllFiles(*args)[source]#
class EmbeddingsDataFrameUtils[source]#

Utility for creating DataFrames compatible with multimodal embedding models (e.g., E5VEmbeddings) for text-only scenarios. Provides:

  • imageSchema: the expected schema for Spark image DataFrames

  • emptyImageRow: a dummy image row for text-only embedding

imageSchema[source]#
emptyImageRow[source]#