sparknlp.common.storage#

Classes#

class HasStorageRef[source]#

storageRef[source]#

setStorageRef(value)[source]#

Sets unique reference name for identification.

Parameters:

valuestr

Unique reference name for identification

getStorageRef()[source]#

Gets unique reference name for identification.

Returns:

str

Unique reference name for identification

class HasStorageOptions[source]#

includeStorage[source]#

enableInMemoryStorage[source]#

setIncludeStorage(value)[source]#

Sets whether to include indexed storage in trained model.

Parameters:

valuebool

Whether to include indexed storage in trained model

getIncludeStorage()[source]#

Gets whether to include indexed storage in trained model.

Returns:

bool

Whether to include indexed storage in trained model

setEnableInMemoryStorage(value)[source]#

Sets whether to load whole indexed storage in memory (in-memory lookup)

Parameters:

valuebool

Whether to load whole indexed storage in memory (in-memory lookup)

getEnableInMemoryStorage()[source]#

class HasStorageModel[source]#

saveStorage(path, spark)[source]#

Saves the current model to storage.

Parameters:

pathstr

Path for saving the model.

sparkpyspark.sql.SparkSession

The current SparkSession

abstract static loadStorage(path, spark, storage_ref)[source]#

static loadStorages(path, spark, storage_ref, databases)[source]#

class HasStorage[source]#

storagePath[source]#

setStoragePath(path, read_as)[source]#

Sets path to file.

Parameters:

pathstr

Path to file

read_asstr

How to interpret the file

Notes

See ReadAs for reading options.

getStoragePath()[source]#

Gets path to file.

Returns:

str

path to file