Packages

package sat

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. trait ReadSaTDLModel extends ReadOnnxModel with ReadSentencePieceModel
  2. trait ReadablePretrainedSaTModel extends ParamsAndFeaturesReadable[SentenceDetectorSaTModel] with HasPretrained[SentenceDetectorSaTModel]
  3. class SentenceDetectorSaTModel extends AnnotatorModel[SentenceDetectorSaTModel] with HasBatchedAnnotate[SentenceDetectorSaTModel] with WriteOnnxModel with WriteSentencePieceModel with HasEngine

    Spark NLP sentence detection annotator based on the wtpsplit / SaT transformer models.

    Spark NLP sentence detection annotator based on the wtpsplit / SaT transformer models.

    Supports segment-any-text/sat-12l-sm and segment-any-text/sat-12l (and any other SaT model exported as ONNX with an XLM-R SentencePiece tokenizer).

    Example

    import com.johnsnowlabs.nlp.base.DocumentAssembler
import com.johnsnowlabs.nlp.annotators.sbd.sat.SentenceDetectorSaTModel
import org.apache.spark.ml.Pipeline

val documentAssembler = new DocumentAssembler()
  .setInputCol("text")
  .setOutputCol("document")

val satModel = SentenceDetectorSaTModel
  .loadSavedModel("/path/to/sat-12l-sm", spark)
  .setInputCols("document")
  .setOutputCol("sentence")
  .setThreshold(0.25f)

val pipeline = new Pipeline().setStages(Array(documentAssembler, satModel))
val result   = pipeline.fit(data).transform(data)
result.selectExpr("explode(sentence.result)").show(false)

Value Members

  1. object SentenceDetectorSaTModel extends ReadablePretrainedSaTModel with ReadSaTDLModel with Serializable

Ungrouped