Packages

class SentimentDetectorModel extends AnnotatorModel[SentimentDetectorModel] with HasSimpleAnnotate[SentimentDetectorModel]

Rule based sentiment detector, which calculates a score based on predefined keywords.

This is the instantiated model of the SentimentDetector. For training your own model, please see the documentation of that class.

A dictionary of predefined sentiment keywords must be provided with setDictionary, where each line is a word delimited to its class (either positive or negative). The dictionary can be set in either in the form of a delimited text file or directly as an ExternalResource.

By default, the sentiment score will be assigned labels "positive" if the score is >= 0, else "negative". To retrieve the raw sentiment scores, enableScore needs to be set to true.

For extended examples of usage, see the Examples and the SentimentTestSpec.

See also

ViveknSentimentApproach for an alternative approach to sentiment extraction

Linear Supertypes
HasSimpleAnnotate[SentimentDetectorModel], AnnotatorModel[SentimentDetectorModel], CanBeLazy, RawAnnotator[SentimentDetectorModel], HasOutputAnnotationCol, HasInputAnnotationCols, HasOutputAnnotatorType, ParamsAndFeaturesWritable, HasFeatures, DefaultParamsWritable, MLWritable, Model[SentimentDetectorModel], Transformer, PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. SentimentDetectorModel
  2. HasSimpleAnnotate
  3. AnnotatorModel
  4. CanBeLazy
  5. RawAnnotator
  6. HasOutputAnnotationCol
  7. HasInputAnnotationCols
  8. HasOutputAnnotatorType
  9. ParamsAndFeaturesWritable
  10. HasFeatures
  11. DefaultParamsWritable
  12. MLWritable
  13. Model
  14. Transformer
  15. PipelineStage
  16. Logging
  17. Params
  18. Serializable
  19. Serializable
  20. Identifiable
  21. AnyRef
  22. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Parameters

A list of (hyper-)parameter keys this annotator can take. Users can set and get the parameter values through setters and getters, respectively.

  1. val decrementMultiplier: DoubleParam

    Multiplier for decrement sentiments (Default: -2.0)

  2. val enableScore: BooleanParam

    if true, score will show as a string type containing a double value, else will output string "positive" or "negative" (Default: false)

  3. val incrementMultiplier: DoubleParam

    Multiplier for increment sentiments (Default: 2.0)

  4. lazy val model: PragmaticScorer

  5. val negativeMultiplier: DoubleParam

    Multiplier for negative sentiments (Default: -1.0)

  6. val positiveMultiplier: DoubleParam

    Multiplier for positive sentiments (Default: 1.0)

  7. val reverseMultiplier: DoubleParam

    Multiplier for revert sentiments (Default: -1.0)

  8. val sentimentDict: MapFeature[String, String]

    Sentiment dict

Annotator types

Required input and expected output annotator types

  1. val inputAnnotatorTypes: Array[AnnotatorType]

    Input annotation type : TOKEN, DOCUMENT

    Input annotation type : TOKEN, DOCUMENT

    Definition Classes
    SentimentDetectorModel → HasInputAnnotationCols
  2. val outputAnnotatorType: AnnotatorType

    Output annotation type : SENTIMENT

    Output annotation type : SENTIMENT

    Definition Classes
    SentimentDetectorModel → HasOutputAnnotatorType

Members

  1. type AnnotatorType = String
    Definition Classes
    HasOutputAnnotatorType
  1. def annotate(annotations: Seq[Annotation]): Seq[Annotation]

    Tokens are needed to identify each word in a sentence boundary POS tags are optionally submitted to the model in case they are needed Lemmas are another optional annotator for some models Bounds of sentiment are hardcoded to 0 as they render useless

    Tokens are needed to identify each word in a sentence boundary POS tags are optionally submitted to the model in case they are needed Lemmas are another optional annotator for some models Bounds of sentiment are hardcoded to 0 as they render useless

    annotations

    Annotations that correspond to inputAnnotationCols generated by previous annotators if any

    returns

    any number of annotations processed for every input annotation. Not necessary one to one relationship

    Definition Classes
    SentimentDetectorModel → HasSimpleAnnotate
  2. final def clear(param: Param[_]): SentimentDetectorModel.this.type
    Definition Classes
    Params
  3. def copy(extra: ParamMap): SentimentDetectorModel

    requirement for annotators copies

    requirement for annotators copies

    Definition Classes
    RawAnnotator → Model → Transformer → PipelineStage → Params
  4. def dfAnnotate: UserDefinedFunction

    Wraps annotate to happen inside SparkSQL user defined functions in order to act with org.apache.spark.sql.Column

    Wraps annotate to happen inside SparkSQL user defined functions in order to act with org.apache.spark.sql.Column

    returns

    udf function to be applied to inputCols using this annotator's annotate function as part of ML transformation

    Definition Classes
    HasSimpleAnnotate
  5. def explainParam(param: Param[_]): String
    Definition Classes
    Params
  6. def explainParams(): String
    Definition Classes
    Params
  7. final def extractParamMap(): ParamMap
    Definition Classes
    Params
  8. final def extractParamMap(extra: ParamMap): ParamMap
    Definition Classes
    Params
  9. val features: ArrayBuffer[Feature[_, _, _]]
    Definition Classes
    HasFeatures
  10. final def get[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  11. final def getDefault[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  12. def getInputCols: Array[String]

    returns

    input annotations columns currently used

    Definition Classes
    HasInputAnnotationCols
  13. def getLazyAnnotator: Boolean
    Definition Classes
    CanBeLazy
  14. final def getOrDefault[T](param: Param[T]): T
    Definition Classes
    Params
  15. final def getOutputCol: String

    Gets annotation column name going to generate

    Gets annotation column name going to generate

    Definition Classes
    HasOutputAnnotationCol
  16. def getParam(paramName: String): Param[Any]
    Definition Classes
    Params
  17. final def hasDefault[T](param: Param[T]): Boolean
    Definition Classes
    Params
  18. def hasParam(paramName: String): Boolean
    Definition Classes
    Params
  19. def hasParent: Boolean
    Definition Classes
    Model
  20. final def isDefined(param: Param[_]): Boolean
    Definition Classes
    Params
  21. final def isSet(param: Param[_]): Boolean
    Definition Classes
    Params
  22. val lazyAnnotator: BooleanParam
    Definition Classes
    CanBeLazy
  23. val optionalInputAnnotatorTypes: Array[String]
    Definition Classes
    HasInputAnnotationCols
  24. lazy val params: Array[Param[_]]
    Definition Classes
    Params
  25. var parent: Estimator[SentimentDetectorModel]
    Definition Classes
    Model
  26. def save(path: String): Unit
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  27. final def set[T](param: Param[T], value: T): SentimentDetectorModel.this.type
    Definition Classes
    Params
  28. final def setInputCols(value: String*): SentimentDetectorModel.this.type
    Definition Classes
    HasInputAnnotationCols
  29. def setInputCols(value: Array[String]): SentimentDetectorModel.this.type

    Overrides required annotators column if different than default

    Overrides required annotators column if different than default

    Definition Classes
    HasInputAnnotationCols
  30. def setLazyAnnotator(value: Boolean): SentimentDetectorModel.this.type
    Definition Classes
    CanBeLazy
  31. final def setOutputCol(value: String): SentimentDetectorModel.this.type

    Overrides annotation column name when transforming

    Overrides annotation column name when transforming

    Definition Classes
    HasOutputAnnotationCol
  32. def setParent(parent: Estimator[SentimentDetectorModel]): SentimentDetectorModel
    Definition Classes
    Model
  33. def toString(): String
    Definition Classes
    Identifiable → AnyRef → Any
  34. final def transform(dataset: Dataset[_]): DataFrame

    Given requirements are met, this applies ML transformation within a Pipeline or stand-alone Output annotation will be generated as a new column, previous annotations are still available separately metadata is built at schema level to record annotations structural information outside its content

    Given requirements are met, this applies ML transformation within a Pipeline or stand-alone Output annotation will be generated as a new column, previous annotations are still available separately metadata is built at schema level to record annotations structural information outside its content

    dataset

    Dataset[Row]

    Definition Classes
    AnnotatorModel → Transformer
  35. def transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" )
  36. def transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" ) @varargs()
  37. final def transformSchema(schema: StructType): StructType

    requirement for pipeline transformation validation.

    requirement for pipeline transformation validation. It is called on fit()

    Definition Classes
    RawAnnotator → PipelineStage
  38. val uid: String
    Definition Classes
    SentimentDetectorModel → Identifiable
  39. def write: MLWriter
    Definition Classes
    ParamsAndFeaturesWritable → DefaultParamsWritable → MLWritable

Parameter setters

  1. def setDecrementMultipler(v: Double): SentimentDetectorModel.this.type

    Multiplier for decrement sentiments (Default: -2.0)

  2. def setEnableScore(v: Boolean): SentimentDetectorModel.this.type

    If true, score will show as a string type containing a double value, else will output string "positive" or "negative" (Default: false)

  3. def setIncrementMultipler(v: Double): SentimentDetectorModel.this.type

    Multiplier for increment sentiments (Default: 2.0)

  4. def setNegativeMultipler(v: Double): SentimentDetectorModel.this.type

    Multiplier for negative sentiments (Default: -1.0)

  5. def setPositiveMultipler(v: Double): SentimentDetectorModel.this.type

    Multiplier for positive sentiments (Default: 1.0)

  6. def setReverseMultipler(v: Double): SentimentDetectorModel.this.type

    Multiplier for revert sentiments (Default: -1.0)

  7. def setSentimentDict(value: Map[String, String]): SentimentDetectorModel.this.type

    Path to file with list of inputs and their content, with such delimiter, readAs LINE_BY_LINE or as SPARK_DATASET.

    Path to file with list of inputs and their content, with such delimiter, readAs LINE_BY_LINE or as SPARK_DATASET. If latter is set, options is passed to spark reader.