Description
Automatically identify Joy, Surprise, Fear, Sadness in Tweets using out pretrained Spark NLP DL classifier.
Predicted Entities
surprise, sadness, fear, joy.
Live Demo
Open in Colab
Download
Copy S3 URI
How to use
document_assembler = DocumentAssembler()\
.setInputCol("text")\
.setOutputCol("document")
use = UniversalSentenceEncoder.pretrained('tfhub_use', "en") \
.setInputCols(["document"])\
.setOutputCol("sentence_embeddings")
classifier = ClassifierDLModel.pretrained('classifierdl_use_emotion', 'en') \
.setInputCols(["document", "sentence_embeddings"]) \
.setOutputCol("sentiment")
nlpPipeline = Pipeline(stages=[document_assembler,
use,
classifier])
data = spark.createDataFrame([["@Mira I just saw you on live t.v!!"],
["Just home from group celebration - dinner at Trattoria Gianni, then Hershey Felder's performance - AMAZING!!"],
["Nooooo! My dad turned off the internet so I can't listen to band music!"],
["My soul has just been pierced by the most evil look from @rickosborneorg. A mini panic attack and chill in bones followed soon after."]]).toDF("text")
result = nlpPipeline.fit(data).transform(data)
val documentAssembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("document")
val use = UniversalSentenceEncoder.pretrained('tfhub_use', "en")
.setInputCols(Array("document"))
.setOutputCol("sentence_embeddings")
val classifier = ClassifierDLModel.pretrained("classifierdl_use_emotion", "en")
.setInputCols(Array("document", "sentence_embeddings"))
.setOutputCol("sentiment")
val pipeline = new Pipeline().setStages(Array(documentAssembler,
use,
classifier))
val data = Seq(Array("@Mira I just saw you on live t.v!!",
"Just home from group celebration - dinner at Trattoria Gianni, then Hershey Felder's performance - AMAZING!!",
"Nooooo! My dad turned off the internet so I can't listen to band music!",
"My soul has just been pierced by the most evil look from @rickosborneorg. A mini panic attack and chill in bones followed soon after.")).toDS.toDF("text")
val result = pipeline.fit(data).transform(data)
import nlu
text = ["""@Mira I just saw you on live t.v!!"""]
emotion_df = nlu.load('en.classify.emotion.use').predict(text, output_level='document')
emotion_df[["document", "emotion"]]
Results
+-------------------------------------------------------------------------------------------------------------------------------------+---------+
|document |sentiment|
+-------------------------------------------------------------------------------------------------------------------------------------+---------+
|@Mira I just saw you on live t.v!! |surprise |
|Just home from group celebration - dinner at Trattoria Gianni, then Hershey Felder's performance - AMAZING!! |joy |
|Nooooo! My dad turned off the internet so I can't listen to band music! |sadness |
|My soul has just been pierced by the most evil look from @rickosborneorg. A mini panic attack and chill in bones followed soon after.|fear |
+-------------------------------------------------------------------------------------------------------------------------------------+---------+
Model Information
| Model Name | classifierdl_use_emotion |
| Model Class | ClassifierDLModel |
| Spark Compatibility | 2.5.3 |
| Spark NLP Compatibility | 2.4 |
| License | open source |
| Edition | public |
| Input Labels | [document, sentence_embeddings] |
| Output Labels | [class] |
| Language | en |
| Upstream Dependencies | tfhub_use |
Data Source
This model is trained on multiple datasets inlcuding youtube comments, twitter and ISEAR dataset.
PREVIOUSCyberbullying Classifier