twitter_xlm_roberta_base_sentiment(Cardiff nlp) (Veer)

Description

This is a multilingual XLM-roBERTa-base model trained on ~198M tweets and finetuned for sentiment analysis. The sentiment fine-tuning was done on 8 languages (Ar, En, Fr, De, Hi, It, Sp, Pt) but it can be used for more languages (see paper for details).

Paper: XLM-T: A Multilingual Language Model Toolkit for Twitter. Git Repo: XLM-T official repository. This model has been integrated into the TweetNLP library.

Predicted Entities

Download Copy S3 URI

How to use

from pyspark.ml import Pipeline

document_assembler = DocumentAssembler() \
    .setInputCol('text') \
    .setOutputCol('document')

tokenizer = Tokenizer() \
    .setInputCols(['document']) \
    .setOutputCol('token')

sequenceClassifier = XlmRoBertaForSequenceClassification.pretrained('twitter_xlm_roberta_base_sentiment')\
  .setInputCols(["document",'token'])\
  .setOutputCol("class")

pipeline = Pipeline(stages=[
    document_assembler, 
    tokenizer,
    sequenceClassifier    
])

# couple of simple examples
example = spark.createDataFrame([['사랑해!'], ["T'estimo! ❤️"], ["I love you!"], ['Mahal kita!']]).toDF("text")

result = pipeline.fit(example).transform(example)

# result is a DataFrame
result.select("text", "class.result").show()

Model Information

Model Name: twitter_xlm_roberta_base_sentiment
Compatibility: Spark NLP 5.0.0+
License: Open Source
Edition: Community
Input Labels: [document, token]
Output Labels: [class]
Language: en
Size: 1.0 GB
Case sensitive: true
Max sentence length: 512