Edit on GitHub
  • John Snow Labs
  • Nov 06, 2021

BERT Sequence Classification - Identify Antisemitic texts

  • en
  • open_source

Description

This model is imported from Hugging Face-models and it was trained on 4K tweets, where ~50% were labeled as antisemitic. The model identifies if the text is antisemitic or not.

  • 1 : Antisemitic
  • 0 : Non-antisemitic

Predicted Entities

1, 0

Download Copy S3 URI

How to use

document_assembler = DocumentAssembler() \
    .setInputCol('text') \
    .setOutputCol('document')

tokenizer = Tokenizer() \
    .setInputCols(['document']) \
    .setOutputCol('token')

sequenceClassifier = BertForSequenceClassification \
      .pretrained('bert_sequence_classifier_antisemitism', 'en') \
      .setInputCols(['token', 'document']) \
      .setOutputCol('class')

pipeline = Pipeline(stages=[document_assembler, tokenizer, sequenceClassifier])

example = spark.createDataFrame([["The Jews have too much power!"]]).toDF("text")
result = pipeline.fit(example).transform(example)

val document_assembler = DocumentAssembler() 
    .setInputCol("text") 
    .setOutputCol("document")

val tokenizer = Tokenizer() 
    .setInputCols("document") 
    .setOutputCol("token")

val tokenClassifier = BertForSequenceClassification.pretrained("bert_sequence_classifier_antisemitism", "en")
      .setInputCols("document", "token")
      .setOutputCol("class")

val pipeline = new Pipeline().setStages(Array(document_assembler, tokenizer, sequenceClassifier))

val example = Seq.empty["The Jews have too much power!"].toDS.toDF("text")

val result = pipeline.fit(example).transform(example)

import nlu
nlu.load("en.classify.bert").predict("""The Jews have too much power!""")

Results

['1']

Model Information

Model Name: bert_sequence_classifier_antisemitism
Compatibility: Spark NLP 3.3.2+
License: Open Source
Edition: Official
Input Labels: [token, sentence]
Output Labels: [label]
Language: en
Case sensitive: true

Data Source

https://huggingface.co/astarostap/autonlp-antisemitism-2-21194454