NER Pipeline for 10 African Languages

Description

This pretrained pipeline is built on xlm_roberta_large_token_classifier_masakhaner ner model which is imported from HuggingFace.

Live Demo Open in Colab Download Copy S3 URI

How to use

masakhaner_pipeline = PretrainedPipeline("xlm_roberta_large_token_classifier_masakhaner_pipeline", lang = "xx")

masakhaner_pipeline.annotate("አህመድ ቫንዳ ከ3-10-2000 ጀምሮ በአዲስ አበባ ኖሯል።")
val masakhaner_pipeline = new PretrainedPipeline("xlm_roberta_large_token_classifier_masakhaner_pipeline", lang = "xx")

val masakhaner_pipeline.annotate("አህመድ ቫንዳ ከ3-10-2000 ጀምሮ በአዲስ አበባ ኖሯል።")

Results

+----------------+---------+
|chunk           |ner_label|
+----------------+---------+
|አህመድ ቫንዳ      |PER      |
|ከ3-10-2000 ጀምሮ|DATE      |
|በአዲስ አበባ       |LOC      |
+----------------+---------+

Model Information

Model Name: xlm_roberta_large_token_classifier_masakhaner_pipeline
Type: pipeline
Compatibility: Spark NLP 3.4.0+
License: Open Source
Edition: Official
Language: xx
Size: 1.8 GB

Included Models

  • DocumentAssembler
  • SentenceDetector
  • TokenizerModel
  • XlmRoBertaForTokenClassification
  • NerConverter
  • Finisher