Edit on GitHub
  • John Snow Labs
  • Dec 13, 2023

Esperanto roberta_sayula_popoluca_esperberto_small_sayula_popoluca RoBertaForTokenClassification from julien-c

  • roberta
  • eo
  • open_source
  • token_classification
  • onnx

Description

Pretrained RoBertaForTokenClassification model, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.roberta_sayula_popoluca_esperberto_small_sayula_popoluca is a Esperanto model originally trained by julien-c.

Download Copy S3 URI

How to use



documentAssembler = DocumentAssembler() \
    .setInputCol("text") \
    .setOutputCol("document")
    
tokenizer = Tokenizer() \
        .setInputCols(["document"]) \
        .setOutputCol("token")
        
    
tokenClassifier = RoBertaForTokenClassification.pretrained("roberta_sayula_popoluca_esperberto_small_sayula_popoluca","eo") \
            .setInputCols(["document","token"]) \
            .setOutputCol("ner")

pipeline = Pipeline().setStages([documentAssembler, tokenizer, tokenClassifier])

pipelineModel = pipeline.fit(data)

pipelineDF = pipelineModel.transform(data)




val documentAssembler = new DocumentAssembler()
    .setInputCol("text") 
    .setOutputCol("document")

val tokenizer = Tokenizer() \
        .setInputCols(Array("document")) \
        .setOutputCol("token")

val tokenClassifier = RoBertaForTokenClassification  
    .pretrained("roberta_sayula_popoluca_esperberto_small_sayula_popoluca", "eo")
    .setInputCols(Array("document","token")) 
    .setOutputCol("ner") 

val pipeline = new Pipeline().setStages(Array(documentAssembler, tokenizer, tokenClassifier))

val pipelineModel = pipeline.fit(data)

val pipelineDF = pipelineModel.transform(data)

Model Information

Model Name: roberta_sayula_popoluca_esperberto_small_sayula_popoluca
Compatibility: Spark NLP 5.2.1+
License: Open Source
Edition: Official
Input Labels: [documents, token]
Output Labels: [ner]
Language: eo
Size: 311.4 MB

References

https://huggingface.co/julien-c/EsperBERTo-small-pos