Edit on GitHub
  • John Snow Labs
  • Sep 03, 2024

Multilingual fine_tuned_hf_language_identification_model_pipeline pipeline XlmRoBertaForSequenceClassification from Joshi-Aryan

  • xx
  • open_source
  • pipeline
  • onnx

Description

Pretrained XlmRoBertaForSequenceClassification, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.fine_tuned_hf_language_identification_model_pipeline is a Multilingual model originally trained by Joshi-Aryan.

Download Copy S3 URI

How to use


pipeline = PretrainedPipeline("fine_tuned_hf_language_identification_model_pipeline", lang = "xx")
annotations =  pipeline.transform(df)   



val pipeline = new PretrainedPipeline("fine_tuned_hf_language_identification_model_pipeline", lang = "xx")
val annotations = pipeline.transform(df)

Model Information

Model Name: fine_tuned_hf_language_identification_model_pipeline
Type: pipeline
Compatibility: Spark NLP 5.5.0+
License: Open Source
Edition: Official
Language: xx
Size: 854.4 MB

References

https://huggingface.co/Joshi-Aryan/Fine_Tuned_HF_Language_Identification_Model

Included Models

  • DocumentAssembler
  • TokenizerModel
  • XlmRoBertaForSequenceClassification