Edit on GitHub
  • John Snow Labs
  • Nov 12, 2023

Multilingual bert_qa_part_1_mbert_model_e1 BertForQuestionAnswering from horsbug98

  • bert
  • xx
  • open_source
  • question_answering
  • onnx

Description

Pretrained BertForQuestionAnswering model, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.bert_qa_part_1_mbert_model_e1 is a Multilingual model originally trained by horsbug98.

Download Copy S3 URI

How to use



document_assembler = MultiDocumentAssembler() \
    .setInputCol(["question", "context"]) \
    .setOutputCol(["document_question", "document_context"])
    
    
spanClassifier = BertForQuestionAnswering.pretrained("bert_qa_part_1_mbert_model_e1","xx") \
            .setInputCols(["document_question","document_context"]) \
            .setOutputCol("answer")

pipeline = Pipeline().setStages([document_assembler, spanClassifier])

pipelineModel = pipeline.fit(data)

pipelineDF = pipelineModel.transform(data)




val document_assembler = new MultiDocumentAssembler()
    .setInputCol(Array("question", "context")) 
    .setOutputCol(Array("document_question", "document_context"))
    
val spanClassifier = BertForQuestionAnswering  
    .pretrained("bert_qa_part_1_mbert_model_e1", "xx")
    .setInputCols(Array("document_question","document_context")) 
    .setOutputCol("answer") 

val pipeline = new Pipeline().setStages(Array(document_assembler, spanClassifier))

val pipelineModel = pipeline.fit(data)

val pipelineDF = pipelineModel.transform(data)

Model Information

Model Name: bert_qa_part_1_mbert_model_e1
Compatibility: Spark NLP 5.2.0+
License: Open Source
Edition: Official
Input Labels: [sentence, token]
Output Labels: [embeddings]
Language: xx
Size: 665.0 MB
Case sensitive: true
Max sentence length: 512

References

https://huggingface.co/horsbug98/Part_1_mBERT_Model_E1