Question Pair Classifier Pipeline

Description

This pre-trained pipeline identifies whether the two question sentences are semantically repetitive or different.

Predicted Entities

Live Demo Open in Colab Download Copy S3 URI

How to use

  • The question pairs should be identified with “q1” and “q2” in the text. The input text format should be as follows : text = "q1: What is your name? q2: Who are you?".

from sparknlp.pretrained import PretrainedPipeline

pipeline = PretrainedPipeline("classifierdl_electra_questionpair_pipeline", "en")

result1 = pipeline.fullAnnotate("q1: What is your favorite movie? q2: Which movie do you like most?")
result2 = pipeline.fullAnnotate("q1: What is your favorite movie? q2: Which movie genre would you like to watch?")

import com.johnsnowlabs.nlp.pretrained.PretrainedPipeline

val pipeline = new PretrainedPipeline("classifierdl_electra_questionpair_pipeline", "en")

val result1 = pipeline.fullAnnotate("q1: What is your favorite movie? q2: Which movie do you like most?")(0)
val result2 = pipeline.fullAnnotate("q1: What is your favorite movie? q2: Which movie genre would you like to watch?")(0)

Results


result1 --> ['almost_same']
result2 --> ['not_same']

Model Information

Model Name: classifierdl_electra_questionpair_pipeline
Type: pipeline
Compatibility: Spark NLP 4.3.0+
License: Open Source
Edition: Official
Language: en
Size: 2.5 GB

Included Models

  • DocumentAssembler
  • BertSentenceEmbeddings
  • ClassifierDLModel