English gpt_2_finetuned_common_gen GPT2Transformer from mrm8488

Description

Pretrained GPT2Transformer model, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.gpt_2_finetuned_common_gen is a English model originally trained by mrm8488.

Download Copy S3 URI

How to use

 
documentAssembler = DocumentAssembler() \
      .setInputCol("text") \
      .setOutputCol("document")

seq2seq = GPT2Transformer.pretrained("gpt_2_finetuned_common_gen","en") \
      .setInputCols(["documents"]) \
      .setOutputCol("generation")       
        
pipeline = Pipeline().setStages([documentAssembler, seq2seq])
data = spark.createDataFrame([["I love spark-nlp"]]).toDF("text")
pipelineModel = pipeline.fit(data)
pipelineDF = pipelineModel.transform(data)


val documentAssembler = new DocumentAssembler() 
    .setInputCol("text") 
    .setOutputCol("document")
    
val seq2seq = GPT2Transformer.pretrained("gpt_2_finetuned_common_gen","en") 
    .setInputCols(Array("documents")) 
    .setOutputCol("generation")

val pipeline = new Pipeline().setStages(Array(documentAssembler, seq2seq))
val data = Seq("I love spark-nlp").toDF("text")
val pipelineModel = pipeline.fit(data)
val pipelineDF = pipelineModel.transform(data)

Model Information

Model Name: gpt_2_finetuned_common_gen
Compatibility: Spark NLP 5.5.1+
License: Open Source
Edition: Official
Input Labels: [document]
Output Labels: [generation]
Language: en
Size: 467.8 MB

References

https://huggingface.co/mrm8488/GPT-2-finetuned-common_gen