Google T5 (Text-To-Text Transfer Transformer) Small

Description

The T5 transformer model is described in the seminal paper “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer”. This model can perform a variety of tasks, such as text summarization, question answering, and translation. More details about using the model can be found in the paper (https://arxiv.org/pdf/1910.10683.pdf).

Predicted Entities

Live Demo Open in Colab Download Copy S3 URI

How to use

Either set the following tasks or have them in line with your input:

• summarize:
• translate English to German:
• translate English to French:
• stsb sentence1: Big news. sentence2: No idea.

The full list of tasks is in the Appendix of the paper: https://arxiv.org/pdf/1910.10683.pdf

document_assembler = DocumentAssembler() \
.setInputCol("text") \
.setOutputCol("documents")

t5 = T5Transformer() \
.pretrained("t5_small") \
.setTask("summarize:")\
.setMaxOutputLength(200)\
.setInputCols(["documents"]) \
.setOutputCol("summaries")

pipeline = Pipeline().setStages([document_assembler, t5])
results = pipeline.fit(data_df).transform(data_df)

results.select("summaries.result").show(truncate=False)

val documentAssembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("documents")

val t5 = T5Transformer
.pretrained("t5_small")
.setTask("summarize:")
.setInputCols(Array("documents"))
.setOutputCol("summaries")

val pipeline = new Pipeline().setStages(Array(documentAssembler, t5))

val model = pipeline.fit(dataDf)
val results = model.transform(dataDf)

results.select("summaries.result").show(truncate = false)

import nlu
nlu.load("en.t5.small").predict("""Put your text here.""")

Model Information

References

https://huggingface.co/t5-small