Description
This is a text-to-text model fine tuned to correct grammatical errors when the task is set to “gec:”. It is based on Prithiviraj Damodaran’s Gramformer model.
Predicted Entities
Live Demo Open in Colab Download Copy S3 URI
How to use
import sparknlp
from sparknlp.base import *
from sparknlp.annotator import *
spark = sparknlp.start()
documentAssembler = DocumentAssembler() \
.setInputCol("text") \
.setOutputCol("documents")
t5 = T5Transformer.pretrained("t5_grammar_error_corrector") \
.setTask("gec:") \
.setInputCols(["documents"]) \
.setMaxOutputLength(200) \
.setOutputCol("corrections")
pipeline = Pipeline().setStages([documentAssembler, t5])
data = spark.createDataFrame([["He are moving here."]]).toDF("text")
result = pipeline.fit(data).transform(data)
result.select("corrections.result").show(truncate=False)
import spark.implicits._
import com.johnsnowlabs.nlp.base.DocumentAssembler
import com.johnsnowlabs.nlp.annotators.seq2seq.T5Transformer
import org.apache.spark.ml.Pipeline
val documentAssembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("documents")
val t5 = T5Transformer.pretrained("t5_grammar_error_corrector")
.setTask("gec:")
.setMaxOutputLength(200)
.setInputCols("documents")
.setOutputCol("corrections")
val pipeline = new Pipeline().setStages(Array(documentAssembler, t5))
val data = Seq("He are moving here.").toDF("text")
val result = pipeline.fit(data).transform(data)
result.select("corrections.result").show(false)
import nlu
nlu.load("en.t5.grammar_error_corrector").predict("""He are moving here.""")
Results
+--------------------+
|result |
+--------------------+
|[He is moving here.]|
+--------------------+
Model Information
| Model Name: | t5_grammar_error_corrector |
| Compatibility: | Spark NLP 3.4.0+ |
| License: | Open Source |
| Edition: | Official |
| Input Labels: | [documents] |
| Output Labels: | [corrections] |
| Language: | en |
| Size: | 926.2 MB |
Data Source
Model originally from the transformers library: https://huggingface.co/prithivida/grammar_error_correcter_v1