Description
The clean_slang is a pretrained pipeline that we can use to process text with a simple pipeline that performs basic processing steps and recognizes entities . It performs most of the common text processing tasks on your dataframe.
How to use
pipeline = PretrainedPipeline('clean_slang', lang='en')
testDoc = '''
yo, what is wrong with ya?
'''
val pipeline = new PretrainedPipeline("clean_slang", lang = "en")
val result = pipeline.fullAnnotate("Hello from John Snow Labs ! ")(0)
import nlu
text = [""Hello from John Snow Labs ! ""]
result_df = nlu.load('en.clean.slang').predict(text)
result_df
Results
['hey', 'what', 'is', 'wrong', 'with', 'you']
Model Information
| Model Name: | clean_slang |
| Type: | pipeline |
| Compatibility: | Spark NLP 3.3.4+ |
| License: | Open Source |
| Edition: | Official |
| Language: | en |
| Size: | 19.1 KB |
Included Models
- DocumentAssembler
- TokenizerModel
- NormalizerModel