Description
Pretrained CLIPForZeroShotClassification model, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.clip_vit_large_patch14 is a English model originally trained by openai.
How to use
imageDF = spark.read \
.format("image") \
.option("dropInvalid", value = True) \
.load("src/test/resources/image/")
candidateLabels = [
"a photo of a bird",
"a photo of a cat",
"a photo of a dog",
"a photo of a hen",
"a photo of a hippo",
"a photo of a room",
"a photo of a tractor",
"a photo of an ostrich",
"a photo of an ox"]
ImageAssembler = ImageAssembler() \
.setInputCol("image") \
.setOutputCol("image_assembler")
imageClassifier = CLIPForZeroShotClassification.pretrained("clip_vit_large_patch14","en") \
.setInputCols(["image_assembler"]) \
.setOutputCol("label") \
.setCandidateLabels(candidateLabels)
pipeline = Pipeline().setStages([ImageAssembler, imageClassifier])
pipelineModel = pipeline.fit(imageDF)
pipelineDF = pipelineModel.transform(imageDF)
val imageDF = ResourceHelper.spark.read
.format("image")
.option("dropInvalid", value = true)
.load("src/test/resources/image/")
val candidateLabels = Array(
"a photo of a bird",
"a photo of a cat",
"a photo of a dog",
"a photo of a hen",
"a photo of a hippo",
"a photo of a room",
"a photo of a tractor",
"a photo of an ostrich",
"a photo of an ox")
val imageAssembler = new ImageAssembler()
.setInputCol("image")
.setOutputCol("image_assembler")
val imageClassifier = CLIPForZeroShotClassification.pretrained("clip_vit_large_patch14","en") \
.setInputCols(Array("image_assembler")) \
.setOutputCol("label") \
.setCandidateLabels(candidateLabels)
val pipeline = new Pipeline().setStages(Array(imageAssembler, imageClassifier))
val pipelineModel = pipeline.fit(imageDF)
val pipelineDF = pipelineModel.transform(imageDF)
Model Information
| Model Name: | clip_vit_large_patch14 |
| Compatibility: | Spark NLP 5.5.0+ |
| License: | Open Source |
| Edition: | Official |
| Input Labels: | [image_assembler] |
| Output Labels: | [label] |
| Language: | en |
| Size: | 1.1 GB |
References
https://huggingface.co/openai/clip-vit-large-patch14