Edit on GitHub
  • John Snow Labs
  • Oct 30, 2024

English yi_coder_1.5b_q4_k_m AutoGGUFModel from lmstudio-community

  • en
  • open_source
  • onnx
  • conversational
  • text_generation
  • text_to_text
  • llamacpp

Description

Pretrained AutoGGUFModel model, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.yi_coder_1.5b_q4_k_m is a English model prepared by lmstudio-community.

Download Copy S3 URI

How to use

             
document = DocumentAssembler() \
       .setInputCol("text") \
       .setOutputCol("document")
    
autoGGUFModel = AutoGGUFModel.pretrained("yi_coder_1.5b_q4_k_m","en") \
    .setInputCols(["document"]) \
    .setOutputCol("completions") \
    .setBatchSize(4) \
    .setNPredict(20) \
    .setNGpuLayers(99) \
    .setTemperature(0.4) \
    .setTopK(40) \
    .setTopP(0.9) \
    .setPenalizeNl(True)

pipeline = Pipeline().setStages([document, autoGGUFModel])
data = spark.createDataFrame([["Hello, I am a"]]).toDF("text")
result = pipeline.fit(data).transform(data)
result.select("completions").show(truncate = False)



val document = new DocumentAssembler()
     .setInputCol("text")
     .setOutputCol("document")
    
val autoGGUFModel = AutoGGUFModel.pretrained("yi_coder_1.5b_q4_k_m", "en")
  .setInputCols("document")
  .setOutputCol("completions")
  .setBatchSize(4)
  .setNPredict(20)
  .setNGpuLayers(99)
  .setTemperature(0.4f)
  .setTopK(40)
  .setTopP(0.9f)
  .setPenalizeNl(true)
                                                                       
val pipeline = new Pipeline().setStages(Array(document, autoGGUFModel))
                                                                       
val data = Seq("Hello, I am a").toDF("text")
val result = pipeline.fit(data).transform(data)
result.select("completions").show(truncate = false)

Model Information

Model Name: yi_coder_1.5b_q4_k_m
Compatibility: Spark NLP 5.5.1+
License: Open Source
Edition: Official
Input Labels: [document]
Output Labels: [completions]
Language: en
Size: 944.8 MB

References

https://huggingface.co/lmstudio-community/Yi-Coder-1.5B-GGUF