Llama-2 text-to-text model 7b int8

Description

Meta developed and publicly released the Llama 2 family of large language models (LLMs), a collection of pretrained and fine-tuned generative text models ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama-2-Chat, are optimized for dialogue use cases. Llama-2-Chat models outperform open-source chat models on most benchmarks we tested, and in our human evaluations for helpfulness and safety, are on par with some popular closed-source models like ChatGPT and PaLM.

Live Demo Open in Colab Download Copy S3 URI

How to use

documentAssembler = DocumentAssembler() \
.setInputCol("text") \
.setOutputCol("documents")

llama2 = LLAMA2Transformer \
    .pretrained("llama_2_7b_chat_hf_int8") \
    .setMaxOutputLength(50) \
    .setDoSample(False) \
    .setInputCols(["documents"]) \
    .setOutputCol("generation")

pipeline = Pipeline().setStages([documentAssembler, llama2])
data = spark.createDataFrame([["My name is Leonardo."]]).toDF("text")
result = pipeline.fit(data).transform(data)
result.select("summaries.generation").show(truncate=False)

val documentAssembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("documents")

val llama2 = LLAMA2Transformer.pretrained("llama_2_7b_chat_hf_int8") 
    .setMaxOutputLength(50) 
    .setDoSample(False) 
    .setInputCols(["documents"]) 
    .setOutputCol("generation")

val pipeline = new Pipeline().setStages(Array(documentAssembler, llama2))

val data = Seq("My name is Leonardo.").toDF("text")
val result = pipeline.fit(data).transform(data)
results.select("generation.result").show(truncate = false)

Model Information