sparknlp.annotator.seq2seq.phi4_transformer#
Contains classes for the Phi4Transformer.
Module Contents#
Classes#
Phi-4: State-of-the-art open model by Microsoft Research |
- class Phi4Transformer(classname='com.johnsnowlabs.nlp.annotators.seq2seq.Phi4Transformer', java_model=None)[source]#
Phi-4: State-of-the-art open model by Microsoft Research
phi-4 is a 14B parameter, dense decoder-only Transformer model trained on 9.8T tokens, designed for advanced reasoning, code, and general NLP tasks. For more details, see: https://huggingface.co/microsoft/phi-4
- Parameters:
- configProtoBytes
ConfigProto from tensorflow, serialized into byte array.
- minOutputLength
Minimum length of the sequence to be generated, by default 0
- maxOutputLength
Maximum length of output text, by default 60
- doSample
Whether or not to use sampling; use greedy decoding otherwise, by default False
- temperature
The value used to modulate the next token probabilities, by default 1.0
- topK
The number of highest probability vocabulary tokens to keep for top-k-filtering, by default 40
- topP
Top cumulative probability for vocabulary tokens, by default 1.0
If set to float < 1, only the most probable tokens with probabilities that add up to
topPor higher are kept for generation.- repetitionPenalty
The parameter for repetition penalty, 1.0 means no penalty. , by default 1.0
- noRepeatNgramSize
If set to int > 0, all ngrams of that size can only occur once, by default 0
- ignoreTokenIds
A list of token ids which are ignored in the decoder’s output, by default []
Notes
This is a very computationally expensive module, especially on larger sequences. The use of an accelerator such as GPU is recommended.
References
arXiv:2412.08905
Input Annotation types
Output Annotation type
DOCUMENTDOCUMENTExamples
>>> import sparknlp >>> from sparknlp.base import * >>> from sparknlp.annotator import * >>> from pyspark.ml import Pipeline >>> documentAssembler = DocumentAssembler() ... .setInputCol("text") ... .setOutputCol("documents") >>> phi4 = Phi4Transformer.pretrained("phi-4") ... .setInputCols(["documents"]) ... .setMaxOutputLength(60) ... .setOutputCol("generation") >>> pipeline = Pipeline().setStages([documentAssembler, phi4]) >>> data = spark.createDataFrame([ ... ( ... 1, ... "<|start_header_id|>system<|end_header_id|> \n"+ ... "You are a helpful assistant! \n" + ... "<|start_header_id|>user<|end_header_id|> \n" + ... "What is Phi-4? \n" + ... "<|start_header_id|>assistant<|end_header_id|> \n" ... ) ... ]).toDF("id", "text") >>> result = pipeline.fit(data).transform(data) >>> result.select("generation.result").show(truncate=False) +------------------------------------------------+ |result | +------------------------------------------------+ |[Phi-4 is a 14B parameter, dense decoder-only Transformer model developed by Microsoft Research for advanced reasoning, code, and general NLP tasks.]| +------------------------------------------------+
- setIgnoreTokenIds(value)[source]#
A list of token ids which are ignored in the decoder’s output.
- Parameters:
- valueList[int]
The words to be filtered out
- setConfigProtoBytes(b)[source]#
Sets configProto from tensorflow, serialized into byte array.
- Parameters:
- bList[int]
ConfigProto from tensorflow, serialized into byte array
- setMinOutputLength(value)[source]#
Sets minimum length of the sequence to be generated.
- Parameters:
- valueint
Minimum length of the sequence to be generated
- setMaxOutputLength(value)[source]#
Sets maximum length of output text.
- Parameters:
- valueint
Maximum length of output text
- setDoSample(value)[source]#
Sets whether or not to use sampling, use greedy decoding otherwise.
- Parameters:
- valuebool
Whether or not to use sampling; use greedy decoding otherwise
- setTemperature(value)[source]#
Sets the value used to module the next token probabilities.
- Parameters:
- valuefloat
The value used to module the next token probabilities
- setTopK(value)[source]#
Sets the number of highest probability vocabulary tokens to keep for top-k-filtering.
- Parameters:
- valueint
Number of highest probability vocabulary tokens to keep
- setTopP(value)[source]#
Sets the top cumulative probability for vocabulary tokens.
If set to float < 1, only the most probable tokens with probabilities that add up to
topPor higher are kept for generation.- Parameters:
- valuefloat
Cumulative probability for vocabulary tokens
- setRepetitionPenalty(value)[source]#
Sets the parameter for repetition penalty. 1.0 means no penalty.
- Parameters:
- valuefloat
The repetition penalty
References
See Ctrl: A Conditional Transformer Language Model For Controllable Generation for more details.
- setNoRepeatNgramSize(value)[source]#
Sets size of n-grams that can only occur once.
If set to int > 0, all ngrams of that size can only occur once.
- Parameters:
- valueint
N-gram size can only occur once
- setBeamSize(value)[source]#
Sets the number of beams to use for beam search.
- Parameters:
- valueint
The number of beams to use for beam search
- setStopTokenIds(value)[source]#
Sets a list of token ids which are considered as stop tokens in the decoder’s output.
- Parameters:
- valueList[int]
The words to be considered as stop tokens
- static loadSavedModel(folder, spark_session, use_openvino=False)[source]#
Loads a locally saved model.
- Parameters:
- folderstr
Folder of the saved model
- spark_sessionpyspark.sql.SparkSession
The current SparkSession
- Returns:
- Phi4Transformer
The restored model
- static pretrained(name='phi-4', lang='en', remote_loc=None)[source]#
Downloads and loads a pretrained model.
- Parameters:
- namestr, optional
Name of the pretrained model, by default “phi-4”
- langstr, optional
Language of the pretrained model, by default “en”
- remote_locstr, optional
Optional remote address of the resource, by default None. Will use Spark NLPs repositories otherwise.
- Returns:
- Phi4Transformer
The restored model