# Ensure any monkey patching is cleaned up for subsequent tests with patch("transformers.models.mllama.modeling_mllama"): from transformers.models.mllama.modeling_mllama import ( MllamaForConditionalGeneration, ) # Instantiate a dummy model config = transformers.models.mllama.configuration_mllama.M...
2. Apply Model-Specific Patching APIs Using thepatching APIs, you can swap Hugging Face models with optimized Liger Kernels. importtransformersfromliger_kernel.transformersimportapply_liger_kernel_to_llama# 1a. Adding this line automatically monkey-patches the model with the optimized Liger kernelsapply...
fused_linear_cross_entropy=False, ) > INFO:liger_kernel.transformers.monkey_patch:Applying Liger kernels for model type: llama with kwargs: {'cross_entropy': True, 'fused_linear_cross_entropy': False} ``` ## Testing Done - Run huggingface example ```python assert modeling_llama.LlamaRMSNor...
CrossEntropy Patching FIx: If you use monkey patch forCrossEntropy(Not FLCE), it is actually not patched after transformers4.46.1. This is becauseCrossEntropywas replaced withF.cross_entropyin the model code. We fixed the issue in the PR (linkedin/Liger-Kernel#375) GroupNorm Kernel: Our ne...
Breaking this contract seems like a great way to generate a lot of confusion in the ecosystem and is no better than monkey patching. Try this: Python 3.8.5 (default, Jul 28 2020, 12:59:40) Type 'copyright', 'credits' or 'license' for more information IPython 7.18.1 -- An enhanced ...
Using the patching APIs, you can swap Hugging Face models with optimized Liger Kernels.import transformers from liger_kernel.transformers import apply_liger_kernel_to_llama # 1a. Adding this line automatically monkey-patches the model with the optimized Liger kernels apply_liger_kernel_to_llama() ...
2. Apply Model-Specific Patching APIsUsing the patching APIs, you can swap Hugging Face models with optimized Liger Kernels.import transformers from liger_kernel.transformers import apply_liger_kernel_to_llama # 1a. Adding this line automatically monkey-patches the model with the optimized Liger ...
Using thepatching APIs, you can swap Hugging Face models with optimized Liger Kernels. importtransformersfromliger_kernel.transformersimportapply_liger_kernel_to_llama# 1a. Adding this line automatically monkey-patches the model with the optimized Liger kernelsapply_liger_kernel_to_llama()# 1b. You...
Using the patching APIs, you can swap Hugging Face models with optimized Liger Kernels. import transformers from liger_kernel.transformers import apply_liger_kernel_to_llama model = transformers.AutoModelForCausalLM.from_pretrained("<some llama model>") # Adding this line automatically monkey-patches...
2. Apply Model-Specific Patching APIsUsing the patching APIs, you can swap Hugging Face models with optimized Liger Kernels.import transformers from liger_kernel.transformers import apply_liger_kernel_to_llama # 1a. Adding this line automatically monkey-patches the model with the optimized Liger ...