研究者也在聊天、医疗、新闻和文本四个领域进行了测试,取得了一定的效果。 目前,NLLB模型及相关数据集已经开源,并集成到HuggingFace的模型库中。 NLLB消耗5万多GPU小时,可能在教育、元宇宙等领域应用 在分享后,观众向Elbayad博士进行了提问,智源社区对部分回答进行了节选。 Q:NLLB模型消耗的算力是多少?如果要将机器...
model=load_and_prune_for_lang_pair(source_lang='eng_Latn',target_lang='fra_Latn',cache_dir='/some_large_disk/.cache/huggingface/hub',low_cpu_mem_usage=True, )# ~28G of GPU memory, ~28G of CPU memoryprune_for_lang_pair(model,source_lang='eng_Latn',target_lang='deu_Latn')# thi...
large model translator Xenova nllb-200-distilled-600M translator transformers transformers.js huggingface hugging face machine learning deep learning artificial intelligence AI ML machutpublished 1.0.7 • 10 months agopublished 1.0.7 10 months ago M Q PFooter...
The function load_and_prune_for_lang_pair downloads the NLLB-200 checkpoints from HuggingFace (200G in total), if they are not in the HuggingFace cache already. Then it loads the model in CPU memory (100G), prunes it for a given language pair and moves it to GPU memory (approx. ...