Python10 Repositories Type Language Sort ckiptaggerPublic CKIP Neural Chinese Word Segmentation, POS Tagging, and NER Python1,649GPL-3.019426(4 issues need help)3UpdatedMay 29, 2024 ckip-transformersPublic CKIP Transformers Python711GPL-3.07450UpdatedApr 21, 2023 ...
Python3.6+ PyTorch1.5+ HuggingFace Transformers3.5+ NLP Tools Usage Seeherefor API details. 詳細的 API 請參見此處。 The complete script of this example ishttps://github.com/ckiplab/ckip-transformers/blob/master/example/example.py. 以下的範例的完整檔案可參見https://github.com/ckiplab/ckip-transf...
transformersnamed-entity-recognitionword-segmentationlanguage-modelckippart-of-speech-tagging UpdatedApr 21, 2023 Python ckiplab/ckipnlp Star118 Code Issues Pull requests CKIP CoreNLP Toolkits nlpnamed-entity-recognitionword-segmentationsentence-segmentationcoreference-resolutionckippart-of-speech-taggingsentence-...
from transformers import AutoTokenizer, AutoModelForCausalLM import torch model_name_or_path = 'ckiplab/CKIP-Llama-2-7b-chat' tokenizer = AutoTokenizer.from_pretrained(model_name_or_path) # int8 model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map='auto', torch_dtype...
CKIP TransformersThis project provides traditional Chinese transformers models (including ALBERT, BERT, GPT2) and NLP tools (including word segmentation, part-of-speech tagging, named entity recognition). 這個專案提供了繁體中文的 transformers 模型(包含 ALBERT、BERT、GPT2)及自然語言處理工具(包含斷詞、...
from transformers import AutoTokenizer, AutoModelForCausalLM import torch model_name_or_path = 'ckiplab/CKIP-Llama-2-7b-chat' tokenizer = AutoTokenizer.from_pretrained(model_name_or_path) # int8 model = AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map='auto', torch_dtype...