AttributeError: 'NoneType' object has no attribute 'int4WeightExtractionHalf' Expected Behavior No response Steps To Reproduce Windows环境加载chatglm-6b-int4-qe模型,GPU启动,提问时报错。 Environment - OS:windows 10 - Python:3.9 - Transformers:4.26.1 - PyTorch:1.10 - CUDA Support (`python -c "...
self.int4WeightCompression = None self.SetNumThreads = None try: if not os.path.exists(default_cpu_kernel_code_path): with open(default_cpu_kernel_code_path, "w", encoding="utf-8") as file: code = default_cpu_kernel_code cpu_quantization_code = bz2.decompress(base64.b64decod...
同3060M,不要说int4-qe了,就是int4都能正常运行,关闭独显直连以减少显存占用,还有就是不要边玩游戏边用chatglm Author zerodegress commented Mar 24, 2023 同3060M,不要说int4-qe了,就是int4都能正常运行,关闭独显直连以减少显存占用,还有就是不要边玩游戏边用chatglm 当时我只开着glm,其他东西都没开...