我创建了text-embedding-ada-002这个模型,但是我使用相同内容,访问gpt原始api。获取到的embedding 向量,不一样。这个是什么原因呢 Azure OpenAI Service Azure OpenAI Service An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities. ...
import{fromPreTrained}from"@lenml/tokenizer-text_embedding_ada002";consttokenizer=fromPreTrained();console.log("encode()",tokenizer.encode("Hello, my dog is cute",null,{add_special_tokens:true,}));console.log("_encode_text",tokenizer._encode_text("Hello, my dog is cute")); Full Tokenizer...
{"embedding_endpoint_type":"openai","embedding_endpoint":"https://api.openai.com/v1","embedding_model":"text-embedding-ada-002","embedding_dim":1536,"embedding_chunk_size":300,"azure_endpoint":null,"azure_version":null,"azure_deployment":null},"organization_id":"org-00000000-0000-4000-...