输入“/”快速插入内容

MiniCPM 3.0 推理

2024年9月29日修改

👇

模型地址：https://huggingface.co/openbmb/MiniCPM3-4B

github地址：https://github.com/OpenBMB/MiniCPM

面向人员：能使用基本的bash，python，能够使用python处理数据即可​

机器要求：最少8G以上内存的机器，其余显存随上下文长度上升​

点击目录直接跳转👇

Transformers

chat方法

代码块

from transformers import AutoTokenizer,AutoModelForCausalLM​
tokenizer = AutoTokenizer.from_pretrained("/root/ld/ld_model_pretrained/minicpm3")​
model=AutoModelForCausalLM.from_pretrained("/root/ld/ld_model_pretrained/minicpm3",trust_remote_code=True).cuda()​
history=[]​
query=input("user:")​
response,history=model.chat(tokenizer, query=query,history=history)​
print("model:",response)​
query=input("user:")​
response,history=model.chat(tokenizer, query=query,history=history)​
print("model:",response)​
#history是列表形式[{"role": "assistant", "content": answer1},{"role": "assistant", "content": response}。。。。]​

Generate 方法

1.
继续生成类(非问答)方法​

代码块

from transformers import AutoTokenizer, AutoModelForCausalLM​
​
model = AutoModelForCausalLM.from_pretrained("/root/ld/ld_model_pretrained/minicpm3",trust_remote_code=True).cuda()​
tokenizer = AutoTokenizer.from_pretrained("/root/ld/ld_model_pretrained/minicpm3",trust_remote_code=True)​
​
prompt = "Hey, are you conscious? Can you tell me "​
inputs = tokenizer(prompt, return_tensors="pt")​
​
# Generate​
generate_ids = model.generate(inputs.input_ids.cuda(), max_length=300,do_sample=False)​
output=tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]​
print(output)​

2.
对话类（问答）方法​

代码块

from transformers import AutoTokenizer,AutoModelForCausalLM​
tokenizer = AutoTokenizer.from_pretrained("/root/ld/ld_model_pretrained/minicpm3")​
model=AutoModelForCausalLM.from_pretrained("/root/ld/ld_model_pretrained/minicpm3",trust_remote_code=True).cuda()​
query="你吃了饭没"​
message=[{"role":"user","content":query}]​
model_inputs = tokenizer.apply_chat_template(message, return_tensors="pt", add_generation_prompt=True).to(model.device)​
generate_ids = model.generate(model_inputs, max_length=300,do_sample=False)​
output=tokenizer.batch_decode(generate_ids, skip_special_tokens=False, clean_up_tokenization_spaces=False)[0]​
print(output)​

Function call 简易实现（更细致、更高效请看​💥【再聊agent】Function call 详解）​

MiniCPM 3.0 推理​

MiniCPM 3.0 推理