diff --git a/README.md b/README.md index a711b78..1b71c01 100644 --- a/README.md +++ b/README.md @@ -226,9 +226,97 @@ python eval_scripts/mplug_owl_2/eval_video_quality.py + + ### InternLM-XComposer-VL -*---coming soon---* +*InternLM-XComposer-VL has been integrated into Huggingface `AutoModel` (remote code mode). You can directly start with the code below without a separate install process.* + +#### Simple Interactive Demos + +
+Example Code (Single Query) + +```python +import torch +from transformers import AutoModel, AutoTokenizer + +torch.set_grad_enabled(False) + +# init model and tokenizer +model = AutoModel.from_pretrained('DLight1551/internlm-xcomposer-vl-7b-qinstruct-full', trust_remote_code=True).cuda().eval() +tokenizer = AutoTokenizer.from_pretrained('DLight1551/internlm-xcomposer-vl-7b-qinstruct-full', trust_remote_code=True) +model.tokenizer = tokenizer + +# Single-Turn Text-Image Dialogue +text = 'Describe and evaluate the quality of the image.' +image = 'fig/sausage.jpg' +response = model.generate(text, image) +print(response) +``` + +
+ +
+Example Code (Multi-Turn Conversation) + +```python +import torch +from transformers import AutoModel, AutoTokenizer + +torch.set_grad_enabled(False) + +# init model and tokenizer +model = AutoModel.from_pretrained('DLight1551/internlm-xcomposer-vl-7b-qinstruct-full', trust_remote_code=True).cuda().eval() +tokenizer = AutoTokenizer.from_pretrained('DLight1551/internlm-xcomposer-vl-7b-qinstruct-full', trust_remote_code=True) +model.tokenizer = tokenizer + +# Multi-Turn Dialogue +text = 'Describe and evaluate the quality of the image.' +image = 'fig/sausage.jpg' +response, history = model.chat(text, image, history=None) +print(f'User: {text}') +print(f'Bot: {response}') + +text = 'Which part of the pan is clearer, the top part of the bottom part?' +response, history = model.chat(text=text, image=None, history=history) +print(f'User: {text}') +print(f'Bot: {response}') +``` + +
+ + +#### Quantitative Evaluations + + +
+Multi-choice question (MCQ) in Q-Bench. + +```shell +python eval_scripts/mplug_owl_2/eval_qbench_mcq.py +``` + +
+ + +
+Image/Video Quality Assessment + +Image Quality Assessment: + +```shell +python eval_scripts/mplug_owl_2/eval_image_quality.py +``` + +Video Quality Assessment: + +```shell +python eval_scripts/mplug_owl_2/eval_video_quality.py +``` + +
+ ## Model Zoo