GPTQ LLM Leaderboard Report #4
This week I tested MythoBoros, MythoLogic, Ouroboros, BlueMethod, and Nous-Hermes-Llama2 in three different configurations.
VRAM usage is tested using Oobabooga's Text Generation WebUI, your usage may vary. VRAM in GB is measured using Windows 11 Task Manager. VRAM in MB is measured using nvidia-smi in command prompt.
Models Tested
- TheBloke/MythoBoros-13B-GPTQ (main)
- TheBloke/MythoBoros-13B-GPTQ (gptq-4bit-32g-actorder_True)
- TheBloke/MythoBoros-13B-GPTQ (gptq-8bit-64g-actorder_True)
- TheBloke/MythoLogic-13B-GPTQ (main)
- TheBloke/MythoLogic-13B-GPTQ (gptq-4bit-32g-actorder_True)
- TheBloke/MythoLogic-13B-GPTQ (gptq-8bit-128g-actorder_False)
- TheBloke/13B-Ouroboros-GPTQ (main)
- TheBloke/13B-Ouroboros-GPTQ (gptq-4bit-32g-actorder_True)
- TheBloke/13B-Ouroboros-GPTQ (gptq-8bit-64g-actorder_True)
- TheBloke/13B-BlueMethod-GPTQ (main)
- TheBloke/13B-BlueMethod-GPTQ (gptq-4bit-32g-actorder_True)
- TheBloke/13B-BlueMethod-GPTQ (gptq-8bit-64g-actorder_False)
- TheBloke/Nous-Hermes-Llama2-GPTQ (main)
- TheBloke/Nous-Hermes-Llama2-GPTQ (gptq-4bit-32g-actorder_True)
- TheBloke/Nous-Hermes-Llama2-GPTQ (gptq-8bit-64g-actorder_False)
Results
You can see all the results here.