A lot of GPTQ models from TheBloke have come up recently. I'm gonna evaluate all of them slowly in the coming weeks/months. With this report, TheBloke/upstage-llama-30b-instruct-2048-GPTQ (gptq-4bit-128g-actorder_True) takes the crown as the highest rated model. From my testing, it's the best instruction following model that I can

I've been playing with SDXL LoRA and Fine-tuning these past few weeks and I'm telling you, it's very hard to get right. LoRA seems easier but there are a lot of tweaks that can make it better. In the mean time, I've finished evaluating a handful of GPTQ models including

This week I tested MythoBoros, MythoLogic, Ouroboros, BlueMethod, and Nous-Hermes-Llama2 in three different configurations. VRAM usage is tested using Oobabooga's Text Generation WebUI, your usage may vary. VRAM in GB is measured using Windows 11 Task Manager. VRAM in MB is measured using nvidia-smi in command prompt. Models Tested * TheBloke/

From now on, my GPTQ Leaderboard Report will include VRAM usage of each models. VRAM usage is tested using two model loaders ExLLaMA_HF and AutoGPTQ. ExLLaMA_HF is used when the model is 4-bit and AutoGPTQ is used when the model is in 8-bit. This blog includes a few

Just a few days ago, LLaMA 2 was released by Meta and boy did it make another huge spark in the open-source LLM landscape. I have tested a bunch of LLaMA 2 quantized models by TheBloke and the older LLaMA 1 models for comparison. I will also include more LLaMA2