<![CDATA[Quantization is a model compression technique that reduces the numerical precision of Large Language Model (LLM) weights to lower memory requirements and accelerate inference. This study presents a comparative analysis of three widely used quantization schemes—INT4, INT8, and FP16—using the Mistral-7B-Instruct-v0.2 model as the subject of study. Evaluation was conducted across five output quality dimensions: BLEU score, ROUGE-L score, perplexity, BERTScore, and average inference latency. Testing utilized 200 instruction prompts covering four task categories: text summarization, factual question answering, code generation, and logical reasoning. The analysis results indicate that INT8 offers the best balance between computational efficiency and output quality, showing an average performance degradation of 2.3% compared to FP16 while achieving a 48% reduction in memory requirements. INT4 exhibited a more significant degradation of 7.8% in logical reasoning tasks, despite successfully reducing memory usage by 74%. These findings provide practical guidance for researchers and practitioners in selecting a quantization scheme suited to available computational resources.]]>
Copyrights © 2026
