ローカルLLMの実測値比較

Last Update 2026/02/19

TOP - 各種テスト - LLM - ローカルLLMの実測値比較 Llama 3.3 [英語プロンプト]

低スペック寄りのPCでローカルLLMを動作させた際の記録です。
LLM以外の仮想マシンなどが起動され、多少負荷がかかった状態で実行しています。
ベンチマークなどでLLMの性能を評価する内容ではありません。

検証用PC

OS	Debian GNU/Linux 12 (bookworm)
CPU	Intel(R) Core(TM) i5-14400F
GPU	GeForce RTX 3060 12GB
メモリ	DDR4 PC4-25600 32GB × 4
SSD	crucial P310 CT1000P310SSD8-JP

構築環境 : Docker + Ollama (特別な設定などは無い状態)

検証用プロンプト

Could you please recommend some great places in the US to see beautiful scenery? Around 10 places in all four directions.

Llama 3.3 [英語プロンプト]

GPU無し

70b-instruct-q2_K(1.46TPS) 70b-instruct-q4_K_M(0.93TPS) 70b-instruct-q5_K_M(0.82TPS)

GPU使用

70b-instruct-q2_K(2.08TPS) 70b-instruct-q4_K_M(1.18TPS) 70b-instruct-q5_K_M(0.98TPS)

・TPS(tokens/s) は eval_count / eval_duration により算出
・モデルロード済みの検証は省略

llama3.3:70b-instruct-q2_K(GPU無し)

Model architecture llama parameters 70.6B context length 131072 embedding length 8192 quantization Q2_K 2026-02-18 total_duration(合計時間) : 299728407321 (299.728s) load_duration(モデルのロード時間) : 11893740630 ( 11.894s) prompt_eval_count(評価されたプロンプトのトークン数) : 34 prompt_eval_duration(プロンプトの評価時間) : 10704058301 ( 10.704s) eval_count(生成トークン数) : 404 eval_duration(生成時間) : 276785169372 (276.785s) real 4m59.747s user 0m0.062s sys 0m0.027s メモリ使用量(RSS) : 27189204 KB

llama3.3:70b-instruct-q4_K_M(GPU無し)

Model architecture llama parameters 70.6B context length 131072 embedding length 8192 quantization Q4_K_M 2026-02-18 total_duration(合計時間) : 753254472747 (753.254s) load_duration(モデルのロード時間) : 40279420736 ( 40.279s) prompt_eval_count(評価されたプロンプトのトークン数) : 34 prompt_eval_duration(プロンプトの評価時間) : 11239222001 ( 11.239s) eval_count(生成トークン数) : 655 eval_duration(生成時間) : 701178618371 (701.179s) real 12m33.265s user 0m0.092s sys 0m0.041s メモリ使用量(RSS) : 42958836 KB

llama3.3:70b-instruct-q5_K_M(GPU無し)

Model architecture llama parameters 70.6B context length 131072 embedding length 8192 quantization Q5_K_M 2026-02-18 total_duration(合計時間) : 617501827366 (617.502s) load_duration(モデルのロード時間) : 24683728501 ( 24.684s) prompt_eval_count(評価されたプロンプトのトークン数) : 34 prompt_eval_duration(プロンプトの評価時間) : 14983468284 ( 14.983s) eval_count(生成トークン数) : 474 eval_duration(生成時間) : 577434073540 (577.434s) real 10m17.545s user 0m0.068s sys 0m0.045s メモリ使用量(RSS) : 50211368 KB

llama3.3:70b-instruct-q2_K(GPU使用)

Model architecture llama parameters 70.6B context length 131072 embedding length 8192 quantization Q2_K 2026-02-18 total_duration(合計時間) : 202597663405 (202.598s) load_duration(モデルのロード時間) : 5959970589 ( 5.596s) prompt_eval_count(評価されたプロンプトのトークン数) : 34 prompt_eval_duration(プロンプトの評価時間) : 1582535027 ( 1.581s) eval_count(生成トークン数) : 405 eval_duration(生成時間) : 194767385600 (194.767s) real 3m22.617s user 0m0.070s sys 0m0.021s +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.261.03 Driver Version: 535.261.03 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce RTX 3060 On | 00000000:01:00.0 On | N/A | | 0% 62C P2 60W / 170W | 11502MiB / 12288MiB | 3% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 1189 G /usr/lib/xorg/Xorg 128MiB | | 0 N/A N/A 1924 G xfwm4 2MiB | | 0 N/A N/A 2424 G /usr/bin/x-www-browser 211MiB | | 0 N/A N/A 48510 C /usr/bin/ollama 11146MiB | +---------------------------------------------------------------------------------------+ メモリ使用量(RSS) : 27018980 KB

llama3.3:70b-instruct-q4_K_M(GPU使用)

Model architecture llama parameters 70.6B context length 131072 embedding length 8192 quantization Q4_K_M 2026-02-18 total_duration(合計時間) : 447424201586 (447.424s) load_duration(モデルのロード時間) : 2410818283 ( 2.411s) prompt_eval_count(評価されたプロンプトのトークン数) : 34 prompt_eval_duration(プロンプトの評価時間) : 2692498635 ( 2.692s) eval_count(生成トークン数) : 521 eval_duration(生成時間) : 441955145121 (441.955s) real 7m27.469s user 0m0.042s sys 0m0.041s +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.261.03 Driver Version: 535.261.03 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce RTX 3060 On | 00000000:01:00.0 On | N/A | | 33% 55C P2 53W / 170W | 11038MiB / 12288MiB | 15% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 1189 G /usr/lib/xorg/Xorg 128MiB | | 0 N/A N/A 1924 G xfwm4 2MiB | | 0 N/A N/A 2424 G /usr/bin/x-www-browser 211MiB | | 0 N/A N/A 58055 C /usr/bin/ollama 10682MiB | +---------------------------------------------------------------------------------------+ メモリ使用量(RSS) : 42987732 KB

llama3.3:70b-instruct-q5_K_M(GPU使用)

Model architecture llama parameters 70.6B context length 131072 embedding length 8192 quantization Q5_K_M 2026-02-18 total_duration(合計時間) : 522678431310 (522.678s) load_duration(モデルのロード時間) : 8927562813 ( 8.928s) prompt_eval_count(評価されたプロンプトのトークン数) : 34 prompt_eval_duration(プロンプトの評価時間) : 3502899891 ( 3.503s) eval_count(生成トークン数) : 498 eval_duration(生成時間) : 509906405160 (509.906s) real 8m42.712s user 0m0.039s sys 0m0.055s +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.261.03 Driver Version: 535.261.03 CUDA Version: 12.2 | |-----------------------------------------+----------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+======================+======================| | 0 NVIDIA GeForce RTX 3060 On | 00000000:01:00.0 On | N/A | | 0% 61C P2 51W / 170W | 10730MiB / 12288MiB | 14% Default | | | | N/A | +-----------------------------------------+----------------------+----------------------+ +---------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=======================================================================================| | 0 N/A N/A 1189 G /usr/lib/xorg/Xorg 128MiB | | 0 N/A N/A 1924 G xfwm4 2MiB | | 0 N/A N/A 2424 G /usr/bin/x-www-browser 211MiB | | 0 N/A N/A 63918 C /usr/bin/ollama 10374MiB | +---------------------------------------------------------------------------------------+ メモリ使用量(RSS) : 50284720 KB