Qwen 3.5 9B vs Ministral 3 14B vs Gemma 4 12B: local LLM benchmark on an RTX 5060 Ti

Small language models have made enormous progress. Models capable of querying a wealth portfolio, reconciling several data points or preparing an action in an application can now run locally on a consumer graphics card.

But one question is more useful than a benchmark ranking:

Key question: which model actually works best in an application such as My Wealth?

We compared three configurations offered by My Wealth's AI profiles:

Tokens per second matter, but the priority is an accurate answer grounded in portfolio data and produced with the right tools. We therefore compare functional success, answer quality, elapsed time and performance under Vulkan and CUDA.

The test machine

All models ran locally on the same machine.

Graphics card

The NVIDIA GeForce RTX 5060 Ti 16 GB belongs to the Blackwell generation. Its 16 GB of GDDR7 can hold all three quantized models in this comparison entirely on the GPU. NVIDIA offers 16 GB and 8 GB versions; we used the former. Official NVIDIA page

Full configuration

Each model's vision module was disabled. The fifteen cases are text-only, preventing an unused component from consuming memory. Peak GPU memory was approximately 7.7 GiB for Qwen, 15.8 GiB for Ministral and 12.0 GiB for Gemma.

The three models

Qwen 3.5 9B — the very fast small model

Qwen 3.5 9B is the most compact of the three. Its official repository presents it as a multimodal conversational model under the Apache 2.0 license. In our text-only test, its smaller size gives it a clear speed advantage. Official Qwen 3.5 9B page

In this test

We use Qwen in Q4_K_M. It is the most accessible and fastest configuration in the comparison. It handles direct requests well, but becomes less consistent when orchestration or writing grows complex.

Ministral 3 14B Instruct 2512 — Mistral AI's edge model

Ministral 3 14B is the largest tested model. Mistral targets it notably at local and edge deployments; its Instruct version supports eleven languages and uses the Apache 2.0 license. Official Ministral 3 14B Instruct page

In this test

We use Ministral in Q5_K_M. To keep the Vulkan/CUDA comparison symmetrical and avoid the memory issue seen with its former runtime, both runs use the same llama.cpp 0.4.0 engine as Gemma, without the vision module. The model uses almost all 16 GB available while remaining stable.

Gemma 4 12B — the highest score in this comparison

Gemma 4 12B is a multimodal Google model under the Apache 2.0 license. Its official page describes text, audio, image and video inputs; only its text component is used here. Official Gemma 4 12B page

In this test

We use Gemma in Q6_K. Its size preserves more precision than Ministral Q5 while staying within the card's memory envelope. It is also the only contender to complete the advanced action successfully.

Quantization as a hardware adjustment variable

The three models do not use the same quantization: Q4_K_M for Qwen, Q5_K_M for Ministral and Q6_K for Gemma. These choices match configurations that are genuinely usable in My Wealth and their hardware targets.

This benchmark therefore compares product configurations. The results do not claim to isolate model architecture alone.

The My Wealth test protocol

My Wealth's complete test bench contains 354 questions and actions. We fixed 15 before taking any measurements:

Each configuration processed exactly the same cases, first under Vulkan and then CUDA. A model remained loaded for its fifteen tests. One warm-up was excluded, and the synthetic portfolio and conversation were reset for every case. We retained a single measured attempt; a model error remains a result.

What we measure

1. Functional success

All 90 results were reviewed manually. For a query, we assess the visible answer for accuracy and completeness against the synthetic data. A redundant tool call does not turn a correct answer into a failure. For a write, we verify the actual recorded effect and every requested field: merely claiming success is not enough.

2. Answer quality

We separately review visible answers for accuracy, clarity, faithfulness to the data and usefulness. This human review determines the published score.

3. Response time

Total time includes tool selection, planning, tool calls and the final answer. It represents the actual wait better than generation throughput alone.

4. Vulkan versus CUDA

For each model, both runs use the same GGUF, template, engine and parameters. Only the GPU backend changes.

Results — answer quality

Overall result

Quality winner

Gemma 4 12B Q6_K achieves the best result under both backends, with 13 successes out of 15. Ministral is second with 11/15. Qwen scores 9/15 under Vulkan and 8/15 under CUDA.

Results by category

Here is one concrete request from each category:

All three models fail both Write cases. The first asked them to add an institution; none completed the mutation. The second asked them to directly change a value derived from the gold price. The models should have explained clearly that this isolated write was unsupported, but their paths continued until generation or tool limits were reached.

A few answers that surprised us

Example 1 — an excellent advanced answer

For the question about the “Retraite sereine” goal, Ministral and Gemma retrieve both expected values. Ministral notably answers:

“Current progress: 29.67% […] Required monthly contribution: €858.88.”

The answer is concise, quantified and based on the correct query path.

Example 2 — a plausible but incorrect answer

Qwen says that liabilities finance 13.70% of assets under Vulkan and 15.94% under CUDA, whereas the expected value is 17.81%. The calculations appear credible, but the model omits some liabilities or chooses the wrong denominator.

This error illustrates why a convincing financial answer is not enough.

Example 3 — when tool use makes the difference

When asked to create the “Compte Travaux” bank account, Gemma applies every requested field exactly under both Vulkan and CUDA: institution, currency, reference, deposit, tracking, owner and note.

Ministral creates an account, but omits the institution and note from the actual recorded effect even though its answer claims otherwise. Qwen applies no mutation. This example shows why write operations must be reviewed against recorded state, not only the final text.

Results — real speed in My Wealth

Response times

Speed winner

Qwen 3.5 9B under CUDA is fastest in both mean and median. Gemma CUDA needs context: its median is slightly better than under Vulkan, but one case generated 8,192 tokens during planning and lasted 4 minutes 33 seconds. This incident raises its mean to 43.1 seconds.

Vulkan vs CUDA

Generation: tokens per second

Throughput is aggregated by dividing total generated tokens by the total generation time measured by llama.cpp. It is not a simple average of each phase's throughput.

CUDA wins on raw throughput for all three models. This does not guarantee a shorter total time: model decisions and tool calls often matter more than a few tokens per second.

Mean response time

CUDA reduces Qwen's mean time by about 7% and Ministral's by about 9%. For Gemma, the planning incident increases the mean by 56%, while the median falls by around 5%. The median better describes typical behaviour here; the maximum reminds us that a local assistant can occasionally enter a very long generation.

The winner is not necessarily the fastest model

Qwen produces almost twice as many tokens per second as the other two models, but its human score is much lower. Ministral delivers a major quality improvement without making the wait excessive. Gemma achieves the highest score and completes the only advanced write under both backends, at the cost of lower raw speed and one long CUDA incident.

We do not calculate an arbitrary composite score. Different uses do not value a saved second, an accurate answer or a correctly executed action equally.

Which model should you choose?

To prioritize speed

Qwen 3.5 9B Q4_K_M under CUDA. It is fast and needs much less memory, but its 8/15 score calls for careful verification whenever a request goes beyond a simple query.

For the best balance

Ministral 3 14B Q5_K_M under CUDA. Its score stays at 11/15 under both backends, and CUDA brings its median down to 18.9 seconds. It is a solid compromise for queries and analysis.

To prioritize quality above all

Gemma 4 12B Q6_K under Vulkan. With 13/15 and a successful advanced action, it wins this comparison. Gemma receives the same score under CUDA, but Vulkan avoids the planning incident that sharply increases the CUDA mean.

Conclusion

On this 16 GB RTX 5060 Ti, no model dominates every criterion.

Qwen is the speed champion. Ministral offers the most consistent balance between waiting time and quality. Gemma under Vulkan provides the best functional results and is the most convincing on complex paths.

The choice therefore depends on the need: fast answers for simple queries, balance for an everyday assistant, or maximum priority on quality and actions. In our comparison, Gemma 4 12B Q6_K wins on quality with both backends; we prefer Vulkan for its consistency. Qwen 3.5 9B Q4_K_M under CUDA wins on speed.