1B, 2B, 4B, or 27B: Choosing a Transliteration Model Size

“How many parameters does Convert need?” sounds like a hardware question. It is really a product question: what text must the model change, what must it preserve, and on which runtime must the result arrive fast enough to use while typing?

Parameter count is one input to that decision. It is not the score.

1B is the smallest serious candidate, not an automatic winner

A model around 1B parameters can make packaging and memory easier. The risk is capacity: the same model must resolve short Pinyin, choose Japanese scripts, handle Hinglish code-switching, interpret Arabizi, and preserve unrelated Latin spans.

That does not prove a 1B model will fail. It means a 1B candidate must pass the same per-language and protected-span tests before its smaller file becomes useful.

2B is attractive when the task is narrower

A 2B model gives more room than 1B while remaining meaningfully smaller than 4B. It may be a good later distillation target if a larger trained model can supply examples and the 2B result retains the same Convert contract.

We should not infer that result from parameter count. The observable variable is exact accepted output on held-out Chinese, Japanese, Hindi, and Arabic cases, followed by runtime memory and latency on the target device.

4B is our first working balance

UnimeType chose a 4B base for the first public model because it was small enough to quantize into a desktop-local artifact and large enough to show strong gains after task-specific tuning.

On our 127-case frozen suite, the unmodified Qwen3.5-4B baseline produced 27 exact accepted outputs. The tuned 6-bit artifact produced 94 through LM Studio. Those numbers compare one base and one tuned model under the same project test, not every 4B model and not general language accuracy.

27B is a teacher or research candidate, not the default Convert target

A 27B model may provide more general capability, but it also increases model storage and runtime memory. Convert does not need most open-ended chat capabilities. A larger model is useful only if its additional capacity changes the failure cases enough to justify the device cost.

For this product, 27B is better treated as a comparison or labeling candidate than as the first local shipping target. The failure condition for that recommendation is simple: if a smaller model cannot meet the accepted-output and preservation gates after tuning, we revisit the size.

The decision needs three measurements

For every size, we need:

1. the same Convert prompt and decoder settings;
2. the same held-out language and edge-case suites;
3. storage, peak memory, and latency from the intended runtime and device.

Only then can “smaller” or “larger” become a product decision. The next article compares a different axis: Qwen versus Gemma and other small model families. Our evaluation method is explained in How UnimeType evaluates transliteration models, and the action boundary is documented in Actions.