fix: give each translation worker its share of the CPUs

Translate runs NumCPU/2 sentence workers and each of them started
GOMAXPROCS goroutines for the logit projection of every step: on 16
cores, 128 goroutines fighting for 16 cores. The model now caps the
goroutines of a step (SetThreads, called once after Load) so that
workers x threads stays within the machine.
This commit is contained in:
Kharec committed 2026-10-15 17:53:38 +02:00
1 parent d6ee57130f
commit 82bfef8cba
4 files changed
+26 -7

No files matched your search

+3 -1
View File
@@ -23,7 +23,9 @@ version is in the `VERSION` file.
replacing the link.
- Translation is faster and uses much less memory: the beam search no longer copies
every beam's decoder state for every candidate at every step, and reuses its
work buffers instead of allocating new ones per layer and step.
work buffers instead of allocating new ones per layer and step. Each sentence
now uses its share of the processors, so translating a script no longer starts
more workers than the machine has.
## [1.4] - 2026-10-15