fix: give each translation worker its share of the CPUs
Translate runs NumCPU/2 sentence workers and each of them started GOMAXPROCS goroutines for the logit projection of every step: on 16 cores, 128 goroutines fighting for 16 cores. The model now caps the goroutines of a step (SetThreads, called once after Load) so that workers x threads stays within the machine.
This commit is contained in:
1 parent
d6ee57130f
commit
82bfef8cba
4 files changed
+26
-7
No files matched your search
+3
-1
@@ -23,7 +23,9 @@ version is in the `VERSION` file.
|
||||
replacing the link.
|
||||
- Translation is faster and uses much less memory: the beam search no longer copies
|
||||
every beam's decoder state for every candidate at every step, and reuses its
|
||||
work buffers instead of allocating new ones per layer and step.
|
||||
work buffers instead of allocating new ones per layer and step. Each sentence
|
||||
now uses its share of the processors, so translating a script no longer starts
|
||||
more workers than the machine has.
|
||||
|
||||
## [1.4] - 2026-10-15
|
||||
|
||||
|
||||
Reference in new issue
Block a user