Translate runs NumCPU/2 sentence workers and each of them started
GOMAXPROCS goroutines for the logit projection of every step: on 16
cores, 128 goroutines fighting for 16 cores. The model now caps the
goroutines of a step (SetThreads, called once after Load) so that
workers x threads stays within the machine.
Each beam-search step deep-cloned the keys and values of every layer
for every candidate it kept, so the cost grew quadratically with the
sentence length and produced garbage faster than the unrolled dot()
could win. Survivors are now chosen first; a beam whose parent has a
single survivor takes its slices over (the parent is dead), and only a
shared cache is cloned. The step's other buffers (query/key/value,
hidden, logits) are reused across layers, beams and steps instead of
being allocated afresh each time. TestRealModel confirms the output is
unchanged.
Kharec