feat: resumable download and cancellable load of the translation model

Downloading the 300 MB model restarted from zero after any failure and
hung forever on a stalled connection. Download into a .partial file
that survives interruptions, resume it with a Range request (falling
back to a fresh download when the server cannot), skip files already
verified, and abort when nothing arrives for 30 s. A model directory
only counts as installed when every file has its pinned size, so a
truncated model is offered for download again instead of failing to
load.

The model load held the translation mutex, so Cancel did nothing for
the seconds it takes to read 300 MB. Load without holding the lock,
make marian.Load context-aware, and let CancelTranslation wait for the
task to wind down so a new one can start at once. The idle release
timer re-arms when it fires while a download holds the task slot,
instead of never freeing the model.
This commit is contained in:
Kharec committed 2026-10-13 18:17:53 +02:00
1 parent 8181c6e885
commit 839e9291d6
5 files changed
+313 -69

No files matched your search

+5 -1
View File
@@ -1,6 +1,7 @@
package marian
import (
"context"
"encoding/binary"
"encoding/json"
"fmt"
@@ -27,7 +28,7 @@ type tensorInfo struct {
// little-endian header length, a JSON header, then the raw data) as
// float32. F32 tensors are read straight into their slice: the file is
// large and a second copy would double the memory needed to load it.
func readSafetensors(path string) (map[string]tensor, error) {
func readSafetensors(ctx context.Context, path string) (map[string]tensor, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
@@ -54,6 +55,9 @@ func readSafetensors(path string) (map[string]tensor, error) {
if name == "__metadata__" {
continue
}
if err := ctx.Err(); err != nil {
return nil, err
}
var info tensorInfo
if err := json.Unmarshal(raw, &info); err != nil {
return nil, fmt.Errorf("tensor %s: %w", name, err)