ModelLoadOptions
ModelLoadOptions =
Pick<ChatRequest,"num_ctx"|"num_gpu"|"num_batch"|"num_thread"|"keep_alive">
Defined in: src/lib/providers/types.ts:175
The request fields that decide how a server loads a model, as opposed to how it samples. A warmup must send the same values the chat request will, or the server loads the model once for the warmup and again for the first chat (#315).