Skip to content

Text generation and chat

Prompt and system instruction

spum run model.gguf --gpu \
  --system "Explain concepts clearly and state uncertainty." \
  -p "Explain the difference between resolution and magnification." \
  -n 192 --temperature 0.4

Prompt files

spum run --model model.gguf --gpu \
  --prompt-file question.txt \
  --system-file instructions.txt \
  --max-tokens 512

Prompt files contain text instructions; this option does not parse PDFs, slide decks or spreadsheets. Do not combine --prompt-file with --prompt.

Save plain text

spum run model.gguf --gpu -p "Summarise the supplied observation." \
  --output answer.txt

The output file is replaced if it exists. Without --output, generated text goes to stdout and run timing messages go to stderr:

spum run model.gguf --gpu -p "What is depth of field?" \
  > answer.txt 2> timings.txt

Export JSON

spum run model.gguf --gpu --prompt-file question.txt \
  --temperature 0 --seed 42 --json --output result.json

The JSON object contains model, response, prompt_tokens, completion_tokens, prefill_tokens_per_second, decode_tokens_per_second and finish_reason. --json controls the CLI envelope. It does not constrain the model response to a user-defined JSON schema.

Quiet runs

spum run model.gguf --gpu -p "Write one sentence about microscopy." --quiet

This suppresses run summaries. Errors are still reported. JSON and file output are buffered until the generation completes; ordinary terminal text streams as it is generated.

Interactive session

spum chat model.gguf --gpu --system-file instructions.txt -c 4096

The conversation lives in the current process. /reset starts fresh. /quit exits. Optional --history on concerns the engine's local output history and is distinct from exporting a readable conversation.