Soft prompting moves adaptation out of the architecture and into token space. Instead of only writing a hand-crafted text prompt, we learn virtual prompt tokens whose embeddings are trained — while the backbone model can stay frozen.
| Approach | Where adaptation happens | Plain-English idea |
|---|---|---|
| Adapters | Architecture space | Add small modules into the network |
| Soft prompts | Token space | Add and learn task-specific virtual tokens / context |
The big idea: good context can steer the language model without changing its weights.
Every word you type is converted into a vector before the model sees it. A soft prompt skips the word stage and learns the vectors directly:
Discrete: "Summarise formally:" -> tokenizer -> [15, 892, 41] -> embeddings
Soft: (no text at all) -> 20 learned vectorsThose 20 vectors are prepended to your real input. They occupy prompt positions like ordinary tokens, but they do not spell anything — if you tried to decode them back into words, you would get nonsense. They exist purely because training found them effective.
The trainable footprint is tiny:
20 virtual tokens x 4096 dimensions ≈ 82,000 parametersCompare that with millions for adapters, or billions for full fine-tuning.
| Kind | Meaning | Catch |
|---|---|---|
| Discrete prompt | A normal string of real tokens written by a person | Small wording changes can swing quality a lot |
| Continuous / soft prompt | A sequence of trainable virtual token embeddings | Not real vocabulary words — learned vectors that act like extra context |
Why continuous prefix-style tuning exists: manual prompts are brittle. Learning a soft prompt can be more stable and compact than rewriting English by hand.
Soft prompts can be sensitive to:
Longer soft prompts also mean more tokens to process — so cost can grow with prompt length even when the backbone is frozen.
Soft prompting steers a frozen model by learning virtual prompt tokens in embedding space, instead of rewriting the network.