Smarter Soft Prompts

A single soft prompt for every input and every layer can be too blunt. These methods make soft prompts smarter and more selective.

Intuition

Different layers do different jobs. Different inputs need different hints. So we should not always:

• Use one prompt for everything
• Give every layer the same prefix length
• Ignore the specific input instance
• Force prompts into every layer

How it works

SMoP — Sparse Mixture of Prompts

Plain-English idea: keep several prompt candidates, and activate only the useful ones for a given input (sparse mixture), instead of forcing one mega-prompt to do all the work.

• Good when the prompt budget is limited
• Uses a routing idea so not every prompt component fires every time

APT — Adaptive Prefix Tuning

Plain-English idea: do not treat all layers equally. Change prefix length (prompt capacity) across layers.

Why that matters:

• Bottom layers tend to capture shallow, phrase-level features → may need longer prefixes
• Higher layers deal more with abstract meaning → often need shorter prefixes on average

APT can also use gating so a layer only “keeps” the prefix tokens it needs.

IDPG — Instance-Dependent Prompt Generation

Plain-English idea: the right soft prompt should depend on this input, not only on the task name.

A small prompt-generator network builds prompt vectors from the input. Useful when one task has many subcases and a static prompt is too generic.

Why a static prompt struggles here — all three of these are "customer support," yet they need different handling:

"My payment failed twice."            -> needs billing care and caution
"How do I change my profile photo?"   -> needs short how-to steps
"This is the fourth time I've asked!" -> needs de-escalation first

One fixed soft prompt must average across all three and serves none of them especially well. IDPG generates a prompt shaped by the message itself, so an angry message and a simple how-to question are not briefed identically.

SPT — Selective Prompt Tuning

Plain-English idea: ask whether every layer needs soft prompting at all. Usually no. Learn to insert prompts only where they help.

What goes wrong

• Jumping to SMoP/APT/IDPG before a simple prompt or adapter baseline is working.
• Assuming “more prompt tokens everywhere” is always better.
• Using one static prompt when inputs vary wildly (IDPG’s motivation).

One-line summary

SMoP, APT, IDPG, and SPT make soft prompts selective — by mixture, by layer, by input instance, or by choosing where prompts are even needed.

Key terms

• SMoP — Sparse mixture of prompts; activate only useful prompts.
• APT — Adaptive prefix tuning; vary prompt length by layer.
• IDPG — Instance-dependent prompt generation from the input.
• SPT — Selective prompt tuning; insert prompts only where needed.