Semantic Residual Prompts for Continual Learning#
October 2024
Continual Learning Prompt Selection Rehearsal-free Pretrain-based
We study the impact of forgetting on the prompt-selection mechanism adopted in most pretrain-based models (e.g., L2P, CODA-Prompt, DualPrompt) and we introduce STAR-Prompt a two-level prompt selection strategy where an initial Vision-Language model (CLIP) is used to select the most relevant second-level prompts. The second-level prompts are then used to guide the learning of the target task using a semantic residual approach.