Related Research
Task vectors and residual instruction porting relevant to vector injection and model editing
Model Editing
Editing Models with Task Arithmetic (Task Vectors)
Task vectors encode a directional delta in weight space computed as the difference between a fine‑tuned model and its pre‑trained baseline. Arithmetic over these vectors (addition, negation, analogies) steers model behavior toward or away from tasks, enabling capability injection or removal without full retraining.
- Vector construction: τ = θfinetuned − θpretrained
- Negation reduces performance on the target task with minimal collateral change
- Addition composes multiple capabilities; analogies enable zero‑data transfer
- Simple, efficient knobs for post‑hoc model editing across modalities
Reference: mlfoundations/task_vectors
Panels: (a) τ = θ_ft − θ_pre; (b) forgetting via negation (−τ); (c) learning via addition (τ_A + τ_B); (d) analogies τ_new = τ_C + (τ_B − τ_A). Source: task_vectors.png
Continual Pretraining
Balancing Continuous Pre‑Training and Instruction Fine‑Tuning
Shows that instruction‑following can be transferred via a residual weight delta between an instruction‑tuned model and its base. Rather than re‑running expensive instruction fine‑tuning after continuous pre‑training (CP), compute the instruction residual once and add it to the updated base.
- Residual (instruction delta): Δinstr = θinstr(old) − θbase(old)
- Port to new base: θinstr(new) ≈ θbase(new, CP) + Δinstr (no new instruction data)
- Finding: CP on instruction models → catastrophic forgetting; avoid
- Finding: CP base then apply residual preserves instruction ability and adds new knowledge
- Validated on LLaMA‑3/3.1 and Qwen‑2/2.5 across varied CP corpus sizes; compute‑efficient