Next Steps & Future Directions
Actionable follow‑ups for rigorous evaluation and scaling
Next Steps
Benchmark rigor:
Evaluate capabilities with standard task benchmarks.
Baseline comparison:
Compare to a normally fine‑tuned model using the same data/compute.
*
Scale experiments:
Try this method on larger and more varied models.
More targeted weight updates:
Is simply adding or subtracting all weights a good way to update the model? Should we do something more targeted?
*
Even if overall results are the same, this implementation was much easier than standard fine‑tuning, so it's still a viable alternate direction.
Future Directions
Thinking vs non‑thinking:
Investigate why refusal rates differ between modes.
Refusal sharpness metric:
Measure how strongly a refusal is expressed, beyond binary.