arXiv Open Access 2026

To Write or to Automate Linguistic Prompts, That Is the Question

Marina Sánchez-Torrón Daria Akselrod Jason Rauchwerk
Lihat Sumber

Abstrak

LLM performance is highly sensitive to prompt design, yet whether automatic prompt optimization can replace expert prompt engineering in linguistic tasks remains unexplored. We present the first systematic comparison of hand-crafted zero-shot expert prompts, base DSPy signatures, and GEPA-optimized DSPy signatures across translation, terminology insertion, and language quality assessment, evaluating five model configurations. Results are task-dependent. In terminology insertion, optimized and manual prompts produce mostly statistically indistinguishable quality. In translation, each approach wins on different models. In LQA, expert prompts achieve stronger error detection while optimization improves characterization. Across all tasks, GEPA elevates minimal DSPy signatures, and the majority of expert-optimized comparisons show no statistically significant difference. We note that the comparison is asymmetric: GEPA optimization searches programmatically over gold-standard splits, whereas expert prompts require in principle no labeled data, relying instead on domain expertise and iterative refinement.

Topik & Kata Kunci

Penulis (3)

M

Marina Sánchez-Torrón

D

Daria Akselrod

J

Jason Rauchwerk

Format Sitasi

Sánchez-Torrón, M., Akselrod, D., Rauchwerk, J. (2026). To Write or to Automate Linguistic Prompts, That Is the Question. https://arxiv.org/abs/2603.25169

Akses Cepat

Lihat di Sumber
Informasi Jurnal
Tahun Terbit
2026
Bahasa
en
Sumber Database
arXiv
Akses
Open Access ✓