Abstract

This thesis presents a systematic evaluation of the robustness of modern Large Language Models
(LLMs) against prompt injection attacks. Prompt injection is formalized as a violation of instruc
tion hierarchy, wherein untrusted user-provided input overrides, modi es, or circumvents trusted
system-level instructions. Rather than relying on informal demonstrations of adversarial prompting,
this work establishes a reproducible benchmarking framework for measuring instruction-separation
failures in a controlled and quanti able fashion. The study builds upon a recent benchmarking
methodology and extends it to newer generations of open-weight LLMs, speci cally LLaMA (Meta
AI) and unexplored Qwen (Alibaba), which currently rival proprietary systems such as GPT models
in reasoning, coding, and multilingual performance. The thesis focuses on simple yet representative
attack classes, including Naive Prompt Injection, Fake Completion attacks, and Escape Character
injection. This deliberate methodological choice enables standardized, interpretable, and repro
ducible robustness evaluation across models and tasks. By demonstrating that even trivial adver
sarial instructions can systematically override trusted prompts, the thesis highlights structural vul
nerabilities in instruction-following architectures rather than weaknesses tied to highly specialized
exploits. Evaluation is conducted across ve task families representing common LLM deployment
settings: (i) Constrained news summarization with strict output-length requirements; (ii) Binary
sentiment classi cation; (iii) Paraphrase identi cation; (iv) SMS spam detection, and; (v) Hate and
offensive content detection. These tasks span both generative settings and structured-output classi
cation scenarios. Different datasets and attack variants are considered to assess consistency and
cross-task robustness patterns. The results reveal that both LLaMA and Qwen exhibit measurable
iii
susceptibility to instruction-hierarchy violations across tasks, even under minimal adversarial per
turbations. These ndings suggest that prompt injection remains a fundamental reliability challenge
in contemporary instruction-tuned LLMs.