Quantifying Positional Encoding Impact on Keyword Attention in Language Models for Short, Temporally-Structured Trauma and Surgical Vignettes

Temporal structure is central to clinical decision making, particularly in trauma and surgical contexts where the sequence and timing of events guide diagnostic and operative priorities. Although large language models increasingly support clinical applications, the influence of positional encoding mechanisms on their internal attention behaviour in temporally structured medical text remains poorly characterized. This work presents a controlled mechanistic comparison of ALiBi and RoPE positional strategies using short trauma and surgical vignettes with explicit temporal cues. Using the Llama 3.2 3B Instruct model, we extract native attention weights and apply post-hoc positional transformations to isolate the operator-level effects of each mechanism across all layers and heads. Across a comprehensive suite of interpretability metrics including entropy, top-k concentration, mean attention distance, and keyword-specific attention, we find that RoPE produces high global focus, low-entropy, and strongly patterned attention with extended receptive fields, but simultaneously exhibits reduced sensitivity to clinically salient tokens. ALiBi, in contrast, remains closer to the baseline windowed attention mechanism, yielding more diffuse, higher-entropy attention, shorter mean attention distances, and the highest inter-head variability, while better maintaining content-driven focus on medically important tokens. These results demonstrate that positional encoding choices introduce distinct inductive biases that substantially reshape attention allocation in temporal clinical text, framing implications for model selection, interpretability, and architectural design in downstream medical reasoning tasks.