Alliteration is one of the most common devices in poetry. Alliteration often helps a poem to sound elevated and poetic, different from and more musical than everyday speech. It can also subtly support a poem's themes, as it does here.
For a good example, take a look at the alliteration on /b/ sounds in lines 19-20:
Where Glory decks the hero's bier,
Or binds his brow.
That repeating /b/ is a strong, percussive sound, and it links significant words at an important moment. In this stanza, the speaker is turning away from rueful thoughts of love and moving toward the glory of war. Discouraging himself from brooding on the death of romance, he uses those forceful /b/s to think of both death and glory: a "bier" is a platform for a dead body to lie on, and Glory might "bind" a heroic "brow" with the laurel wreath of victory. The connected sounds help the reader to feel the import of these ideas. The thought of death and glory intertwined makes a big impression on the speaker—an impression that might strike him with a thump, like a blow.
Alliteration links important words in similar ways all through the poem; many examples are highlighted here. For example, the hard /g/ of "Glory" and "Greece" in line 22 reflects the fact that these two things are closely connected in the speaker's mind; to him, Greece represents glory itself.
At its end, the poem also uses sibilance to come to a rest: the softer /s/s of "seek out," "sought," and "Soldier" suggest the eventual quiet of death after the noisy glory of battle.