Alliteration is one of the most common poetic devices. Those repeating initial sounds don't turn up that often in day-to-day speech, and therefore often makes a poem feel, well, poetic: heightened and artful. But in "Remember," this effect is fairly low-key, gently emphasizing linked ideas.
For instance, take a look at the alliteration on /h/ sounds in lines 3-4:
When you can no more hold me by the hand,
Nor I half turn to go yet turning stay.
That /h/ is a gentle sound, and it subtly draws together images of touch and connection: holding hands, half-leaving but then turning back to embrace one's beloved again. The soft, repeated sounds suggest the tender affection between the speaker and her beloved.
There's similarly subtle alliteration on /f/ sounds in lines 9 and 13:
Yet if you should forget me for a while
[...]
Better by far you should forget and smile
This is an even quieter effect than before: the /f/ sounds of the words "for" and "far" are especially discreet, since those words are pretty unobtrusive ones. But the connection between those words makes a difference in meaning: connecting "forget" and "for a while" makes it clear that the speaker isn't imagining being forgotten forever, and connecting "far" and "forget" emphasizes the speaker's insistence that, really, her beloved's happiness is more important than his constant attention to her memory.