Frequent anaphora makes the language of "Renouncement" sound more emphatic and intense. In the opening lines, for instance, the repetition of the word "I" helps to convey the speaker's passionate conviction:
I must not think of thee; and tired yet strong,
I shun the thought that lurks in all delight—
She declares that she "must not think" of her beloved, and then doubles down on this statement in the very next line: she "shun[s]," or actively, vehemently rejects, "the thought" of this person. This is easier said than done, of course, given that everything "delight[ful] seems to remind the speaker of her beloved—something the anaphora of "and in the" in lines 3-4 makes clear:
[...] and in the blue heaven's height,
And in the sweetest passage of a song.
The thought of this person "lurks" both "in the blue heaven's height [...] And in the sweetest passage of a song." Anaphora (and polysyndeton with that repetition of the word "and") creates the sense that the speaker could go on and on; there are likely plenty more places where thoughts of her beloved hide.
Anaphora also creates a sense of forward momentum here and throughout the poem. Listen to the repetition of "When" in lines 9-10, which calls attention to the shift that happens as day turns to night and the speaker can no longer control her desire:
But when sleep comes to close each difficult day,
When night gives pause to the long watch I keep,
Anaphora drags the transition from night to day out, building anticipation. That anticipation then gets released with the exuberant anaphora in the poem's final line: "I run, I rum, I am gathered to thy heart."