There are a few moments of alliteration in the poem, some of which are due to the poem's use of repetition. Only a few initial sounds are repeated and repeated: /w/, /n/, /d/, and /m/.
The most persistent sound in the poem is the /w/ that appears in "wild," "were," and "with" in the first stanza, and "winds" and "with" in the second. This persistent /w/ might mimic the whooshing of storm winds. It might also link wildness, windiness, and with-ness (i.e., togetherness) in the reader's mind—three ideas that share an intimate and complex connection in the poem's meaning, too.
Later, the /d/ sounds in the repeated "Done" of lines 7 and 8 land with a thump, as if those devices are being thrown down. In line 11, the soft /m/ sounds of "Might" and "moor" fit in with the final stanza's mood of blissful softness: "Rowing in Eden" might well make a person go "mmmmm. (Take a look at the Devices entry on "consonance" for further discussion of how the poem's alliteration weaves a pattern of harder and softer sounds.)
Alliteration is also important to this poem in the places where it stops. Take a look at those few lines where no alliteration appears (4, 6, 9, 10, and 11). The reader may notice that these lines have one thing in common: they're all moments when the speaker is imagining scenes of greater peace, delight, and calm than what she's presently experiencing. The break in the density of alliteration also reflects an all-too-brief break in the speaker's turmoil.