Alliteration is especially prominent in this poem because of the speaker's fondness for not only repeated sounds, but for repetition, period. For instance, in the second stanza, "sing" meets "sing" and "singing," and also matches up with "soul," "school," "studying," "sailed," and "seas." (See the Poetic Devices entry on "sibilance" for more on /s/ sounds specifically.)
Similarly, /g/ sounds appear repeatedly, but across only a few words: "God" and "Grecian" both appear once, and there are five instances of "gold" or a word that starts with "gold" ("goldsmiths" in line 27 and "golden" in line 30). On a broad level, this alliteration (and repetition) makes the poem feel musical and lyrical—that is, like a work of art. This, in turn, reflects the poem's thematic idea that immortality can be achieved through art; the speaker is creating a work of art with this very poem, and in doing so a part of him lives on.
Other times the alliteration serves to draw readers' attention to certain words and phrases. In line 5, for instance, the alliteration of the /f/ sound in "Fish, flesh, or fowl" connects these three words, placing them on the same level and underscoring that they are all subject to the same fate. As the speaker says in the next line, again heightening the phrase with alliteration, "Whatever is begotten, born, and dies." That is, all living things—be they fish, human beings, or birds—must die.