Each of the poem's four couplets provides a different example of how things live on in people's thoughts even after the things themselves have ceased to exist. The poem's opening couplet focuses on music. Just because "voices" stop singing, the speaker begins, that doesn't mean the listener no longer hears them. An echo of those voices "Vibrates" in the listener's "memory."
The comma after "Music" creates a pause (a caesura), making "Music" stand out in the poem much like it stands out in listeners' memories. Consider how differently the poem would read if the poet had instead started with something like, "When soft voices die / Music vibrates in the memory." There would. be less emphasis on the "Music" being discussed.
Part of the poem's own "Music" comes from its steady meter: it's written mostly in trochaic tetrameter, meaning that lines each contain four trochees (metrical feet that follow a stressed-unstressed pattern, creating a DUM-da rhythm). Here are the first two lines scanned:
Music, | when soft | voices | die,
Vibrates | in the | memor- | y—
All these trochees create a propulsive, galloping rhythm. The meter contains variations, however, that keep readers on their toes. For example, readers might scan the second foot of line 1 as an iamb (an unstressed-stressed foot); some might even argue that this foot is a spondee (two stressed beats, "when soft"). Both lines are also catalectic, meaning they're missing their final expected syllables. As a result, they end with firm, stressed beats—perhaps subtly conveying the way the "Music" described lives on rather than fades away.
The poem also follows a straightforward couplet rhyme scheme. These rhyme pairs make the poem feel tightly knit, a subtle nod to the enduring connection between each of the elements described (and, maybe, to the speaker's own partnership with their beloved, something they'll get to later in the poem). The first rhyme (between "die" and "memory") is slant, which prevents the poem from sounding overly stiff or formal. Still, this neat rhyme scheme helps to emphasize the organization of the poem's argument.
In addition to meter and rhyme, these lines also use sibilance: there's the repetition of /s/ and /z/ sounds in "Music," "soft voices," and "Vibrates." This buzzing sibilance mimics those "soft voices" vibrating in listeners' ears.