The poem is filled with parallelism: the speaker returns to the same grammatical structure again and again. This makes the poem's argument easy to follow and adds some emphatic rhythm to its language.
Each of the poem's four couplets begins with a grammatically identical line. There's a noun ("Music," "Odours," "Rose leaves," and "thy thoughts"), followed by a comma (creating a caesura); the second part of the phrase then begins with the word "when," followed by a reference to death. Take lines 1-2:
Music, when soft voices die,
[...]
Odours, when sweet violets sicken,
The odd-numbered lines, meanwhile, each respond to this image of something pleasant dying, insisting that these things continue to exist in the minds of those who survive them. This lends the poem a kind of AB AB AB AB logical structure. Through this steady parallelism, the speaker makes it clear that these images are all meant to illustrate the same point: that love, beauty, and so on live on in people's memories.
Line 7 breaks up the rhythm a little by adding "And so" at the beginning of the line, before jumping right back into the expected pattern:
And so thy thoughts, when thou art gone,
This slight shift in the rhythm adds emphasis to the speaker's final point about love enduring beyond death. That "And so" indicates that the everlasting nature of love was the speaker's point all along, the natural conclusion to all of the poem's previous points.