The poem's opening line sets things firmly in the past tense: the speaker reflecting on the painful separation from his former lover, and their relationship is definitely over. The alliterating /w/ sounds ("When we") in this phrase—as a kind of twosome—subtly hints at the former union between these two people.
By referring to "we," the speaker also introduces the fact that the entire poem is written using the literary device apostrophe. That is, it's addressed directly to his ex-lover, but she doesn't participate in the conversation; instead, it's as if the reader is eavesdropping on a private speech from the speaker to one very specific individual. The repeated use of words like "thy" and "thee" continues to emphasize this throughout the poem.
Then, the speaker explains that the lovers "parted / in silence and tears." The mention of "tears" is clear enough—the lovers were sad when they went their separate ways. The structure of the poem's lines subtly reinforces this idea throughout. Though the lines are short, the phrases tend to run over two lines through the use of enjambment. This means that most phrases are "broken" into two, mirroring the lovers' split. The opening lines follow this pattern, with one distinct phrase running across lines 1 and 2 and another covering lines 3 and 4.
The "silence," though, is more mysterious, and requires the rest of the poem for context. It hints at the secrecy surrounding the lovers' relationship. Most scholars agree that Byron wrote the poem about an affair he had with a married aristocrat, Lady Frances Wedderburn-Webster. Because the poem uses "I," "you," and "we," throughout, it feels as though the reader isn't really invited to witness to this exchange, acting instead as a kind of voyeur eavesdropping on an intimate and likely illicit conversation.
Lines 3 and 4 offer more information about the break-up. The speaker and his lover "sever[ed] for years," with the verb "sever" indicating the almost physical pain of the split. The alliteration of "half broken-hearted" creates another pair of sounds, this time at a greater distance from one another than the alliteration in line 1. This subtle shift reinforces the idea that the lovers are now separated where they were once close.