The poem opens with the speaker's instant separation of Richard Cory from the speaker's community. Cory only goes down town as a visitor, but the speaker's community seems to be part of the down town landscape; in other words, they live there, and cannot come and go as they please.
The speaker is also immediately identified with a community through the first person plural pronoun "we." Unlike the distinguished, attention-attracting Cory, who has the first line of the poem all to himself, no one in the speaker's community, the speaker included, will be given an individual identity in any way throughout the poem. Instead, the community appears as a collective—the "people on the pavement," a phrase which indicates that they are probably working-class individuals, and certainly not riding through the streets. To that end, note that "down town" is usually associated with the part of a city where business happens—and as such is usually more hectic, dirtier, and noisier than "uptown" areas (where Cory might be coming from). From the very start of the poem, even before it is established what makes Cory the object of their attention, the community's contrast with Cory becomes clear.
The use of "whenever" also paints a scene that is routine: Cory repeatedly goes down town and the community watches him each time. This hint of repetitive, perpetual behavior will evolve throughout the poem.
This first pair of lines establishes the iambic pentameter that will continue without variation for the entirety of the poem:
Whenev- | er Rich- | ard Cor- | y went | down town,
This consistent, familiar meter will prove significant in establishing in a sense of security and predictability, a comfort that will be shattered by the poem's final line.
There is also a strong pattern of alliteration and consonance in the first two lines. Richard Cory's entrance features a series of softer /w/ and /n/ sounds: whenever, went, down, town). The community's description seems tougher by contrast, with its hard, percussive /p/ sounds (people, pavement).