"Old Man" uses alliteration for both musical and dramatic effect.
Since he was six, the speaker has seen a "picture painted" (line 2) before his eyes (though really it's in his mind). It's an inviting, rainbow-like vision, though it remains always out of reach—even now, when he's sixty. That /p/ alliteration has a decorative effect, perhaps suggesting the beauty of this strange yet alluring image. This /p/ is also a punchy, plosive sound, hinting at the picture's impact on the speaker's entire life.
The speaker then describes how, as a kid, he "strove" to "reach" that picture. He advanced towards it with "slow step some few short yards" (line 6). These sibilant /s/ sounds "slow" the poem's pace, mirroring the speaker's movement as the picture remains tantalizingly out of reach. As the following line dials up the sibilance ("perceptibly the distance lessening"), the hushed, whispery tones create an atmosphere of mystery and uncertainty—as if the picture is a kind of ghost that haunts the speaker. This sibilant sound returns in line 13 as well ("so steady").
In lines 10 and 11, the poem deploys more plosive sounds:
With all its beauteous colors painted bright,
I'm backward from it further borne each day [...]
As with "picture painted" in line 2, these are bold, compelling sounds that capture the magnetic pull of that far-off picture. But line 11 modifies this effect by using the same sound to describe how the picture remains forever out of reach. The speaker feels he is "backward [...] borne," receding further away from what he yearns for. Here, the plosives convey the strength of the "compulsive force" that imposes itself on the speaker's life—whether at age six or sixty!