Alliteration is used throughout "A Supermarket in California." It begins in the first sentence, linking Walt Whitman's name—which is in itself alliterative—with "walked." As Whitman is functioning as a kind of spiritual guide for the speaker here, it follows that he should be linked with the act of walking—the wandering that creates space for the speaker's reflections.
The poem returns to this idea in the third and final stanza, linking Whitman's name with "where," "we," "which way," and "will we walk" (from line 8 to line 10). This makes the link even stronger between Whitman and moral authority, summed up in the speaker's asking for the "way" to go. It's worth noting that when Whitman is in the supermarket—an environment that seems to baffle him—his name doesn't alliterate with any other words, conveying how alien this world seems to him (in the speaker's imagination).
The third part of the first stanza also uses alliteration. Firstly, it evokes the abundance of choice to be found in the supermarket:
What peaches and what penumbras!
This almost sounds like an advertising slogan, or the kind of thing that might be put on the side of a product to boost its sales. "Penumbras," however, are not something you can buy in a supermarket—they are shadows. So the alliteration is also linking the bright "neon fruit" with a kind of moral and intellectual darkness too. The alliteration of "what were ... watermelons" and "doing down" in the Lorca section of the first stanza adds a touch of absurdity to the speaker's vision.
In the last sentence of the second stanza (line 7 beginning "We strode ..."), the speaker walks with Whitman and samples the various culinary delights of the supermarket:
We strode down the open corridors together in our solitary fancy tasting artichokes, possessing every frozen delicacy, and never passing the cashier.
This alliteration plays with the same idea established by the alliteration go "peaches" and "penumbras" earlier. It suggests the abundance and variety on display in the supermarket, while also conveying the playful naughtiness of two men enjoying produce without any intention of paying for it.
The other key instance of alliteration is in the middle of the third stanza (line 11 starting "will we stroll ..."):
Will we stroll dreaming of the lost America of love past blue automobiles in driveways, home to our silent cottage?
This links loss with love, making it clear what's lacking from 20th-century America according to the speaker. The poem doesn't explore what kind of love society would have to value in order to find this ideal America, but it's certainly not the mundane conformism of the hoards of shopping families.