"The Collar" is built around the conceit of, well, the "collar": the clerical "dog collar" the speaker wears becomes a metaphor for the inescapable clutches of sincere religious faith.
If the speaker wears a clerical collar, he's a Christian priest: a guy who has devoted his whole life to faithful service. As the poem begins, he's fed up with all the sacrifices this career has demanded of him. Rather than living the life of a famous poet crowned with the "bays" (or laurel wreath) of artistic glory, the speaker feels he's stuck "in suit," a mere servant. Worse, he has to spend most of his time "sigh[ing] and pin[ing]" over his sins, dithering over "what is fit and not," obsessing about moral dilemmas.
In this sense, then, his clerical collar is a metaphorical restraint, a thing that holds him back from leading a free-and-easy life of hedonistic pleasure. Like a dog on a leash, he feels prevented from jumping impulsively after the squirrels of possibility.
But even as he rants and raves, threatening to abandon not just his priesthood but his religious faith itself, a single word from God stops him in his tracks: all God has to say is "Child!" for the speaker to answer, almost reflexively, "My Lord." It isn't so easy to slip the "collar" of faith, the poem suggests—and that collar's restraints might be meant more to keep believers from running into spiritual traffic than to deny them all the squirrels they can eat.
That's true not just for priests, the poem hints, but for all Christian believers. Perhaps, then, this collar is also meant to evoke the light, easy "yoke" in a famous biblical parable. Faith might feel heavy sometimes, the speaker suggests, but it's a lighter burden than sin.