The poem begins by specifying its setting. O'Hara seems to have written "The Day Lady Died" on the actual day of Billie Holiday's death, and he establishes that sense of immediacy right up front.
The speaker—O'Hara himself, as later details make clear—narrates events in the present tense, naming the day of the week ("Friday"), the time of day ("12:20" p.m.), the year ("1959"), and his current location ("New York" City). He also specifies that it's "three days after Bastille day"—meaning France's national holiday, which is celebrated annually on July 14. So the poem takes place on the afternoon of July 17, 1959, in the city where Holiday, the "Lady" of the title (her nickname was "Lady Day"), had died of cirrhosis hours earlier.
Right away, the narration style is fast-paced, a rapid blur of detail with very few pauses. Line 2 contains one of only a handful of caesuras (the comma before "yes) in a poem of 29 lines. For the most part, O'Hara omits punctuation where it would normally go: for example, after "New York" and Friday" in line 1 ("It is 12:20 in New York a Friday"). The poem also uses heavily enjambed free verse, with no meter, rhyme scheme, or regular stanza pattern.
These techniques make the poem feel dashed-off and spontaneous, as if O'Hara is jotting down his impressions and experiences as fast as possible. (He once claimed that poetry sounds best if "You just go on your nerve.") The resulting language feels fresh, candid, and also a little anxious.
As the title hints, he's recording his impressions on this particular day because the death of Billie Holiday feels like a historic event. Interestingly, he starts by invoking a famous event from French history: the storming of the Bastille (a political prison) during the French Revolution. This reference suggests that "The Day" of Holiday's death is also, in its way, a major occasion, which he's rushing to document and commemorate.
It might also subtly relate to Holiday's career. She was considered a revolutionary artist; her art was sometimes political; she had trouble with the law (in part due to her art/politics), spent time in prison, etc. The same is true of other artists the poem mentions later, including Jean "Genet" (line 18).
In all these ways, the seemingly random "Bastille day" detail may be a meaningful allusion. If nothing else, it shows that O'Hara is keenly aware of international culture—a trait he'll continue to demonstrate throughout the poem.