These first two lines establish the setting of the poem, describing an extraordinarily deep part of the ocean. The reference to the "upper deep" in line 1 is especially important because it suggests that there are varying degrees of depth in the ocean itself. In other words, it's not enough to say that the ocean is simply "deep." As such, the speaker makes a distinction between the "upper deep" and areas that are even farther down. To that end, the "upper deep" is most likely what comes to mind for most people when they think about the immediate depths of the ocean, which is the oceanic layer in which the most familiar marine animals live.
"The Kraken," however, takes place below these somewhat familiar depths, plunging deep into the "abysmal sea." The word "abysmal" is particularly noteworthy, since it can either mean terrible and appalling or extraordinarily deep. Indeed, the word derives from the "abyss," which refers to endless, void-like chasms. Because "abysmal" can be interpreted both ways, then, the word not only helps the speaker describe just how deep these waters are, but also allows the speaker to associate these depths with an ominous, foreboding quality. In this way, readers feel as though they are being drawn down into an unsettling and remote place, sinking past the familiar and into the unknown.
There is also a muscular sound to these opening lines, which are subtly consonant. For instance, the /b/, /p/, and /r/ sounds work their way throughout both lines:
Below the thunders of the upper deep,
Far, far beneath in the abysmal sea
None of these instances of consonance jump out at readers right away, but their presence in these lines is prominently felt, giving the entire opening a strong sound that aligns with the imagery of the thunderous ocean waters. In this sense, the consonance emboldens the beginning of the poem, giving it a power reminiscent of the low rushing sound that would accompany the act of plunging into the "abysmal sea."