Right after Bram reveals himself as Blue at the carnival, Simon feels horrible for assuming that Blue would be white. This first reinforces that Simon's lack of curiosity about his friends and peers meant that he tackled the issue of discovering Blue's identity with pretty minimal evidence—remember that he suspected Cal at first simply because Cal has blue eyes. Because Simon didn't know much about Bram, even though Bram has eaten lunch at Simon's lunch table every day of school since September, he was unable to piece together that Bram might be Blue, regardless of Bram's skin color.
When Simon…