The narrator indicates that objectively, there was nothing scary about Grenouille, as he was never big, strong, exceptionally ugly, or seemed particularly smart. He didn't begin walking until age two and began to speak at age three. His first words were concrete nouns that were especially odorous.
Here, the narrator gets at the most important premise of the novel: that scent silently and secretly controls everything. The children can't place why Grenouille is scary, but the reader can infer that it's because he has no scent.
One day, Grenouille sits on a cord of wood and says the word "wood" for the first time. He already understood the idea of wood, but it had never truly captured his interest. But on this day, he closes his eyes and simply smells the wood for a half hour and gets lost in the smell. The experience is confusing and intense, and he mutters "wood" over and over again for the next few days.
We see here how engrossed Grenouille is in his olfactory world, and how this immersion influences how he interacts with the outside world. The word "wood" only presents itself as a thing worth learning or remembering once it enters his mind as something of interest scent-wise.
Grenouille learns to speak in this way, but never fully grasps the concepts of words that express objects that don't smell, particularly words that express abstract or moral ideas. Even as an adult he will use them incorrectly. The language he does possess at this point soon becomes inadequate to express his olfactory experiences. He is able to differentiate between milk from different cows, or smoke from different burning materials, and since language doesn't provide enough words to describe these differences, he soon begins to doubt if language makes sense at all.
Grenouille's belief in language only goes as far as it's useful to him. Remember that he's emotionally deficient and hateful of the world, so his unwillingness or inability to learn, understand, or use words that express higher moral concepts demonstrates his emotional state. Further, he doesn't even know for sure that language has a true point, since even the words for smelling objects aren't even enough to articulate the world he experiences.
By age six, Grenouille has grasped his entire surroundings by smell. He has tens of thousands of smells in his memory and can imagine how different scents combine. The narrator likens him to a musical wunderkind (child prodigy), but for scent. He grows more secretive and disappears for days to smell the meadows and vineyards. Punishments have little effect on him.
Grenouille is increasingly portrayed as a kind of genius, but also as extremely emotionally deficient. Punishments don't work because his world just doesn't include anyone aside from himself, and so there's no impetus for him to please anyone but himself.
Madame Gaillard begins to notice that Grenouille has potentially supernatural qualities, such as being able to detect worms in uncut vegetables and an apparent ability to see through walls. At one point, Madame Gaillard forgets where she hid her money, and he finds it within seconds. He also seems to be able to tell the future, as he can predict storms hours away. While we know that he could simply smell all these things, Madame Gaillard believes he has the “second sight” and therefore will bring misfortune and death. She decides he has to go, and coincidentally, the cloister of Saint-Merri ceases their payments when Grenouille is eight.
As the fantastical nature of Grenouille’s genius is emphasized, he is once again believed to be in league with the devil, and again because of his relationship to scent. By addressing the reader directly in this situation, the narrator encourages us to feel sympathy for Grenouille and to feel somewhat superior to Madame Gaillard. It's not yet truly apparent that Grenouille is evil; at this point he seems simply strange and troubled.
Madame Gaillard walks Grenouille to the tanner Grimal, who is always in need of cheap labor to do the most dangerous parts of the job. She knows that Grenouille will likely not survive, but feels no responsibility or remorse. She walks home satisfied.
Again the reader is asked to feel sympathy for Grenouille as he's sold into what's essentially slavery. We know it's Madame Gaillard's injury that makes her feel no remorse, but it's still no less awful.
The narrator says that since Madame Gaillard will not appear again, we should find out how she dies. She lived to be very old and gave up her business and purchased her annuity in 1782, and set about waiting for death. However, the French Revolution came first, and while it didn't affect her at first, she was later forced to auction off her belongings and then her home. By 1797 she'd lost her entire fortune and contracted cancer. She spent her last three weeks in the Hôtel-Dieu in a communal bed with five other women, and was buried in a mass grave in 1799.
Many, many years in the future, Madame Gaillard experiences the death she explicitly wanted to avoid and spent her whole life working against. While a long life is usually celebrated, in Madame Gaillard's case, it ended up being her downfall. The inclusion of Madame Gaillard's death raises the question of Grenouille's part in it. Was she right, and he did bring misfortune upon her?