The story moves back and forth between Ove's childhood and his life as a fifty-nine-year-old man in Sweden. Ove's mother died when he was eight, so Ove lived with his father and began working at the railway with him. One day, Ove is sent to clean out a railway car with Tom, a mean and selfish coworker. Ove finds a wallet with money in it, and decides to turn it in rather than keep it. The railway director gives Ove's father a Saab, and Ove and his father spend Saturdays repairing the car. Ove's father dies when Ove is 16 and leaves him only his house and the Saab. Ove quits school to take his father's job at the railway. When he starts receiving letters informing him that the city wants to buy his house and demolish it, he decides to get a job at a construction site and learn how to build houses so he can renovate his house. Tom accuses Ove of stealing money, and because Ove refuses to accuse Tom, Ove loses his job. The director hires Ove back on as a night cleaner on a long-distance train. Ove finishes renovating his house and weeks later, it catches fire. Men in white shirts stop Ove from putting the fire out.
Ove rents a room in town. Days after his house burned down, Tom confronts Ove at work and steals his father's watch. Ove punches Tom, takes his watch back, and Tom never bothers Ove again. Ove tries to sign up for the military but is denied entrance when he learns he has a congenital heart defect. Soon after, he meets Sonja on a train. She is riding to school to become a teacher. Ove spends three months riding the train with her until she asks him out to dinner. Although he admits to lying about his reasons for riding the train with her, she doesn't care. She insists he start an engineering course so he can build houses. When Sonja and Ove have been in a relationship for three years, Sonja's father dies and Sonja is distraught. Sonja's childhood cat, Ernest, dies a week after her father. Sonja tells Ove she is pregnant and insists they buy a house. They move into a row house on the same day as a couple named Rune and Anita. Anita is also pregnant, and she and Sonja become best friends. Rune and Ove share tools and built cribs for their babies.
Ove and Sonja take a bus tour to Spain. On the way home, the bus crashes. Sonja miscarries and becomes paralyzed from the waist down. Ove insists on rebuilding the house himself to make it wheelchair accessible. Sonja takes a job teaching troubled students and teaches them all to read Shakespeare. Ove fights with “men in white shirts” over his wife’s health until Sonja makes him stop. After Rune's baby boy is born, Ove and Rune begin fighting over everything—from how to mow the lawn to which heating system to install in the houses. The feud spans nearly forty years and the last straw is when Rune buys a BMW after his son moves to America. After Rune buys the BMW, he is diagnosed with Alzheimer's. Four years before the present, Sonja is diagnosed with cancer. She works as long as she can and Ove continues to fight men in white shirts, but she dies anyway.
In the present, six months after Sonja's death, Ove's boss at the housing office forces him to retire. He decides to commit suicide and join Sonja. He makes his morning inspection of the neighborhood as usual and notices that Blond Weed's dog is harassing a cat. He yells at Blond Weed and scares the cat away. Later, as he is installing a hook in his ceiling so he can hang himself, his new neighbors, Patrick and Parvaneh (who is very pregnant), back their moving trailer right into Ove's house. Ove goes outside to yell at them, and maneuvers the trailer for them. Later, their children, Nasanin and the seven-year-old, bring Ove dinner. Ove decides to wait to hang himself until the next day. The next day, Ove confronts a teenager named Adrian and his friend Mirsad about an improperly parked woman's bicycle. Patrick and Parvaneh bring Ove cookies and ask to borrow a ladder and an Allen wrench. Ove angrily fetches the items for them and is annoyed when Anita shows up in his yard. She says that the council is going to take Rune away from her, but Ove insists that that will take years. Patrick notices wheel marks on Ove's floor, and Anita tries to explain to Patrick that Sonja had been in a wheelchair, but Ove yells at Anita. Ove slams the door and tries to hang himself, but his rope breaks.
The next morning, Ove kicks the cat while he’s inspecting the neighborhood as usual. As he's walking back to his garage, a man in a white shirt drives a Škoda (a type of car) drives through the residential area, almost hitting Ove—who yells at the car as it drives off. Ove notices Blond Weed looking suspiciously satisfied while her dog barks, and Ove sees that Patrick is preparing to climb the precariously positioned ladder. Ove worries about the cat but continues with his new plan: to kill himself using car exhaust. As he sits in his car and lets the fumes fill it, he hears an ambulance and then someone banging on the garage door. Ove finally opens the garage door hard into Parvaneh's nose. She asks him to take her and her daughters to the hospital to see Patrick, who fell off the ladder.
At the hospital, Ove argues with the parking attendant about the price of parking. Parvaneh leaves Ove to watch the children, and Ove punches the hospital clown who tries to do a magic trick with one of Ove's coins. Nasanin thinks Ove is fantastic, but Parvaneh and the seven-year-old are less impressed. Ove drives them back home and Parvaneh asks Ove to help with fixing her radiator. Ove agrees.
The next morning, Ove goes to the train station without noticing a cat-shaped hole in a snowdrift outside his house. Ove studies people on the platform, and as the train approaches, a businessman faints and falls onto the track. Ove curses and pulls the man off the track. Ove thinks he'll let the train kill him, but then he makes eye contact with the young train conductor and decides it would be rude to ruin his day. When Ove gets home, the man in the white shirt almost runs him over again. Ove finally notices the frozen cat in the snowdrift. Parvaneh notices as well and yells at Ove to save the cat. She insists on taking the cat into Ove's house to warm it up. Jimmy, Ove's overweight young neighbor, joins them in Ove's living room. He warms the cat up with his body heat. Parvaneh sees Ove's kitchen and realizes that Sonja had been in a wheelchair. Jimmy has an allergic reaction to the cat, and Ove has to take him to the hospital. Parvaneh convinces Ove to keep the cat, and Ove takes it to visit Sonja's grave the next day.
Later that week, a journalist named Lena pesters Ove for an interview about saving the man at the train station. Ove ends up briefly locking Lena in his garage, but is discovered by Parvaneh. Parvaneh tells Lena to call her. Ove agrees to take Parvaneh to the hospital to get Patrick and Jimmy. On the trip home from the hospital Jimmy insists on stopping at McDonald's. Parvaneh asks Ove to teach her to drive so she can get her driver's license. When they return to their street, Parvaneh gives Ove Nasanin's drawing of him, which is done in riotous color.
The next day, Ove lets the cat out and considers killing himself using Sonja's painkillers. When he hears the cat screaming in pain, he goes outside and finds Blond Weed about to kick the cat. Blond Weed stalks away, and Ove goes to Rune and Anita's house to borrow some corrugated iron and rigs a trap for Blond Weed's dog that will electrocute it when it tries to pee on his paving stones, but the cat gives him a look and he disassembles it. Later, Adrian brings Ove’s mail to the door and tells him that Sonja had been his teacher. Ove offers to help Adrian fix his bike and bring it to the cafe where Adrian works. Ove then goes to see Parvaneh and sets a time for their first driving lesson. The lesson starts badly: Parvaneh didn't want to learn to drive a manual transmission. At the first red light, Parvaneh stalls the car and rolls back into an SUV driven by rude men with neck tattoos. She panics and gives up. Ove yells at the SUV drivers and gives Parvaneh a pep talk.
The next day, Ove and Parvaneh drive the bike to the cafe where Adrian works. Ove is officially introduced to Mirsad, Adrian's boss. When Ove notices makeup on Mirsad’s face, he uses offensive words to ask Mirsad whether he's gay. Ove brews his own filter coffee and helps Adrian fix the bike. Adrian asks Ove to not tell Amel, the owner of the cafe, that Mirsad is gay. The next day, Ove gets out a rifle to kill himself, but is interrupted when Parvaneh asks to use Ove's bathroom and then puts him on the phone with Lena. Ove sees the man in the white shirt driving to Rune's house, and Ove yells at him. The man seems unconcerned about Ove. Ove asks Patrick if he can borrow his trailer, and Ove parks it to block the man's Škoda. Later, the man confronts Ove about it and Ove insists he did nothing. Anita tells Ove that the council is taking Rune later that week, and Ove locks himself in his house and cries. Later that night, Ove takes his clothes off and prepares to shoot himself. Mirsad and Adrian appear at Ove's door and ask whether Mirsad can stay over because Mirsad had come out to his father, Amel, who in turn kicked Mirsad out. Ove agrees and thinks that Sonja would've liked having Mirsad stay. The next morning, Mirsad accompanies Ove on his morning inspection. Jimmy joins them and tells Ove that they're taking Rune this week. He reveals that Anita has spent the last two years petitioning to keep Rune but didn't want Sonja to know. Patrick fetches their neighbor, Anders, who has a towing company. Anders tows the man's Škoda and shares that he has broken up with Blond Weed.
Ove and his neighbors spend the next few days planning. Finally, Ove makes a phone call. On the day of Rune's "transfer into care," the man in the white shirt shows up to take Rune away, but the neighborhood stands behind Anita. Lena gives the man a pile of records showing improper conduct, and the man backs down. Ove finally agrees to an interview with Lena. Ove has dinner with Parvaneh and Patrick, and learns that the seven-year-old loves houses. The next day, he goes to speak to Amel and encourages him to accept Mirsad’s homosexuality. The seven-year-old invites Ove to her birthday party and tells him she wants nothing but an iPad, so Ove asks Jimmy to help him purchase an iPad. After the birthday party, Ove discovers burglars at his neighbor's house and has a heart attack. Parvaneh lists herself as next of kin and laughs when the doctor tells them that Ove's heart is too big. When Ove comes home from the hospital, Parvaneh helps him put Sonja's things away. Parvaneh gives birth to a baby boy later that night.
Over the next few years, Mirsad and Jimmy get married and adopt a daughter. Four years after Patrick and Parvaneh moved in, Ove dies. He leaves Parvaneh's children and Jimmy's daughter a million kroner each (roughly $100,000), and Parvaneh and Patrick start a charity for orphaned children with the rest of Ove's money. Over 300 people attend Ove's funeral. Later that night, Parvaneh shows Ove's house to a couple. The wife is pregnant and the husband drives a Saab.
