Jude works a while refinishing old buildings and lettering tombstones. One day he is on a ladder working at a church, and he sees Sue with Miss Fontover at the service inside. His passion for Sue grows stronger in the following days, though he tries to resist it. Jude recognizes that by law he is obligated to love Arabella forever, despite the reality of his feelings.
Hardy furthers his critique of marriage, lamenting that the institution puts a contract on love, which is a fickle, voluntary feeling. Jude is obligated to love Arabella because of one act (their mistaken marriage), and his whole lifetime of feelings following that act are then invalidated by his religion and society.
One afternoon Sue comes to the stonemason’s yard while Jude is away, and she asks for Jude Fawley. One of Jude's coworkers mentions the visit, and Jude recognizes Sue by her description. He then finds a note from her at his lodging, saying she will be leaving Christminster soon. He immediately writes back to her and proposes to meet Sue outside. Sue agrees, but finds the particular place Jude chooses to be “gloomy and inauspicious in its associations.” They greet each other bashfully.
We don’t learn this until later, but Sue associates their meeting place with the curse of the Fawleys regarding marriage. She also has grown up a child of divorce, and heard the same advice to never marry. Sue breaks through Jude’s idealism by coming to see him herself, instead of acting as a distant, almost holy figure.
Jude asks Sue if she knows Mr. Phillotson (whom he assumes is a parson), but she says she only knows a schoolmaster of that name in nearby Lumsdon. Jude suddenly realizes that Phillotson has failed in his ambitions of attending a university. Jude invites Sue to come with him to visit Phillotson, and they set off.
The “defective real” returns again, now crushing Jude’s hopes for Phillotson. Phillotson’s failure is also a blow to Jude’s own hopes of raising his social station.
They find Phillotson, and his “homely complexion” destroys the idealized vision Jude had had of him. Phillotson doesn’t remember Jude, but he vaguely remembers sending him the Greek and Latin grammar books. Phillotson admits that he gave up his lofty goals long ago. He is comfortable as a schoolmaster again, though he is in need of a pupil-teacher. The three talk for a while and then Jude and Sue return to Christminster.
Phillotson was Jude’s precursor and idol, and his failure to be accepted at Christminster does not bode well for Jude. Phillotson is still relatively successful as a schoolmaster – he lacks Jude’s fierce, hopeless ambition, and is willing to settle for something less than ideal.
As they walk home Jude is struck by “what a revelation of woman” Sue is, and he realizes he is even more in love with her than before. Jude asks why Sue is leaving Christminster, and Sue says it is because she had argued with Miss Fontover, who broke her statues when she saw them. Jude proposes that Sue work for Phillotson as a teacher, and she agrees to consider it. The next day Jude visits Phillotson and he agrees to hire Sue, though he says the job is low-paying and so would only be useful to her as an apprenticeship for a teaching career.
Hardy can build a complicated world with only a few characters, and Phillotson returns to play a completely different role than before. We learn that Sue has grown bolder in pushing against Christianity and authority, as she is leaving Christminster because Miss Fontover broke her pagan statues.