Peter Helfgott is a Jewish survivor of the European Holocaust of the Nazis during World War II -- many of them, after surviving the Holocaust, moved to Australia. In the film, there's a brief moment while talking to David's mother, Rachel, where he hints at how the Nazis stole his family when they sent them to a death/concentration camp when he was a boy. Peter was quite plainly traumatized by that event in his life and wants to insure that he won't lose his own family, hence his strict nature toward his wife and kids. He is also afraid that he'll lose David if he leaves to study in London.