Due to the aftereffects of a stroke, James Cagney was unable to properly articulate his dialogue during shooting. In the finished film, his voice is actually dubbed by impressionist Rich Little.
The story was originally intended as a vehicle for Katharine Hepburn, with the main character a retired tennis champion. When she declined the project, it was re-written for James Cagney, with the lead character changed to a retired boxer.
Harry and Tonto, the 1974 story of an elderly man who is turned out of his New York apartment by urban renewal, and ends up on a cross-country road trip to visit his three children, was intended by screenwriter Paul Mazursky as a part for James Cagney. Cagney refused the part, insisting that he had retired from acting. The part then went to Art Carney, who won a Best Actor Oscar for his performance. Ironically, Cagney "unretired" not once but twice in succeeding years -for a small part in 1981's Ragtime, and finally for this film - in which he is paired with Art Carney.