Born in Lemont, Illinois, Paine was the son of Reverend Samuel Delahaye Paine. Rev. Paine was born in London and served in the British Army during the Crimean War before emigrating to America in 1856. Rev. Paine served as a lieutenant in the 2nd Maine Battery during the American Civil War and later was chaplain-in-chief of the Grand Army of the Republic.
While his father was pastor of the Green Street Presbyterian Church in Jacksonville, Florida,:115 Paine worked as a reporter for twelve dollars a week. He also frequented a soda fountain in a cigar shop owned by Cuban revolutionary José Alejandro Huau.
Paine attended Hillhouse High School in New Haven, Connecticut and then Yale University. At Yale he was on the football and rowing teams and was a member of Skull and Bones. At Yale, he covered athletic news for a news syndicate. This, plus the money he saved reporting in Jacksonville, paid for his education. He graduated in 1894.