Smith was born in a Romani bender tent in Epping Forest, six miles northeast of London. Today the site is marked with a large, commemorative stone in the woods near Waterworks Corner, Woodford Green. Smith received no education, and his family made a living selling baskets, tinware, and clothespegs. His father, Cornelius (1831-1922), and his mother, Mary (Polly) Welch (c1831-1865), provided a home that was happy in their vardo. Smith was a child when his mother died from smallpox near Baldock in Hertfordshire. She is buried in the nearby churchyard of St Nicholas church in Norton, now part of Letchworth Garden City. Cornelius Smith was later buried with her. The Smith children numbered four girls and two boys (Rodney was the fourth child).
Cornelius Smith was in and out of jail for various offences. There, he heard the gospel from a prison chaplain; later, he and his brothers were converted at a mission meeting. From 1873 on, "The Converted Gypsies" were involved in numerous evangelistic efforts.