Brinton was born in Thornbury Township, Chester County, Pennsylvania. After graduating from Yale University in 1858, Brinton studied at Jefferson Medical College for two years and spent the next year travelling in Europe. He continued his studies at Paris and Heidelberg. From 1862 to 1865, during the American Civil War, he was a surgeon in the Union army, acting during 1864-1865 as surgeon-in-charge of the U.S. Army general hospital at Quincy, Illinois. Brinton was sun-stroked at Missionary Ridge (Third Battle of Chattanooga) and was never again able to travel in very hot weathers. This handicap affected his career as an ethnologist.
After the war, Brinton practiced medicine in West Chester, Pennsylvania for several years; was the editor of a weekly periodical, the Medical and Surgical Reporter, in Philadelphia from 1874 to 1887; became professor of ethnology and archaeology in the Academy of Natural Sciences in Philadelphia in 1884; and was professor of American linguistics and archaeology in the University of Pennsylvania from 1886 until his death.
He was a member of numerous learned societies in the United States and in Europe and was president at different times of the Numismatic and Antiquarian Society of Philadelphia, of the American Folklore Society, the American Philosophical Society, and of the American Association for the Advancement of Science.
At his presidential address to the American Association for the Advancement of Science in August 1895, Brinton advocated theories of scientific racism that were pervasive at that time. As Charles A. Lofgren notes in his book, The Plessy Case, although Brinton "accepted the 'psychical unity' throughout the human species," he claimed "all races were 'not equally endowed,' which disqualified [some of] them from the atmosphere of modern enlightenment." He asserted some have "...an inborn tendency, constitutionally recreant to the codes of civilization, and therefore technically criminal." Further, he said the characteristics of "races, nations, tribes...supply the only sure foundations for legislation, not a priori notions of the rights of man."