Perry was born in Williamstown, Massachusetts and was educated at Williams College, Williamstown, as well as the universities of Berlin and Strasbourg.
Perry taught at Williams from 1886 until 1893. He then taught at Princeton University, where he became acquainted with future US president Woodrow Wilson, Dean Andrew West, and former US President Grover Cleveland, about whom he wrote entertainingly in his autobiographical work, And Gladly Teach. Perry taught at Harvard University between 1907 and 1930 and was the Harvard lecturer at the University of Paris from 1909 to 1910. From 1899 to 1909 he was the editor of The Atlantic Monthly.
Perry was awarded the Legion of Honour by the French. He edited the works of Edmund Burke, Sir Walter Scott, and Ralph Waldo Emerson. From 1905 until 1909 he was general editor of the Cambridge edition of the major American poets. He wrote extensively, including monographs on Walt Whitman, John Greenleaf Whittier, Thomas Carlyle and Emerson. He was also a prolific writer of novels, short fiction, essays, studies in poetry, and an autobiography.
Perry is famed in certain Vermont lore for "establishing" the "summer colony" of Greensboro, Vermont. He enjoyed its tranquil setting and its distance from the cares of the busy world of the Atlantic Monthly and his professorships. Fly fishing was one of his key hobbies, which led to the publication of "Fishing With a Worm.