Parrish was born in the city of Kewanee, the only son of Rufus Parker and Frances Adeline (Hollis) Parrish. He was born in "Rose Cottage" on June 10, 1858, at what was later the site of the city's Methodist Episcopal church. The old family home was at Gilmanton, New Hampshire, but the parents removed to Kewanee from Boston, where Rufus Parker Parrish had been engaged in business and was prominently associated with William Lloyd Garrison and others in the anti-slavery cause. Both parents had a wide acquaintance with the famous Boston citizens of that era, including Longfellow, Holmes, Whittier, Wendell Phillips and Emerson. They came to Kewanee, then the merest excuse of a village, in April, 1855, the husband becoming connected with the pioneer store of Morse & Willard, then situated at the corner of Main and Fourth streets.