Born into an elite Virginia family in Richmond, Virginia, the young Glasgow developed in a different way from that traditional to women of her class. Due to poor health, she was educated at home in Richmond. She read deeply in philosophy, social and political theory, and European and British literature. She spent her summers at her family's Bumpass, Virginia, estate, the historic Jerdone Castle plantation, a setting that she used in her writings. Her father, Francis Thomas Glasgow, was the son of Arthur Glasgow and Catherine Anderson. He was raised in Rockbridge County, Virginia, and graduated from Washington College, now Washington and Lee College, in 1847.