Mackenzie was born in Logierait, Perthshire, Scotland, the son of Mary Stewart (Fleming) and Alexander Mackenzie Sr. He was the third of ten children. At the age of 13, Mackenzie's father died, and he was forced to end his formal education in order to help support his family. At the age of 16 he apprenticed as a stonemason and by the age of 20 he had reached journeyman status in this field. Mackenzie immigrated to Canada in 1842 to seek a better life as well as to follow his sweetheart, Helen Neil. Shortly thereafter, he converted from Presbyterianism to Baptist beliefs. Mackenzie's faith was to link him to the increasingly influential temperance cause, particularly strong in Canada West where he lived, a constituency of which he was to represent in the Parliament of Canada.