Perry was born October 9, 1916 in Lake Charles, Louisiana. At the age of 13, he entered the seminary of the Society of the Divine Word and in 1938 he took his vows as a member. In 1944, Perry was ordained to the priesthood and he then served as assistant pastor at a number of churches before returning to Louisiana as founding pastor of St. Joseph’s Church.
During his six years as pastor, he built the church, a rectory, and a school. From 1958 to 1964, he served as rector at the Divine Word Seminary. In 1964, he became the first African American to deliver the opening prayer in the United States Congress. In 1965, Perry was appointed auxiliary bishop of the Archdiocese of New Orleans, making him the first African American of the modern era to become a Catholic bishop. He remained in that position until his death.