Kilgore was born February 20, 1913 in Woodruff, South Carolina. He earned his Bachelor of Arts degree from Morehouse College in 1935 and earned his Bachelor of Divinity degree from Union Theological Seminary in 1957.
Kilgore began his fight for equality in the 1940s, registering voters and organizing tobacco workers in North and South Carolina. He moved to New York City in 1947 and as pastor of Friendship Baptist Church raised bail money for civil rights workers jailed in the South. He also served as founding president of the Heart of Harlem Neighborhood Church Association which was organized in 1957 to fight segregation in New York City.
Kilgore was also an organizer of the Prayer Pilgrimage for Freedom in 1957 and the March on Washington for Jobs and Freedom in 1963. In 1963, he moved to Los Angeles and became pastor of Second Baptist Church, the oldest Black Baptist church in the city. In 1969, he was elected president of the American Baptist Churches USA, the organizations first Black president, and from 1976 to 1978, Kilgore served as president of the Progressive National Baptist Convention. The Morehouse College Campus Center is named in his honor.