Jack was born Sidney Walker on April 1, 1921 in Waynesboro, Georgia but raised in Augusta, Georgia. At the age of 15, he began to fight in battles royal, fights funded by rich white men for their entertainment in which six black men fought blindfolded until only one remained standing.
Jack turned professional in 1940 and won the Lightweight World Championship in December, 1942. In 1943, he lost and regained the title before losing it for good in 1944. Jack was the 1944 Ring Magazine Fighter of the Year. Jack retired from boxing in 1958 with a record of 88 wins, 24 losses and 5 draws.
In 1991, he was inducted into the International Boxing Hall of Fame.