Patterson was born January 4, 1935 in Waco, North Carolina. He started boxing at the age of 14 and won the Gold medal at the 1952 Helsinki Olympic Games as a middleweight. Following the Olympics, Patterson turned professional and on November 30, 1956 became the youngest World Heavyweight champion in boxing history.
After losing his title on June 26, 1959, on June 20, 1960 he became the first man to regain the title. Patterson retired from boxing in 1972 with a career record of 55 wins, 8 losses, and 1 draw. After retiring, Patterson became chairman of the New York State Athletic Commission and in 1991 he was inducted into the International Boxing Hall of Fame. Patterson published his autobiography, “Victory Over Myself”, in 1962.