Williams was born April 6, 1886 in Seguin, Texas. He began playing baseball professionally in 1905. Then known as Smokey Joe Williams, he spent 5 years with southern black baseball before moving in 1910 to the Chicago Giants. When playing for the Cuban Winter League, he was known as Cyclone Joe and continued his winning styles.
Although records are sketchy, Williams was credited with 41 wins and 3 losses in 1914. He also pitched many games against major league stars in post-season barnstorming exhibitions and is credited with posting a record of 20 wins and 7 losses in those games.
In 1928 at the age of 44, while playing for Homestead, he officially joined the American Negro League.
Williams retired in 1932 and although most modern sources consider Satchel Paige to be the greatest Negro league pitcher, a 1952 poll by the Pittsburgh Courier gave that title to Williams. Williams record is listed as 125-56 and 9-2-1 against white major league teams with 4 shutouts. He was also a better hitter than Paige.
Williams was inducted into the Baseball Hall of Fame in 1999.