Jack Elam was born and raised in Greensboro, North Carolina. He enrolled at the University of North Carolina in 1943, at the age of sixteen. In 1944, Elam joined the navy. When his service ended in 1946, Elam returned to the university and completed his degree. He earned a law degree from UNC School of Law in 1952 and became an assistant professor of public law and government and the assistant director of the Institute of Government at UNC. In April of 1954, he returned to Greensboro to work as an assistant city attorney. He became the city attorney in 1957 and held the position until December of 1961. Elam then served as corporate council for Cone Mills Corporation.
Elam was elected to the Greensboro City Council in May of 1965. He went on to serve as mayor from 1969 to 1971 and as mayor pro tempore from 1971 to 1973.