John McNeill Smith, Jr., was born in Rowland, North Carolina, on 9 April 1918. He attended the University of North Carolina, where he served as editor of The Daily Tar Heel, graduating in 1938. He received his law degree from Columbia University in 1941 and briefly worked as an associate at Root Clark Buckner & Ballantine in New York before serving in the navy during World War II as a bomb disposal officer. Smith then returned to North Carolina and practiced law in Greensboro with Smith, Moore, Smith, Schell, and Hunter.
Smith became known as an advocate of human rights and equal justice, frequently taking cases that other lawyers refused. From 1958 to 1962, Smith was co-counsel for Junius Irving Scales, a Communist Party member charged with violating the Smith Act. In 1960, McNeill Smith was the negotiator between black and white leaders during the lunch counter sit-in in Greensboro, and he continued acting as a negotiator in the desegregation of local hotels, motels, and restaurants. In 1963, he was the attorney for University of North Carolina students in the Speaker Ban case. Smith was also a member of the United World Federalists Movement and chair of the NC Civil Rights Commission.
Smith was elected to the North Carolina House of Representatives in 1970, and in 1972, to the North Carolina Senate. In 1978, Smith unsuccessfully ran for the United States Senate. He then returned to civil litigation and also taught constitutional law at University of North Carolina at Chapel Hill. Smith also served at the North Carolina Center for Public Policy Research from 1980 until 1992 and was chair of the North Carolina Board of Ethics from 1980 to 1984. In 1989, Smith was ranked as one of the nation's most influential lawyers by the National Law Journal for his work in establishing the North Carolina Civil Liberties Union. From 1992 to 1993, Smith traveled to Estonia to help the Soviet republic in its legal preparations to become self governing.