Evander Gilmer, Jr., was born on September 22, 1943, in Greensboro, North Carolina. He graduated from Dudley High School in 1961 and then he attended North Carolina A&T State University. While there, Gilmer participated in the civil rights demonstrations of 1962 and 1963 and was arrested several times for his participation. He served as treasurer of the local CORE chapter during that time.
In 1967 Gilmer graduated from A&T with a degree in architectural engineering. He subsequently moved to Maryland and worked with the U.S. Navy as an architect within a public works engineering group. He later worked in program management, and in 1979 became a deputy Equal Employment Opportunity officer.