Robert Tyrone "Pat" Patterson was born in Laurinburg, North Carolina, around 1941. He attended the Laurinburg Institute, a school for African Americans, and then local public schools. He graduated from high school in 1959 and enrolled at North Carolina A&T State University, where he was friends with the Greensboro Four.
As a student at A&T, Patterson was very active in the civil rights movement, and in 1963, he served as vice chairman of the Greensboro chapter of CORE, the Congress of Racial Equality. Patterson remained very involved in the local activities through his completion of a bachelor’s degree at A&T in 1963.
Following graduation, Patterson married and worked briefly as a taxi driver and fireman. In 1967, he moved to New York for a job with Mobil Oil, but returned to North Carolina six months later to work in Winston-Salem.