First Calvary Baptist Church descended from African congregants who left First Baptist Church following the Civil War. These founding members, like many African Americans at the time, sought greater autonomy by breaking from white-controlled churches. The congregation of First Calvary first organized under a brush arbor and later met in the home of Celia Mann, now the Mann-Simons Cottage.
The congregation built a permanent home, a frame structure, on Richland Street (c. 1870). They remained at that location until building a new stone sanctuary at Pine and Washington Streets, which was completed in 1950. After more than 50 years of useful service, that church was replaced by a modern brick sanctuary, (built on the same site as the 1950 building), which was dedicated in 2005.