The First Baptist Peddie Memorial church was formed when part of the congregation of the Lyons Farms Baptist Church split on June 6, 1801. The Lyons Farms Baptist Church had given permission to form this new church the year before on July 24, 1800. The present building was dedicated in 1890 by the generous gift of Thomas B. Peddie, a businessman who also served two terms as Mayor of Newark.

The interior of the church is round with the pews set in theatre style seating. The balcony goes around the entire church with seating in the three quarters in front of the altar. Each window on the lower floor is made of Tiffany glass. Gaslights that look like dragons once illuminated the sanctuary and the exterior vestibule. Originally a statue of Gabriel stood on the roof but it was struck by lightning and never replaced.