On October 4, 1884, according to historic records, the church was “up and partly enclosed and the floorboards laid loosely” when the first worship service was held at what was then called the Congregational Church of Sunol.  With the help of the American Home Mission Society of the Congregational Church, the church was built to serve the several families who used the Town of Sunol as their summer resort. In 1890, the church’s Ladies Aid Society, the Glenside Helpers, was organized. In 1903, it helped furnish the church with gas lighting.

The church parsonage was added in 1907—the same year that the Congregational Church in Mission San Jose donated their pews to the church. Between 1916 and 1949, the church appeared in several feature films of the Silent Picture Era.

In 1954, the church was painted brown and renamed “The Little Brown Church of Sunol” after the popular song of the time, “The Little Brown Church in the Vale.” The Little Brown Church celebrated its 130th Anniversary in 2015.