The first home of the Stoughton Public Library was in Morton's Square (1874). The building is still standing today, and is located near the train station. It's currently used as a residential apartment.

The library then moved to a building near the old First Congregational Church, which was located near where the VFW Hall on Washington St. is today.

From 1881 until 1904, the library was actually housed in the Town Hall, before moving again to the Lucius Clapp Building on the corner of Park St. and Pleasant St. in 1904.

The Library remained in the Clapp building until 1969 when the current library on the corner of Park St. and Walnut St. opened. The Wales French Room in this library, where many meetings and activities are held, is named for the first librarian and founder of the Stoughton Public Library.