Building commenced on Stone's Public House in 1832 by John Stone, who called his business the Railroad House. John was a farmer and a captain in the militia, but he was also a savvy businessman, owning most of the land in the center of what was then called Unionville. When he heard that the railroad was to be built through the center of town (on his own land) he decided to build a hotel right alongside the tracks. The Railroad House (the property also included a barn and a cow-yard and later a home for his family) opened on September 20, 1834 , to an enthusiastic crowd of (some say) 300 people.

John operated the Railroad House for less than two years (though he continued to live on the property), then leasing it to a long list of innkeepers. John died in 1858, and W.A. Scott bought the business in 1868.

Over the years the building fell into disrepair and disrepute. The man credited with helping to return the building to its former glory is Leonard "Cappy" Fournier, who bought the building in 1976. Cappy is also the man credited with first exploring the paranormal side of the building.