Sterling was once part of the territory of the Nashaway Indians, at one time the most numerous and powerful tribe in Massachusetts.  Owing to an abundance of lakes and streams, the land that is now Sterling was home to native Americans as far back as 7000 BC.

In 1643, the Nashaway Sachem, Nashawhonan, now known as Sholan, sold an 80 square mile tract of land to the founders of Lancaster.  This land included the eastern portion of Sterling.  In 1702, Tahanto, a nephew of Sholan, sold additional land which included both Waushacum Lakes and the western part of town.  The first settlers lived around the Waushacum Lakes in the 1640's.  The first permanent settler was Gamaliel Beaman of Lancaster, who in 1720 settled near the present day Beaman Road.  By 1740 about 50 families had settled.  In 1743, a meeting house, a school and a pound were built in the Center on three acres of land donated by the Sawyer family.  The Town Common is a remnant of that land.  Over 850 families lived in the town by 1764 which at that time was called Chocksett, "land of the foxes".

In 1781, Chocksett separated from Lancaster and was incorporated as Sterling, in honor of William Alexander, Lord Stirling, a Scottish lord who helped the colonists during the Revolutionary War.  

Sterling was primarily an agricultural community.  In addition, many residents engaged in cottage industries.  Among these were the making of hats, needles, clocks, leather goods, shirts, chairs and pottery.  Lack of sufficient water to power industrial equipment limited their growth and eventually caused many to move.  The railroads came to town in the 1850's.  Their coming opened up new markets for both agricultural products and manufactured goods.

Sterling has had its share of prominent residents and their names can be found in the street names and places in the town and among present residents.  Lt. Joel Pratt was an aide to George Washington during the Revolution.  Pratt's Junction was named in his honor.  In 1828, Silas Lamson invented the curved snath for the scythe which improved harvesting of hay and grains.  Silas Stuart invented a machine to make sewing machine needles in 1860.  In 1863, Ebenezer Butterick developed dress patterns.  Mary Sawyer whose pet lamb was the inspiration for the poem, "Mary Had a Little Lamb" was born in Sterling in 1806.  A statue commemorating the lamb stands on the Town Common.

Many traditions still continue in Sterling.  The annual Town Fair was first held on the Town Common as a farm showing in 1859. The Sterling Cadet Band first played during the Civil War.  

Today the town retains much of its small town atmosphere, while its agricultural lands are giving way to residential and industrial development.  For about 100 years, the population remained at 2,000.  Since 1950, it has grown rapidly to over 8,000.