Scarborough Village began in the 1830's as a crossroads village, centred around Markham and Kingston Roads and Eglinton Avenue. In 1832, it became the first community in the former Township of Scarborough to have its own post office.

Robert R. Bonis, in A History of Scarborough describes Scarborough Village in 1896 as "consisting only of a brick schoolhouse, a general store, a blacksmith shop, a building for the sale of farm implements, the Methodist parsonage, less than a dozen dwellings, and a large railway hotel converted to other uses."

Historical Scarborough Village landmarks include the Washington Manse, circa 1875 which is located at 14 Centre Street, and the Fred S. Cornell House, circa 1850, which was relocated to Thompson Memorial Park in 1962, when the present day neighbourhood was being developed.