Tattoo Shop in North Hills, NY

The glacial action that formed much of Long Island deposited Shelter Rock in what is now North Hills approximately 11,000 years ago. The Matinecock had a village in the area. Farming developed in North Hills in the middle part of the 17th century. Around this time a long fence was built along the road later known as Northern Boulevard, to the north of North Hills. The lands of the Cow Neck Peninsula enclosed by the fence (present day Manhasset and Port Washington) were used for grazing.

During the 19th Century, one of the largest farms in the area was owned by Isaac Underhill Willets.

The village was incorporated in 1929 as a tax haven to prevent the Town of North Hempstead (seated in nearby Manhasset) from raising taxes on the property, as well as to gain and maintain home rule powers. The name North Hills was chosen due to how the village is located in the northern part of Nassau County and because the terrain is very hilly in the area.