



The Knarn Kolak Aboriginal people have long inhabited the area.
In the early 19th century whalers and seal hunters used the coast in this region. The bay was named by the crew of the whaler The Fairy in 1828.
John Griffiths established a whaling station in 1835 and a store was opened in 1839. In 1843, James Atkinson, a Sydney solicitor, purchased land in the town by special survey. He drained the swamps, subdivided and leased the land, and built a harbour on the Moyne River. He renamed the town 'Belfast' after his hometown in Northern Ireland. The Post Office opened on 1 July 1843 as "Port Fairy" but was renamed "Belfast" on 1 January 1854 before reverting to the original name 20 July 1887.
Agriculture developed in the region, and Belfast became an important transport hub. By 1857 the town had a population of 2,190. In the mid-to-late 19th century, Belfast was one of Australia's largest ports, catering to the whaling industry. In 1887 the town went back to the original name Port Fairy, as a result of an Act of Parliament.
A railway was extended to the town in 1890, but closed in 1977. It has been converted into the Port Fairy to Warrnambool Rail Trail.