Palermo Terrace was built as investment property in the mid-1880s for Caroline and Joseph Nelson. The houses were later named separately as Wallan-Billan, Irene, Wendovir and Indarra. Caroline Nelson and her son Joseph continued to own Polermo Terrace until it was resumed by the government in 1903. They lived next door in number 43 Lower Fort Street, known as Bligh House or Clyde Bank.