Christchurch Mansion is built on the site of the Holy Trinity Priory, founded in the 12th century, which was suppressed by King Henry VIII.

In 1545 Paul Withypoll, a successful London merchant, purchased the site. In 1547 his son Edmund began building Christchurch Mansion. In 1645 the estate passed to Elizabeth, the daughter and heir of Sir William Withypoll and wife to Leicester Devereux.

In 1649 Devereux inherited the title 6th Viscount Hereford. He was responsible for many alterations, particularly the frontage of the house and the surrounding gardens.

In 1735 Claude Fonnereau, a wealthy London merchant of Huguenot descent, purchased the house and 70 acre estate. Over the next hundred years the Fonnereau family increased the estate to 95  acres of land. The public were allowed to enjoy the private grounds and community events were held in the park. In 1892 William Neale Fonnereau put the Mansion and estate up for sale.

A property syndicate bought the site in 1894 and sold some of the land for development. Felix Cobbold, a member of the syndicate, presented Christchurch Mansion to the town as a gift and the Ipswich Corporation purchased the remainder of the park.