Brome Hall in Suffolk, Seat of Lord Cornwallis