House prices in one of the most affluent neighbourhoods are declining as people can't afford overpriced coastal locations.