The impact of escalating interest rates throughout April dampened home buying, even with more inventory in the market.