More potential home buyers may be getting priced out of second homes. Rising prices and loan increases are slowing demand after a pandemic-driven boom in the sector.