When lenders have holdings that are vulnerable to higher yields, they're punished by investors for the unrealized losses.
