Swalwell says they’re not going away and Kevin McCarthy will regret giving the three of them more time on their hands.