The problem is that this guy's entire life revolved around kink. It wasn't just something he did in the bedroom.
From what I've found, people who define themselves by kink tend to have serious issues, whereas those who just enjoy kink recreationally tend to be much more normal.
While this guy has a right to enjoy whatever weird shit he's into, it doesn't mean he should be appointed by the President to an important position. Clearly there was some mental/emotional instability going on here, and it didn't take this suitcase theft to see that.
Regarding the suitcase theft, it's different than if he were caught stealing an iPhone from Best Buy. Both would be legit grounds for dismissal, but at least stealing an item from a store would be unrelated to his kink lifestyle. This theft is highly likely to have been motivated by whatever clothing he thought was in the bag. I doubt it was a coincidence that he stole a Vera Bradley bag (one more likely to contain designer women's clothes than a normal bag), and I don't think it's a coincidence that he claimed to have left the clothes in the hotel when questioned over the phone. He probably wanted to get rid of the evidence that he wore them. This crime was directly related to his weirdness (maybe he gets a thrill out of stealing women's clothes and wearing them), and not financially motivated.
I think Biden is probably hoping this story just dies, or maybe that they drop the charges in Minnesota and everyone forgets about it. I'm sure Biden's people told him that right now is the worst time to dismiss him, as it will both look bad for the Administration, as well as validate the right's narrative that this freak shouldn't have been hired in the first place.