Stop assuming you know anything about what hurts me and what doesn't.

If you want me to leave you alone because I irritate you, or because I'm a nuance, or a waste of your time, or a burden on you, I will. If you want me out of your life from some fault of mine, I'll never speak to you again. End of story.

But, you said it yourself. I don't get to tell you whether the energy you spent on me is worthwhile. Don't expect me to allow you something you wouldn't allow anyone else.