As a salesperson, how do you handle when someone tries to sell you?

My thought process:

When I get cold called, I usually try to let them down easy because I know the struggle. I'm straight forward and just tell them I'm not interested for *insert-random reason.*

The downside of this is that you get a lot of pushback. The gist of my question: what is the most tactful way you've found to reject some other hustler without destroying them.