I agree it's mainly fear. Fear of what? embarrassment, rejection, ridicule... Is this fear justified? It's possible those fears may be realized, but think of what you have to gain- How great shall be your joy if you bring but one soul unto God, how great shall be your joy if you bring many souls. It really is what brings me the greatest joy to see people embrace eternally important truth and change their lives for the better. I really ought to do it more.
And although unprepared people might reject or try to embarrass you, you don't have to let that get to you - and they're not really going to think less of you. If they do, then you know they're not inclined to spiritual things and they're really in need of help... so pray for them, and take confidence in the fact that you have been richly blessed with the Spirit of the Lord, and count your many blessings!
I'm preaching to myself as much as anyone.