I wouldn't get down on yourself. For the most part, unless you started skiing when you were young and fearless, we all started that way, to a degree.
Anxiety over falling and speed is normal when you start out -- even when you advance and move on to more challenging terrain. It would be hard...