Men afraid of having children?

I'm 43. Was married for a couple years, never had children. If we had sex, she was on the pill, I used a condom, AND pulled out. Just terrified of it. Long term relationship a few years later. She couldn't have kids, but had 2 from previous marriage. It was easier with those kids. Now very interested in a girl who desperately wants children someday, and I'm absolutely terrified. Do any of you experience this? Did you ever get over it?? Thanks for any words of advice.