Pacifiers and nursing question?

My son is fed and is only 3weeks old. When he has a freak out we found that he likes to comfort suck and I learned with my first born that that is not the greatest thing to start with a baby. I let him suck on my finger for a bit and that seems to soothe him, even puts him to sleep sometimes....im curious though if it will ruin his latch and if he will become addicted. I am a "natural" Mama and really dont like the idea of man made materials in my sons mouth. Im noticing already that he gets lazy sometimes and latches on only to spit out the nipple and the milk he got with it. Dont really know what to do at this point. Any tips?