New outdoor grow pest control

Hi all- noob grower here about to try a first time outdoor grow and want to keep it as organic as possible. I'm a Santa Cruz native and cannabis connoisseur for nearly 30 years, finally at a place to give a little home grow a shot. I've been doing a lot of reading in the forums about organic soil mixes, nutes, teas etc, but need a little advice on prepping the area prior to building raised beds and mixing up some organic soil.

I've noticed some aphids and other pests in my yard on some of the surrounding foliage. I was thinking about spraying the yard with neem oil prior to setting up my raised beds and bringing in the soil mix. Is this a good plan? Or should I use something else, or just not worry about it? I've read of using neem cake in the soil mix to help prevent gnats & other soil pests etc from having a feast on the roots, is that enough? I'd like to take care of some of these pests ahead of time to try and provide the best opportunity for the plants to succeed, am I on the right track? Any advice or kick in the right direction or forum would be greatly appreciated!