I know with my "best" and "average" customer LTV that my current cost per acquisition (CPA) is good.
However, if I was just looking at my "worst" customer LTV the current amount I'm paying in CPA is way over.
How do I judge what I should be spending on advertising based on these 3 LTV segments (best, average, worst) I have for my customers?
To build on what Sarah said - you can also look within channel if you have access to someone with the technical chops.
Some ad groups or campaigns could be driving "worst" leads and others could be driving "best" ones. To cut off the channel is throwing the baby out with the bathwater.
UTM tags in adwords (or any other platform these days) will be key to figuring this out. If you get leads on the phone, there are awesome solutions out there as well.
Just as you're averaging CPA across all the customers you're acquiring, you should also be looking at your average customer LTV and using that to determine the strength of your overall CPA.
That said, if you have the data to be able to break down CPA by best, average, and worst customers AND can see the source of each of these customers, you may be able to glean some insights around which sources or campaigns yield the best and worst customers. (For example, if your highest-CPA and lowest-LTV customers are mostly coming from AdWords, you should focus your budget/efforts on a better-performing channel.) This will help you cut out high CPA channels and generate higher-quality customers, thereby reducing your average CPA and bringing your ROI up.
In my experience the easiest way is to change what you're optimizing towards, at present your CPA appears to be that best/average/worst are all equal to 1. Why not change out your acquisition model to optimize towards the LTV you've defined.