Caster is a main culprit, but it’s never the only, or sometimes even the main culprit.
As I’ve said many times here, my 71 has less than 2° on one and just over 2 on the other and it tracked straight and true its whole life.
I never measured caster when it was stock height. I didn’t measure when it was 1 1/2 inches of lift with stock c-bushings, or 2 1/2 inches of lift with stock c-bushings. I only measured after it was 3 1/2 inches of lift with 7° c-bushings.
I’m sure I’ll come across the printout again sometime so I can quote the exact numbers. But they were pretty low, which led me to believe that it didn’t have the specified settings even from the factory.
As has been the experience of many others here over the years as well.
So while it is a culprit, we always have to look at the entire package and optimize everything.
Sounds like you’ve already done most of that, but maybe something is still missing.
And maybe that something is as simple as getting that alignment checked.
Well, we can hope anyway!