Before I did my CAGE 3.5" lift, there was a hodge-podge of stuff the previous owner had done, but it drove pretty well with about 2.5 to 3 degrees of caster and 33x12.5 tires, but didn't return to center as hard as I'd have liked, especially for small steering inputs.
After the CAGE 3.5" lift, long-arms and 4 degree bushings, I gained about 1.5 degrees of caster and it drove quite well so long as I kept it in 2WD. In 4WD, the front pinion angle issue became readily apparent at anything over 10-15 MPH, and I avoided running around in 4WD at speed due to the unsettling vibration coming from the front shaft.
Since you're going to be running in 4WD at speed on the highway for extended periods of time, I'd again suggest that you turn your knuckles and fix both your caster and pinion angles at the same time so that you can just be done with it. Keep in mind that you may need to rework your track bar mounts at the axle end when you roll the pinion up...I'm swapping in a different axle, so track bar mounts and steering linkages were already on the list for me.