A picture of your setup would help. But in lieu of that, how steep is the angle? A 3.5" lift should not be overly steep. I've found that when they are, the rear pinion is pointing too far down and needs corrective measures such as shims under the leaf springs.
Another thing, probably one of the most common reasons actually, is the slip joint in the driveshaft binding up. The only way I could get rid of my clunk was to pull it apart and manually grease the splines.
However, most of the time I think a slip joint is more of a clunk when you start. But not so much when you let off the gas.
So you'll have to be the one to do all the detective work. We can't see it or hear it, so it's all on you. We can only point you in different directions.
One of those directions would be to do with the body lift.
Look at your transfer case shifter to make sure it's not just touching the tunnel where it comes through.
Where is the exhaust run? Any part of it close to the body or frame?
What about the back of the driver's side cylinder head? Close to the firewall?
Steering shaft? Do you feel the clunk through the wheel, or is it all in the seat-of-the-pants coming through the floor?
Paul