There are a lot of mis-understandings about the steering damper. If you are putting one on to try and fix a shimmy, you are not fixing the problem and probably not even hiding it very well. It's not power steering, won't make the steering any easier. Nor should a properly installed one make the steering any more difficult.
It is a damper. It arrests spike inputs. I remember reading an article from the 60s where ford was putting them on as factory equipment because impacts were flattening the needle bearings in the steering box. Considering the price of a good steering box these days, cheap insurance. It also take those impact loads away from the frame where the steering box mounts to, reduced chances of frame cracking. Those are all wins.
So you took it off and never noticed it was gone. That is entirely possible. I broke the frame side bracket one weekend and never noticed it on the street. It was only off-road (rocky AZ trails) where one front tire would clip a rock where I would notice it. For me, it was a violent spin of the steering wheel that didn't exist with the damper working. Not that it would stop the spin of the steering wheel, just reduce the violence of it. I liked my steering box, I wanted it to live, so I fixed the broken bracket.
Going to a larger steering box, say the 4x4x2, it probably has enough hydraulic ram inside to dampen the inputs to the steering wheel. But they are still hitting the frame rail.
The one place I would call a steering damper a misplaced accessory is if there is ram assist. The steering ram will take the impact load before it gets through the linkage and up to the box.