On the street there is not as much debris going to get in anyway, but over time it's still a thing. But off-road and especially if you don't have slingers on the inner axles to block some of it, a whole lotta gunk can find it's way into the tube. Most of it just sits there being ugly and eventually helping the rust thrive, but it's not that detrimental even in the long run. It's the stuff that gets down to the inner seals that is what you like to avoid.
Not an earth shattering problem either way, but if my axle tube accepted outer seals, I'd use them.
Paul