In my experience, the stock carrier is OK if you plan on carrying a stock (or close to it) spare....anything larger really seems to stress the carrier, making it rattle & squeak even more than it did before. Yes, it can pull the rear corner from the qtr panel. Also, I've had a few that had to be tied, bolted, or otherwise "doubly" secured as the latch mechanism was worn out & the thing would pop loose just driving down the road.
Having said all that, I've still got the stock carrier on my '73 that's carried 32" & 33" spares for the last 8+ years without much hassle. Until recently it saw mostly on-road use and very light trails. First real trail I took it on, sure enough, I had issues. Since I hope to do more off-roading with it, I do plan to upgrade to something else to make it more solid, secure, & take all of that stress off of the body.