I think you are rather dramatically underestimating what goes into the current systems of robotics. Curiosity and Perseverance are the result of decades of research by teams of hundreds of scientists with decades of work and research and budgets in the billions of dollars. 'We just need to make them orders of magnitude better' is nonsensical. You're looking at the best that we can do, and it's nowhere even remotely near to what's needed for robotics to be effective. And fragility is just one aspect - mobility, power, and many others also lag far behind what they'd need for robotics to be viably competitive with humans.

So now you're left with some argument that ends up like 'Okay - so we can't do it with robotics right now, but I want to do it with robotics, so we shouldn't do it until we can do it with robotics.' I mean the answer is quite obvious. Humans can do it, humans want to do it, and thus humans will do it. I don't entirely understand why you're so averse to this. To not be coy, one obvious explanation is a political thing about Musk. Outside of that being petty, it's also largely irrelevant - China's a major player in this game as well, and it's conceivable that they get there before we do. And I don't think this is just going to be a two pony race this time either, especially when the goal isn't just footprints and flags.