Arbitrary backtracking in a recursive descent parser is very easy with exceptions.
However, there's also an argument here that if the grammar is too complicated to be parsed with recursive descent, it's probably just too complicated in general and should be simplified if possible. Obviously you don't always have this option when you're dealing with an external grammar, but for your own PL, you can design around that. Most Wirth's languages are good examples; Pascal is famously LL(1).
Recursive descent parsers can have ambiguities, but like PEGs, they just resolve the ambiguity by arbitrarily picking one of the alternatives.
A famous example is ALGOL 60, which had the dangling else ambiguity (https://en.wikipedia.org/wiki/Dangling_else), but this was not discovered until the language had already been published. If they had been using a parser generator tool, it would have warned them that the grammar was not LL or LR.
Overall I do still prefer hand-written recursive descent parsers, but I do find this to be one of the biggest downsides of not using parser generators.
Of course the Algol 60 committee could not have used a parser generator tool because they were not invented until later in the 1960s, but your point is valid for anyone developing a language after that time.
However, there's also an argument here that if the grammar is too complicated to be parsed with recursive descent, it's probably just too complicated in general and should be simplified if possible. Obviously you don't always have this option when you're dealing with an external grammar, but for your own PL, you can design around that. Most Wirth's languages are good examples; Pascal is famously LL(1).