There are multiple tools to do this on your own, with CPAN::Mini and friends, or Carton.
I also don't know how long it's been since you last tried, but cpantesters and the other odds and ends of the ecosystem have helped tremendously in reducing published bugs. ( http://matrix.cpantesters.org/?dist=System-Command%201.115 )
Checkout Pinto, http://perlmaven.com/pinto-tutorial, which allows you to host your own CPAN mirror and have stacks. So you could have a "development" stack and a "production" stack, or a "perl-5.8" stack and a "perl-5.16" stack. Whenever you add or upgrade a module, it only affects one stack.
I also don't know how long it's been since you last tried, but cpantesters and the other odds and ends of the ecosystem have helped tremendously in reducing published bugs. ( http://matrix.cpantesters.org/?dist=System-Command%201.115 )