Using handles instead of proper references is still a huge hack IMO, because you lose all the soundness checks that the compiler can do for references. It's a good thing that immovable objects are coming soon.
You can, but that only solves part of the problem: keeping pointer types from being used as the wrong type. You still have the equivalent of use-after-free to guard against which the lifetime system tracks for you for pointers, but not for handles.