You might also enjoy ranger[0] which is a console file manager with vi key bindings, I use it as my primary file manager and it's pretty convenient when SSHing as well.
Edit: I should also clarify that if I will work with a remote directory for a non-trivial amount of operations or time, I just mount it locally via sshfs[1] which also saves me the pain of having to install ranger on the remote machine.
Edit: I should also clarify that if I will work with a remote directory for a non-trivial amount of operations or time, I just mount it locally via sshfs[1] which also saves me the pain of having to install ranger on the remote machine.
[0] https://news.ycombinator.com/item?id=10090834
[1] http://fuse.sourceforge.net/sshfs.html