You are almost right. With enough RAM, a system will work perfectly without swap. There is, however, a small performance penalty. Even with enough RAM, Linux will swap out processes, trying to free memory for use as disk cache. On most cases, this yields performance gains, which would be lost if the swap file is disabled.