Yes, but they may not always work. While generally true there are still some edge cases.

SQL Server connections are one example where I do get .exe with the .pdb in the publish directory but the .exe won't run correctly without the "Microsoft.Data.SqlClient.SNI.dll" file.

Another example are any libraries that have "RequiresDynamicCodeAttribute" requirements.