For a while the GOOS=nacl port and the Google App Engine ports of Go disallowed unsafe pointer manipulation too, so there is some precedent. Throughout some of the ecosystem you can see pieces of "nounsafe" build tag support (e.g. in easyjson).
Most programming languages that offer unsafe, either as language keyword, or meta package (unsafe/SYSTEM/UNSAFE whatever the name), have similar option, that doesn't make it less of a feature.