>Can't you do this in Rust by writing a #[derivative] proc macro without having to modify the language?
That question can be generalized to "since <Turing Complete language> can do <anything-we-want-via-library/function/macro/codegen/customcode/extension/whatever>, can't we just do that instead of modifying the core language?"
Well yes, but the advantage of having 1st-class language syntax includes:
+ more helpful compiler error messages because it has intelligence of the semantics
+ more helpful IDE because the UI has intelligence about the intentions of the programmer
+ interoperability because programmers are all using the same syntax instead of Programmer Bob using his custom macro-dialect and Progammer Alice using her idiosyncratic macro-dialect.
Points 1 and 2 are about introspection, which are hugely language- and implementation-dependent. IME, the languages with great tooling and error messages didn't achieve it by integrating every possible feature into the compiler. (Usually, it's closer to the opposite.) Even different compilers for exactly the same language differ greatly in their ability to extract intention, and present it to users in a comprehensible way.
Point 3 is simply that programmers aren't sharing interfaces, which is indeed a shame, but that's independent of whether you choose to put the interface in the language or the library. Didn't TensorFlow fork the Swift compiler?
> That question can be generalized to "since <Turing Complete language> can do <anything-we-want-via-library/function/macro/codegen/customcode/extension/whatever>, can't we just do that instead of modifying the core language?"
Not really, since that usually requires using weird syntax or APIs, but the Rust proc-macro solution has the exact same syntax as the Swift one:
@differentiable
...cube function
let grad = gradient(at: cube);
vs
#[derivative]
...cube function....
let grad = gradient(cube);
>, but the Rust proc-macro solution has the exact same syntax as the Swift one
You're focusing on syntax but I was talking about the higher-level semantics. When programmers write custom code (e.g. custom macros), as far as the compiler/IDE is concerned, it's just an opaque string that it checks for valid syntax. Those tools have no "intelligence" of what the higher concepts of derivatives/deltas etc.
E.g. the IntelliJ plugin for Rust will have no idea what a the higher semantics of a differentiable function is. It just sees a recursive "gradient()" which is just an arbitrary string of code to parse and it might as well be spelled "abcdefgzyxwvut123()".
I.e. the blog post is talking about an symbiotic ecosystem of tools (e.g. compilers, IDEs, etc) that treats differentiable programming as a 1st-class language feature. The ultra flexibility of Rust or Lisp macros to reproduce the exact syntax doesn't really solve the semantics understood by the ecosystem of tools.
edit reply to: >You seem to be at least suggesting that if one adds differentiable programming as a first class language feature, IDEs would automatically be intelligent about it,
No, that's an uncharitable interpretation. I never claimed that IDEs will automagically understand higher level semantics with no extra work required to enhance the IDE's parser.
I'm only saying that your macro proposal only recreates the syntax but not the semantics -- so you're really not fully solving what the blog author is talking about.
You seem to be at least suggesting, that if one adds differentiable programming as a first class language feature, IDEs would either automatically be intelligent about it, or that making these tools know about the feature would somehow be simpler.
That's not true. IDEs would need to add support for that feature, and whether the feature is a language feature, or a custom Rust macro, for Rust at least, the amount of work required is the same.
Rust IDEs already understand many ecosystem macros (cfg_if, serde derives, etc.). Somebody would just need to put in the work for a new macro, but that would need to be the case anyways.
The same would apply to Rust static analysis tools like clippy. Somebody would need to write lints, and whether those target an ecosystem macro or a keyword doesn't really matter with respect to the amount of work that this requries.
> I'm only saying that your macro proposal only recreates the syntax but not the semantics
What do you think these semantics are? The only semantics I see is being able to compute the derivative of a function at some particular point (for some particular inputs). The proc macro solution has this semantics: it automatically computes the derivative of a function, and gives you a way to query its value at a particular point.
> No, that's an uncharitable reading. I never claimed that IDEs will automagically understand higher level semantics.
So what are you saying? Because that's what I'm honestly understanding from your post.
That somehow this feature being a Rust macro means that it cannot be as good as if it was a first class feature. You are making this claim in general, as if first class features are always strictly better than user-defined features, yet this claim isn't true, and you haven't mention any particular aspect of automatic differentiation for which this is the case.
Swift doesn't have proc macros, so they have to make the feature first class. For languages with proc macros, the question that one should be asking is: what value does a first class language feature adds over just using a proc macro.
That question can be generalized to "since <Turing Complete language> can do <anything-we-want-via-library/function/macro/codegen/customcode/extension/whatever>, can't we just do that instead of modifying the core language?"
Well yes, but the advantage of having 1st-class language syntax includes:
+ more helpful compiler error messages because it has intelligence of the semantics
+ more helpful IDE because the UI has intelligence about the intentions of the programmer
+ interoperability because programmers are all using the same syntax instead of Programmer Bob using his custom macro-dialect and Progammer Alice using her idiosyncratic macro-dialect.