Are they throwing away 40-50 years of work on things like LAPACK/BLAS? Or do they still build on the basic foundations and implement higher level things in Julia?
No --- I thought about adding that caveat but didn't. Most of the non-Julia code that I'm aware of is scientific libraries: linear algebra (LAPACK/BLAS), random number generators, etc. But lots of very basic foundations and data types are straight Julia --- array addition (as a very trivial example) is implemented by looping over the elements of the arrays in Julia, not by passing the loop to a C library.[1]
Reimplementing LAPACK or BLAS would be a lot of unnecessary work, but I think that one goal of the language is to be fast enough that a reimplementation would not be worse than the existing versions. (I'm not affiliated w/ the project, though, so I'm just guessing.)
There's an embedding API that's not too bad to work with. There are pyjulia and rjulia projects, but they're not quite as actively developed as the other directions of PyCall.jl or RCall.jl. Making standalone libraries out of Julia code isn't easy to do yet, but should eventually be better-supported.