A student with their times tables down isn't going to do any better in a calculus course than one who relies on a calculator.
A linear algebra course, on the other hand, is going to make mincemeat out of any student who can't do basic arithmetic in their head at high speed. Gaussian elimination, matrix multiplication, determinants, diagonalization; all of these tasks are extremely arithmetic-intensive. If someone needs a calculator to multiply 8x6 then they are not going to be able to solve a linear system in 4 unknowns on an exam that disallows calculators (which is all math courses at my university, besides stats and act sci).
Such a math course is being taught wrong if passing a test relies on churning through a million arithmetic steps instead of demonstrating understanding. I did fine in my course. Had to use the technique of mentally placing objects around a familiar path to memorize a silly list of matrix properties though.
The computational part was only 60% of the exams. The rest was proofs. For most of us, we needed every mark we could get in the computations because the proofs were really hard.
A linear algebra course, on the other hand, is going to make mincemeat out of any student who can't do basic arithmetic in their head at high speed. Gaussian elimination, matrix multiplication, determinants, diagonalization; all of these tasks are extremely arithmetic-intensive. If someone needs a calculator to multiply 8x6 then they are not going to be able to solve a linear system in 4 unknowns on an exam that disallows calculators (which is all math courses at my university, besides stats and act sci).