Unfortunately some people are taking you seriously. The whole scrolls are a huge amount of data, but for this First Letters prize using more than one GPU would be overkill. The amount of training data available is actually too small for deep learning to even work well.

That's the capital cost of the cluster, not of the compute cost of this operation. Significant and cool, but it over-represents the amount of compute required.

Where do they say that the winners used that cluster?

It is an assumption based on the fact that the codebase uses cuda and the main backer of the project owns the cluster.

> It is an assumption

Then don't say "certainly"

Why not? If they had bought the compute themselves, it might cost more than $700k.

I am absolutely certain that the compute might have cost more than 100 trillion dollars.

Many amateurs compete in Kaggle, most of them use whatever hardware they have to hand, and a lot of them will use CUDA directly or indirectly.