So soft-PSA: the following is more than a little misleading:
“The fact that it can compress such an enormous quantity of visual information into such a small space is itself a fascinating detail.”
This is not a detail: it’s the principle mechanism. The ability to compress something is conferred by the identification and exploitation of structure, conversely the scarcity or absence of structure inhibits or prohibits compression. You can eyeball check an RNG with compression techniques.
This has counter-intuitive consequences that you can test on your laptop! Even using off-the-shelf codecs it only takes a modest corpus to see that pop music compresses better than eclectic jazz, which compresses better than white noise. The same thing holds for headshots of people: a big pile of headshots drawn from a reasonably broad corpus of humans will enjoy a noticeably lower compression ratio than a subset selected by any plausible “conventional attractiveness” filter. “Conventional attractiveness” (defined any common-sense way) correlates sharply with bilateral symmetry, with obvious implications for storage space.
Information theory is the thread that ties together all this AI craze stuff!
“The fact that it can compress such an enormous quantity of visual information into such a small space is itself a fascinating detail.”
This is not a detail: it’s the principle mechanism. The ability to compress something is conferred by the identification and exploitation of structure, conversely the scarcity or absence of structure inhibits or prohibits compression. You can eyeball check an RNG with compression techniques.
This has counter-intuitive consequences that you can test on your laptop! Even using off-the-shelf codecs it only takes a modest corpus to see that pop music compresses better than eclectic jazz, which compresses better than white noise. The same thing holds for headshots of people: a big pile of headshots drawn from a reasonably broad corpus of humans will enjoy a noticeably lower compression ratio than a subset selected by any plausible “conventional attractiveness” filter. “Conventional attractiveness” (defined any common-sense way) correlates sharply with bilateral symmetry, with obvious implications for storage space.
Information theory is the thread that ties together all this AI craze stuff!