I think what it's referring to is this idea that it's quite beautiful, somehow, that all of mathematics arises with just a very few premises. You don't need to memorize because nothing is arbitrary, everything naturally pops out once you accept just the bare minimum axioms.
Compare that to something like, say, specializing in European history. In European history, no amount of axioms of history can help you derive who was the monarch of Luxembourg in 1722. You either know the fact, or you do not.
There's something beautiful about the fact that in Mathematics, all truths are in principle accessible and derivable.