“If man chooses oblivion, he can go right on leaving his fate to his political leaders. If he chooses Utopia, he must initiate an enormous education program - immediately, if not sooner.”
-R B Fuller

  • 1 Post
  • 13 Comments
Joined 1 year ago
cake
Cake day: August 25th, 2025

help-circle







  • We train in bursts because we need a ground truth/scoring function to guide the learning

    These are completely orthogonal things.

    We train in bursts (“epochs”) because empirically it leads to more rapid convergence on an acceptable error minimum.

    The fact that a scoring function is needed is almost a tautology. You have to have some differentiable function to say what direction is lower error. Typically you segment your training corpus, and use the scoring function to get the error for one batch at a time to a certain threshold before proceeding to the next batch. Then go to the first batch and push the error bar even lower, and so on.

    Kinda like tightening nuts on a car wheel. You don’t fully tighten the first one first; you have to wiggle the system into its final configuration.


  • Can you prove that humans don’t?

    This, this this. This is something the prevailing armchair opinion is really missing. Their blatantly-wrong assumptions about human cognition.

    It feels as proudly ignorant as my grandmother’s generation who proudly declared Evolution to be patently ridiculous (“my grerat-gramma was no monkey swingin’ from a tree!”)

    I am not a researcher in the area, but I am the kind of nerd who watches university colloquia and sometimes does in fact read the paper.

    “I don’t hallucinate all the time”

    YES, YES YOU DO, do you not understand how the sensory system works? You interpolate between inputs. This is why optical illusions work! Your fovea is a tiny tiny spot and is the only area that’s actually in focus!

    Learning by statistical sampling, reward/repress feedback loops, production system-based computation giving rise to “fast” (System I) thinking, “hallucinating” to interpolate between sensory inputs … the list goes on and on