• Nibodhika@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    arrow-down
    1
    ·
    3 months ago

    Every time I read something like this I realize just how little people understand how these things work.

    The tiniest LLMs you can run on your machine are 1B, those take around 10TB of unrepeatable and well varied text to train. Let me put this into perspective, if you downloaded the whole Wikipedia, you would get 1% of the amount of data needed to train the tiniest LLM you can run. Do you think you can ethically source the remaining 99%?