I like to think of it like solar panels. Sure at first it takes a lot to make, but over time it evens out. I was pretty clear local AI is still morally grey at best, but its a real solution to some problems, an I would argue its better to have local control and use small energy efficient models to do assisting tasks than let the open AI’s of the world decide for us.
It’s strange to think selfhosted models are systematically more efficient than cloud hosted ones, have you never heard of economy of scale? Of batching?
While using minimal energy to run, but not minimal energy to make.
I like to think of it like solar panels. Sure at first it takes a lot to make, but over time it evens out. I was pretty clear local AI is still morally grey at best, but its a real solution to some problems, an I would argue its better to have local control and use small energy efficient models to do assisting tasks than let the open AI’s of the world decide for us.
It’s strange to think selfhosted models are systematically more efficient than cloud hosted ones, have you never heard of economy of scale? Of batching?