#humanloop — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #humanloop, aggregated by home.social.
-
Based on a now deleted interview of #SamAltman by #Humanloop it seems like OpenAI struggles to keep up with progress of the field. The factuality of the notes are questionable though, considering they were deleted since publication.
Things I would lift up from these:
- The post claims that Altman said that they are going to work on producing a session aware #ChatGPT API to reduce costs. This is surprising to me because it is trivial to cache #LLM state and retrieve it on subsequent requests. This would be one of the first things any machine learning engineer would do. I don't believe for a second they don't already do this so there must be some sort of a miscommunication here.
- Being limited by #GPU availability is certainly true, but if the other notes are correct they are also heavily limited by number of engineers. See the next point.
- There is a claim that they don't have a solution for quadratic scalability of #Transformer architectures. There are numerous solutions to it published. It is just about summarizing and bottlenecking the recurrent state representation. They don't yet seem to use any of those solutions in the currently available models though. But they are nearly trivial to deploy. Don't they have enough engineers to do that?
- Another claim is that OpenAI doesn't use #LoRa for fine-tuning. Why not? Again, trivial to deploy.However, let's remember that they are a tiny company, and they don't have enough engineers to do these things apparently.
If these claims are true, and I suspect they aren't completely true, it would imply that #OpenAI is already being left behind by even open source, let alone competing organizations with more machine learning engineers on deck.
All in all, it seems people buying billions of dollars worth of aftermarket Microsoft stock in a frenzy didn't lead to any part of that money to go into actually hiring or training machine learning engineers.
That should be surprising to no one.
People, invest your money. I mean, make your money actually do machine learning work. Don't buy collector hockey cards of corporations and play casino with them. Those are only depictions of work, not work.
-
Based on a now deleted interview of #SamAltman by #Humanloop it seems like OpenAI struggles to keep up with progress of the field. The factuality of the notes are questionable though, considering they were deleted since publication.
Things I would lift up from these:
- The post claims that Altman said that they are going to work on producing a session aware #ChatGPT API to reduce costs. This is surprising to me because it is trivial to cache #LLM state and retrieve it on subsequent requests. This would be one of the first things any machine learning engineer would do. I don't believe for a second they don't already do this so there must be some sort of a miscommunication here.
- Being limited by #GPU availability is certainly true, but if the other notes are correct they are also heavily limited by number of engineers. See the next point.
- There is a claim that they don't have a solution for quadratic scalability of #Transformer architectures. There are numerous solutions to it published. It is just about summarizing and bottlenecking the recurrent state representation. They don't yet seem to use any of those solutions in the currently available models though. But they are nearly trivial to deploy. Don't they have enough engineers to do that?
- Another claim is that OpenAI doesn't use #LoRa for fine-tuning. Why not? Again, trivial to deploy.However, let's remember that they are a tiny company, and they don't have enough engineers to do these things apparently.
If these claims are true, and I suspect they aren't completely true, it would imply that #OpenAI is already being left behind by even open source, let alone competing organizations with more machine learning engineers on deck.
All in all, it seems people buying billions of dollars worth of aftermarket Microsoft stock in a frenzy didn't lead to any part of that money to go into actually hiring or training machine learning engineers.
That should be surprising to no one.
People, invest your money. I mean, make your money actually do machine learning work. Don't buy collector hockey cards of corporations and play casino with them. Those are only depictions of work, not work.