home.social
Sign in Create account

#scientivism — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #scientivism, aggregated by home.social.

  1. Leshem Choshen @[email protected] ·

    Solar mixes two base-model copies
    to create a larger one
    Then train it a bit more and beat other open models out there.
    How? and my thoughts 🧵

    (no author with a handle?!)
    arxiv.org/abs/2312.15166
    #scientivism #LLM #LLMS #pretraining #MIXTRAL #SOLAR #machinelearning #ml #NLP

    #scientivism #llm #llms #pretraining #mixtral #solar
  2. Leshem Choshen @[email protected] ·

    We want to pretrain🤞
    Instead we finetune🚮😔
    Could we collaborate?🤗

    ColD Fusion:
    🔄Recycle finetuning to multitask
    ➡️evolve pretrained models forever

    On 35 datasets
    +2% improvement over RoBERTa
    +7% in few shot settings
    🧵

    #NLProc #MachinLearning #NLP #ML #modelRecyclying #collaborativeAI #scientivism #pretrain

    #nlproc #machinlearning #nlp #ml #modelrecyclying #collaborativeai