home.social

#babylm — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #babylm, aggregated by home.social.

  1. LTG Oslo @[email protected] ·

    4. #BabyLM challenge description paper, co-authored by Lucas Georges Gabriel Charpentier

    babylm.github.io/

    #babylm
  2. LTG Oslo @[email protected] ·

    4. #BabyLM challenge description paper, co-authored by Lucas Georges Gabriel Charpentier

    babylm.github.io/

    #babylm
  3. Leshem Choshen @[email protected] ·

    👶BabyLM will be back, but what did we learn?
    the best papers from this year🍼

    We know one mechanism that learns from 100M words (us), what are the main boosts to reach this in an LLM?

    #EMNLP #babyLM #LLMs #ML #machinelearning #NLProc #NLP

    #emnlp #babylm #llms #ml #machinelearning #nlproc
  4. Leshem Choshen @[email protected] ·

    👶BabyLM will be back, but what did we learn?
    the best papers from this year🍼

    We know one mechanism that learns from 100M words (us), what are the main boosts to reach this in an LLM?

    #EMNLP #babyLM #LLMs #ML #machinelearning #NLProc #NLP

    #emnlp #babylm #llms #ml #machinelearning #nlproc
  5. Leshem Choshen @[email protected] ·

    👶BabyLM will be back, but what did we learn?
    the best papers from this year🍼

    We know one mechanism that learns from 100M words (us), what are the main boosts to reach this in an LLM?

    #EMNLP #babyLM #LLMs #ML #machinelearning #NLProc #NLP

    #emnlp #babylm #llms #ml #machinelearning #nlproc
  6. Leshem Choshen @[email protected] ·

    👶BabyLM will be back, but what did we learn?
    the best papers from this year🍼

    We know one mechanism that learns from 100M words (us), what are the main boosts to reach this in an LLM?

    #EMNLP #babyLM #LLMs #ML #machinelearning #NLProc #NLP

    #nlp #nlproc #machinelearning #ml #llms #babylm
  7. Leshem Choshen @[email protected] ·

    👶BabyLM will be back, but what did we learn?
    the best papers from this year🍼

    We know one mechanism that learns from 100M words (us), what are the main boosts to reach this in an LLM?

    #EMNLP #babyLM #LLMs #ML #machinelearning #NLProc #NLP

    #emnlp #babylm #llms #ml #machinelearning #nlproc
  8. Leshem Choshen @[email protected] ·

    Residual connections are 🔥, right?
    Wait, so why do we only use them to skip 1 layer?

    Not only Lucas Georges Gabriel Charpentier &
    @davidsamuelcz checked it.

    They found that this provided huge gains - winning the babyLM challenge🍼

    arxiv.org/abs/2311.02265
    #babyLM #LLMs #LLM #GPT #NLP #ML #machinelearning #nlproc #NLP #AI #generativeAI #pretraining

    #babylm #llms #llm #gpt #nlp #ml
  9. Leshem Choshen @[email protected] ·

    Residual connections are 🔥, right?
    Wait, so why do we only use them to skip 1 layer?

    Not only Lucas Georges Gabriel Charpentier &
    @davidsamuelcz checked it.

    They found that this provided huge gains - winning the babyLM challenge🍼

    arxiv.org/abs/2311.02265
    #babyLM #LLMs #LLM #GPT #NLP #ML #machinelearning #nlproc #NLP #AI #generativeAI #pretraining

    #babylm #llms #llm #gpt #nlp #ml
  10. Leshem Choshen @[email protected] ·

    Residual connections are 🔥, right?
    Wait, so why do we only use them to skip 1 layer?

    Not only Lucas Georges Gabriel Charpentier &
    @davidsamuelcz checked it.

    They found that this provided huge gains - winning the babyLM challenge🍼

    arxiv.org/abs/2311.02265
    #babyLM #LLMs #LLM #GPT #NLP #ML #machinelearning #nlproc #NLP #AI #generativeAI #pretraining

    #babylm #llms #llm #gpt #nlp #ml
  11. Leshem Choshen @[email protected] ·

    Residual connections are 🔥, right?
    Wait, so why do we only use them to skip 1 layer?

    Not only Lucas Georges Gabriel Charpentier &
    @davidsamuelcz checked it.

    They found that this provided huge gains - winning the babyLM challenge🍼

    arxiv.org/abs/2311.02265
    #babyLM #LLMs #LLM #GPT #NLP #ML #machinelearning #nlproc #NLP #AI #generativeAI #pretraining

    #pretraining #generativeai #ai #nlproc #machinelearning #ml
  12. Leshem Choshen @[email protected] ·

    Residual connections are 🔥, right?
    Wait, so why do we only use them to skip 1 layer?

    Not only Lucas Georges Gabriel Charpentier &
    @davidsamuelcz checked it.

    They found that this provided huge gains - winning the babyLM challenge🍼

    arxiv.org/abs/2311.02265
    #babyLM #LLMs #LLM #GPT #NLP #ML #machinelearning #nlproc #NLP #AI #generativeAI #pretraining

    #babylm #llms #llm #gpt #nlp #ml
  13. Leshem Choshen @[email protected] ·

    Welcome the new babies!
    19 pretrained models on the loose track
    24 on the strict
    118 on strict-small
    dynabench.org/babylm

    We are proud of >30 pretraining teams submitting papers to babyLM!

    FOMO?
    Get updated on CoNLL or
    participate next year
    babylm.github.io

    #NLP #nlproc #babyLM #CoNLL #machinelearning #llm #llms #pretraining

    #nlp #nlproc #babylm #conll #machinelearning #llm
  14. Leshem Choshen @[email protected] ·

    Welcome the new babies!
    19 pretrained models on the loose track
    24 on the strict
    118 on strict-small
    dynabench.org/babylm

    We are proud of >30 pretraining teams submitting papers to babyLM!

    FOMO?
    Get updated on CoNLL or
    participate next year
    babylm.github.io

    #NLP #nlproc #babyLM #CoNLL #machinelearning #llm #llms #pretraining

    #nlp #nlproc #babylm #conll #machinelearning #llm
  15. Leshem Choshen @[email protected] ·

    Welcome the new babies!
    19 pretrained models on the loose track
    24 on the strict
    118 on strict-small
    dynabench.org/babylm

    We are proud of >30 pretraining teams submitting papers to babyLM!

    FOMO?
    Get updated on CoNLL or
    participate next year
    babylm.github.io

    #NLP #nlproc #babyLM #CoNLL #machinelearning #llm #llms #pretraining

    #nlp #nlproc #babylm #conll #machinelearning #llm
  16. Leshem Choshen @[email protected] ·

    Welcome the new babies!
    19 pretrained models on the loose track
    24 on the strict
    118 on strict-small
    dynabench.org/babylm

    We are proud of >30 pretraining teams submitting papers to babyLM!

    FOMO?
    Get updated on CoNLL or
    participate next year
    babylm.github.io

    #NLP #nlproc #babyLM #CoNLL #machinelearning #llm #llms #pretraining

    #pretraining #llms #llm #machinelearning #conll #babylm
  17. Leshem Choshen @[email protected] ·

    Welcome the new babies!
    19 pretrained models on the loose track
    24 on the strict
    118 on strict-small
    dynabench.org/babylm

    We are proud of >30 pretraining teams submitting papers to babyLM!

    FOMO?
    Get updated on CoNLL or
    participate next year
    babylm.github.io

    #NLP #nlproc #babyLM #CoNLL #machinelearning #llm #llms #pretraining

    #nlp #nlproc #babylm #conll #machinelearning #llm
  18. Leshem Choshen @[email protected] ·

    TinyStories: Tiny models are coherent and understand instructions
    If their data is very simple

    What is simple?
    What 3-4 year old vocabularies allow (according to LLMs...)

    arxiv.org/abs/2305.07759
    #NLProc #LLM #babyLM

    #nlproc #llm #babylm
  19. Leshem Choshen @[email protected] ·

    TinyStories: Tiny models are coherent and understand instructions
    If their data is very simple

    What is simple?
    What 3-4 year old vocabularies allow (according to LLMs...)

    arxiv.org/abs/2305.07759
    #NLProc #LLM #babyLM

    #nlproc #llm #babylm
  20. Leshem Choshen @[email protected] ·

    TinyStories: Tiny models are coherent and understand instructions
    If their data is very simple

    What is simple?
    What 3-4 year old vocabularies allow (according to LLMs...)

    arxiv.org/abs/2305.07759
    #NLProc #LLM #babyLM

    #nlproc #llm #babylm
  21. Leshem Choshen @[email protected] ·

    TinyStories: Tiny models are coherent and understand instructions
    If their data is very simple

    What is simple?
    What 3-4 year old vocabularies allow (according to LLMs...)

    arxiv.org/abs/2305.07759
    #NLProc #LLM #babyLM

    #babylm #llm #nlproc
  22. Leshem Choshen @[email protected] ·

    TinyStories: Tiny models are coherent and understand instructions
    If their data is very simple

    What is simple?
    What 3-4 year old vocabularies allow (according to LLMs...)

    arxiv.org/abs/2305.07759
    #NLProc #LLM #babyLM

    #nlproc #llm #babylm