home.social

#slidingwindowattention — Public Fediverse posts

Live and recent posts from across the Fediverse tagged #slidingwindowattention, aggregated by home.social.

fetched live
  1. Korea @[email protected] ·

    Kakao Open-Sources ‘Kanana-2’ Lightweight AI Models with 30% Better Korean

    Performance comparison of four lightweight language models Kakao released as open source on the 28th. Photo courtesy of…
    #EuropeSays #Korea #KR #Kakao #HuggingFace #Kanana-2 #Korean-specializedtokenizer #On-deviceAI #open-sourceAI #slidingwindowattention #smalllanguagemodel
    europesays.com/korea/101933/

    #smalllanguagemodel #slidingwindowattention #open #on #korean #kanana
  2. AI Daily Post @[email protected] ·

    New research shows how Test‑Time Training equips Transformers with a dual‑memory system: a sliding‑window for short‑term context and a persistent long‑term store, all while keeping inference cheap. The TTT‑E2E framework uses targeted weight updates to boost performance on streaming data. Open‑source friendly, it could reshape real‑time NLP pipelines. #TestTimeTraining #DualMemory #Transformers #SlidingWindowAttention

    🔗 aidailypost.com/news/test-time

    #testtimetraining #dualmemory #transformers #slidingwindowattention