Raito Kiya

I am a second-year master's student at the Graduate School of Information Sciences, Tohoku University, and a member of the Tohoku NLP Group (Sakaguchi-Inui Lab).

Coffee siphon

Research Interest

  • Artificial Intelligence
  • Natural Language Processing
  • Large Language Models (LLMs)
    • Internal Mechanisms
    • Attention Sink
    • Massive Activations

Publications

International Conferences (Peer-Reviewed)

It’s Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention
Raito Kiya, Satoki Ohashi, Kosuke Sato, Go Kamoda, Ryosuke Takahashi, Yuji Yamamoto, Daiki Shiono, Keisuke Sakaguchi, Goro Kobayashi
The 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP 2026) (Acceptance rate: 15.4%)
2026
arXiv EMNLP (coming soon)

Domestic Conferences

Attention SinkおよびMassive Activationの発生機序の分解 🏆 Young Researcher Encouragement Award
木谷 頼斗, 大橋 諭貴, 佐藤 宏亮, 鴨田 豪, 高橋 良允, 山本 悠士, 塩野 大輝, 坂口 慶祐, 小林 悟郎
言語処理学会第32回年次大会 (NLP2026)
March 2026
paper
注意機構における Attention Sink のバイアス項的解釈 🏆 Young Researcher Encouragement Award
大橋 諭貴, 木谷 頼斗, 佐藤 宏亮, 高橋 良允, 鴨田 豪, 山本 悠士, 塩野 大輝, 坂口 慶祐, 小林 悟郎
言語処理学会第32回年次大会 (NLP2026)
March 2026
paper
説明文生成による予測精度の向上に着目した解釈可能なフェイクニュース検出
木谷 頼斗，張 建偉
令和6年度情報処理学会東北支部研究会 (IPSJ-TOHOKU)
February 2025

Experiences

NLP Engineer (Part-time) Future Corporation
December 2025 - Present
Assistant for developing online experimental environments for LLMs in Japanese NLP (Administrative Assistant) Graduate School of Information Sciences, Tohoku University
July 2025 - March 2026
Data Scientist (Internship) NTT EAST, Inc.
September 2025
Research & Development in NLP (Internship) Future Corporation
August 2025 - September 2025

Awards / Grants / Qualifications

Young Researcher Encouragement Award Attention SinkおよびMassive Activationの発生機序の分解
言語処理学会第32回年次大会 (NLP2026) (21/517 = 4.1%)
March 2026
Kusakari Award Awarded to students at Iwate University (Studies in Computer, Intelligence and Media Technology or Studies in Electrical, Electronic, and Communication Engineering) who lead an exemplary student life and serve as distinguished role models for their juniors.
March 2025
Grant for Bachelor's Thesis Research in Fiscal Year 2024 Iwate Katada Foundation
Nozaki Wakaba Foundation Competitive Scholarship (April 2023 - March 2024).
Nissho Bookkeeping Test 2nd Grade

Education

Master of Information Science Graduate School of Information Sciences, Tohoku University
April 2025 - Present
Bachelor of Science and Engineering Department of Systems Innovation Engineering, Faculty of Science and Engineering, Iwate University
April 2021 - March 2025

Hobbies

Skiing SAJ Level 1 🎿
Bouldering RP: 1Q (V5-V6)
Coffee Drip, Siphon, Flannel Drip
Onsen Relaxation
Photography Canon EOS 5D Mark III
GLAY Music & Concerts
Manga SLAM DUNK, Rurouni Kenshin, DEATH NOTE

Citation

BibTeX