WEKO3
アイテム
Pose-based Model for Continuous Japanese Sign Language Recognition with Transformer
https://ipsj.ixsq.nii.ac.jp/records/241043
https://ipsj.ixsq.nii.ac.jp/records/241043afc292ec-0a59-4bc5-bc2b-f94abea333df
| 名前 / ファイル | ライセンス | アクション |
|---|---|---|
IPSJ-AL24200027.pdf (2.2 MB)
2026年11月19日からダウンロード可能です。
|
Copyright (c) 2024 by the Information Processing Society of Japan
|
|
| 非会員：¥660, IPSJ:学会員：¥330, AL:会員：¥0, DLIB:会員：¥0 | ||
| Item type | SIG Technical Reports(1) | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 公開日 | 2024-11-19 | |||||||||
| タイトル | ||||||||||
| タイトル | Pose-based Model for Continuous Japanese Sign Language Recognition with Transformer | |||||||||
| タイトル | ||||||||||
| 言語 | en | |||||||||
| タイトル | Pose-based Model for Continuous Japanese Sign Language Recognition with Transformer | |||||||||
| 言語 | ||||||||||
| 言語 | eng | |||||||||
| 資源タイプ | ||||||||||
| 資源タイプ識別子 | http://purl.org/coar/resource_type/c_18gh | |||||||||
| 資源タイプ | technical report | |||||||||
| 著者所属 | ||||||||||
| Muroran Institute OF Technology | ||||||||||
| 著者所属 | ||||||||||
| Muroran Institute OF Technology | ||||||||||
| 著者所属(英) | ||||||||||
| en | ||||||||||
| Muroran Institute OF Technology | ||||||||||
| 著者所属(英) | ||||||||||
| en | ||||||||||
| Muroran Institute OF Technology | ||||||||||
| 著者名 |
Ren, Wang
× Ren, Wang
× He, Li
|
|||||||||
| 著者名(英) |
Ren, Wang
× Ren, Wang
× He, Li
|
|||||||||
| 論文抄録 | ||||||||||
| 内容記述タイプ | Other | |||||||||
| 内容記述 | This research focuses on developing a model to recognize Japanese Sign Language (JSL) gestures using pose-based data processed through a transformer module. The dataset includes 401 JSL words from single signer videos, with hand gesture landmarks extracted via Mediapipe. These landmarks are transformed into an 18-dimensional vector, normalized, and tokenized for input into a BERT-based model. The architecture supports hands-to-gloss recognition, utilizing masked unit modeling for pre-training. Training is divided into isolated word-level recognition and continuous hands-to-gloss recognition. The model's performance was evaluated using accuracy, precision, recall, and F-measure. It achieved 0.746 accuracy in single-word recognition and improved to 0.808 in continuous sign recognition using cosine similarity and fine-tuning techniques. | |||||||||
| 論文抄録(英) | ||||||||||
| 内容記述タイプ | Other | |||||||||
| 内容記述 | This research focuses on developing a model to recognize Japanese Sign Language (JSL) gestures using pose-based data processed through a transformer module. The dataset includes 401 JSL words from single signer videos, with hand gesture landmarks extracted via Mediapipe. These landmarks are transformed into an 18-dimensional vector, normalized, and tokenized for input into a BERT-based model. The architecture supports hands-to-gloss recognition, utilizing masked unit modeling for pre-training. Training is divided into isolated word-level recognition and continuous hands-to-gloss recognition. The model's performance was evaluated using accuracy, precision, recall, and F-measure. It achieved 0.746 accuracy in single-word recognition and improved to 0.808 in continuous sign recognition using cosine similarity and fine-tuning techniques. | |||||||||
| 書誌レコードID | ||||||||||
| 収録物識別子タイプ | NCID | |||||||||
| 収録物識別子 | AN1009593X | |||||||||
| 書誌情報 |
研究報告アルゴリズム（AL） 巻 2024-AL-200, 号 27, p. 1-5, 発行日 2024-11-19 |
|||||||||
| ISSN | ||||||||||
| 収録物識別子タイプ | ISSN | |||||||||
| 収録物識別子 | 2188-8566 | |||||||||
| Notice | ||||||||||
| SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc. | ||||||||||
| 出版者 | ||||||||||
| 言語 | ja | |||||||||
| 出版者 | 情報処理学会 | |||||||||
