Unsupposable Test-data Generation for Machine-learned Software

• Item type: Symposium(1)
• 公開日: 2020-09-03
• タイトル: Unsupposable Test-data Generation for Machine-learned Software
• タイトル / 言語: en
• タイトル: Unsupposable Test-data Generation for Machine-learned Software
• 言語: eng
• キーワード / 主題Scheme: Other
• キーワード / 主題: 機械学習工学
• 資源タイプ / 資源タイプ識別子: http://purl.org/coar/resource_type/c_5794
• 資源タイプ: conference paper
• 著者所属: Research & Development Group, Hitachi, Ltd.
• 著者所属: Research & Development Group, Hitachi, Ltd.
• 著者所属: Research & Development Group, Hitachi, Ltd.
• 著者所属(英): en
• 著者所属(英): Research & Development Group, Hitachi, Ltd.
• 著者所属(英): en
• 著者所属(英): Research & Development Group, Hitachi, Ltd.
• 著者所属(英): en
• 著者所属(英): Research & Development Group, Hitachi, Ltd.
• 著者名: Naoto, Sato; Hironobu, Kuruma; Hideto, Ogawa
• 著者名(英): Naoto, Sato; Hironobu, Kuruma; Hideto, Ogawa
• 論文抄録 / 内容記述タイプ: Other
• 論文抄録 / 内容記述: As for software development by machine learning, a trained model is evaluated by using part of an existing dataset as test data. However, if data with characteristics that differ from the existing data is input, the model does not always behave as expected. Accordingly, to confirm the behavior of the model more strictly, it is necessary to create data that differs from the existing data and test the model with that different data. The data to be tested includes not only data that developers can suppose (supposable data) but also data they cannot suppose (unsupposable data). To confirm the behavior of the model strictly, it is important to create as much unsupposable data as possible. In this study, therefore, a method called “unsupposable test-data generation” (UTG)—for giving suggestions for unsupposable data to model developers and testers—is proposed. UTG uses a variational autoencoder (VAE) to generate unsupposable data. The unsupposable data is generated by acquiring latent values with low occurrence probability in the prior distribution of the VAE and inputting the acquired latent values into the decoder. If unsupposable data is included in the data generated by the decoder, the developer can recognize new unsupposable features by referring to the data. On the basis of those unsupposable features, the developer will be able to create other unsupposable data with the same features. The proposed UTG was applied to the MNIST dataset and the House Sales Price dataset. The results demonstrate the feasibility of UTG.
• 論文抄録(英) / 内容記述タイプ: Other
• 論文抄録(英) / 内容記述: As for software development by machine learning, a trained model is evaluated by using part of an existing dataset as test data. However, if data with characteristics that differ from the existing data is input, the model does not always behave as expected. Accordingly, to confirm the behavior of the model more strictly, it is necessary to create data that differs from the existing data and test the model with that different data. The data to be tested includes not only data that developers can suppose (supposable data) but also data they cannot suppose (unsupposable data). To confirm the behavior of the model strictly, it is important to create as much unsupposable data as possible. In this study, therefore, a method called “unsupposable test-data generation” (UTG)—for giving suggestions for unsupposable data to model developers and testers—is proposed. UTG uses a variational autoencoder (VAE) to generate unsupposable data. The unsupposable data is generated by acquiring latent values with low occurrence probability in the prior distribution of the VAE and inputting the acquired latent values into the decoder. If unsupposable data is included in the data generated by the decoder, the developer can recognize new unsupposable features by referring to the data. On the basis of those unsupposable features, the developer will be able to create other unsupposable data with the same features. The proposed UTG was applied to the MNIST dataset and the House Sales Price dataset. The results demonstrate the feasibility of UTG.
• 書誌情報: ソフトウェアエンジニアリングシンポジウム2020論文集; 巻 2020, p. 153-160, 発行日 2020-09-03
• 出版者 / 言語: ja
• 出版者: 情報処理学会