2024-03-29T03:25:16Zhttps://ipsj.ixsq.nii.ac.jp/ej/?action=repository_oaipmhoai:ipsj.ixsq.nii.ac.jp:001912162023-04-27T10:00:04Z01164:01165:09553:09554
DevOpsを活用した協調型データ準備プロセスの提案Proposal of Collaborative Data Preparation Process Utilizing DevOps MethodjpnDB基盤技術http://id.nii.ac.jp/1001/00191128/Technical Reporthttps://ipsj.ixsq.nii.ac.jp/ej/?action=repository_action_common_download&item_id=191216&item_no=1&attribute_id=1&file_no=1Copyright (c) 2018 by the Information Processing Society of Japan(株)日立製作所研究開発グループHitachi America Ltd., Research & Development Group(株)日立製作所研究開発グループ(株)日立製作所研究開発グループ(株)日立製作所研究開発グループ樫山, 俊彦保田, 弘武角井, 健太郎北脇, 淳鹿野, 裕明データ利活用ニーズが高まる中,ビジネス環境変化のスピードに追随するため,ソリューション開発工数の 8 割を占めるデータ準備の迅速化が求められている.そのため,ドメイン知識 ・ IT スキルの異なる複数ロール間において,データ準備要件を決定し,検証済み加工データを提供する際の手戻り最小化が課題であった.本研究では,インタラクティブなデータ理解 ・ 加工を通じた早期の要件決定,および DevOps 活用による迅速な環境構築 ・ 開発 ・ 検証によるデータ準備プロセスを提案する.過去のデータ分析案件に本プロセスを適用した結果,過去実績工数の最大 1/3 に低減できた.Data-utilizing applications are being attracted to improve company's operational key performance indicators or to create new business. Due to faster changes in business environments, such applications need to be released more quickly. However, data preparation is very time-consuming and hindering quick deployment of such applications. Since requirements of data used for analytics applications cannot be fixed before a project starts, or they change even in development, collaboration between multiple persons who have different level of domain knowledge and IT skills is a challenge to determine specification of data transformation. In this research, collaborative data preparation process utilizing DevOps method is proposed. This process realizes quick agreement of concrete specification through interactive data understanding / transformation with data wrangling tools. In addition to that, the process enables to build development environment rapidly, and develop/test ETL efficiently by utilizing multiple plugins of an ETL tool and container technology. Initial evaluation with the existing data and the same goal of data preparation that was used for actual analytics application, data preparation time reduced to about up to 1/3.AN10112482研究報告データベースシステム(DBS)2018-DBS-16710162018-09-052188-871x2018-09-04