| Item type |
SIG Technical Reports(1) |
| 公開日 |
2021-07-13 |
| タイトル |
|
|
タイトル |
Towards Compute Flexibility for Genome Analysis in the Hybrid Cloud |
| タイトル |
|
|
言語 |
en |
|
タイトル |
Towards Compute Flexibility for Genome Analysis in the Hybrid Cloud |
| 言語 |
|
|
言語 |
eng |
| キーワード |
|
|
主題Scheme |
Other |
|
主題 |
コンテナ・クラウド |
| 資源タイプ |
|
|
資源タイプ識別子 |
http://purl.org/coar/resource_type/c_18gh |
|
資源タイプ |
technical report |
| 著者所属 |
|
|
|
IBM Research-Tokyo |
| 著者所属 |
|
|
|
IBM Research-Tokyo |
| 著者所属(英) |
|
|
|
en |
|
|
IBM Research-Tokyo |
| 著者所属(英) |
|
|
|
en |
|
|
IBM Research-Tokyo |
| 著者名 |
Takeshi, Yoshimura
Tatsuhiro, Chiba
|
| 著者名(英) |
Takeshi, Yoshimura
Tatsuhiro, Chiba
|
| 論文抄録 |
|
|
内容記述タイプ |
Other |
|
内容記述 |
Genome analysis has become an emerging research area in medical and life science. Genome Analysis Toolkit (GATK), an industry-standard genome analysis tool, enables to run and speed up genome analysis in the cloud. Cromwell, a workflow engine for GATK enables to define and reproducible pipelines for complex data processing as files written in WDL, a domain-specific language for defining genome workflows. Their motivation is to efficiently process a huge amount of genome data in the cloud. However, the current design of Cromwell and GATK has less flexibility in terms of choices of the cloud vendors and storage due to vendor lock-in. In this paper, we extend Cromwell to support OpenShift and multiple cloud object storage to avoid vendor lock-in. We also characterize performance overheads and resource underutilization of existing Cromwell. It leads to our design leveraging copy bypassing with container storage interface and cost-efficient pod scheduling with cluster autoscaling. This paper demonstrates early experimental results of copy overheads and resource utilization under managed Red Hat OpenShift 4.6 on IBM Cloud. The experiments show that copy reductions of our backend reduced the elapsed time for an example workflow by 14% and 20% compared to existing backends. Also, cluster autoscaling reduced the cost of a best-practice workflow by 31%. |
| 論文抄録(英) |
|
|
内容記述タイプ |
Other |
|
内容記述 |
Genome analysis has become an emerging research area in medical and life science. Genome Analysis Toolkit (GATK), an industry-standard genome analysis tool, enables to run and speed up genome analysis in the cloud. Cromwell, a workflow engine for GATK enables to define and reproducible pipelines for complex data processing as files written in WDL, a domain-specific language for defining genome workflows. Their motivation is to efficiently process a huge amount of genome data in the cloud. However, the current design of Cromwell and GATK has less flexibility in terms of choices of the cloud vendors and storage due to vendor lock-in. In this paper, we extend Cromwell to support OpenShift and multiple cloud object storage to avoid vendor lock-in. We also characterize performance overheads and resource underutilization of existing Cromwell. It leads to our design leveraging copy bypassing with container storage interface and cost-efficient pod scheduling with cluster autoscaling. This paper demonstrates early experimental results of copy overheads and resource utilization under managed Red Hat OpenShift 4.6 on IBM Cloud. The experiments show that copy reductions of our backend reduced the elapsed time for an example workflow by 14% and 20% compared to existing backends. Also, cluster autoscaling reduced the cost of a best-practice workflow by 31%. |
| 書誌レコードID |
|
|
収録物識別子タイプ |
NCID |
|
収録物識別子 |
AN10444176 |
| 書誌情報 |
研究報告システムソフトウェアとオペレーティング・システム（OS）
巻 2021-OS-153,
号 7,
p. 1-9,
発行日 2021-07-13
|
| ISSN |
|
|
収録物識別子タイプ |
ISSN |
|
収録物識別子 |
2188-8795 |
| Notice |
|
|
|
SIG Technical Reports are nonrefereed and hence may later appear in any journals, conferences, symposia, etc. |
| 出版者 |
|
|
言語 |
ja |
|
出版者 |
情報処理学会 |
IPSJ-OS21153007.pdf (738.8 kB)
