Introducing
Your new presentation assistant.
Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.
Trending searches
What?
System Libraries
Bioinformatics Applications
What is OpenStack and private cloud computing?
xvfb
virtualenv
Poorly Defined Boundaries
Cron Jobs
Init Scripts
Proxy Webserver
API
Tool Wrappers
Workflow Engine
Job Scheduler
User Interface
Log Management
Display Application
Bio Data
Two people or teams need
to be intimately familiar with
Galaxy and must frequently
communicate.
Replace e-mail with API calls,
documentation with scripts.
`
(at least)
Opportunity to reduce workload by building Galaxy using common community template.
Businesses large and small are flocking to Amazon et. al. because they are cheap.
There is no GalaxyAdmin walk through - each
large Galaxy installation represents numerous innovations.
https://github.com/chapmanb/cloudbiolinux
"A fully automated infrastructure installs software and data, with packages specified in simple configuration files."
Do not wasting effort manually installing software, automate it.
Why?
Building more of the stack on community templates, provides a way to share more innovations back with community.
"CloudMan is a cloud manager that orchestrates all of the steps required to provision a complete compute cluster environment on a cloud infrastructure; subsequently, it allows one to manage the cluster, all through a web browser. "
Saving money however is not the only reason to employ cloud computing, as I will argue for the specific case of Galaxy - cloud computing can also help manage complexity.
Deploy open source cloud infrastructure on your own hardware.
Why deploy Galaxy in a (private) cloud?
A block of YAML text used to
configure VM at launch time.
CloudMan uses to configure virtual machine - Galaxy, nginx, NFS, arbitrary other files.
Full details @ ReadTheDocs
How?
User-Data
user-data
CloudBioLinux must compile nginx with LDAP
nginx.conf
https://bitbucket.org/galaxy/galaxy-central/pull-request/44/
Very useful in non-cloud contexts as well. Allows universe_wsgi.ini to be split into a directory of
files (ala /etc/sudoers.d or /etc/apache/conf.d).
Setup nginx.conf
Allow some properties set in repository others in runtime environment.
Easier for configuration management tools such as Puppet or Chef to work with.
Separate development/production properties and/or developer/admin properties.
Hack to fix admin panel.
Powerful tool provides a wealth of valuable data on every job that Galaxy has run as well as disk usage accounting, etc....
user-data
Implemented CloudMan "service" for this...
Configure arbitrary config files on VM
nginx.conf
user-data
no postgres
user-data
Run arbitrary commands on master and worker nodes at startup...
https://lwr.readthedocs.org/
How to build production Galaxy instances for the cloud.
https://bitbucket.org/jmchilton/lwr/
Run normal Galaxy jobs from normal tools on a remote server without requiring shared file systems.
Run jobs on *nix or Windows.
user-data
Any tool id
Any job runner URL, not just LWR urls
Ephemeral cloud VM submitting jobs to a dedicated compute resources... a little backwards but it works.
Finer grain grid engine configuration
Related Work
CloudBioLinux + Galaxy without CloudMan (formerly galaxy-vm-launcher)
http://galaxy.nbic.nl/
http://www.nbic.nl/about-nbic/news-press/bioinformatics-news/detail/article/galaxy-on-cloud/
https://github.com/jmchilton/cloudbiolinux/tree/deploy
Enis Afgan, Ph.D.
Monitoring
Brad Chapman, Ph.D.
How does one do nagios in the Cloud?
Nate Coraor and Dannon Baker
https://github.com/jmchilton/cloudbiolinux/tree/chef
libcloud based deployment scripts
https://bitbucket.org/galaxy/cloudman/pull-request/23/added-optional-loggly-based-cloud-logging/diff
Jim Johnson; Pratik Jagtap, Ph.D.; Daniel Debertin; Kevin Silverstein; Ph.D.; Anne-Françoise Lamblin, Ph.D.; Benjamin Lynch, Ph.D.
novnc for web accessible console access
https://bitbucket.org/galaxyp/galaxyp-central
https://github.com/jmchilton/cloudbiolinux/tree/proteomics
https://github.com/jmchilton/proteomics-wine-env
Configured with chef recipes, Globus online integration.
Timothy Griffin, Ph.D.
Deploy Galaxy from git
http://www.globus.org/provision/
http://www.cse.buffalo.edu/faculty/tkosar/datacloud2012/papers/datacloud2012_paper_6.pdf
Ability to package tool shed installs
CloudBioLinux + Galaxy without CloudMan (formerly galaxy-vm-launcher)
https://github.com/jmchilton/cloudbiolinux/tree/deploy
Good for cloud & traditional staging/production environments.
This work was funded by Minnesota Partnership for Biotechnology and Medical Genomics and the National Science Foundation.