Closed
Description
Main Galaxy Roadmap
as described in issues.rst
The next major review of this roadmap will be done by the Galaxy Project after the 18.01 release.
The discussion under this issue is moderated to keep the roadmap clear, but please feel free to comment on any part of it.
Priorities
Main/usegalaxy.org
- Mull conda environments and disable env copying - mostly done but still need to upgrade Conda @natefoo @jmchilton [Done on test, not main]Fix data issues - Create a data manager to retrieve liftOver files from UCSC #1904 @natefoo @jennaj [New Roadmap]Two-tiered jobs ready to run query (Two tiered jobs-ready-to-run-query #2270) @natefoo [Killed]Bridges optimization (improve dynamic core/runtime calculation) @natefoo [In progress]
UI rewrite
Dataset collections
- More collection operations (Implement Collection Operation to Relabelling Identifiers #3603 , Fix collection operations not having a history. #3786) @jmchiltonImprove tool collection input UI @guerlerRobust handling of cluster failures @jmchiltonRerunning a failed dataset collection element does substitute the failed element Rerunning a failed dataset collection element should substitute the failed element #2235Rerunning entire collection @jmchiltonCleanup representation in history panel @jmchilton Simplified User-Facing Dataset Collection Model #1810 *~Generalization of collections i.e. paired lists to generic tuple lists ~ Implement records - heterogenous collections (Issue Implement Record Types (generalization of tuple types) #3834) @jmchiltonPagination / Search (Enhancement: dataset search should work within a dataset collection view #4115)
Interactive environments
[Move to new Roadmap]
- GIE hardening GIE Hardening #4651Allow running more than one GIE per sessionBring RStudio to on MainWorking IEs out of the box + uwsgi switchRStudioTest/alerts for live GIEsJupyter cannot be uploaded from other places (ipynb files cannot be uploaded into Galaxy #1922)Render notebooks in history (serve user content from separate domain [Planning] Allow serving user generated content from a separate domain #1932)Improving provenance: IEs as tools (IE Provevance #1923)Developing tests
Workflows and Job Running
- Common workflow language (CWL) - at least for GA4GH challenge Phase 2. [New Roadmap, Future]Scaling collection performance (Scaling Collection Performance - A meta issue #3939) @jmchiltonuWSGI mules as job handlers @natefooUser feedback on scheduling @natefoo [New Roadmap]Status updates (stdout, stderr, output) while jobs are running e.g. by displaying output file sizes @natefoo [New Roadmap]Clarify the path for users to request missing workflow tool @martensonGalaxy workflow format version 2 [New Roadmap]Publish workflows to Github @jmchilton @martenson [Killed]Decision points (issue will be written by @blankenberg) (Allow Decision Points / Control Statements in Workflows #4798) [New Roadmap, Future]
- Assertions (Assertion checking in workflows #3954)
Subworkflow Enhancements [New Roadmap, Future]
Data Libraries
- uploading structured data (mainly for training-material folders/datasets) Hierarchical upload API optimized for folders & collections. #5220
Tool Shed / tool administration
- Improve admin Galaxy interface to TS @martenson @davebx
- Allow mass update
- Switch default installation interface to API-enabled.
Data Table duplication bugs (Data fixes to-do summarized usegalaxy-playbook#55) [New Roadmap]
Galaxy Hub and Training Materials
Metrics collection
Uploading large numbers of datasets
- support chunked uploads Chunked file uploads #573 [New Roadmap]upload directly to collections [New Roadmap]
Cloud
- Galaxy-on-the-Cloud merges with the GVL → 17.09 releaseCloudLaunch
- Move out of beta
- Write/update docs [WIP]
CloudBridge support for Azure & GCECloudMan- Core framework; integration w/ CloudBridge
- Ability to deploy K8S [New Roadmap]
- Launchable in containerized mode via CloudLaunch [Cloud Board]
- UI [Cloud Board]
Containerized Galaxy deployment [Cloud Board]ObjectStore- User-Based ObjectStore (User-based ObjectStore #4840) [New Roadmap]
- Integration with CloudBridge (Cloud ObjectStore #4487)
OAuth
- Authentication and Authorization
- OIDC rp; a PSA-based implementation (User authentication using OpenID Connect protocol #4474) [New Roadmap]
- Integration with AWS authentication
Guided tours
- easier to shareCreate initial set of tours for 101, rna-seq, etc**. @nekrut [WIP]
Histories
[New Roadmap]
- Visualizations and workflow invocations in history panelGraph view - better view of the histories (Tree view of histories #1929)Improve access to visualizations/GIEs/charts/display applications from history History Panel Style Access to Workflow Invocations #5142Meta-history: showing many more histories in a multi-history viewFeatures (archiving, org, folders, etc) (Allow archiving histories and workflows #1734)Bundles
main / performance
- Main performance stats @dannon
- Build reports
- Analyze/fix
Tools
- Make it easier to keep Main’s toolset in top shape
- Make Test/Main use the Ansible tools role to manage installed tools ** @natefoo
- Extend Use the API to install repositories instead of loading the toolshed in an iframe. #1392 to install repositories discovered from another galaxy instance’s /api/tools. @davebx
Find high quality WF on other Galaxies and mirror the toolset and import the WF as well** @nekrutInstall tools needed for a workflow (Installing tools+versions needed for a workflow #559) ** @martenson @afganeProvide admins with framework and interface to test installed tools @martenson @jmchiltonData Updates Genome Indexes, Known DM Issues and Ephemeris for Main?
Main Maintenance/Operations
- Separate the presentation layer of the tool panel to an extra file?
Labels on tool sections (derive labels dynamically)Automate Installation AND maintenance (updates, deprecations, labels) @afgane @martenson
New / enhanced tools suites
- Metagenomics *** @blankenbergRNA-seq **Long reads (PB and ONT) -- assembly, polishing, hybrid, variant calling, iso-seq *** @jxtx @nekrut @jgoecks @blankenbergMethylation analyses - this is available mostlyR tool integration * @blankenbergR tool integration wiki/github instruction page with Galaxy + Best practices to handle bioconductor tool integrationChromatin Conformation @msauria
Workflows and Job Running
- Job Piping/streaming from tool to tool
TS/dependencies
- Remove support for uploading repos with more than one tool * @martensonAutomatically increment tool versions and don't expose changeset revisions.*Offer version management to the tool author.
Workflow hosting service
- Architecture and UX study @martenson @jmchilton
- This set of priorities was developed and extensively discussed during 2016 Galaxy Team meeting (March 14-15, Williamsport, PA).
- Post 16.04 Review occurred on 5/24/2016 via hangouts.
- Post 16.07 Review occurred on 9/xx/2016 via hangouts.
- Post 16.10 Review occurred on 12/9/2016 via hangouts.
- Post 17.01 Review occurred on 4/20/2017 after SAB meeting.
- Post 17.05 Review occurred on 6/xx/2017 at GCC.
- Post 17.09 Review occurred on 11/9/2017 on zoom. And again on 12/7.
Metadata
Assignees
Labels
No labels
Activity
jmchilton commentedon Oct 11, 2016
@nekrut Wanted me to summarize n Slack conversation that ran away with itself on this issue. If I mis-characterize anyone - feel free to just edit my comment on Github.
jxtx commentedon Oct 11, 2016
I don't want to make that strong a comment. I think we should evaluate whether we can benefit from adopting (and potentially improving) a scheduling / planning library that is shared with others. The usual tradeoffs apply, but I think we should evaluate options.
18 remaining items