Contributed by Lisa N. Cao, Datastrato
What is the DataOps Initiative?
The DataOps Initiative puts together technical material and guides for users interested in the end-to-end process of deploying Machine Learning (ML) applications and models within their organizations.
By developing an inclusive set of DataOps and DevOps best practices for engineers, we can empower developers, architects, and decision-makers to effectively leverage open source tools and frameworks for streamlined, secure, and scalable ML application deployment.
Why are we putting this together?
Organizations face complex challenges in managing technical and data debt when deploying data-intensive applications and machine learning models, from initial development to operational maintenance. This process requires seamless integration of CI/CD practices, containerization, data infrastructure, MLOps, and security measures. We believe that agile methodologies, infrastructure as code, and cloud native development will be the foundation of modern Data Reliability Engineering and Machine Learning Platforms.
Roadmap
- Develop a series of high-level blog posts to raise awareness and flesh out the course material, test out ideas, in conjunction with OPEA and the CD Foundation
- Develop the course materials, including practical implementations and code checks, and set up environments for developer and end users
- Publish the course on Linux Foundation Training as a certification
Curriculum
Here’s the proposed curriculum for the initiative:
01. Fundamentals
- DataOps vs DevOps
- DataOps Philosophy
- Organizational DataOps
02. Platform
- Example Architecture
- Team Organization
- Data Architecture
03. Operations I
- Pipeline Orchestration
- CI/CD for Data Pipelines
- Data Quality
- Data Contracts
04. Operations II
- Data Governance
- Observability
- Cloud Native Data
- Securing your Data Pipelines
05. AI/ML
- Realtime ML
- MLOps and Monitoring Models
- Security for AI/ML
What are we looking for?
We are looking for various folks to fill the following roles for each part of the curriculum.
- Technical Writer
- Technical Editor (code reviews)
- Proofreader (content reviews)
Who can participate?
We are actively looking to collaborate with other members and organizations within and outside of the Linux Foundation.
How can you contribute?
To get started, fill out the application form and we’ll send you details to onboard you into the initiative.
Fortnightly meetings
Our next fortnightly meeting will take place on January 29, 2025. View the CDF community calendar for details.
