-
Notifications
You must be signed in to change notification settings - Fork 56
Transformation
a_git_a edited this page Feb 7, 2024
·
6 revisions
A typical Processing job:
- Creates a materialized view
- Data comes from a source database
- Data goes to a target database
- Data in the target database is in a star schema
- Schema is populated using standard fact/dimension loading strategies (relevant ones are implemented in the platform, so it is 1-liner in terms of Data Job code)
- SQL and Python parameterized transformations.
- Extensible templates for data modeling.
- Creates a dataset or table as a product.
➡ Data Modeling: Treating Data as a Product
➡ Processing data using SQL and local database
➡ Processing data using Kimball warehousing templates
VDK provides SQL Data Processing Templates:
➡ Slowly Changing Dimension Type 1
➡ Slowly Changing Dimension Type 2
➡ Append Strategy
➡ Insert Strategy
➡️ Next Section: Data Processing Templates
SDK - Develop Data Jobs
SDK Key Concepts
Control Service - Deploy Data Jobs
Control Service Key Concepts
- Scheduling a Data Job for automatic execution
- Deployment
- Execution
- Production
- Properties and Secrets
Operations UI
Community
Contacts