Open source platform for the machine learning lifecycle
-
Updated
Dec 25, 2024 - Python
#
apache-spark
Apache Spark is an open source distributed general-purpose cluster-computing framework. It provides an interface for programming entire clusters with implicit data parallelism and fault tolerance.
Here are 1,832 public repositories matching this topic...
Simple and Distributed Machine Learning
microsoft http opencv data-science machine-learning scala big-data ai spark apache-spark deep-learning azure ml pyspark lightgbm cognitive-services databricks synapse model-deployment onnx
-
Updated
Dec 2, 2024 - Scala
lakeFS - Data version control for your data lake | Git for data
go golang apache-spark aws-s3 google-cloud-storage data-engineering data-lake azure-storage data-version-control object-storage datalake hadoop-filesystem data-quality data-versioning azure-blob-storage apache-sparksql git-for-data lakefs datalakes
-
Updated
Dec 24, 2024 - Go
酷玩 Spark: Spark 源代码解析、Spark 类库等
-
Updated
May 18, 2022 - Scala
Interactive and Reactive Data Science using Scala and Spark.
-
Updated
May 16, 2023 - JavaScript
Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
kubernetes spark apache-spark kubernetes-operator kubernetes-controller kubernetes-crd google-cloud-dataproc
-
Updated
Dec 23, 2024 - Go
BigDL: Distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray
python scala apache-spark pytorch keras-tensorflow bigdl distributed-deep-learning deep-neural-network analytics-zoo
-
Updated
Nov 1, 2024 - Jupyter Notebook
Apache Spark docker image
-
Updated
Apr 21, 2023 - Shell
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
microsoft emr machine-learning streaming spark apache-spark csharp fsharp analytics dotnet azure bigdata spark-streaming dotnet-core dotnet-standard databricks spark-sql tpcds hdinsight tpch
-
Updated
Dec 20, 2024 - C#
Feathr – A scalable, unified data and AI engineering platform for enterprise
data-science machine-learning apache-spark azure artificial-intelligence data-engineering feature-engineering data-quality mlops feature-store feature-management feature-marketplace feature-governance feature-metadata feature-platform
-
Updated
Apr 4, 2024 - Scala
Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning
-
Updated
Aug 16, 2021 - Java
A curated list of awesome Apache Spark packages and resources.
-
Updated
Oct 24, 2024 - Shell
An end-to-end GoodReads Data Pipeline for Building Data Lake, Data Warehouse and Analytics Platform.
python airflow spark apache-spark scheduler s3 data-engineering data-lake warehouse redshift data-migration livy etl-framework apache-airflow emr-cluster etl-pipeline etl-job data-engineering-pipeline airflow-dag goodreads-data-pipeline
-
Updated
Mar 9, 2020 - Python
SQL data analysis & visualization projects using MySQL, PostgreSQL, SQLite, Tableau, Apache Spark and pySpark.
mysql python postgres sql apache-spark sqlite postgresql challenges pyspark mysql-database data-analysis exercises tableau sql-queries pgadmin mysqlworkbench mysql-notes digital-music-store sql-data-analysis
-
Updated
Jul 18, 2022 - Jupyter Notebook
This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
-
Updated
May 8, 2024 - Scala
PySpark + Scikit-learn = Sparkit-learn
-
Updated
Dec 31, 2020 - Python
(Deprecated) Scikit-learn integration package for Apache Spark
-
Updated
Dec 3, 2019 - Python
MapReduce, Spark, Java, and Scala for Data Algorithms Book
python java machine-learning scala apache-spark distributed-computing design-patterns pyspark mapreduce reducers partitioning hadoop-mapreduce distributed-algorithms mappers data-algorithms apache-hadoop
-
Updated
Oct 14, 2024 - Java
R interface for Apache Spark
-
Updated
Oct 28, 2024 - R
Created by Matei Zaharia
Released May 26, 2014
- Followers
- 425 followers
- Repository
- apache/spark
- Website
- spark.apache.org
- Wikipedia
- Wikipedia