This release covers the following:

• add databricks sql connector functionality so that querying from local machine is made possible

This release covers the following:

• removal of all AWS EMR and Sagemaker related utils
• add Databricks utils
• add MLflow utils
• add common requests functions for our endpoint integration tests

This release includes the following patch fix:

• change dependencies version

This release includes the following patch fix:

• change autoscaling ondemand limit to 0 so that only spot ec2 instances are spun up

This release includes the following patch fix:

• add EMR version as part of the function parameter to prevent hardcoding of versions
• add auto scaling for number of ec2 instances of cluster; min 60 and max 120
• changed the logic to check if the master DNS address with polling request every 15 minutes

This release includes the following patch fix:

• shortening spot instances timeout duration to 5 minutes
• increasing the time to check if the EMR cluster has a master DNS address to 10.5 minutes

This release includes the following:

• Fix the bug to correctly store int and float data type as numpy int and numpy float when querying and storing data as pandas dataframe.

This release includes the following:

• loosening dependency requirements to be compatible with other ml repos

This release includes the following:

• fixes EMR pytests to use botocore stubber, moto throws an malformatted inputs and not quite sure the cause of it

This release includes the following:

• pyathena client query engine
• boto3 client for EMR
• boto3 client for AWS Sagemaker Model Registry
• minor bug fixes and code optimisation for pyathena
• fix up requirements