Bigquery jobs
...- specify use case, e.g., appointment recommendations, demand forecasting) and BigQuery for efficient data management. * Implement Telnyx integration for SMS communication, including appointment reminders, notifications, and updates. * Build RESTful APIs to facilitate communication between the backend and frontend applications. * Ensure code quality, security, and performance optimization. * Collaborate effectively with the frontend development team and other stakeholders. Qualifications: * Proven experience in backend development using Django Python. * Strong understanding of relational databases and data modeling. * Hands-on experience with Google Cloud Platform (GCP), specifically Vertex AI and BigQuery. * Familiarity with Telnyx API for SMS integration. * Expe...
Develop a cost-effective, fully Google-powered workflow enhanced with external data sources to collect, validate, and manage data for local small-to-medium businesses (SMBs) not active on LinkedIn. The focus is on gathering business owner names, email addresses, and price levels (average ticket size). This workflow leverages Google Places API, Google Sheets, Google Apps Script, Google BigQuery, and Google Gemini AI, along with supplementary data sources for broader coverage. Scope: Data Sources: Primary: Google Places API: Core source for structured business data. Facebook Graph API: Supplementary owner and contact information. Yelp API: For reviews, ratings, and price levels. Secondary: Chamber of Commerce Directories: For local owner and email details. Yellow Pages API: Traditio...
More details: What type of data processing do you need on GCP? This question was skipped by the user Which GCP services do you plan to use? BigQuery, Dataflow, Cloud Pub/Sub What is the main goal of your data processing project? Data migration
...all collected data to a database (preferably BigQuery or MySQL). Ensure efficient storage of large datasets (100,000+ products with multiple competitor prices). Include fields: Product name SKU Our sell price Competitor name Competitor sell price Source URL Timestamp Data Updates: Automate scraping at regular intervals (daily or weekly). Append new data to the database while keeping a history of older price data. Technical Requirements: Scalable and optimized to handle large product datasets. Compliant with website terms of service. Tool should allow cloud deployment for continuous operation. Additional Notes: We will provide: A list of predefined competitor sites. Product data (e.g., Norton 271R). Database credentials for testing (BigQuery/MySQL). Deliverables: ...
I'm looking for an intermediate to advanced level expert in Google Cloud Platform (GCP) to assist me with a job. The specific area in GCP I need help with has not been decided yet, so the ideal freelancer should have a broad understanding of all GCP components, including Compute Engine, Cloud Storage, and BigQuery. Skills and experience required: - Extensive knowledge of Google Cloud Platform - Proficiency in Compute Engine, Cloud Storage, and BigQuery - Intermediate to Advanced level expertise - Ability to assist in various GCP areas Your main tasks will be to provide guidance, share insights, and help me navigate GCP effectively. A flexible approach and strong problem-solving skills will be essential to adapt to the needs of the job as they arise.
Seeking an AWS Glue expert to assist with fetching analytical data from Google BigQuery and storing it on S3 in Parquet or CSV format. The job includes setting up an incremental data extraction process that runs daily. Current Status: Query: Already prepared. Connection: The connection to BigQuery is set up and ready from within Glue Studio. Challenge: 1- I need assistance configuring Glue ( Glue Notebook or visual Job) to handle date-partitioned tables in BigQuery and load data from there incrementally. 2- configure S3 crawler to scan the bucket and push new data to DB Background: I previously implemented this workflow using QlikView script, but I am now transitioning to AWS. Looking for guidance on best practices specific to AWS. Ideal candidates should have:...
Setup Android phone with Android studio, Facilitate code to run in the APP, Setup and connect to my bigquery warehouse with the data. ------------- Familiar with settings to facilitate the function, important!
...all collected data to a database (preferably BigQuery or MySQL). Ensure efficient storage of large datasets (100,000+ products with multiple competitor prices). Include fields: Product name SKU Our sell price Competitor name Competitor sell price Source URL Timestamp Data Updates: Automate scraping at regular intervals (daily or weekly). Append new data to the database while keeping a history of older price data. Technical Requirements: Scalable and optimized to handle large product datasets. Compliant with website terms of service. Tool should allow cloud deployment for continuous operation. Additional Notes: We will provide: A list of predefined competitor sites. Product data (e.g., Norton 271R). Database credentials for testing (BigQuery/MySQL). Deliverables: ...
...with the website details. Schedule Regular Checks: Run the Zap every 24 hours to ensure data is consistently monitored. Step 4: Set Up BigQuery for Scalable Data Analysis If you manage over 100 websites, BigQuery is an ideal tool for handling large-scale data: Export Google Analytics Data: Link your Google Analytics properties to BigQuery for automatic data exports. GA4 properties come with built-in support for BigQuery. Create Automated Queries: Write SQL queries to check for missing data or zero activity for any website. Schedule queries to run daily. Alert System Integration: Use tools like Looker Studio or Zapier to visualize and send alerts based on BigQuery query results. Step 5: Use Monitoring Tools For non-technical solutions, integrate third...
I'm looking for an expert to create insightful reports in Google Sheets utilizing BigQuery data. Your task will be primarily focusing on sales data, but you will also incorporate marketing data and product performance data. The goal is to tie together various metrics and provide a comprehensive view of our sales landscape. Ideal skills for this project: - Proficiency in Google Sheets and BigQuery - Experience in data visualization and reporting - Ability to interpret and correlate different datasets - Familiarity with sales, marketing, and product performance metrics The final product should be clear, concise, and capable of providing meaningful insights into our sales performance.
We are looking for a senior professional with extensive experience in BigQuery, Alteryx and Looker Studio to assist us in a consulting project. We are currently storing and consuming information in BigQuery, integrating the data with Alteryx for analytics and Looker Studio for visualization. The objective is to carry out a detailed analysis of our data environment, identify good practices that can be adopted and suggest improvements to reduce operational costs. The freelancer must offer strategic insights and propose solutions to optimize our processes. The optimization project should start immediately. The duration of the project is expected to be less than 1 month.
I have a Python script that extracts data from an API and loads it into Google Big Query. The script takes 30 minutes to run. I've de...looking for someone with strong skills in Python, Google Cloud Function, and IAM to help me troubleshoot this issue. Please note that this can only be done via Zoom. Key Information: - Trigger Type: HTTP trigger - IAM Roles Assigned: Editor, BigQuery User, Cloud Functions Invoker - API Authentication Method: API key Ideal Skills: - Proficient in Python - Experienced with Google Cloud Function - Knowledgeable about IAM settings and permissions - Able to troubleshoot permission errors - Comfortable with Zoom consultations - Deep knowledge of Google BigQuery: Writing and optimizing SQL queries, managing datasets, and ensuring efficient d...
...relatórios personalizados e registro de logs. O front-end será desenvolvido em Angular, o back-end em Node.js, e o banco de dados PostgreSQL será usado para armazenar os logs e dados relacionados. Requisitos Funcionais 1. Gerenciamento de Pontos de Auditoria (PA) 1.1. Cadastro de PA Campos obrigatórios: Nome do PA. Query SQL que será executada. Banco de dados onde a query será executada (Postgres ou BigQuery). Setor responsável pelo ponto. Instruções para corrigir erros encontrados. Horário de execução programada. Estado do PA (Ativo/Inativo). Responsáveis vinculados: Permitir a seleção de responsáveis pelo recebimento de relatórios. Divisão automá...
We're seeking an experienced external consultant to assist with integrating data from Google Ads (Performance Max campaigns) into BigQuery and creating an aggregated merge with our Bubble app data, which is also integrated into BigQuery via API. Issue: We have attempted to transfer Performance Max (Pmax) campaign data from Google Ads to BigQuery. However, certain key data fields are missing despite selecting all necessary configurations to include Pmax campaign data. Requirement: Diagnose and resolve the issue causing missing Pmax campaign data in BigQuery. Specifically, ensure we can extract the following fields: Campaign Name Campaign ID Date Cost Clicks Impressions Note: Google Ads and Google Analytics support teams have not been able to assist with this...
Summary We are looking for a Senior MLOps Engineer to support the AI CoE in building and scaling machine learning operations...to write code in ML ⮚ Excellent analytical and problem-solving skills for technical challenges related to MLOps ⮚ Excellent English proficiency, presentation, and communication skills ⮚ Proven experience in deploying, monitoring, and managing machine learning models on GCP / AWS /Azure ⮚ Hands-on experience with data catalog tools ⮚ Expert in GCP / AWS / Azure services such as Vertex AI, GKE, BigQuery, and Cloud Build, Endpoint etc for building scalable ML infrastructure (GCP / AWS / Azure official Certifications are a huge plus) ⮚ Experience with model serving frameworks (e.g., TensorFlow Serving, TorchServe), and MLOps tools like Kubeflow, MLflow, or TFX
I am in need of a skilled developer with profound knowledge of the Google Cloud Suite, specifically for the integration of Vertex AI into my WordPress webs...within the real estate industry. Key Integration Requirements: - Full integration of Vertex AI into the website - Enhancing user interaction modules with AI capabilities - Utilizing AI to optimize property listings - Implementing AI-driven insights for marketing and analytics Currently, I am utilizing the following Google Cloud services: - Google Cloud Storage - Google Cloud Functions - Google BigQuery Skills and Experience Needed: - Proficient in Google Cloud Suite - Expertise in integrating AI into websites - Familiar with WordPress and real estate applications - Experience in utilizing Google Cloud services for website de...
Seeking a developer to create a system to store and manage 2-3 million rows of da...to filter and export data by polygon, zip code, city, etc. This project is for development only; no ongoing maintenance required. Business Context: We work with construction companies targeting storm-damaged areas. We need the ability to draw polygons to pull data from specific storm paths, ensuring we can efficiently target affected areas. Requirements: Proven experience with Google Cloud, BigQuery, Google Maps API, or similar tools. Ability to handle and process large datasets efficiently. Strong skills in SQL and geospatial functions. Experience in developing web applications with data visualization capabilities. Additional Information: We have the data ready and just need help creating a manage...
...telematics devices with cloud infrastructure to enable real-time tracking, analytics, and data visualization for fleet management. Responsibilities Device Connectivity: Set up Azure IoT Hub or Google Cloud IoT Core to connect and manage telematics devices securely. Data Ingestion and Storage: Implement data ingestion pipelines and select optimal storage solutions, such as Azure Cosmos DB or Google BigQuery, for real-time and historical data. Data Processing: Configure real-time analytics using Azure Stream Analytics or Google Cloud Dataflow, enabling fleet monitoring, alerting, and analytics. Data Visualization: Build interactive dashboards with Power BI or Looker, offering real-time maps and analytics on fleet performance and vehicle health. Notifications and Alerts: Set up a ...
...Educational Resources and Strategy Marketplace: Tutorials, webinars, and a collection of pre-built, customizable strategies available to all users. Tech Stack:- Frontend: React with Material UI (MUI) and TypeScript for a robust, scalable, and type-safe interface. Backend: Python (Flask/FastAPI) preferred, with Google Cloud Platform (GCP)/Firebase integration. Database: Firestore for real-time data, BigQuery for historical data. Deployment: Firebase/Google Cloud Platform, secured with GCP API Gateway, managed through our GitHub repository. Expected Deliverables: A fully functional platform with all core features. Source code in our GitHub repository with CI/CD pipelines. Complete documentation covering all APIs, data flows, and architectural elements. Ongoing post-launch support...
...need a Python project deployed on Google Cloud Platform (GCP). The project involves a BigQuery database and a virtual machine (VM) that I've already set up. The successful freelancer will need to ensure that the login process operates seamlessly within the GCP environment. Key Requirements: - Implement basic authentication with username and password for VM and BigQuery access - Facilitate read/write access to the BigQuery database - Configure admin-level permissions within the BigQuery database Ideal Skills: - Proficient in Python and GCP - Experience with BigQuery and VM setup - Knowledge of implementing basic authentication systems - Familiarity with configuring user roles and permissions in BigQuery I look forward to your bids and hop...
I am seeking a seasoned Data Engineer to conduct technical mock interviews focusing on data pipeline development. The ideal candidate should be well-versed in: - Data Pipeline Development: Mock interviews will center primarily on this skill. - Data Warehousing: Familiarity and practical knowledge with Amazon Redshift, Google BigQuery, and Snowflake will be crucial. - Data Pipeline Development Tools: Proficiency and experience with Apache Kafka and Apache Spark are required. Your role will be to assess my skills, provide constructive feedback, and help me improve my technical knowledge and interview readiness. The aim is to enhance my capabilities in data pipeline development and prepare me for real-world technical interviews. Working hours will be in evening IST after 7:30 PM IST
I need a skilled data scientist to assist me with the storage and management of sales data sourced from third-party APIs in BigQuery. Key responsibilities include: - Setting up and optimizing a data warehouse in BigQuery - Ensuring smooth integration with various third-party APIs - Implementing effective data storage solutions for sales data Ideal candidates should possess: - Extensive experience with BigQuery and data warehousing - Proven track record working with third-party APIs - Strong understanding of sales data and its implications Your skills in Google Cloud Platform, SQL language, AI, data analysis, and your ability to handle complex data sets will be essential in delivering a robust and efficient data warehousing solution.
Quote for building a dashboard with Tableau integrating BigQuery, Skyvia,etc.
I am looking for a skilled data engineer with experience in dbt, Airflow and BigQuery to help optimize and enhance my existing data model. The main goal of this project is to improve the data transformation and modeling process. Key Requirements: - Experience with dbt, Airflow and BigQuery. - Ability to optimize query performance. - Skills in integrating new data sources. - Expertise in enhancing data accuracy. Your primary task will be to: - Use dbt for data transformation and modeling. - Schedule automated data pipeline using Airflow. - Store and query data in BigQuery. The ideal candidate would be a proactive problem-solver with a strong understanding of data engineering principles and practices. If you're able to deliver high-quality work under tight deadl...
I need a strong Oracle and PLSQL developer tor the Food Services INdustry and Supply Chain Management (SCM) software. Ideal Skills: - Strongin Oracle and PLSQL - Experience with Custom Application Development Desirable Skills: - Experience with BigQuery would be desirable
I'm looking for an expert who can create a Slack chatbot that connects with Python code to fetch data from my existing Google BigQuery dataset. Key Requirements: - The chatbot should be able to fetch data from BigQuery upon command. - It should be capable of executing pre-defined queries. Ideal Skills: - Proficient in Python and Slack API - Experienced with Google BigQuery - Capable of designing user-friendly chatbots Please note: The dataset is already set up. The queries will be pre-defined, so there's no need for the chatbot to handle user-defined queries.
...Manage data migration, data warehousing, and cloud integration processes. Oversee the implementation of BigQuery, Google Cloud SQL, Apache Airflow, and Dataproc services. Ensure the development of data warehousing layers and reporting dashboards for data visualization and analytics. Coordinate cross-functional teams and manage resources to ensure timely delivery of projects. Develop project plans, manage timelines, and provide regular updates to stakeholders. Ensure adherence to cloud security best practices and regulatory compliance. Requirements: Proven experience as an IT Project Manager in multi-cloud environments. Strong understanding of Google Cloud and Azure services, especially BigQuery, Google Cloud SQL, Apache Airflow, and Dataproc. Experience in data migration, ...
I am experiencing issues with data retrieval from Google BigQuery using Pentaho Report Designer. While I have successfully connected Pentaho to BigQuery, the data retrieval process is not functioning as expected. I have verified the SQL syntax and the validity of the query in BigQuery and confirmed that the SQL is correct. - Issue: I am encountering a 404 error with the query during execution. - Resolution Needed: I need an expert who can troubleshoot these connection errors during query execution and provide a solution.
I need a skilled data professional to help me analyze structured data using Talend, Spark, and Scala. Key Responsibilities: - Utilizing Talend for data integration and tr...data professional to help me analyze structured data using Talend, Spark, and Scala. Key Responsibilities: - Utilizing Talend for data integration and transformation - Employing Spark for big data processing - Applying Scala for data analysis Ideal Skills: - Proficiency in Talend, Spark, and Scala - Experience with structured data analysis - Familiarity with data warehouses like Redshift and BigQuery - Strong analytical skills to extract insights from data This project is focused on deriving insights from structured data stored in data warehouses. Your expertise will help convert raw data into meaningful in...
I'm looking for an experienced Python developer to create a chatbot for me. The chatbot will need to: - Interact with users via a web interface - Provide text-based replies - Fetch data from Google BigQuery, specifically custom data sets The ideal candidate will have: - Extensive experience with Python and chatbot development - Familiarity with Google BigQuery - Skills in web interface design and development Please provide examples of previous chatbot projects you've worked on, particularly those that utilized Google BigQuery.
...Project Goal: Build a data extraction workflow to create a database of information from insurance documents. The workflow should handle both initial policy documents and mid-term adjustments. Minimum Requirements: Document AI: Proficiency in using Google Cloud's Document AI to extract data from PDF documents. Data Storage: Experience with storing structured data in a database (e.g., Cloud SQL, BigQuery). Data Manipulation: Familiarity with data manipulation techniques (e.g., using Python, SQL, or data transformation services) to append data to existing entries or create new rows. Workflow Automation: Understanding of how to automate workflows using Google Cloud Functions or similar tools. Data Validation: Ability to implement data validation checks to ensure data consisten...
...logging, and alerting solutions on GCP’s Operations suite - Automated Resource Deployment o Provide production-capable IaC scripts to support repeatable deployment of the GCP landing zone - Security o Provide recommended standard Organization Policy constraints - Phase 2 - Analytics Pipeline and Looker o Create pipeline to capture conversation history in Cloud Logging and BigQuery o Deliver four (4) pre-built, Looker analytics dashboards for Customer’s internal stakeholders (executive team) with visibility across the workflow. The dashboards will be created then handed over to the Customer for Customer’s use o Data and reports will be provided based on the virtual agent implemented by INI across three phases namely ...
I'm looking to develop a text-based chatbot that will interact with users on Slack, providing information from my GCP BigQuery tables. The chatbot should answer simple queries, handle complex questions with drill-down options, and present data visualizations when needed. Instead of users directly querying BigQuery, they will interact with this chatbot. Key Requirements: - Development of a text-based Slack chatbot. - Capable of simple query response and handling complex queries with drill-down options. - Able to present data visualizations. Ideal Skills: - Experience in chatbot development, particularly for Slack. - Proficiency in working with GCP BigQuery. - Skills in data visualization and making complex data understandable.
I'm looking for a skilled data analyst who can integrate data from our Warehouse Management, Sales, and Inventory systems into BigQuery. Key Responsibilities: - Data Integration: Combine data from the three specified systems into BigQuery for comprehensive analysis. - Performance Metrics Analysis: Utilize the integrated data to derive key performance metrics. The final reports generated will primarily be used by our Supply Chain team, so an understanding of supply chain dynamics will be beneficial. Ideal Skills and Experience: - Proficient in BigQuery and data integration. - Strong background in performance metrics analysis. - Familiarity with supply chain data and requirements. - Experience with warehouse management and sales system data.
I need a seasoned WebFocus Developer to facilitate a data migration from Teradata to BigQuery. Key Responsibilities: - Execute a seamless report transition from Teradata to BigQuery. - Utilize WebFocus for efficient migration processes. - Ensure report and dashboard integrity throughout the migration process. Ideal Skills and Experience: - Proficient in WebFocus, with a strong track record in data migration projects. - Extensive experience working with Teradata and BigQuery. - Excellent understanding of data integrity and quality assurance measures.
I am in need of a skilled GCP Cloud Engineer who can help manage and optimize my current GCP services. The key GCP services to focus on include Compute Engine, Cloud Storage, and BigQuery. Specific tasks for Compute Engine will encompass: - Instance management - Load balancing - Scaling and auto-scaling Ideal candidates should have solid experience with GCP, particularly with the aforementioned services. Strong knowledge of Compute Engine's instance management, load balancing, and scaling processes is crucial. Your main goal will be to ensure optimal performance and cost-efficiency of our cloud infrastructure.
I'm in need of a skilled data engineer who has good exposure to Kafka connect and its integration with Kafka (broker - schema registry) with security setup. I am doing real time data migration from Oracle -> Kafka -> BigQuery. USing No Authentication, Everything is working end to end for me (using PLAINTEXT) I am looking to advance the configuration to use SASL_PLAINTEXT or SASL_SCRAM. FYI: Using below connectors (Kafka connect) - OracleCDC Connector - BQSink Connector Evaluation Criteria (MUST) - Using SASL_PLAINTEXT/SASL_SCRAM the environment should be setup and data should sync end to end. Note: Without any security part, everything is working for me already. - Once above is done, I'll use Confluent cloud to host Broker and Schema Registry. Rest will be hosted ...
We are encountering some problems with our Power BI dashboard. The latest data we updated isn't showing up despite our best efforts to verify and ensure consistency in the formats used. Here's what you'll need for a successful project...you'll need for a successful project outcome: - Experience in troubleshooting data behavioural issues on Power BI dashboards. - Solid understanding of API Integration, as we are presently updating our data automatically. - Demonstrable skills in deciphering updates in data extraction scripts since we believe this could be a potential reason for our present struggles. - The ability to work with BigQuery and dbt, our current systems. If you're a problem-solver with skills in these areas, we'd love to talk to you about...
I'm looking for a highly-skilled data scientist or AI expert who can develop an intelligent system for my solar energy setup. The system should be capable of id...based on both historical data and real-time measurements. Key concerns: - Equipment malfunctions - Consistent underperformance - Soiling The primary objective of this project is to optimize the energy output of my solar system. Therefore, the ideal candidate for this job will have: - Extensive experience in AI and machine learning - A strong background in data analysis - Background with Google BigQuery & Looker Studio or similar - Prior experience with solar energy systems is a plus - Ability to deliver a user-friendly system Your proposal should include examples of similar projects you've completed and ...
I'm seeking a professional with at least 3-5 years of experience in Google Cloud, particularly in Google BigQuery, Google Forms, and Vertex DB. The candidate should be able to assist with a broad range of tasks in these platforms. This is a support task and you will need to be available 4 hours from mon to fri. The timings will be 10 pm to 2 am IST . The ideal candidate for this role: - Should have a minimum of 5 years of experience working with Google Cloud, especially in Google BigQuery, Google Forms, and Vertex DB - Must be adept at troubleshooting and providing effective solutions - Should be able to carry out performance optimization tasks efficiently - Should be capable of supporting migration and integration tasks - Must be proactive in identifying and resolvin...
...interface. 10. **Compliance:** - Ensure compliance with relevant local and international regulations related to cryptocurrency trading and bank transfers. ### Google Services Integration: - **Google Cloud Platform (GCP):** - Hosting and server management. - Google Cloud Storage for storing user data and documents. - Google Firebase for authentication and real-time database. - Google BigQuery for advanced data analysis and reporting. - Google Kubernetes Engine (GKE) for container orchestration if needed. ### Scalability: - Ensure the website architecture is scalable to accommodate future growth and additional features. ### Additional Information: - The website should be built using modern web development technologies. - Please provide examples of similar projects ...
I'm looking for a skilled freelancer to connect PowerBI to Google Cloud data sources and ensure daily data refreshes. Key Responsibilities: - Connect PowerBI to Google Cloud Platform - Set up daily data refresh in PowerBI Ideal Skills and Experience: - Proficient in PowerBI - Experience with Google Cloud platform, Google Sheets, Google Analytics & Google BigQuery - Skilled in data integration and working with APIs - Prior experience in setting up scheduled data refreshes This project will involve close communication with me to ensure the successful integration of vital data sources and their consistent updates on a daily basis.
I need a skilled developer to connect my Salesforce and API data with Google BigQuery. Specifically, the focus is on 'Leads' and 'Opportunities' from Salesforce, along with 'Medical record data MRN system' from the Lightning Step API. This project demands a thorough understanding of working with APIs, Salesforce, and Google BigQuery. Furthermore, because I plan to utilize Looker for reporting, proficiency with this platform is key. The final deliverable I'm looking for is a robust and efficient data schema in Google BigQuery, which enables easy report generation in Looker. Key skills and experience: - Expertise in Salesforce - Experience with Google BigQuery - Proficiency in using APIs - Understanding of data schemas - Pr...
...experienced professional to assist with various tasks in Google Cloud, specifically Google BigQuery, Google Forms, and Vertex DB. The ideal candidate should have over 5 years of experience in the field. Key Responsibilities: - Data Analysis and Querying: Your primary task will be to perform robust data analysis and querying, potentially identifying patterns and trends to help improve business operations. - Form Creation and Integration: You will also be responsible for creating and integrating Google Forms as required. - Database Management and Optimization: Lastly, you should be adept at managing and optimizing Vertex DB. Skills and Experience: - Over 5 years of hands-on experience with Google Cloud, Google BigQuery, Google Forms, and Vertex DB. - Expertise in data anal...
I need an experienced PHP developer to help fix a specific bug in CSV parsing functionality. Creating synthetic data is a process used to balance an ML/AI model for a specific purpose. Thi...can be produced in various ways. Even though the synthetic data represents genuine scenarios, there are multiple methods to create these datasets. This flexibility allows for the generation of numerous true variants of data batches, each suitable for different aspects of model training and validation. - CSV Parsing: - I need some help un Bigquery integration CSV parsing functionality. The data that needs to be parsed is in CSV format. we use Bigquery for log data analisys . The freelancer should be proficient in php data parsing and have experience in handling such data for machine l...
...over 6 years of experience in BigQuery, GCP and SQL. The ideal candidate must be able to work onsite full-time. Key Objectives: - Data Analysis & Reporting: You will be responsible for analyzing and reporting on our data, providing insights and actionable information for the business. - Data Integration & Transformation: Your role will involve integrating and transforming data from various sources, ensuring a smooth flow and consistency in data usage. - Database Optimization & Performance Tuning: You will also be responsible for optimizing our databases, ensuring they perform efficiently and meet the necessary performance standards. Duration: This is a full-time onsite permanent role. ONSITE IS A MUST. Ideal Qualifications: - 6+ years of experience in BigQuery...
I need something built to emulate the YouTube home feed. I'd like it built on the Google Cloud Platform. I'd like control over the model and to be able to provide it with content that influences my recommendations. There may be YouTube data available on BigQuery or something similar, or you may need to crawl YouTube. If we need to crawl YouTube, you should use whatever tools are available in the cloud to minimize the effort required. If YouTube uses open models for it's home feed, we should use the same. I'm looking for your expertise on the details. Thank you.
The objective of this project is to set up a Kafka ecosystem on Google Kubernetes Engine (GKE) to facilitate Change Data Capture (CDC) from MySQL and PostgreSQL databases. The captured data will be pushed to Kafka topics, and from there, it will be ingested into Google BigQuery for analytics and reporting purposes. Components Kafka: A distributed streaming platform for building real-time data pipelines and streaming applications. Kafka Connect: A tool for scalably and reliably streaming data between Apache Kafka and other systems. Debezium: A CDC tool that captures row-level changes in databases and streams them to Kafka topics. GKE: Google Kubernetes Engine, a managed Kubernetes service that simplifies deploying, managing, and scaling containerized applications. Setup Steps 1. Cre...
I need an ETL developer who is well versed in Google Cloud Platform. This individual should have: Skills & Experience: - Extensive experience in BigQuery. - Proven expertise in Dataflow. - Familiarity with Cloud Storage. Project Details: - The project involves processing data, approximately less than 1 GB. - The task will involve both data transfer and transformation. The right candidate should highlight their prior experience handling equivalent tasks. Fluent Spanish will be a bonus, as it is the preferred language for this project.
