Azure Databrick SME at BayRock Labs – Milpitas, California

About BayRock Labs

At BayRock Labs, we pioneer innovative tech solutions that drive business transformation. As a leading product engineering firm based in Silicon Valley, we provide full-cycle product development, leveraging cutting-edge technologies in AI, ML, and data analytics. Our collaborative, inclusive culture fosters professional growth and work-life balance. Join us to work on ground-breaking projects and be part of a team that values excellence, integrity, and innovation. Together, let's redefine what's possible in technology.

We are seeking a highly skilled Data Platform Engineer (SME level) with deep hands-on expertise in Azure Databricks, data engineering, and platform engineering. The ideal candidate will have strong experience in Microsoft Azure, DevOps, and automation, along with working knowledge of AI/ML Ops platforms. This role is critical in designing, building, and maintaining scalable, secure, and high-performance enterprise data solutions.

Key Responsibilities

• Build and maintain enterprise-grade data platforms leveraging Azure Databricks.
• Design, develop, and optimize data pipelines, ETL workflows, and orchestration processes.
• Support platform engineering initiatives to enhance scalability, reliability, and performance.
• Integrate AI/ML Ops capabilities into data platforms to enable advanced analytics and machine learning.
• Implement DevOps and automation best practices for CI/CD, monitoring, and infrastructure as code.
• Ensure security, compliance, and governance across all data systems.
• Collaborate with cross-functional teams (data scientists, architects, business stakeholders) to deliver end-to-end solutions.

Required Skills & Experience

• Expert-level hands-on experience with Azure Databricks (clusters, notebooks, Delta Lake, MLflow).
• Strong background in data engineering (ETL, data modeling, big data frameworks).
• Proven expertise in platform engineering and enterprise-scale data solutions.
• Proficiency in Microsoft Azure services (Data Lake, Synapse, Key Vault, Functions, Logic Apps).
• Solid understanding of DevOps practices (CI/CD pipelines, Git, Terraform/ARM templates, automation).
• Working knowledge of AI/ML Ops platforms (model deployment, monitoring, ML lifecycle management).
• Strong skills in SQL, Python, and Spark for data processing and analytics.
• Excellent problem-solving, communication, and stakeholder management skills.
• 10+ years of experience in enterprise data engineering/platform roles preferred.

Preferred Qualifications

• Experience with enterprise supply chain or manufacturing data platforms.
• Familiarity with data governance frameworks and compliance standards.
• Certifications in Azure Data Engineer Associate or Azure Solutions Architect Expert.

The pay range for this role is:
80 - 90 INR per hour(Hybrid (Milpitas, California, US))