Databricks Engineer in Pittsburgh, Pennsylvania at Techstra Solutions LLC

• Develop and maintain data pipelines using Databricks and Apache Spark.
• Assist in building scalable ETL/ELT processes to ingest, transform, and curate data from multiple sources.
• Collaborate with senior engineers to design and implement Lakehouse solutions following Medallion Architecture best practices.
• Write efficient PySpark and SQL code to process large datasets.
• Support data validation, testing, and troubleshooting to ensure data quality and reliability.
• Participate in performance tuning and optimization of Databricks workloads.
• Assist with implementing data governance, security, and access controls.
• Work closely with data analysts, AI engineers, and business stakeholders to understand data requirements.
• Contribute to CI/CD processes, source code management, and deployment automation.
• Participate in Agile ceremonies including sprint planning, daily standups, retrospectives, and code reviews.
• Document technical designs, data flows, and implementation standards.

• Master's degree in Computer Science, Data Engineering, Data Science, Information Systems, or a related technical discipline.
• Strong understanding of data engineering concepts and relational databases.
• Experience with Python and SQL through academic projects, internships, or personal work.
• Knowledge of Apache Spark fundamentals.
• Familiarity with Databricks or cloud-based analytics platforms.
• Understanding of data modeling and ETL/ELT processes.
• Knowledge of Git and version control best practices.
• Strong analytical, problem-solving, and debugging skills.
• Excellent communication and collaboration abilities.

Preferred Qualifications

• Hands-on experience with Databricks through coursework, internships, certifications, or personal projects.
• Experience with Microsoft Azure, AWS, or Google Cloud Platform.
• Familiarity with Delta Lake and Medallion Architecture.
• Knowledge of Azure Data Factory, Microsoft Fabric, or similar data integration tools.
• Exposure to data warehousing concepts and dimensional modeling.
• Experience with Power BI or Tableau.
• Databricks certifications are a plus.
• Understanding of data governance and data quality principles.
• Passionate about modern data engineering technologies.
• Eager to learn from experienced engineers and continuously improve technical skills.
• Strong team player who enjoys collaborating across technical and business teams.
• Curious mindset with a willingness to explore new tools and technologies.
• Highly organized with strong attention to detail.
• Self-motivated, adaptable, and receptive to feedback.
• Committed to delivering high-quality, scalable solutions.