Big Data Engineer (Databricks)

Required qualifications

• Experience with Databricks Asset Bundles (DAB).
• Experience using Azure DevOps.
• Knowledge of incremental processing and schema evolution.
• Healthcare or Insurance industry experience.
• Experience with cloud-based data platforms.

Key responsibilities

• Develop and maintain data pipelines using Databricks.
• Support the design and optimization of Delta Lake tables.
• Work with Unity Catalog to support data governance and access management.
• Participate in CI/CD processes using Azure DevOps and Databricks Asset Bundles (DAB).
• Implement incremental data processing and support schema evolution requirements.
• Collaborate with team members to troubleshoot data quality and performance issues.
• Assist with data modeling activities and platform optimization efforts.
• Document solutions and follow engineering best practices.
• Ideally 5 YOE
• Strong expertise in Apache Spark, including delivery on Databricks

Nice to have

• Azure Databricks certifications.
• Exposure to Medallion Architecture.
• Data quality and monitoring framework

Conditions

• Contrato a largo plazo.
• 100% Remoto.
• Vacaciones y PTOs
• Posibilidad de recibir 2 bonos al año.
• 2 revisiones salariales al año.
• Clases de inglés.
• Equipamiento Apple.
• Plataforma de cursos en linea
• Budget para compra de libros.
• Budget para compra de materiales de trabajo
• mucho mas..