- Solid practical experience in the implementation of Lakehouse architectures as well as in the construction and operation of Databricks.
- In-depth knowledge of Delta Lake, Unity Catalog, DLT/SDP, authorization models and governance mechanisms.We are looking for a Databricks Data Engineer (m/f/d) for our end customer starting in November.
As a Databricks Data Engineer, you will work closely with business units and other data engineers, delivering robust, scalable data solutions – iteratively, customer-centric, and within a Scrum setup.
On the path to our Azure/Databricks target architecture, you will migrate the existing Oracle/Informatica data warehouse to a modern Azure Lakehouse architecture. You will migrate ETL processes to high-performance Databricks pipelines, ensure data quality and governance, and support testing, parallel operation, and the cutover.
Requirements:
Databricks & Lakehouse experience –
Programming & Data Competence –
- Solid experience in Python, PySpark and SQL, as well as in performance optimization of large data sets.
- AI-based support is being used effectively to increase efficiency.
Migration expertise – ideally, experience in replacing traditional ETL processes and migrating to Databricks.
Data modeling & DWH expertise – solid experience in a traditional business intelligence environment: data storage, data modeling, data transport, and SCD2-compliant versioning.
Agile working style – experience as a developer within a Scrum team.
Communication skills – excellent written and spoken German skills and the ability to communicate confidently and clearly with business departments and other development teams.