Senior Data Engineer

Drustech is looking for an enthusiastic and skilled Senior Data Engineer to join our team on immediate basis. This role is an exciting opportunity for professionals with 5-8 years of experience to work on cutting-edge cloud data solutions. As a Data Engineer, you will collaborate closely with our data and analytics team to design, build, and maintain scalable data pipelines enabling our clients to harness the power of their data.

Note: Immediate joiners will be preferred.

Key Responsibilities:

• Extensive experience in Data & Analytics field with 5-8 years of hands on experience in Databricks, PySpark, Python and SQL.
• Proven experience in implementing data solutions on the Databricks platform with hands on experience in setting up Databricks cluster, working in Databricks modules, data pipelines for ingesting and transforming data from various sources into Databricks
• Configure, set up, and manage Databricks clusters, workspaces, and notebooks to ensure optimal performance, scalability, and resource allocation.
• Implement data pipelines for ingesting, cleansing, and transforming data from various sources into Databricks, ensuring data quality, consistency, and reliability. Develop ETL (Extract, Transform, Load) processes as needed.
• Develop, optimize, and maintain Spark-based ETL workflows and data pipelines for data preparation and transformation within Databricks.
• Knowledge of Performance Optimization, Monitoring and Automation would be a plus.
• Understanding of data governance, compliance, and security best practices.
• Strong Proficiency in PySpark, Databricks Notebooks, SQL, Python.
• Proficiency in SQL for data querying and manipulation.
• Experience with data modelling, ETL processes, and data warehousing concepts.
• Strong problem-solving and troubleshooting skills.