Senior Data Engineer

We are seeking a Senior Data Engineer to play a pivotal role in building our data platform from the ground up. You will be responsible for designing, developing, and deploying a scalable, secure, and open-source based data platform on the AWS cloud. This is a unique opportunity to shape the future of our data infrastructure and empower data-driven decision making across the organization.
Design and implement a robust data platform architecture on AWS, leveraging open-source technologies whenever possible.
Evaluate and select appropriate open-source tools and services for data ingestion, transformation, storage, and analytics.
Develop and maintain data pipelines for efficient movement and processing of large datasets.
Ensure data quality and consistency throughout the data lifecycle.
Implement data security best practices to protect sensitive information.
Collaborate with data scientists, analysts, and other stakeholders to understand data needs and translate them into technical requirements.
Automate data platform tasks and processes for improved efficiency and reliability.
Monitor and troubleshoot data platform performance to identify and resolve issues proactively.
Stay up-to-date on the latest trends and technologies in the data engineering space.
Document data platform design, architecture, and processes for future reference.

Proven experience 4+ years as a Senior Data Engineer or similar role.
Strong understanding of data engineering principles and best practices.
Expertise in designing and building scalable data pipelines.
Proficient in working with open-source data technologies (e.g., Apache Spark, Apache Airflow, Kafka, etc.).
Experience with cloud platforms, preferably AWS.
Solid knowledge of SQL and experience with relational databases.
Excellent written and verbal communication skills.
Ability to work independently and as part of a cross-functional team.
Bonus Points
Experience with machine learning and data science workflows.
Experience with data governance and data security frameworks.
Experience with DevOps principles and practices.
Experience building data platforms for similar industries.