Our client, a leading global logistics company, is embarking on a new initiative to build and enhance their data platform. They are currently seeking a Data Engineer to join their rapidly expanding data team. The successful candidate will enjoy excellent benefits, including a flexible work-from-home policy!
Â
Key Responsibilities:
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into scalable data solutions.
Design and develop robust data pipelines, ETL processes, and data integration frameworks to collect, transform, and store large volumes of structured and unstructured data from various sources.
Implement and maintain data warehouses, data lakes, and data marts to support reporting, analytics, and business intelligence initiatives.
Identify, assess, and incorporate new tools, technologies, and best practices to drive continuous improvement in data engineering processes.
Ensure the integrity, security, and privacy of data by implementing appropriate data governance standards and procedures.
Perform data profiling, cleansing, validation, and enrichment activities to optimize the quality and usability of data.
Collaborate with infrastructure teams to ensure optimal performance, scalability, and availability of the data infrastructure.
Monitor data pipelines, identify performance bottlenecks, and troubleshoot issues to ensure smooth and uninterrupted data flow.
Document data engineering processes, data models, and architectural decisions to facilitate knowledge sharing and enable effective collaboration.
Stay updated with emerging trends, technologies, and industry practices in the data engineering domain, and provide recommendations for improvements and enhancements.
Â
Qualifications and Skills:
Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
3+ years of experience as a Data Engineer or in a similar role, preferably within the logistics industry.
Strong proficiency in SQL and experience working with relational databases (e.g., MySQL, PostgreSQL) and big data technologies (e.g., Hadoop, Spark).
Proficiency with Python and familiarity with data processing frameworks (e.g., Apache Beam, Apache Flink).
Experience with data modeling, data integration, and data warehousing concepts and technologies (e.g., Snowflake, Redshift).
Solid understanding of ETL processes, workflow orchestration tools (e.g., Airflow, Luigi), and version control systems (e.g., Git).
Familiarity with cloud platforms (e.g., AWS, GCP, Azure) and working knowledge of containerization technologies (e.g., Docker, Kubernetes).
Strong analytical, problem-solving, and critical-thinking skills with a keen attention to detail.
Excellent communication and collaboration skills, with the ability to explain complex technical concepts to non-technical stakeholders.