Our client, a listed conglomerate, is looking for a data engineer for their upcoming real-time big data projects. The selected candidate will be working in their Innovative Tech Hub (young and dynamic team, startup culture) and will be responsible for the new development and design of the cloud-based data platform. Great opportunity for data engineers to grow in a data-driven team.
Responsibilities:
Design, build and optimize data pipelines for the real-time data platform with the use of AWS and its big data tools
Build and test the infrastructure for data management on AWS
Work closely with software engineers to build backend applications
Build analytical tools to leverage the data pipeline and provide insights on key business metrics.
Perform POC projects
Requirements:
3+ years of hands-on experience in data engineering and ETL development
Strong experience in Python and Spark
Strong experience in using public cloud platforms (AWS/ Azure/ GCP).
Experience with Apache Kafka or other real-time streaming technology will be a huge advantage
Good spoken and written skills in Cantonese and English