Cpl are supporting a dynamic client in their search for a Senior Data Engineer. In the role you will play a key role in developing and scaling our strategic data platform on Azure Cloud. You'll focus on building a robust reference data system for securities and pricing, with opportunities to expand into other domains. Bring your passion for innovation and help shape our evolving data landscape.
Key Responsibilities:
- Collaborate with cross-functional teams to enhance our Azure-based data platform.
- Lead the creation of a reference data system for securities and pricing, with future expansion into other areas.
- Design, implement, and maintain scalable data pipelines and storage solutions.
- Utilize Azure tools like Data Factory, Lake Gen 2, and Databricks for efficient data processing.
- Develop and integrate APIs using frameworks such as FastAPI.
- Ensure security, scalability, and performance best practices in cloud solutions.
- Align data solutions with business goals and evolving requirements.
- Continuously optimize data architecture and engineering processes.
Must-Have Qualifications:
- 10+ years of experience with deep expertise in Python and PySpark.
- Proven experience as a Data Engineer with extensive knowledge of Azure Cloud services is essential
- Proficiency with Azure Data Factory, Lake Gen 2, Databricks, Functions, and API Gateway.
- Hands-on experience with SQL across relational and NoSQL databases.
- Proven skills in developing APIs with FastAPI or similar frameworks.
- Expertise in ETL/ELT processes and DevOps tools (e.g., Git, Jenkins).
- Ability to thrive in a fast-paced, collaborative environment.
#LI-LDM
