Job Responsibilities :
- Create and maintain optimal data pipeline architecture
- Implement data products curated by our Chief Data Office, as well as custom data models for fit for use.
- Ensure data quality and integrity across various data sources and systems to ensure data accuracy, completeness, and reliability.
- Optimize data pipelines for performance and scalability.
- Provide technical support to promptly resolve escalated incidents / outages.
- Develop and document a detailed solution design, impart your subject matter expertise throughout life cycle.
- Take business, Enterprise Architecture, system performance and development standards requirements, then develop functional, technical and user interface designs for an application and / or system.
- Find ways to keep costs low, help come up with strategic solutions to support cost effectiveness and enhance stakeholder experience.
- Conduct code reviews to address quality, standards compliance, reusability and ease of maintenance, Operational Readiness Reviews, and support gating and review signoffs for solution design.
- Ensure design leverages existing reusable components, traces back to business requirements, and that new modules are designed with reusability in mind.
- Keep up to date with the latest industry trends and technologies related to data engineering.
Skills and Experience Required :
Required :
- 10+ years of relevant experience in a related field of job function.
- Experience with : Java, Spring,
- Experience with big data tools : Hadoop, HDFS, ADLS, ADF, Spark, Kafka, Databricks, Dremio etc.
- Experience with relational SQL and NoSQL databases, including Cassandra.
- Experience designing production grade, scalable applications and microservices.
- 5+ years of Capital Markets experience.
- Experience working on Agile Teams
Desired :
- Experience in Python and / or Scala
Sr. Data Engineer-Java
Implement data products curated by our Chief Data Office, as well as custom data models for fit for use. Ensure data quality and integrity across various data sources and systems to ensure data accuracy, completeness, and reliability. Keep up to date with the latest industry trends and technologies ...
SR Backend Java Engineer; Distributed Systems
We’re hiring a SR Backend Java Engineer with a strong background in large scale distributed systems. Our customer is embarking on a transition from data centers to Public cloud infrastructure, and you will work on a complete Java stack that serves hundreds of billions of transactions daily. You’ll w...
Staff Software Engineer with Java (Springboot) to build out features on a Master Data Management product from start to finish.
Staff Software Engineer with Java (Springboot) to build out features on a Master Data Management product from start to finish. ...
Sr Data Engineer Lead
Data Engineer with AWSGlue Lambda SQL Python Dovps Redshift. Perform data cleansing data validation etc. Must have working knowledge in designing and implementingdata pipelines on any of the cloud providers (AWS is preferred). Must be able to work with large volumes of data coming from varioussource...
Senior Data Engineer, Java
As part of the Data Engineering team, you will be working with a team of smart, friendly, and dedicated Data Engineers, ML Engineers, and Data Scientists determined to develop high-quality data products that power core features of our app. The ideal candidate has a passion for data, a strong backgro...
Sr Data Engineer
The company invests in all major asset classes, including public equity, private equity, real estate, infrastructure and fixed-income instruments, and currently has offices in Toronto, UK, EU, USA, Asia and South America.Client is a professional investment management organization that globally inves...
Sr. AWS ETL Data Engineer (Amazon Redshift ETL)
AWS ETL Data Engineer (Amazon Redshift ). Design, develop, and maintain ETL processes using AWS Glue to extract, transform, and load data from various sources (including S, ORC/Parquet/Text files) into AWS Redshift. Transform, clean, and preprocess data from various sources to load into Redshift and...
Java Fullstack Developer + Data Engineer
Role: Java Full stack Developer + Data Engineer. Combination of Data Engineer and Java Fullstack Dev API, AWS, Kubernetes, API, No SQL. API development, Java script, Kafka Stream, Kafka Connectors/ IBM MQ, Active MQ, AWS. Strong foundation in programming languages like JavaScript, Python, or Ruby. ...
Data Engineer with Java Development
Document data engineering processes, creating data flow diagrams, and maintaining metadata for data lineage and cataloging. This role requires a blend of a Data Engineer and a Developer who will be responsible for designing, building, monitoring, tuning, and troubleshooting data pipelines for Global...
Sr. Java Dev / Big Data Developer
The primary focus of this role is to provide re-engineering expertise, covering design, implementation and support of the RM and FRTB applications, namely Aggregation of Market Risk and Market Data Shocks and Scenarios generation, and Risk Application Service Layer. Big Data applications using Java/...