We are seeking a Senior Data Engineer with specialized experience in Data Lakehouse and AWS technologies to design, develop, and optimize our data management systems.
The ideal candidate will have a strong foundation in data architecture and data engineering principles, with extensive experience in AWS cloud services.
This role involves managing large datasets, ensuring data integrity, and providing scalable data storage, processing, and analytics solutions.
Key Responsibilities :
- Develop and maintain scalable and secure data architectures using AWS services like S3, Redshift, Glue, and Athena
- Design and implement efficient data pipelines within the AWS environment, facilitating seamless data integration and accessibility
- Leverage AWS analytics and machine learning services to enhance business insights and decision-making processes
- Ensure data governance, quality, and compliance within our AWS-based data lakehouse infrastructure
- Experience building and optimizing big data’ data pipelines, architectures, and data sets
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
- Collaborate with various teams to understand data needs and deliver innovative cloud-based data solutions
- Stay updated with the latest AWS technologies and data lakehouse trends to keep our data infrastructure at the forefront of innovation
Knowledge, Skills, Abilities and Competencies :
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field
- Extensive experience with AWS services, particularly in data warehousing and analytics (certifications preferred)
- Proficiency in SQL, Python, or Scala for data processing and pipeline development
- Strong understanding of data lakehouse architecture and experience in implementing such systems
- Solid background in data engineering practices, including data modeling, ETL development, and data warehousing
- Excellent problem-solving abilities and strong communication skills
Software Engineer, User Data (Intermediate/Senior)
Working with large data sets and various databases including Aerospike, Elasticsearch, Redis, ScyllaDB, Redshift, TiDB, MariaDB. Our real-time advertising bidding system handles over 3,000,000 requests per second and stores several terabytes of data every day. Very strong problem solving skills in d...
Senior Risk Data Engineer / Risk Management Data Analyst - Toronto, ON
Senior Risk Data Engineer / Risk Management Data Analyst . Strong technical skills (working with SQL, data handling, data mapping). Strong data analysis skills are required including analyzing large data sets in using SQL queries and other modeling tools. Producing and using Data Catalogues, underst...
Senior Azure Data Engineer
Overall experience includes about 5+ years of experience in 1) data project development experience, Azure Data Engineer with hands-on experience in Azure Synapse, Databricks, SQL data warehouse and data visualization models 2) ETL experience in data warehousing concepts - data extraction, transforma...
Senior Data Engineer
Big Data Management, Cloud Computing, Database Development, Data Mining, Data Warehousing (DW), ETL Processing, Group Problem Solving, Quality Management, Requirements Analysis. You are accountable for architecting, implementing and managing data models and data pipelines, developing, maintaining an...
Senior Engineer - Data Governance and Quality - Deloitte Global Technology
The data governance and data quality engineer is an individual contributor that provides specialized technical and functional expertise that works in collaboration with functional teams and stakeholders to modeling, developing, enhancing and deploying technical components related to the data governa...
Senior Data Engineer
Are you passionate about using data to shape the future of Faire's product features for small and medium businesses (SMBs)? If so, we want to talk to you! We're looking for a Senior Data Engineer with a deep expertise in designing and building data systems and a dedication to applying software devel...
Senior Data Engineer
Ness is a full lifecycle digital engineering firm offering digital advisory through scaled engineering services. As your tech partner, we help engineer your company’s future with cloud and data. Combining our core competence in engineering with the latest in digital strategy and technology, we seaml...
Senior Data Engineer Lead to oversee critical data engineering activities within the Digital Health sector
Senior Data Engineer Team Lead to oversee critical data engineering activities within the Digital Health sector. Provide strategic oversight of data engineering activities, shaping platform strategy and data architecture. Implement and manage data access policies in coordination with the Data Govern...
Senior Data Engineer
You will contribute to the development and implementation of data pipelines and ETL processes to extract, transform, and load data into Manulife's data warehouse and utilize your expertise in building pipelines using, Python, Azure Data Factory, Databricks, Synapse, SQL server, and Azure functions t...
Senior QA Engineer - Data Platform
As a Senior QA Engineer in Opendata, you will lead the effort for building the QA framework from the ground up and the overall QA of the Data Platform for Opendata. Check the data source locations and formats, perform a data count, and verify that the columns and data types meet the requirements. Ve...