Are you a passionate Data Engineer with expertise in building and maintaining large-scale data pipelines? Do you thrive in a cloud environment and enjoy tackling complex data challenges?
If so, we want you on our team!We are seeking a talented Staff Software Engineer to join our growing Data Infrastructure team.
In this role, you will leverage your past experiences and deep understanding of data warehousing and data lake concepts to help shape and execute Achievers’s roadmap technologies to design, develop, and maintain our data pipelines on Google Cloud Platform (GCP).
You will collaborate closely with Data Architects and Data Scientists to translate business needs into robust and scalable data solutions.
Responsibilities :
- Design, develop, and implement scalable and reliable data pipelines using Dataflow, Airflow, and Apache Beam on GCP.
- Work with Data Architects to define data requirements and translate them into technical specifications.
- Write high-quality, maintainable, and efficient code using Python, PHP, and Go.
- Collaborate with data engineers and data scientists to troubleshoot data pipelines and ensure data quality.
- Automate data pipeline deployment and monitoring processes.
- Stay up-to-date on the latest trends and technologies in the big data landscape.
- Document data pipelines and code for maintainability.
- Identify and fill gaps in the team and create the processes necessary for the team's success.
Qualifications :
- Minimum 5+ years of experience as a Software Engineer with a focus on data infrastructure.
- Proven experience in designing and developing data pipelines on cloud platforms (preferably GCP).
- Strong expertise in software development best practices (clean code, version control, testing).
- In-depth knowledge of distributed processing frameworks like Apache Beam.
- Experience with containerization technologies like Kubernetes.
- Proficiency in programming languages like Python, PHP, and Go.
- You're comfortable with the building blocks of modern back-end systems, such as horizontally scalable data infrastructure, event-driven architecture, and beyond and can clearly articulate the pros / cons of different approaches, while also providing a recommended solution based on the current context.
- You have familiarity with databases and analytics technologies in the industry, including Data Warehousing, Data Lakes, ETL and Relational Databases.
- Excellent communication, collaboration, and problem-solving skills.
Senior Director, Enterprise Data & Analytics
Define target state for enterprise data and analytics capabilities, build support for and take-up of innovative, technology and data-led approaches to a highly strategic multi-year enterprise-wide transformation delivering market leading data and analytics platform. Drive and tailor adoption of data...
Senior Data Engineer (Reasoning ML)
As our Senior Data Engineer (Reasoning ML), you will be responsible for creating and curating the datasets for training and evaluating our ML planner components. You will join a growing team of computer science and robotics experts leveraging machine learning, data and cloud infrastructure to build ...
Rx Data Enrichment Engineer - Pharmacy Domain
RX data enrichment - BPS resource with 3-6 years experience Preferred experience in Pharmacy/healthcare domain specially drug data entry and enrichment/formulary updates. ...
Data Engineer
Collaborate with data architects, data scientists, analysts and other business consumers to quickly and thoroughly analyze business requirements to populate the data warehouse, optimized for reporting and analytics. In depth knowledge and hands-on experience with Data Engineering, Data Warehousing a...
Data Analytics Auditor, Global Wealth & Insurance Audit
The ideal candidate will have working Wealth Business knowledge, and technical knowledge and experience in analytics and data acquisition, and will have the ability to provide data, execute data analytics, and develop continuous monitoring MIS dashboards. The incumbent will be working directly with ...
2024 Customer Operations Graduate Leadership Program- AI & Data Engineering
We lead applied data science projects to support contact centre operations, sales, marketing, media, and more! Bell's forward-thinking AI & Data Engineering team is helping create the ultimate service experience for our residential, wireless and small business consumers. Collaborate with business, o...
Cloud Data Engineers
Must have working knowledge in designing and implementing data pipelines on any of the cloud providers (AWS is preferred). Perform data cleansing, data validation etc. Must be able to work with large volumes of data coming from various sources. Hands on ETL developer who is good at python, SQL. ...
Electrical Engineer (Data Centre)
American engineering company based in Dallas, specializing in the design and digital construction of data centres across the USA and Canada. Our projects involve complex electrical, mechanical, automation, and security systems necessary to power, cool, and secure these high-demand facilities, often ...
Sr. QA Analyst to create and execute automation tests for Data/ETL projects using TOSCA for a large insurance client - 5152
Experience with data warehouse, enterprise data lake, and other big data tools and concepts. Applications Operations Services & Quality Engineering (AOSQE) is looking for a Senior Software Development Engineer in Test (Data/ETL Testing) with a proven track record of leading testing activities to joi...
UHIP Records and Data Administrator
Reporting to the Assistant Director, International Student Services, the UHIP Records and Data Administrator ensures the maintenance of student UHIP records, databases and data transfers by overseeing the data collection and analysis processes, creating and updating student records, and verifying an...