Are you a passionate Data Engineer with expertise in building and maintaining large-scale data pipelines? Do you thrive in a cloud environment and enjoy tackling complex data challenges?
If so, we want you on our team!We are seeking a talented Staff Software Engineer to join our growing Data Infrastructure team.
In this role, you will leverage your past experiences and deep understanding of data warehousing and data lake concepts to help shape and execute Achievers’s roadmap technologies to design, develop, and maintain our data pipelines on Google Cloud Platform (GCP).
You will collaborate closely with Data Architects and Data Scientists to translate business needs into robust and scalable data solutions.
Responsibilities :
- Design, develop, and implement scalable and reliable data pipelines using Dataflow, Airflow, and Apache Beam on GCP.
- Work with Data Architects to define data requirements and translate them into technical specifications.
- Write high-quality, maintainable, and efficient code using Python, PHP, and Go.
- Collaborate with data engineers and data scientists to troubleshoot data pipelines and ensure data quality.
- Automate data pipeline deployment and monitoring processes.
- Stay up-to-date on the latest trends and technologies in the big data landscape.
- Document data pipelines and code for maintainability.
- Identify and fill gaps in the team and create the processes necessary for the team's success.
Qualifications :
- Minimum 5+ years of experience as a Software Engineer with a focus on data infrastructure.
- Proven experience in designing and developing data pipelines on cloud platforms (preferably GCP).
- Strong expertise in software development best practices (clean code, version control, testing).
- In-depth knowledge of distributed processing frameworks like Apache Beam.
- Experience with containerization technologies like Kubernetes.
- Proficiency in programming languages like Python, PHP, and Go.
- You're comfortable with the building blocks of modern back-end systems, such as horizontally scalable data infrastructure, event-driven architecture, and beyond and can clearly articulate the pros / cons of different approaches, while also providing a recommended solution based on the current context.
- You have familiarity with databases and analytics technologies in the industry, including Data Warehousing, Data Lakes, ETL and Relational Databases.
- Excellent communication, collaboration, and problem-solving skills.
Senior Data Engineer
A period of over 4 years and up to and including 6 years in data ingestion, data modelling, data engineering, and software development. Design, build and productionize modular and scalable data pipelines and data infrastructure leveraging the wide range of data sources across the organization. Work ...
Staff Software Engineer - Data Platforms
Are you passionate about using data to shape the future of Faire's product features for small and medium businesses (SMBs)? If so, we want to talk to you! We're looking for a leader in our Core Data Infrastructure team to work closely with Product and Strategy Analysts, Data Scientists, and Software...
Data & Analytics - Senior Engineer, Digital Platform
As the Senior Engineer, Digital Platform, you will leverage modern software development and operational practices to support a comprehensive approach to Digital Analytics-ensuring the right data is collected, and the appropriate tools and technologies are in place to support in-depth understanding a...
Customer & Product Support / Test Engineer / Technical Sales Engineer
Coordinate with On-Board and Ground Based Product Engineering Departments on new opportunities requiring engineering/ application development. Manage Domestic and International sales, customer support, and product support efforts as directed by the Business & Operations Manager. Prepare and update g...
Senior Director, Business Development - Data & AI
Our services include digital strategy consulting, product development, brand and experience design, digital marketing, and data and AI services. While working with Business Development and Marketing leadership on a combination of product, brand, market, and sales strategies, the Senior Director will...
Sr. SDE, Measurement, Measurement, Ad Tech, and Data Science (MADS)
Work with the data to understand advertiser needs and develop most efficient ways of handling it. You'll work closely with our top notch team of product managers and scientists to invent and build new approaches, and iterate on what works best. You'll get feedback from principal and sr engineers, as...
Associate Director- Lead Data Engineer
Knowledge of Docker/ Kubernetes and Strong Computer Science Knowledge; Proficiency in writing clean, maintainable, and efficient Python code; Self-motivated and pro-active with proven ability to work accurately and under pressure to meet deadlines; Strong communication skills - Ability to summarize ...
Director, Data Platforms, Reference, Total Fund Management & Risk
Work cross-functionally: with upstream teams to guide and consult them on how to structure their data to expose good data contracts, and with downstream including data consumers and data teams to expose well-designed and well-documented data models and metrics. Experience in data modeling, data arch...
Manager, Master and Reference Data (MRD)
The Manager, Master and Reference Data (MRD), leads a team responsible for defining and implementing a corporate vision and strategy to enhance master and reference data capabilities for operational, analytical and reporting needs with a focus on key master and reference data domains (e. Leads visio...
BIG DATA Application Developer & Product Trainer | Macropus Global Ltd.
Architect, implement, and test data processing pipelines and data mining / data science algorithms on a variety of hosted settings such as AWS, Azure, client technology stacks. Candidate must have a deep understanding of logical and physical data modeling for OLTP and OLAP systems. Hands on developm...