Are you a passionate Data Engineer with expertise in building and maintaining large-scale data pipelines? Do you thrive in a cloud environment and enjoy tackling complex data challenges?
If so, we want you on our team!We are seeking a talented Staff Software Engineer to join our growing Data Infrastructure team.
In this role, you will leverage your past experiences and deep understanding of data warehousing and data lake concepts to help shape and execute Achievers’s roadmap technologies to design, develop, and maintain our data pipelines on Google Cloud Platform (GCP).
You will collaborate closely with Data Architects and Data Scientists to translate business needs into robust and scalable data solutions.
Responsibilities :
- Design, develop, and implement scalable and reliable data pipelines using Dataflow, Airflow, and Apache Beam on GCP.
- Work with Data Architects to define data requirements and translate them into technical specifications.
- Write high-quality, maintainable, and efficient code using Python, PHP, and Go.
- Collaborate with data engineers and data scientists to troubleshoot data pipelines and ensure data quality.
- Automate data pipeline deployment and monitoring processes.
- Stay up-to-date on the latest trends and technologies in the big data landscape.
- Document data pipelines and code for maintainability.
- Identify and fill gaps in the team and create the processes necessary for the team's success.
Qualifications :
- Minimum 5+ years of experience as a Software Engineer with a focus on data infrastructure.
- Proven experience in designing and developing data pipelines on cloud platforms (preferably GCP).
- Strong expertise in software development best practices (clean code, version control, testing).
- In-depth knowledge of distributed processing frameworks like Apache Beam.
- Experience with containerization technologies like Kubernetes.
- Proficiency in programming languages like Python, PHP, and Go.
- You're comfortable with the building blocks of modern back-end systems, such as horizontally scalable data infrastructure, event-driven architecture, and beyond and can clearly articulate the pros / cons of different approaches, while also providing a recommended solution based on the current context.
- You have familiarity with databases and analytics technologies in the industry, including Data Warehousing, Data Lakes, ETL and Relational Databases.
- Excellent communication, collaboration, and problem-solving skills.
Platform Engineer, SF Data & Reporting
At Astellas, we are relentless in our pursuit of scientific progress and in tackling unmet medical needs, demonstrated by our legacy in oncology, overactive bladder and transplant and our impressive pipeline in women's health, blindness and regeneration, genetic regulation, immuno-oncology, mitochon...
Azure Data Engineer
Any other certificate in Azure/AWS/GCP and data engineering hands-on experience in cloud. Azure Data Engineer (minimum years exp) NO junior profile. Azure Data Factory , Azure Data bricks Spark (PySpark or scala), SQL. EEO) to all employees and applicants for employment without regard to race, color...
Data Engineer, Business Intelligence & Analytics
As a Data Engineer, you will help drive the direction of our Data Warehouse, work hands-on with teammates across the organization to get access to relevant data, and empower teams to make data driven decisions about the direction of our business. Partner with other Data Engineers and other stakehold...
Data Engineer (31990)
The ideal candidate will be experienced in building and optimizing data pipelines on Microsoft Azure, with expertise in Azure Data Fabric and other Azure data services. Work closely with data scientists, analysts, and other stakeholders to understand requirements and implement solutions. Design, bui...
Senior Data Engineer
Implement data engineering technologies, primarily stream and batch processing systems with Apache Beam/Dataflow, Data Warehouses including BigQuery, queuing and message systems with Pub/Sub, Data Lakes with BigLake and Cloud Storage, application databases including SQL and Postgres, Redis, and Comp...
GCP Data Engineer NT1025A2
Theideal candidate will have a deep understanding of data warehousingand cloud technologies particularly within the Google CloudPlatform (GCP) ecosystem. Dataproc BigQuery Cloud Spanner Cloud Runfunctions DataflowPub/Sub. Solid understanding of the software development lifecycle and therelease proce...
Data Engineer
Participate in the implementation of data engineering and transformation solutions to ensure highly scalable, extensible, and performant solutions. Contribute to the delivery of data ingestion, transformation and extraction solutions in Enterprise Data Lake (EDL) Big Data platform. Accessibility and...
Coding and Data Management for Business Analytics
Minimum Qualifications Must have completed and achieved a minimum grade of “A” MGT301 equivalentcourse with programming/codingexperience and familiarity withPython/SQL or equivalent. Update material and grades on Quercus courseware. Attend weekly lecture and/or review lecture material. ...
Azure Data Engineer
Experience in developing Data Factory pipelines that are parametrized and reusable and worked on most of the ADF control flow activities like Get Metadata, Lookup, Copy, For Each, Databricks, Stored Procedure, If Web etc. ETL / data engineering within DWH / Data Lake / BI reporting systems. As your ...
Cyber and Data Analytics Audit, Assistant Vice President
Strong knowledge of Cyber and Cloud technologies and tools (Azure, AWS, Snowflake, Databricks, Alteryx), Identity and Access Management, Security Incident and Event Management (SIEM) technologies and cyber operations, and incident and response processes. Design, coordinate, and implement data analyt...