Job Description
Role Description :
- Designing and developing data pipelines using Cloud Technologies based on business requirements and technical design.
- Collaborating with Stakeholders, Business Analysts and Data Architects to assist and translate business requirements to technical solutions
- Develop big data and analytic solutions leveraging new or existing technology to advance client’s all lines of business
- Exploratory data analysis by querying and processing data stores and databases and generate reports.
- Design, upgrade and implement new data workflows.
- Write and maintain technical documentation
- Executes updates, patches, and other activities required to maintain and enhance the operations of on-premise or cloud-based environments
Competencies : Data Structures
Experience (Years) : 4-6
Essential Skills :
- Experience working as Data Engineer with focus on big data processing and / or relational databases with good understanding of SQL
- Experienced with working on Structured and Semi-Structured datasets.
- Experience working with Microsoft Azure Data Platform, specifically Azure SQL Database, Azure Data Factory, ADLS storage account, Azure Data Bricks and DevOps Experienced in creating data pipelines & developing complex and optimized queries
- Experienced with Workflow Management Tools : Airflow, Crontab, CA Workload Automation
- Experience with implementing CI / CD pipeline on GitHub, Jenkins / JGP.
- Experience writing complex SQL and NoSQL jobs to analyze data in traditional DBMS (MS-SQL, Oracle) environments
- Experience with integrating to back-end / legacy environments
- Experience integrating business and technology teams
- Experienced in any of the following programming / scripting languages (SQL, Python, Shell, Scala) Excellent organizational and time management skills, strong business presence with ability to multi-task and effectively deal with competing priorities
Desirable Skills :
- Previous work experience in the financial sector (insurance or benefits industry).
- Knowledgeable or at least have the basic concept of Data Visualization in any of the following tools : Tableau, PowerBI, QlikView / QlikSense
Senior Azure Data Engineer
Overall experience includes about 5+ years of experience in 1) data project development experience, Azure Data Engineer with hands-on experience in Azure Synapse, Databricks, SQL data warehouse and data visualization models 2) ETL experience in data warehousing concepts - data extraction, transforma...
Azure Lead Data Architect
Utilize Azure services like Azure SQL Database, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Cosmos DB, Azure Blob Storage, and more to build scalable and secure data solutions. Experience in configuring and implementing Azure Storage, Azure Data Factory, Azure Data Lake, Azu...
Big Data Developer
Design best architecture and select the most appropriate modeling techniques and data visualization for big data analysis. Bachelor’s Degree or Master’s in Computer Science, Engineering, Software Engineering or a relevant field. Strong experience in Python, Kafka, Spark, SQL, Hive, Couchbase, Impala...
Data Scientist/ML Engineer
You have at least 3 years experience, ideally within a Data Analyst or Data Engineer role. Processing & Interpreting data to get actionable insights. Demonstrated experience working with large and complex data sets as well as experience analyzing volumes of data. Developing scripts to process struct...
Senior Azure Cloud Engineer - Teraform, Kubernetes
Senior Azure Cloud Engineer - Teraform, Kubernetes. Knowledge transfer, training, mentoring of cloud practices across the engineering org. Proficiency in one or more major cloud platforms, such as Azure (preference), Google Cloud, AWS or others. Azure Functions, AWS Lambda etc). ...
Software Development Manager - Big Data, MADS Traffic Reporting
Passionate about Big Data and Distributed Systems? Interested in building new state-of-the-art products at Petabyte scale? Be part of a team of industry leading experts that operates one of the largest big data analytics systems at Amazon. We are applying the latest machine learning and AWS big data...
QE Big Data Lead
Data QE Lead with 8+ years QE and lead experience on ETL, Big Data & Python skills. Strong Test Lead with 5+ years of functional testing experience, including 3+ years of testing in ETL, database, Big Data environment. Complex query writing, procedures, routines in Big Query. ...
data processing consultant
Develop and maintain computer databases. Oversee the analysis of employee data and information. Setup the IT infrastructure to allow applications to access databases. ...
Network Engineer - Azure Cloud, NSG Security Rules
Resources will use NSG flow logs as well as Azure Migrate logs to gather network flows that workloads are using for Azure East/West communications. Further network analysis is required for North/South traffic that is ingress/egress from the Azure Internet Edge. This would effectively block any east/...
Data Engineer
Collaborate with data analysts and stakeholders to define data requirements, and design efficient and effective data models within BigQuery. We're looking for a skilled and enthusiastic Data Engineer to join our growing team! We're a close-knit group building and maintaining mission-critical data pi...