Talent.com
Business Intelligence Specialist

Business Intelligence Specialist

MillenilinkGreater Toronto Area, Canada
Il y a 8 jours
Type de contrat
  • Temps plein
Description de poste

Job Title : Business Intelligence Specialist (Data Modeler & ETL Developer – Hybrid )

Overview : We are seeking a highly skilled Data Modeler & ETL Developer to design, develop, and implement an efficient data ingestion and transformation framework. The ideal candidate will have expertise in both data modeling and ETL processes, leveraging Azure Data Factory , Databricks , and Oracle to load and transform data into a data lake and data warehouse. In this hybrid role, you will collaborate with both business and IT teams to build solutions that enable seamless data processing and reporting.

Key Responsibilities :

1. Data Modeling & Design :

  • Analyze the existing data model and propose optimizations to meet evolving business requirements and improve reporting efficiency.
  • Design physical data models, ensuring the correct mapping of data from sources to reporting destinations.
  • Collaborate with business analysts and IT teams to assess data requirements, ensuring that the physical model supports ETL processes and reporting needs.
  • Reverse engineer and document existing SQL logic to enhance model design and improve ETL efforts.
  • Maintain source-to-target mapping documentation and dimensional models , including slowly changing dimensions for data warehouse solutions.

2. Data Ingestion & ETL Development :

  • Design and implement ETL frameworks to ingest data from Oracle to Azure Data Lake (using Azure Data Factory ), focusing on both initial load and incremental ETL.
  • Use Azure Databricks , PySpark , and SQL to build transformations of raw data into curated data in the data lake, ensuring data quality and consistency.
  • Develop, maintain, and troubleshoot data pipelines to move data into the Databricks SQL Warehouse and update data marts as required.
  • Perform performance tuning of ETL jobs and troubleshoot ETL issues, optimizing pipeline efficiency and ensuring minimal latency.

    • 3. Change Data Capture & Data Transformation :

  • Implement Change Data Capture (CDC) processes using Oracle Golden Gate and Azure Data Factory for seamless and efficient data synchronization.
  • Work with IT partners to configure Oracle Golden Gate and provide guidance on best practices.
  • Create and optimize data transformation scripts using SQL , PySpark , and Databricks to convert raw data into usable formats for reporting and analytics.

    • 4. Testing & Quality Assurance :

  • Conduct unit testing and end-to-end integrated testing for both full and incremental data loads , ensuring data accuracy and integrity throughout the pipeline.
  • Perform data consistency checks and troubleshoot performance bottlenecks, focusing on optimizing data load and transformation operations.
  • Ensure high-quality ETL mappings, transformation logic, and testing results are documented and reviewed.

    • 5. Go-Live & Production Deployment :

  • Plan, configure, and execute production deployment steps , including the creation of production deployment documentation and scripts for go-live.
  • Provide support during go-live and monitor post-deployment performance to address any issues that may arise.

    • 6. Documentation & Knowledge Transfer :

  • Develop and document detailed ETL designs , transformation logic, troubleshooting steps, and configuration details.
  • Lead knowledge transfer sessions to educate business and IT stakeholders on the ETL processes, transformations, and best practices.
  • Assist with training and support for internal teams, ensuring successful adoption of the new platform and tools.
  • Create and maintain comprehensive documentation for ETL pipeline designs , data models , and source-to-target mappings .

    • 7. Ongoing Support & Optimization :

  • Continuously monitor ETL pipeline performance, providing recommendations for improvements to reduce load times and improve data processing.
  • Work closely with business analysts , data architects , and other developers to address new requirements and improve existing processes.
  • Proactively suggest infrastructure and process improvements to enhance data processing and reporting performance.

    • Skills & Experience Requirements :

  • Technical Expertise :
  • 7+ years of experience working with SQL Server , T-SQL , Oracle , PL / SQL , and similar relational databases.
  • 2+ years of experience working with Azure Data Factory , Databricks , and Python / PySpark to build scalable data pipelines.
  • Strong experience in designing dimensional data models , including slowly changing dimensions for data warehouses.
  • Hands-on experience with Change Data Capture (CDC) and data ingestion, specifically with Oracle Golden Gate (nice-to-have).
  • Experience in designing and developing ETL pipelines using tools like SQL Server SSIS , Azure Data Factory , and Databricks .
  • Key Tools & Technologies :
  • Experience with Azure Data Studio , SQLDeveloper , SSIS , and Databricks SQL .
  • Proficiency in SQL , Python , PySpark , and other relevant programming languages.
  • Understanding of data lake architecture and the Delta Lake framework.
  • Soft Skills :
  • Strong communication skills to present technical requirements and solutions to business stakeholders.
  • Ability to collaborate across teams, working closely with business analysts , IT teams , and project managers .
  • Problem-solving skills to address performance issues and optimize complex ETL processes.
  • Ability to work independently and manage multiple tasks in an agile development environment.

    • Evaluation Criteria :

  • Design Documentation and Analysis Skills (30%) :
  • Experience in creating Functional Design Documents (FDD) and Detailed Design Documents (DDD) .
  • Ability to conduct Fit-Gap analysis and document solution complexity.
  • Experience in maintaining deliverable plans and providing status reports.
  • Development, Database, and ETL Experience (60%) :
  • Demonstrated experience in ETL development, including database design, data ingestion, and transformation processes.
  • Experience working in an Azure DevOps environment.
  • Ability to implement logical and physical data models and develop effective ETL pipelines
  • Expert Knowledge of data warehouse design methodologies, dimensional modeling in particular
  • Understanding of Extract / Transform / Load processes to transform data for reporting / BI purposes
  • Ability to define schema for reporting databases
  • Knowledge of BI tools for metadata modeling and report design (e.g. PowerBI, Cognos)
  • Good knowledge and experience in MS SQL Server technology, Azure Databricks SQL Warehouse, Azure Data Lake
  • Experience using T-SQL, PL / SQL for development of Business Intelligence applications. Demonstrated skills in writing and reverse engineering SQL stored procedures and packages for datamarts and reporting
  • Knowledge Transfer (10%) :
  • Proven ability to facilitate knowledge transfer sessions, providing training and documentation for internal teams.
  • Ability to develop training materials and lead both virtual and in-person training sessions.

    • Additional Information :

  • This is a hybrid position, with a requirement to work a minimum of 3 days per week at the office.
  • The role involves both hands-on development and collaborative team work , ensuring seamless data ingestion, transformation, and reporting.
    • Créer une alerte d'emploi pour cette recherche

    Business Intelligence Specialist • Greater Toronto Area, Canada