Sr. Data Architect - GOAPRDJP00000616
Eleventh Floor
9942 - 108 Street
Edmonton
Primarily work remotely but must be available for on-site meetings as required (Meetings may occur up to 3-4 times per fiscal month, but the actual frequency will depend on the specific initiative and will be determined on an on-demand basis.)
Duties:
- Design and implement scalable, secure, and high-performance data architecture on Microsoft Azure, supporting both cloud-native and hybrid environments.
- Lead the development of data ingestion, transformation, and integration pipelines using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Architect and manage data lakes and structured storage solutions using Azure Data Lake Storage Gen2, ensuring efficient access and governance.
- Integrate data from diverse source systems including ServiceNow, and geospatial systems, using APIs, connectors, and custom scripts.
- Develop and maintain robust data models and semantic layers to support operational reporting, analytics, and machine learning use cases.
- Build and optimize data workflows using Python and SQL for data cleansing, enrichment, and advanced analytics within Azure Databricks.
- Design and expose secure data services and APIs using Azure API Management for downstream systems.
- Implement data governance practices, including metadata management, data classification, and lineage tracking.
- Ensure compliance with privacy and regulatory standards (e.g., FOIP, GDPR) through role-based access controls, encryption, and data masking.
- Collaborate with cross-functional teams to align data architecture with business requirements, program timelines, and modernization goals.
- Monitor and troubleshoot data pipelines and integrations, ensuring reliability, scalability, and performance across the platform.
- Provide technical leadership and mentorship to data engineers and analysts, promoting best practices in cloud data architecture and development.
- Other duties as needed
Must-Have's:
- A college or Bachelor's degree in Computer Science or a related field of study.
- Hands-on experience managing Databricks workspaces, including cluster configuration, user roles, permissions, cluster policies, and applying monitoring and cost optimization for efficient, governed Spark workloads - 3 years
- Experience as a Data Architect in a large enterprise, designing and implementing data architecture strategies and models that align data, technology, and business goals with strategic objectives - 8 years
- Experience designing data solutions for analytics-ready, trusted datasets using tools like Power BI and Synapse, including semantic layers, data marts, and data products for self-service, data science, and reporting - 4 years
- Experience in Github/Git for version control, collaborative development, code management, and integration with data engineering workflows - 4 years
- Experience with Azure services (Storage, SQL, Synapse, networking) for scalable, secure solutions, and with authentication (Service Principals, Managed Identities) for secure access in pipelines and integrations - 5 years
- Experience in Python (including PySpark) and SQL, applied to developing, orchestrating, and optimizing enterprise-grade ETL/ELT workflows in a large-scale cloud environment - 6 years
- Experience building scalable data pipelines with Azure Databricks, Delta Lake, Workflows, Jobs, and Notebooks, plus cluster management. Extending solutions to Synapse Analytics and Microsoft Fabric is a plus - 3 years
Nice-to-Have's:
- Certification in The Open Group Architecture Framework (TOGAF).
- Use of AI-Experienced in using AI for code generation, data analysis, automation, and enhancing productivity in data engineering workflows - 1 year
- Direct, hands-on experience performing business requirement analysis related to data manipulation/transformation, cleansing and wrangling - 8 years
- Experience and strong technical knowledge of Microsoft SQL Server, including database design, optimization, and administration in enterprise environments - 8 years
- Experience building scalable ETL pipelines, data quality enforcement, and cloud integration using TALEND technologies - 2 years
- Experience in data governance, security, and metadata management within a Databricks-based platform - 2 years
- Skilled in building secure, scalable RESTful APIs for data exchange, with robust auth, error handling, and support for real-time automation - 3 years
- Experience in Message Queueing Technologies, implementing message queuing using tools like ActiveMQ and Service Bus for scalable, asynchronous communication across distributed systems - 3 years
- Experience working with cross-functional teams to create software applications and data products - 5 years
- Experience working with ServiceNow- Azure-based Data Management Platform Integrations - 1 year
Originally posted on Himalayas
