What are the responsibilities and job description for the Sr. Azure Data Engineer position at Jobs via Dice?
Dice is the leading career destination for tech experts at every stage of their careers. Our client, Avtech Solutions, is seeking the following. Apply via Dice today!
Title Sr. Azure Data Engineer with Data Bricks, Python and Spark
Location Minneapolis, MN (Onsite)
Healthcare experience MANDATORY.
Experience Min 10 years of experience
Job Description:
Key Responsibilities
Required Skills & Qualifications
Title Sr. Azure Data Engineer with Data Bricks, Python and Spark
Location Minneapolis, MN (Onsite)
Healthcare experience MANDATORY.
Experience Min 10 years of experience
Job Description:
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Databricks, Spark, and Python
- Build and optimize ETL/ELT workflows ingesting data from multiple healthcare data sources (EHR, claims, HL7, FHIR, etc.)
- Implement Azure Data Lake (ADLS Gen2), Azure Synapse, Azure SQL solutions
- Optimize Spark jobs for performance, scalability, and cost efficiency
- Collaborate with data scientists, analysts, and business stakeholders to enable analytics and ML use cases
- Ensure HIPAA compliance, data privacy, security, and governance best practices
- Implement data quality checks, monitoring, and logging
- Work with CI/CD pipelines and infrastructure-as-code (Azure DevOps, Git)
- Mentor junior engineers and provide technical leadership
- Participate in architecture design and technology decision-making
Required Skills & Qualifications
- Strong experience with Azure Data Engineering stack
- Azure Databricks
- Azure Data Lake (ADLS Gen2)
- Azure Synapse Analytics
- Hands-on expertise in Apache Spark (PySpark)
- Advanced Python programming for data engineering
- Strong SQL skills (performance tuning, complex queries)
- Experience building batch and streaming pipelines
- Knowledge of data modeling (star/snowflake, dimensional models)
- Experience working with healthcare data
- EHR / EMR systems
- Claims data
- HL7, FHIR, ICD-10, CPT (nice to have)
- Strong understanding of HIPAA and healthcare data security requirements.