What are the responsibilities and job description for the Azure Databricks Engineer/ Architect position at STAFFWORXS?
Title: Consultant – Data Engineering (Azure Databricks Data Architect)
Location: Nashville, Tennessee, United States (ONLY LOCAL CANDIDATES)
Duration of the Contract: 12 / Long-term project
Client Industry: Manufacturing
Skills Required
Core Technical Skills
- Advanced Apache Spark with strong hands-on expertise in PySpark and/or Scala, including performance tuning using Spark UI, partitioning strategies, joins, caching, and shuffle optimization
- Deep Azure Databricks platform expertise, including notebooks, jobs/workflows, cluster configuration, Databricks SQL, and implementation of Medallion Architecture (Bronze/Silver/Gold)
- Strong experience with Delta Lake, including ACID transactions, schema enforcement and evolution, MERGE operations, time travel, and performance-optimized Delta table design
- Unity Catalog and data governance experience: catalogs, schemas, tables, permission models, secure access patterns, and data lineage across workspaces
- Advanced SQL skills for data modeling, complex transformations, aggregations, and BI-ready lakehouse structures
- Solid Azure platform experience, including:
- Azure Data Lake Storage (ADLS)
- Azure Data Factory or Synapse Pipelines
- Event Hubs / IoT integrations
- Networking, identity and access management
- Azure Key Vault and monitoring
- Data modeling and warehousing expertise, including dimensional modeling, star/snowflake schemas, and analytics-focused performance design
- CI/CD and DevOps for data engineering, using Git-based workflows with Azure DevOps or GitHub Actions for Databricks pipelines and artifacts
Core Functional & Client-Facing Skills
- Ability to conduct client meetings and stakeholder interviews
- Experience creating structured questionnaires for client discovery sessions
- Strong capability in gathering user requirements and producing data architecture and design documents for manufacturing applications on Azure and Databricks
- Basic project management skills to plan, track, and manage data engineering initiatives
- Excellent communication skills, with the ability to collaborate effectively with both client teams and internal delivery teams
Job Description
We are seeking a highly experienced Azure Databricks Data Architect / Consultant – Data Engineering with 9–12 years of experience to support manufacturing data platforms. The ideal candidate brings deep hands-on Databricks and Azure engineering expertise, combined with strong architecture, governance, and client-facing skills. This role involves designing scalable lakehouse architectures, implementing robust data pipelines, enforcing governance with Unity Catalog, and working closely with stakeholders to translate business requirements into high-quality data solutions.
Salary : $60 - $65