What are the responsibilities and job description for the Senior Software Engineer position at WIT Recruiting?
WIT Recruiting is hiring a Senior Software Engineer to work with a leading provider of financial data. This is a 12-month contract.
Overview:
We are seeking a Senior Data Engineer to join our Cloud Solutions team focused on designing and building a scalable data platform. This role involves developing cloud-native data ingestion workflows, automating data delivery pipelines, and enabling data analysts and scientists with robust tools and infrastructure.
You’ll collaborate cross-functionally with product owners, analysts, and engineering teams to migrate legacy pipelines, build new solutions, and implement automation across the data lifecycle.
Key Responsibilities:
- Migrate legacy data ingestion pipelines to a shared cloud-based data management system.
- Design and develop serverless data workflows using modern cloud technologies (e.g., AWS Lambda, Step Functions, Databricks).
- Build internal tools to empower data analysts to validate, cleanse, and manage datasets.
- Deliver enriched datasets to cloud-based storage (e.g., AWS S3) and Delta Lake tables.
- Collaborate with analysts and data scientists to identify automation opportunities and improve data processing efficiency.
- Write infrastructure as code (IaC) and apply DevOps best practices to support scalable deployment.
- Monitor and optimize data workflows using observability tools (e.g., CloudWatch).
Required Qualifications:
- 4 years of professional experience programming in Python or Java.
- Strong SQL and database experience (Oracle, PostgreSQL, or similar).
- Hands-on experience with AWS services including S3, Lambda, and CloudWatch.
- Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field—or equivalent work experience.
Preferred Qualifications:
- Experience migrating data workflows from on-premise or legacy systems to the cloud.
- Familiarity with Databricks, Spark, or Delta Lake.
- Proficiency with infrastructure as code tools such as Terraform or AWS CloudFormation.
- Exposure to Jupyter Notebooks/Workflows and data analysis environments.
