What are the responsibilities and job description for the Junior Data Engineer - Databricks position at Jobright.ai?
Verified Job On Employer Career Site
Job Summary:
Steampunk, Inc. is a Change Agent in the Federal contracting industry, focusing on innovative solutions for clients in various sectors. They are seeking a seasoned Data Engineer to lead the development of enterprise-grade data platforms and pipelines using Databricks, while collaborating with teams to solve complex data challenges.
Responsibilities:
• Lead and architect migrations of data using Databricks with focus on performance, reliability, and scalability.
• Assess and understand ETL jobs, workflows, data marts, BI tools, and reports.
• Address technical inquiries concerning customization, integration, enterprise architecture and general feature/functionality of data products.
• Experience working with database/data warehouse/data mart solutions in cloud (Preferably AWS. Alternatively Azure, GCP).
• Key must have skill sets – Databricks, SQL, PySpark/Python, AWS.
• Support an Agile software development lifecycle.
• You will contribute to the growth of our AI & Data Exploitation Practice!
Qualifications:
Required:
• Ability to hold a position of public trust with the US government.
• 2-4 years industry experience coding commercial software and a passion for solving complex problems.
• 2-4 years direct experience in Data Engineering with experience in tools such as:
• Big data tools: Databricks, Apache Spark, Delta Lake, etc.
• Relational SQL (Preferably T-SQL. Alternatively pgSQL, MySQL).
• Data pipeline and workflow management tools: Databricks Workflows, Airflow, Step Functions, etc.
• AWS cloud services: Databricks on AWS, S3, EC2, RDS (or Azure equivalents).
• Object-oriented/object function scripting languages: PySpark/Python, Java, C , Scala, etc.
• Experience working with Data Lake house architecture and Delta Lake/Apache Iceberg.
• Advanced working SQL knowledge and experience working with relational databases, query authoring and optimization (SQL) as well as working familiarity with a variety of databases.
• Experience manipulating, processing, and extracting value from large, disconnected datasets.
• Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.
• Experience manipulating structured and unstructured data.
• Experience architecting data systems (transactional and warehouses).
• Experience the SDLC, CI/CD, and operating in dev/test/prod environments.
• Commitment to data governance.
• Experience working in an Agile environment.
• Experience supporting project teams of developers and data scientists who build web-based interfaces, dashboards, reports, and analytics/machine learning models.
Preferred:
• Experience with data cataloging tools such as Informatica EDC, Unity Catalog, Collibra, Alation, Purview, or DataZone is a plus.
Company:
Steampunk is anchored by a startup culture with a customer-centered delivery approach, we put our Federal government clients in the center of everything we design, develop, and deliver to drive high-quality mission impacts and user experiences at speed. Founded in 2019, the company is headquartered in Washington, District of Columbia, USA, with a team of 201-500 employees. The company is currently Growth Stage.
