What are the responsibilities and job description for the Databricks - Data Engineer position at Lyteworx Automation Systems?
Lyteworx Automation Systems (Lyteworx) is seeking talented individuals to join our team of technology pioneers. With over 16 years of experience, we have honed our core capabilities in IT infrastructure engineering, cyber security, data science, and artificial intelligence. Our comprehensive suite of engineering and technical support services caters to clients across defense, intelligence, and commercial sectors. If you have a passion for cutting-edge technology and a drive to push the boundaries of innovation, Lyteworx is the place for you. Join us and be part of a team that is shaping the future of technology.
We are seeking a talented and experienced Databricks Data Engineer to join our dynamic team. The successful candidate will play a crucial role in building and managing data pipelines, processing large datasets, and ensuring efficient and secure data management practices. They will also be responsible for inspiring the adoption of advanced analytics and data science across the organization.
Requirements
Security Clearance: None/Secret/Top Secret
Location : Remote
Certifications:
We believe that our employees are our greatest asset, and we are dedicated to providing them with the resources and support they need to achieve their goals and succeed in their careers.
- Currently Collecting Resumes ---
We are seeking a talented and experienced Databricks Data Engineer to join our dynamic team. The successful candidate will play a crucial role in building and managing data pipelines, processing large datasets, and ensuring efficient and secure data management practices. They will also be responsible for inspiring the adoption of advanced analytics and data science across the organization.
Requirements
Security Clearance: None/Secret/Top Secret
Location : Remote
Certifications:
- Databricks Certified Data Engineer Associate certification
- Bachelor's degree in a relevant field or equivalent combination of education and experience
- 7 years of experience in the data engineering field
- 3 years of experience in a data analytics environment, preferably in DoD or the intelligence community
- Master's degree in a relevant field (may substitute for 3 years of general experience)
- Previous experience in a data engineering role or similar
- Experience with data processing and pipeline management
- Familiarity with industry-standard data engineering platforms and tools
- Strong data manipulation and processing skills
- Proficiency in data engineering tools such as Databricks, Apache Spark, Delta Lake, MLflow, and SQL
- Understanding of the Databricks Lakehouse Platform, its workspace, architecture, and capabilities
- Ability to perform multi-hop architecture ETL tasks using Apache Spark SQL and Python
- Knowledge of incremental data processing in batch and streaming mode
- Familiarity with open-source tools, cloud computing, machine learning, and data visualization
- Strong interpersonal skills and a collaborative work style
- Understanding of security and governance best practices
- Strong problem-solving and analytical skills
- Excellent written and verbal communication abilities
- Build and manage data pipelines for data engineering applications
- Process and manipulate large datasets efficiently and securely
- Work with data engineering platforms and tools such as Databricks, Apache Spark, Delta Lake, MLflow, and SQL
- Maintain best practices around security and governance
- Model data management solutions
- Manage, test, and deploy code
- Provide expertise on data concepts to the advanced analytics group
- Install continuous pipelines of filtered information for data analysts and scientists to access relevant datasets
- Collaborate with cross-functional teams to ensure effective data utilization
We believe that our employees are our greatest asset, and we are dedicated to providing them with the resources and support they need to achieve their goals and succeed in their careers.
- 401K Plan
- Vacation and Paid Time Off (PTO)
- Health, Dental & Vision Insurance
- Life & Supplemental Life Insurance
- Disability & Accidental Death & Dismemberment
- Mental Health Care
- Health Saving Account (HSA)