What are the responsibilities and job description for the Senior Big Data Engineer position at Deegit Inc?
Company Description
Deegit Inc.
Job Description
Job description:
Responsibilities
· Participate in the engineering and administration of big data systems.
· Apache Storm/Java development for both data transformation and augmentation.
· Employ best practices to ensure high availability and scalability.
· Install, configure, and maintain big data technologies and systems.
· Maintain documentation and troubleshooting playbooks.
· Respond to and resolve access and performance issues.
· Ability to work non-peak hours when needed.
· Assist in capacity planning.
· Proactively monitor performance.
· Collaborate with engineering teams to optimize data collection.
· Maintain change control and testing processes for all modifications and deployments.
· Conduct research and make recommendations on big data products, services, and standards.
Preferred Qualifications
· Experience with operational analytics and log management using ELK or Splunk.
· Understanding and experience with SQL and MPP databases such as Greenplum.
· Experience with SaltStack or comparable deployment automation tools.
· Working knowledge of Zenoss, NewRelic, PagerDuty or other monitoring technologies.
· Experience with Apache Drill, Apache Hbase, Hive, Phoenix, or other Hadoop with SQL technologies.
· Experience working with batch-processing and tools in Hadoop technology stack (MapReduce, Yarm, Pig, Hive, HDFS)
Qualifications
Basic Requirements
· BS in Computer Science, Information Systems, or related technical degree.
· 5 or more years’ experience with data transformation pipelines preferably Storm
· 5 or more years’ experience in messaging frameworks preferably Kafka.
· 5 or more years’ experience with Java programming.
Additional Information
All your information will be kept confidential according to EEO guidelines.