Hadoop Developer

We are seeking a skilled Hadoop Developer to build and manage large-scale data storage and processing systems using Hadoop.

About the Position

Hadoop has become the cornerstone of large data systems. Indeed, Hadoop developers know how to write applications that interact with Hadoop; but they also know how to build, operate, and troubleshoot large Hadoop clusters.

Hadoop is a large software stack. On one hand, it deals with low-level hardware resources, and on the other, it provides a high-level API to build software. As a result, a Hadoop developer not only develops software but also operates it.

Furthermore, a good grasp of algorithms and their runtime characteristics is essential to developing for Hadoop efficiently.

Responsibilities

• Write software to interact with HDFS and MapReduce.
• Assess requirements and evaluate existing solutions.
• Build, operate, monitor, and troubleshoot Hadoop infrastructure.
• Develop tools and libraries, and maintain processes for other engineers to access data and write MapReduce programs.
• Develop documentation and playbooks to operate Hadoop infrastructure.
• Evaluate and use hosted solutions on AWS / Google Cloud / Azure.
• Write scalable and maintainable ETLs.
• Understand Hadoop’s security mechanisms and implement Hadoop security.
• Write software to ingest data into Hadoop.

Skills

• Knowledge of the JVM runtime, the Java language, and ideally another JVM-based programming language.
• Knowledge of computer science fundamentals, particularly algorithmic complexity.
• Knowledge of trade-offs in distributed systems.
• Proficiency at software engineering principles that produce maintainable software.
• Experience with a Hadoop distribution.
• Experience with one or more computation frameworks, such as Spark.
• Familiarity with HBase, Kafka, ZooKeeper, or other Apache software.
• Knowledge of Linux and its operation, networking, and security.
• Ability to efficiently move large data around.

We Offer

• Opportunity to work with cutting-edge technology.
• Collaborative and innovative work environment.
• Competitive salary and benefits package.
• Remote working possibilities.

About the Company

{{ Write a short and catchy paragraph about your company. Make sure to provide information about the company’s culture, perks, and benefits. Mention office hours, remote working possibilities, and everything else that you think makes your company interesting. }}