Tech Stack
KafkaPythonAWSSQL
Job Description, Responsibilities & Requirements
Description language:
About the Position
EPAM Systems is looking for a Senior Big Data Software Engineer to develop and maintain internal data storage systems for our client's products and applications.
Responsibilities
- Development and maintenance of complex data storage systems for processing large volumes of data
- Designing and optimizing database architecture and data schemas
- Developing and implementing efficient SQL queries for data analysis and processing
- Developing and implementing software solutions using the Hadoop ecosystem (Yarn, MapReduce, Hive, HDFS)
- Using Spark and Kafka for real-time data processing and transmission
- Configuring and managing AWS cloud services (EMR, S3, Lambda)
- Optimizing existing solutions to reduce time and resource consumption
- Collaborating with other engineers and teams to create scalable and efficient solutions
- Performing data analysis, creating reports, and providing recommendations for system improvements
- Identifying and implementing best practices to ensure data security and integrity
- Participating in the preparation of technical documentation and supporting user documentation
Requirements
- More than 4 years of experience in database development and database design
- Expert in SQL with at least 2+ years of experience
- Good Python development skills
- Software development using components of the Hadoop ecosystem (Yarn, MapReduce, Hive, HDFS)
- Practical experience with Spark and Kafka
- Experience with AWS cloud services (EMR, S3, Lambda)
- Excellent analytical and problem-solving skills
- Good communication and presentation skills
- Attention to detail
- Ability to meet tight deadlines and prioritize tasks
Nice to Have
- Experience in database administration
- Experience with AWS Redshift
- Practical experience with PySpark
- Understanding of service/application scalability approaches
We Offer
- Remote work from Lviv, Ukraine
About the Company
EPAM Systems is a global software engineering and product development company that partners with many of the world's leading businesses to transform their businesses through technology.