Tech Stack
PythonSQLAWSKafka
Job Description, Responsibilities & Requirements
Description language:
About the Position
EPAM Systems is looking for a Senior Big Data Software Engineer to develop and support internal data warehousing systems for our client's products and applications. This role is available in a remote format from Lviv, Ukraine.
Responsibilities
- Development and support of complex data warehousing systems for processing large volumes of data
- Designing and optimizing database architecture and data schemas
- Developing and implementing efficient SQL queries for data analysis and processing
- Developing and implementing software solutions using the Hadoop ecosystem (Yarn, MapReduce, Hive, HDFS)
- Using Spark and Kafka for real-time data processing and transmission
- Configuring and managing AWS cloud services (EMR, S3, Lambda)
- Optimizing existing solutions to reduce time and resource consumption
- Collaborating with other engineers and teams to create scalable and efficient solutions
- Performing data analysis, creating reports, and providing recommendations for system improvements
- Identifying and implementing best practices to ensure data security and integrity
- Participating in the preparation of technical documentation and supporting user documentation
Requirements
- More than 4 years of experience in database development and database design
- Expert in SQL with at least 2+ years of experience
- Good Python development skills
- Software development experience using Hadoop ecosystem components (Yarn, MapReduce, Hive, HDFS)
- Practical experience with Spark and Kafka
- Experience with AWS cloud services (EMR, S3, Lambda)
- Excellent analytical and problem-solving skills
- Good communication and presentation skills
- Attention to detail
- Ability to meet tight deadlines and prioritize tasks
Nice to Have
- Experience in database administration
- Experience with AWS Redshift
- Practical experience with pyspark
- Understanding of service/application scaling approaches
About the Company
EPAM Systems is a global software engineering and product development company, delivering digital platforms and solutions to the world's leading companies.