Lead Pyspark Developer

About the Position

We are seeking a Lead Pyspark Developer with over 10 years of experience in data-intensive Pyspark development to join our team in Bengaluru, India. This role is pivotal in enhancing our surveillance platform to detect potential instances of market misconduct, manipulation, and abuse.

Responsibilities

• Develop software applications based on business requirements
• Maintain software applications and make enhancements according to project specifications
• Participate in requirement analysis, design, development, testing, and implementation activities
• Propose new techniques and technologies for software development
• Perform unit testing and user acceptance testing to evaluate application functionality
• Ensure to complete the assigned development tasks within the deadlines
• Work in compliance with coding standards and best practices
• Provide assistance to Junior Developers when needed
• Perform code reviews and recommend improvements
• Review business requirements and recommend changes to develop reliable applications
• Develop coding documentation and other technical specifications for assigned projects
• Act as primary contact for development queries and concerns
• Analyze and resolve development issues accurately

Requirements

• Must have

  • 10+ years of experience in data intensive Pyspark development
  • Must have experience in leading a Development Team
  • Proven experience as a Core Python developer
  • Good understanding of Gen AI
  • Experience developing Classes, OOPS, exception handling, parallel processing
  • Strong knowledge of DB connectivity, data loading, transformation, calculation
  • Extensive experience in Pandas/Numpy dataframes, slicing, data wrangling, aggregations
  • Lambda Functions, Decorators
  • Vector operations on Pandas dataframes /series
  • Application of applymap, apply, map functions
  • Concurrency and error handling data pipeline batch of size [1-10 gb]
  • Ability to understand business requirements and translate them into technical requirements
  • Ability to design architecture of data pipeline for concurrent data processing
  • Familiar with creating/designing RESTful services and APIs
  • Familiar with application unit tests
  • Working with Git source control Service-orientated architecture, including the ability to consider integrations with other applications and services
  • Debugging application

• Nice to have

  • Knowledge of web backend technology - Django, Python, PostgreSQL
  • Apache Airflow
  • Atlassian Jira
  • Understanding of Financial Markets Asset Classes (FX, FI, Equities, Rates, Commodities & Credit), various trade types (OTC, exchange traded, Spot, Forward, Swap, Options) and related systems is a plus
  • Surveillance domain knowledge, regulations (MAR, MIFID, CAT, Dodd Frank) and related Systems knowledge is certainly a plus

We Offer

• Competitive salary
• Opportunity to work in a dynamic and innovative environment
• Professional growth and development opportunities

About the Company

Luxoft is a global provider of digital transformation and technology services, empowering businesses to thrive in the digital age. Our team is dedicated to delivering cutting-edge solutions and fostering a culture of excellence and innovation.

Apply

Interested candidates are encouraged to apply with their updated resume and relevant certifications.