Data Engineer
Primary stack
Nice to have's
Job description
Data Engineer
About the Position
PixelPlex is currently looking for an experienced Data Engineer to join our new project. It is a Business Intelligence and Data Visualization service designed especially for NFT enthusiasts. The service uses on-chain, social, and other metrics, enriches it to deliver unique insights ready for decisions.
Main Focus
- Data Quality: Automating the DQ process using Great Expectations.
- Data Marts: Presenting data marts, optimizing analyst code if necessary.
- Ensuring the collection and data loading into analytical and hot databases, including obtaining structured and unstructured data from various sources, preparing, cleaning, and pre-processing data from external sources, and building aggregates.
Responsibilities
- Creating and improving data processing processes (optimizing processor groups, optimizing data marts).
- Participating in the development of data processing solutions within ML projects and deploying pipelines.
- Developing and optimizing procedures for generating detailed data layers (raw to ods) and data mart layers in DataLake and hot database for online reporting.
- Implementing CI/CD processes and monitoring the developed data processing processes (Grafana, Prometheus).
- Handling ad-hoc queries.
Requirements
- Knowledge of database principles and HD design.
- Experience in ETL process development.
- Good understanding of ML pipelines and their deployment to production.
- Experience using Airflow (or other industry-standard pipeline orchestrators, such as Luigi, Dagster, etc.).
- Experience with high-load distributed data storage and processing systems.
- Excellent knowledge of SQL, query optimization experience.
- Willingness to learn, grow, and be motivated to achieve goals.
Nice to Have
- Experience in Python or Node.js development.
- Knowledge of DevOps tools.
- Experience with cloud services.
We Offer
- Competitive salary
- Full-time position
- Opportunity to work on innovative projects in the NFT space
About the Company
PixelPlex is a leading company in blockchain, big data, artificial intelligence, and AR/VR technologies. We are dedicated to delivering cutting-edge solutions tailored to the unique needs of your business.
© PixelPlex. This job description was sourced from the employer's public career page. TheJob is not the employer — we index the posting and route candidates to the source. All content rights and hiring decisions belong to the employer.
FinTech & banking,Retail & eCommerce,Supply chain & logistics,Healthcare,Real estate
About the Company PixelPlex is an award-winning custom software development company with a proven track record, a mix of broad technology expertise, and over 100 talented developers. Founded in 2007, PixelPlex has delivered over 300 projects. Products, Services & Tech Stack Core Solutions Blockchain Development Services: dApps, NFT marketplaces, complex blockchain protocols, risk management systems, smart contract development & audit, DApp development, STO development, and Web3 project launch. Custom Software Development: Web development, UI/UX design, accessible web design, DevSecOps, and custom solutions. AI Development Services: AI consulting, ML development, computer vision implementation, predictive analytics model creation. Data Analytics & Visualization: Data visualization, business intelligence, data analytics, big data engineering. Technologies & Frameworks Blockchain Artificial Intelligence Big Data Web Development Mobile App Development Project Scope & Target Clients Client Base FinTech & Banking: Digital finance challenges, risk management, compliance software, STO/tokenization platforms. Retail & eCommerce: Digital marketplaces, retail POS software, goods order and delivery platforms. Supply Chain & Logistics: Route optimization, intelligent warehouse management, fleet management, certification and logistics platforms. Healthcare: EHR/EMR systems, telemedicine software, ML-powered analysis and reporting solutions. Real Estate: Real estate marketplaces, document management solutions, immersive tours, real estate tokenization solutions. Oil & Gas: Safety monitoring apps, BI-powered systems, energy resource tokenization platforms. Manufacturing: Inventory tracking, quality control, predictive maintenance. Automotive: Dealership management solutions, automotive analytics, transportation management apps, telematics systems. Insurance: Claim management, billing, customer-related processes. Engineering Scale Startups: Aiding through launch challenges, collecting best practices for earning investor trust and sparking user interest. Small Businesses: Optimizing operations, streamlining processes, implementing growth strategies. Enterprises: Guiding ventures from pilots to value-adding solutions.