NBBL- Lead Innovation

About the Role We are seeking a highly skilled Lead Data Engineer to design and implement the data foundation for the Innovation vertical within BBPS. This role focuses on building scalable, high-performance data pipelines and data products that enable AI/ML-driven solutions and real-time analytics for BBPS and the broader ecosystem. Responsibilities Pipeline Architecture: Design, build and optimize high-scale data pipelines (Batch & Streaming) using Spark/PySpark, Kafka, and Airflow/similar on-premise or cloud-based infrastructure. Develop and maintain scalable data processing workflows that integrate with various data sources, such as relational databases, NoSQL databases, and file systems. Collaborate with cross-functional teams to design and implement data products that meet business requirements. Data Engineering: Design, build, and deploy large-scale data engineering solutions using technologies like Apache Spark, Hadoop, and AWS Glue. Develop and maintain scalable data processing workflows that integrate with various data sources, such as relational databases, NoSQL databases, and file systems. Collaborate with cross-functional teams to design and implement data products that meet business requirements. Data Products: Design, build, and deploy AI/ML-driven solutions using technologies like TensorFlow, PyTorch, and scikit-learn. Develop and maintain scalable data processing workflows that integrate with various data sources, such as relational databases, NoSQL databases, and file systems. Collaborate with cross-functional teams to design and implement data products that meet business requirements. Real-time Analytics: Design, build, and deploy real-time analytics solutions using technologies like Apache Kafka, Apache Storm, and AWS Kinesis. Develop and maintain scalable data processing workflows that integrate with various data sources, such as relational databases, NoSQL databases, and file systems. Collaborate with cross-functional teams to design and implement data products that meet business requirements. Requirements: Bachelor's degree in Computer Science or related field. 5+ years of experience in data engineering, data science, or a related field. Strong understanding of big data technologies like Hadoop, Spark, and NoSQL databases. Experience with cloud-based infrastructure such as AWS, Azure, or Google Cloud Platform. Strong programming skills in languages like Python, Java, or Scala. Experience with data visualization tools like Tableau, Power BI, or D3.js. Certifications in data engineering, data science, or a related field are a plus. Benefits: Competitive salary and benefits package. Opportunities for professional growth and development. Collaborative and dynamic work environment. Access to cutting-edge technologies and tools. Flexible working hours and remote work options.