Senior Data Engineer

We are looking for a senior Data Engineer to join our BI team and take an active role in designing, building, and maintaining the end-to-end data pipeline, architecture and design that powers our warehouse, LLM-driven applications, and AI-based BI.

• 8+ years of experience as a Data Engineer.
• Demonstrated experience in utilizing Python for data engineering tasks, including transformation, advanced data manipulation, and large-scale data processing.
• Hands-on experience with big data technologies including Apache Spark, Hadoop, and Kafka for distributed processing and real-time data ingestion.
• Experience designing complex data pipelines extracting data from RDBMS, JSON, API, and Flat file sources.
• Demonstrated skills in SQL and PLSQL programming, with advanced mastery in Business Intelligence and data warehouse methodologies, along with hands-on experience in one or more relational database systems and cloud-based database services such as Snowflake/Redshift
• Understanding of software engineering principles and skills working on Unix/Linux/Windows Operating systems, and experience with Agile methodologies.
• Proficiency in version control systems, with experience in managing code repositories, branching, merging, and collaborating within a distributed development environment.
• Interest in business operations and comprehensive understanding of how robust BI systems drive corporate profitability by enabling data-driven decision-making and strategic insights.
• Effective oral and written English communication skills with BI team and user community.

Pluses

• Experience with vector databases such as DataStax AstraDB, and developing LLM-powered applications using popular open source frameworks like LangChain and LlamaIndex–including prompt engineering, retrieval-augmented generation (RAG), and orchestration of intelligent workflows.
• Familiarity with evaluating and integrating open-source LLM frameworks–such as Hugging Face Transformers/LLaMA-4 across end-to-end workflows, including fine-tuning and inference optimization.
• Knowledge of MLOps tooling and CI/CD pipelines to manage model versioning and automated deployments.

Originally posted on Himalayas