Role Summary
We are looking for a skilled Data Engineer to design, build, and maintain our core data infrastructure. In this role, you will make data clean, reliable, and accessible. You will work closely with data scientists, analysts, and product teams to power business intelligence and machine learning initiatives.
Key Responsibilities
- Pipeline Development: Build, test, and maintain scalable ETL (Extract, Transform, Load) pipelines to move data from various sources into centralized storage.
- Data Warehousing: Design and optimize data warehouses, data lakes, and database systems for high performance.
- Data Quality: Implement validation checks, error-handling routines, and monitoring tools to ensure high data integrity and reliability.
- Collaboration: Partner with data analysts and data scientists to understand data needs and deliver clean, analysis-ready datasets.
- Optimization: Fine-tune big data queries, storage structures, and processing workflows to reduce latency and infrastructure costs.
- Security & Governance: Apply security protocols, access controls, and data privacy policies to keep enterprise data safe.
Required Skills & QualificationsEducation:
- Bachelor’s degree or Master's degree in CS, IT, EC, Data Engineering, or a related technical field (or equivalent work experience).
- Programming: Strong proficiency in Python, plus expert-level SQL skills.
- Database Knowledge: Experience with relational databases (PostgreSQL, MySQL, SQL Server, etc)
- Big Data & Cloud Tools: Must have AWS knowledge. Familiarity with other cloud platforms (GCP, & Azure) and big data frameworks (Apache Spark).
- ETL: Must have Databricks knowledge. Familiarity with AWS Glue.
- Workflow Automation: Experience with orchestration tools like AWS Lambda, Step Function, Airflow, etc.
- Problem-Solving: Strong analytical mindset with an ability to troubleshoot complex data bottlenecks.
- Minimum 5 years of Relevent Experience.
- Experience of having worked on education project(s) or eGovernance projects at State / Central govt. programs, preferred
