Data Engineer (AI)

Required Qualifications

• 6+ years of experience in Data Engineering.
• Strong proficiency in Python and SQL.
• Hands-on experience with Spark, Databricks, or distributed data processing frameworks.
• Experience building batch and streaming data pipelines.
• Strong knowledge of cloud platforms (AWS, Azure, or GCP).
• Experience with data warehouses such as Snowflake, Redshift, BigQuery, or Synapse.
• Familiarity with orchestration tools such as Airflow, Prefect, or Azure Data Factory.
• Experience with Git, CI/CD, and Infrastructure as Code.
• Understanding of data modeling, performance optimization, and data governance.

AI & Generative AI Experience

Candidates should have practical exposure to some of the following:

• Building applications using Large Language Models (OpenAI, Anthropic, Gemini, or open-source models).
• Experience with LangChain, LlamaIndex, or similar AI orchestration frameworks.
• Working knowledge of vector databases such as Pinecone, Chroma, Weaviate, FAISS, or Milvus.
• Understanding of embeddings, semantic search, and Retrieval-Augmented Generation (RAG).
• Prompt engineering and evaluation of AI applications.
• Experience integrating AI APIs into production systems.
• Basic understanding of AI observability, model evaluation, and responsible AI practices.