Lead design and fine-tuning of advanced LLM and conversational AI systems. Focus on scalable, secure, and contextual NLP/LLM integration into enterprise workflows.
Core Responsibilities
Architect custom NLP pipelines using transformer and LLM frameworks.
Fine-tune models using PEFT, LoRA, QLoRA on domain datasets.
Implement RAG pipelines with semantic memory and retrieval scoring.
Design multi-agent conversational frameworks for domain reasoning.
Establish LLM monitoring, safety, and prompt audit pipelines.
Required Technical Skills
Libraries: Hugging Face Transformers, spaCy, NLTK, sentence-transformers.
Models: BERT, RoBERTa, T5.
LLMs: GPT, Claude, LLaMA, Mistral (API usage & fine-tuning).
LLM Adaptation: PEFT, LoRA/QLoRA, prompt optimization, RLHF.
RAG Pipelines: LangChain, LlamaIndex, FAISS, ChromaDB.
Deployment: FastAPI, MLflow, Docker.
Evaluation: BLEU, ROUGE, F1, perplexity.
Infrastructure: Pinecone, Elasticsearch.
MLOps for LLMs: Model serving, context caching, feedback loops.
Advanced Skills: Semantic retrieval, embedding optimization, tokenizer customization.
Hugging Face Transformers, Spacy, Nltk, Sentence-Transformers, Bert, Roberta, T5, Gpt, Claude, Llama, Mistral, Peft, Lora, Qlora, Prompt Optimization, Rlhf, Langchain, Llamaindex, Faiss, Chromadb, Fastapi, Mlflow, Docker, Bleu, Rouge, F1, Perplexity, Pinecone, Elasticsearch, Model Serving, Context Caching, Feedback Loops, Semantic Retrieval, Embedding Optimization, Tokenizer Customization