Senior Databricks ML & AI Engineer
- Location: No Location Set
- Type: Contract
- Job #35337
- 5+ years of hands-on Machine Learning Engineering, MLOps, or AI Engineering experience within enterprise production environments.
- Extensive experience designing and deploying Databricks Lakehouse solutions using Delta Lake, Unity Catalog, MLflow, Databricks SQL, Workflows, and Delta Live Tables.
- Strong programming expertise in Python, PySpark, Pandas, NumPy, and modern software engineering best practices.
- Experience building, training, deploying, and monitoring machine learning models using PyTorch, TensorFlow, scikit-learn, XGBoost, or similar ML frameworks.
- Proven experience implementing end-to-end MLOps pipelines including experiment tracking, model registry, automated retraining, model deployment, and production monitoring.
- Hands-on experience developing Generative AI (GenAI) and Large Language Model (LLM) solutions, including RAG architectures, prompt engineering, LangChain, LlamaIndex, or Databricks Mosaic AI.
- Experience implementing Vector Search, embeddings, semantic search, and AI retrieval pipelines using Databricks or similar vector database technologies.
- Strong understanding of Apache Spark internals, distributed computing, performance tuning, partitioning, memory management, and large-scale data processing.
- Experience with cloud platforms including AWS, Azure, or Google Cloud Platform, supporting enterprise AI and machine learning workloads.
- Experience implementing CI/CD pipelines, GitHub Actions, Azure DevOps, Databricks Asset Bundles, and modern DevOps automation practices.
- Hands-on experience with Docker, Kubernetes, Terraform, or Pulumi supporting Infrastructure-as-Code and scalable AI platform deployments.
- Experience working with Delta Lake optimization, Unity Catalog governance, data lineage, access controls, and enterprise data governance practices.
- Strong experience monitoring model performance, model drift, data quality, observability, and production SLAs using tools such as Prometheus, Grafana, or Databricks Lakehouse Monitoring.
- Demonstrated ability to collaborate with Data Scientists and Data Engineers to productionize AI and machine learning solutions while conducting code reviews, architecture reviews, and technical mentoring.
- Experience supporting modern AI ecosystems including Feature Store, Databricks Model Serving, Kafka, Spark Structured Streaming, Lakehouse Architecture, and responsible AI or ML governance practices.