AI Engineer Job Description
About the Role
We are seeking an experienced AI Engineer to join our team. In this role, you will be responsible for developing and designing specialized DSPy optimizers for Large Language Models (LLMs), creating custom optimization techniques to enhance the performance of our LLM agents.
What You'll Do
- Design and implement optimizers for LLM-based applications.
- Create novel optimization approaches to improve AI agent capabilities.
- Develop evaluation methodologies to measure agent effectiveness.
- Implement trace-based optimization and programmatic prompt tuning strategies.
- Collaborate on improving our AI agent infrastructure.
Technical Must-Haves
- Experience designing LLM-based agents with retrieval-augmented generation (RAG).
- Proficiency in AI evaluation frameworks, NLP fundamentals (tokenization, embeddings, LLM architecture), and bias handling.
- Systematic prompt engineering experience with experimental frameworks (tools like Weave or Langsmith).
- Strong Python back-end skills for data-intensive systems and data processing.
- Demonstrable ability to balance token limits, cost, and speed while producing structured outputs.
Requirements (Must Have)
- 3+ years of professional industry experience working with AI/ML technologies (internships not counted).
- Experience building industry-focused, commercial AI applications.
- Proven track record deploying and maintaining LLM-based applications in production with medium to large user bases (1000+ users).
- Experience handling real-world challenges in AI agent performance, reliability, and cost optimization.
- Track record of systematically improving AI agent performance through iterative optimization.
- Hands-on experience with debugging and fixing AI agent failures in production environments.
- Experience analyzing user interactions with AI systems to drive improvements.
Nice to Have
- Experience with DSPy or similar LLM optimization frameworks.
- Familiarity with building custom optimization techniques.
- Background in ML/AI research.
- Experience with multiple LLM providers and models.
Apply
Apply if you've successfully deployed and optimized AI applications serving real users at scale in industry settings and can share concrete examples of how your engineering improved performance in production environments.
Job Highlights
- Experience designing LLM-based agents with retrieval-augmented generation (RAG)
- Proficiency in AI evaluation frameworks and NLP fundamentals including tokenization, embeddings, LLM architecture, and bias handling.
- Systematic prompt engineering with experimental frameworks like Weave or Langsmith.
- Strong Python skills for data-intensive systems.
- Ability to balance token limits, cost, and speed.
- 3+ years of industry experience in AI/ML.
- Proven deployment of LLM-based applications in production.
- Experience addressing AI performance, reliability, and cost optimization in a business environment.
- Experience with optimization frameworks such as DSPy.
- Familiarity with custom optimization techniques.
- Background in ML/AI research.
- Experience with various LLM providers.
Tags
AI, LLM, Optimization, Python, NLP