What are the responsibilities and job description for the AI Engineer, Mid-Level position at Jobright.ai?
Verified Job On Employer Career Site
Job Summary:
Relias LLC is a company dedicated to improving healthcare outcomes through innovative technology solutions. They are seeking a highly skilled AI Engineer to design and deploy scalable LLM-based systems, focusing on building robust infrastructure for generative AI applications. The role requires collaboration with cross-functional teams to enhance AI capabilities and ensure high standards for system performance and validation.
Responsibilities:
• LLM-based System Implementation: Architecting, building, and deploying generative AI systems using large language models. Applying MCP and A2A protocols to ensure dynamic, context-aware, and multi-agent capabilities across AI workflows.
• LLM-Agnostic Infrastructure Development: Developing modular, reusable infrastructure components that support integration with various foundation models (e.g., OpenAI, Azure OpenAI, open-source LLMs). Enabling prompt chaining, context caching, and retrieval-augmented generation (RAG) in a scalable manner.
• Testing and Validation Frameworks: Designing and implementing frameworks for automated evaluation, regression testing, and performance monitoring of LLMs. Ensuring high standards for factual consistency, bias mitigation, hallucination control, and user alignment.
• Cross-functional Integration: Collaborating with Data Scientists, Engineers, and Product Managers to translate AI capabilities into real-world applications. Supporting the integration of LLM functionality into user-facing tools and backend systems.
• Documentation and Maintainability: Writing high-quality, maintainable code with clear documentation to support auditability, version control, and team collaboration. Contributing to internal knowledge sharing and onboarding materials.
Qualifications:
Required:
• 5 years of experience in AI/ML engineering, with 2 years specifically in building and deploying LLM-based systems in production.
• Bachelor’s degree in Data Science, Computer Science, or a related field.
• Proficiency in leveraging Model Context Protocol (MCP) and Agent-to-Agent (A2A) architecture to enable intelligent coordination between AI systems.
• Strong experience building prompt management, context orchestration, and evaluation frameworks that are model-agnostic and production-ready.
• Skilled in designing and operationalizing LLM testing pipelines using synthetic evaluation, unit prompting, and human-in-the-loop feedback loops.
• Strong coding skills in Python and AI toolchains (e.g., LangChain, LlamaIndex, MLflow, FastAPI, etc.), with experience in cloud platforms (Azure preferred) and containerization (Docker, Kubernetes).
Preferred:
• Master’s degree or Ph.D. in Data Science, Computer Science, or a related field.
• Experience with evaluating multiple LLMs and integrating open-source model.
Company:
Relias provides workforce enablement solutions for 12,000 healthcare organizations and 4.5 million caregivers to drive measurable outcomes. Founded in 2002, the company is headquartered in Cary, North Carolina, USA, with a team of 501-1000 employees. The company is currently Late Stage. Relias has a track record of offering H1B sponsorships.
