Principal Software Engineering Manager - AI Frameworks

Overview

As a Principal Software Engineering Manager – AI Frameworks on the team, you will lead and grow a group of engineers working across the AI software serving stack while remaining technically hands-on. You will set direction and contribute to critical engineering decisions across developer infrastructure, build systems, runtimes, libraries, release pipelines, and application programming interfaces (APIs). You will be responsible for improving engineering velocity and build reliability, establishing effective system and release practices, and ensuring the team ships high-quality software to production for large-scale model training and inference.

In this role, you will guide the team’s work on benchmarking OpenAI and other large language models (LLMs) across GPUs and Microsoft hardware, translating performance findings into production-ready builds and releases. You will drive automation across build, validation, benchmarking, regression detection, and deployment workflows, including advanced AI-assisted automation and closed-loop systems that accelerate diagnosis and remediation. You will partner closely with researchers, product teams, and platform owners to improve developer velocity, reduce time-to-deployment and hardware footprint, and support Microsoft Azure’s capex efficiency goals.

At Microsoft, our mission to empower every person and every organization on the planet to achieve more guides how we partner with customers to deliver trusted, impactful solutions. With a growth mindset culture, we innovate responsibly and measure success by shared progress people, teams, and customers. Join us to do meaningful work that changes the world and helps shape what’s next for everyone.

Responsibilities

• Lead and develop a team of engineers working across multiple layers of the AI software stack, while staying technically engaged in architecture, design reviews, debugging, and critical implementation decisions that enable large-scale training and inference.
• Set technical vision and execution strategy for developer infrastructure, build and release systems, model performance benchmarking, optimization, and production deployment across GPUs and Microsoft hardware.
• Drive performance and production outcomes by prioritizing and overseeing efforts to build, benchmark, profile, debug, validate, and optimize training and inference workloads, from developer workflows through production release.
• Own build, release, and performance health by establishing best practices for build reliability, system observability, regression monitoring, release quality, impact measurement, developer velocity, time-to-deploy, and hardware efficiency.
• Partner cross-functionally with research, product, infrastructure, and hardware teams to deliver scalable, production-ready AI performance improvements.
• Balance short-term delivery and long-term investments by advancing build, release, and validation automation, including AI-assisted workflows and closed-loop systems that identify issues, recommend or implement fixes, and verify outcomes. Ensure these investments align with organizational goals, platform roadmaps, and Azure capex objectives.
• Build a strong engineering culture through coaching, feedback, hiring, and career development, enabling the team to operate with increasing autonomy and impact.

Qualifications Qualifications

Minimum/Required Qualifications:

• Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.

Other Requirements:

• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications:

• Master’s Degree in Computer Science or related technical field AND 10+ years of software engineering experience, including 6+ years in engineering management, + OR Bachelor’s Degree in Computer Science or related technical field AND 12+ years of software engineering experience, including 6+ years in engineering management, + or equivalent experience.
• 4+ years people management experience.
• Strong technical foundation in software engineering principles, computer architecture, GPU architecture, and hardware acceleration for neural networks, combined with hands-on experience in developer infrastructure such as build systems, dependency management, Bazel or comparable tooling, CI/CD, release engineering, and developer productivity.
• Experience leading teams responsible for end-to-end performance analysis and optimization of LLMs, AI systems, or HPC workloads, including GPU profiling and performance analysis tools, and shipping validated builds through release pipelines into production environments.
• Demonstrated ability to lead cross-team initiatives, align stakeholders, and translate research or platform capabilities into scalable, production-ready solutions.
• Proven people leadership skills, including hiring, coaching, performance management, and career development, with a track record of building high-performing, inclusive teams.
• Working knowledge of AI and ML systems across training, inference, evaluation, and benchmarking, with experience in at least one modern deep learning framework such as PyTorch, TensorFlow, or ONNX Runtime. The candidate should be able to reason about how models, frameworks, runtimes, hardware, and developer infrastructure interact across the end-to-end lifecycle.
• Familiarity with GPU software stacks and acceleration technologies such as CUDA, ROCm, Triton, or equivalent, sufficient to guide technical direction and evaluate tradeoffs. Experience designing advanced automation for build, test, release, benchmarking, and production validation workflows is preferred, including AI-assisted tooling and closed-loop systems that use observed results to drive subsequent actions.

#AIInfra

Software Engineering M5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

https://careers.microsoft.com/us/en/us-corporate-pay

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process.