Technical Product Manager, AI Inference & Software

$200K - $350K Remote Mid Level AI Product Manager

Interested in this AI Product Manager role at Rippling?

Apply Now →

About This Role

AI job market dashboard showing open roles by category

About Positron AI

=====================

Positron AI is building next\-generation AI inference accelerators designed from the ground up for low\-latency, high\-throughput large language model inference. Our first\-generation ASIC, Asimov, is a cutting\-edge accelerator targeting frontier AI workloads, with additional generations already underway.

Role Overview

=================

Positron AI is looking for a Technical Product Manager to own AI inference and software technical product planning end to end. In this role, you will be the person who translates where models and inference systems are heading into concrete, well\-scoped requirements for our inference software stack, spanning model coverage, numerics, inference\-engine features and modes, serving\-stack capabilities, and our managed service.

This is a deeply technical planning role that sits at the intersection of engineering, go\-to\-market, and the broader inference ecosystem. You will track the model frontier as a discipline, convert that movement into engineering requests before it becomes a customer escalation, and serve as the connective tissue between our engineering organization, our GTM teams, and our ecosystem partners. You will also be expected to use agentic AI daily as a core part of how the planning function operates.

Key Responsibilities

------------------------

Product Requirements and Roadmap

  • Serve as the leader at Positron for all aspects of AI inference and software technical product planning.
  • Write requirements for the inference software stack spanning model coverage, numerics, inference\-engine features and modes, serving\-stack features and modes, and managed\-service capabilities.
  • Partner with engineering and GTM to build and communicate a clear, defensible roadmap.
  • Create scope and feasibility frameworks that convert model and inference\-system innovations into tangible engineering requests.

Market and Frontier Tracking

  • Keep planning ahead of where models and inference systems are going, tracking the model frontier as a discipline and converting movement into requirements before it surfaces as a customer escalation.
  • Work closely with GTM teams to understand customer and market needs and feed them back into the roadmap.
  • Create competitive briefings covering inference providers, serving stacks, and adjacent hardware platforms.
  • Engage key ecosystem partners, including model labs, open\-source runtimes, and serving and orchestration partners, to understand their technology roadmaps.

Lifecycle, Documentation, and Process

  • Define and manage the software product lifecycle, including versions and release trains, feature modes, model catalog, and deprecation policy.
  • Ensure our software products are well documented for both internal and customer\-facing audiences.
  • Streamline and automate the product planning process using agentic AI.

Required Qualifications

---------------------------

  • 10\+ years of experience across ML systems, inference infrastructure, or serving\-stack engineering, with direct ownership of performance or architecture trade\-offs.
  • Deep expertise in transformer internals at the operator level, including attention variants, MoE routing, KV\-cache mechanics, and quantization formats along with their hardware implications.
  • Hands\-on experience with production inference serving at scale, covering multi\-tenancy, latency SLAs such as TTFT and TPOT, batching and scheduling, disaggregated serving, KV\-cache management, and observability.
  • Working fluency in the open\-source inference ecosystem, including vLLM and SGLang\-class runtimes, kernels, model ingestion, and how models are released, quantized, and adopted in practice.
  • Strong performance analysis skills spanning models and systems, including utilization reasoning, tokens per dollar and tokens per watt arithmetic, and benchmark design, with the ability to build and defend the math personally.
  • Demonstrated experience in competitive landscaping and analysis of inference providers and serving stacks, gained at a model lab, an inference API provider, or an AI hardware company.
  • A proven ability to learn quickly and span the full stack, from model\-architecture details up to fleet\-scale serving systems, while staying current with the model and inference landscape.
  • Excellent communication and interpersonal skills, with comfort navigating uncertainty and driving a process of idea and decision socialization.
  • Confidence being the most technically grounded person in a GTM room and the most market\-aware person in an engineering room.
  • A strong instinct for owning decision history, serving as the documented answer to "why did we choose X," including with executive leadership.
  • Daily, hands\-on use of agentic AI in real technical work, building and running agent workflows for research, analysis, and requirements drafting, with the judgment to verify and own everything the agents produce.

Preferred Qualifications

----------------------------

  • Prior experience at an AI hardware or custom silicon company, with exposure to the realities of bringing a new accelerator platform to market.
  • Direct contribution to or close engagement with open\-source inference runtimes or serving projects.
  • Experience defining and operating a managed inference service, including model catalog and deprecation policy.
  • A track record of building internal automation or agentic workflows that measurably improved a planning or research function.

### Leveling \& Scope

While this role is currently posted at a specific level, we are a growth\-oriented organization and are open to hiring at a more senior level for the right candidate. Please note that this job description serves as a focused but generalized overview of the role; specific responsibilities and impact expectations will be tailored to the experience and seniority of the final hire.

Why Join Us?

----------------

  • You will shape the software roadmap for a purpose\-built inference accelerator, working at the layer where model architecture, systems performance, and real customer workloads meet.
  • You will have unusually direct influence, defining what gets built across the inference stack and seeing it land in silicon\-backed products that compete on performance per dollar and performance per watt.

Compensation \& Benefits

----------------------------

The base salary range for this role is $200,000 – $350,000\.

Please note that the figures provided represent the base salary range only and do not include other elements of our total compensation package, equity, or comprehensive benefits.

At Positron AI, we value the unique expertise each candidate brings. While the range above reflects our typical expectation for the position, we reserve the flexibility to exceed this range for candidates whose specialized skills, significant experience, or unique qualifications fall outside the standard scope of the role. Final offers are determined based on a variety of factors, including internal equity, and individual impact.

Benefits \& Perks

---------------------

We want you to do your best work and feel confident that you and your family are taken care of. That means comprehensive coverage, real time to rest, and support for your future.

Health and wellness

  • Fully company\-paid medical, dental, and vision insurance for you and your dependents
  • Company\-paid life and disability coverage, with voluntary options to add more
  • Supplemental hospital, critical illness, and accident coverage available

Time off and flexibility

  • Unlimited paid time off, we encourage everyone to truly unplug and recharge
  • 13 paid company holidays
  • Remote\-first culture with a company\-provided computer and home office setup

Compensation and future

  • Competitive salary and equity
  • 401(k) with company matching, eligible from day one

Visa Support

----------------

This position is open to candidates currently authorized to work in the U.S. We cannot provide new visa sponsorship for this role but are open to facilitating H\-1B visa transfers for eligible candidates.

Equal Opportunity Employer. If you're excited about the role but don't meet every bullet, we'd still love to hear from you.

Salary Context

This $200K-$350K range is above the 75th percentile for AI Product Manager roles in our dataset (median: $185K across 167 roles with salary data).

View full AI Product Manager salary data →

Role Details

Company Rippling
Title Technical Product Manager, AI Inference & Software
Location Remote, US
Experience Mid Level
Salary $200K - $350K
Remote Yes

About This Role

AI Product Managers define what AI features get built and why. They translate business problems into ML-solvable tasks, work with engineering to scope model requirements, and own the metrics that determine if an AI feature is working. The role requires a rare combination of technical fluency and product instinct.

Unlike traditional product management, AI PM work involves managing uncertainty at a fundamental level. Your model might work 90% of the time. What happens the other 10%? What's the user experience when the AI is wrong? How do you measure 'good enough' for a probabilistic system? These questions don't have easy answers, and the AI PM is the person responsible for finding them.

Across the 4,317 AI roles we're tracking, AI Product Manager positions make up 4% of the market. At Rippling, this role fits into their broader AI and engineering organization.

AI Product Manager roles are growing as companies realize that shipping AI features requires different product thinking than traditional software. The best candidates combine product management experience with enough technical depth to have productive conversations with ML engineers about model capabilities and limitations.

What the Work Looks Like

A typical week includes: reviewing model evaluation results with the ML team, defining success metrics for a new AI feature, conducting user research on how customers respond to AI-generated outputs, writing product requirements that include accuracy thresholds and fallback behaviors, and presenting the AI roadmap to leadership. You're the translator between technical capability and business value.

AI Product Manager roles are growing as companies realize that shipping AI features requires different product thinking than traditional software. The best candidates combine product management experience with enough technical depth to have productive conversations with ML engineers about model capabilities and limitations.

Skills in Demand for This Role

Python (52% of roles) Aws (28% of roles) Azure (22% of roles) Rag (21% of roles) Gcp (15% of roles) Pytorch (15% of roles) Prompt Engineering (14% of roles) Kubernetes (13% of roles)

Technical fluency with ML concepts is essential, though you won't be writing models. Expect to understand training data, evaluation metrics, model limitations, and responsible AI practices. SQL and basic Python are increasingly expected. Experience with A/B testing, data analysis, and product analytics is baseline. Understanding LLM capabilities and limitations is now a core requirement.

The differentiator is AI-specific product thinking: knowing when to use ML vs. heuristics, understanding the cost of training data collection, designing graceful degradation for model failures, and building products that improve with usage data. Experience with AI safety, bias mitigation, and responsible AI deployment is increasingly important.

Strong postings describe specific AI products the PM will own, mention the ML team structure, and talk about measurement methodology. Look for companies that have already shipped AI features. Roles at companies that are 'exploring AI' often mean you'll spend a year defining the strategy before any building happens.

Compensation Benchmarks

AI Product Manager roles pay a median of $217,100 based on 471 positions with disclosed compensation. Mid-level AI roles across all categories have a median of $194,400. This role's midpoint ($275K) sits 27% above the category median. Disclosed range: $200K to $350K.

Across all AI roles, the market median is $215,000. Top-quartile compensation starts at $266,300. The 90th percentile reaches $320,790. For comparison, the highest-paying categories include AI Safety ($287,500) and Research Engineer ($272,100). By seniority level: Entry: $110,000; Mid: $194,400; Senior: $227,400; Director: $274,554; VP: $241,000.

Rippling AI Hiring

Rippling has 12 open AI roles right now. They're hiring across AI/ML Engineer, AI Software Engineer, AI Product Manager, AI Agent Developer. Positions span Melville, NY, US, Columbia, MD, US, Remote, US. Compensation range: $130K - $350K.

Remote Work Context

Remote AI roles pay a median of $180,000 across 1,196 positions. About 15% of all AI roles offer remote work.

Career Path

Common paths into AI Product Manager roles include Product Manager, Data Analyst, Technical Program Manager.

From here, career progression typically leads toward Director of AI Product, VP Product, Head of AI.

The most effective path is PM experience plus self-directed AI education. Take Andrew Ng's courses, build a small ML project, and learn enough Python to read model evaluation code. The goal isn't to become an ML engineer. It's to have credibility in technical conversations and to understand what's possible, what's hard, and what's a bad idea.

What to Expect in Interviews

AI interviews typically combine coding challenges (Python-focused), system design questions tailored to the role, and discussions about your experience with relevant tools and frameworks. Strong candidates demonstrate both technical depth and the ability to make pragmatic engineering tradeoffs. Prepare portfolio projects that demonstrate end-to-end capability rather than isolated skills.

When evaluating opportunities: Strong postings describe specific AI products the PM will own, mention the ML team structure, and talk about measurement methodology. Look for companies that have already shipped AI features. Roles at companies that are 'exploring AI' often mean you'll spend a year defining the strategy before any building happens.

AI Hiring Overview

The AI job market has 4,317 open positions tracked in our dataset. By seniority: 138 entry-level, 2,071 mid-level, 1,655 senior, and 453 leadership roles (Director, VP, C-Level). Remote roles make up 15% of the market (635 positions). The remaining 3,657 roles require on-site or hybrid attendance.

The market median for AI roles is $215,000. Top-quartile compensation starts at $266,300. The 90th percentile reaches $320,790. Highest-paying categories: AI Safety ($287,500 median, 34 roles); Research Engineer ($272,100 median, 227 roles); AI Engineering Manager ($244,000 median, 23 roles).

AI Product Manager roles are growing as companies realize that shipping AI features requires different product thinking than traditional software. The best candidates combine product management experience with enough technical depth to have productive conversations with ML engineers about model capabilities and limitations.

The AI Job Market Today

The AI job market spans 4,317 open positions across 15 role categories. The largest categories by volume: AI/ML Engineer (3,004), Data Scientist (345), AI Software Engineer (309). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.

The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (138) are outnumbered by mid-level (2,071) and senior (1,655) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 453 positions, representing the bottleneck between technical execution and organizational strategy.

Remote work availability sits at 15% of all AI roles (635 positions), with 3,657 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.

AI compensation is structured in clear tiers. The market median sits at $215,000. Top-quartile roles start at $266,300, and the 90th percentile reaches $320,790. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.

Category matters for compensation. AI Safety roles lead at $287,500 median, while Prompt Engineer roles sit at $145,000. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.

The most in-demand skills across all AI postings: Python (2,249 postings), Aws (1,224 postings), Azure (938 postings), Rag (915 postings), Gcp (660 postings), Pytorch (640 postings), Prompt Engineering (624 postings), Kubernetes (559 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.

Frequently Asked Questions

Based on 471 roles with disclosed compensation, the median salary for AI Product Manager positions is $217,100. Actual compensation varies by seniority, location, and company stage.
Technical fluency with ML concepts is essential, though you won't be writing models. Expect to understand training data, evaluation metrics, model limitations, and responsible AI practices. SQL and basic Python are increasingly expected. Experience with A/B testing, data analysis, and product analytics is baseline. Understanding LLM capabilities and limitations is now a core requirement.
About 15% of the 4,317 AI roles we track offer remote work. Remote availability varies by company and seniority level, with senior and leadership roles more likely to offer location flexibility.
Rippling is among the companies actively hiring for AI and ML talent. Check our company profiles for detailed breakdowns of open roles, salary ranges, and hiring trends.
Common next steps from AI Product Manager positions include Director of AI Product, VP Product, Head of AI. Progression depends on whether you lean toward technical depth, people management, or product strategy.

Get Weekly AI Career Intelligence

Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.