Senior Software Development Engineer in Test, ML

$152K - $190K Madison, WI, US Senior AI Product Manager

Interested in this AI Product Manager role at Sony Interactive Entertainment?

Apply Now →

Skills & Technologies

AwsDockerGcpKubernetesPython

About This Role

AI job market dashboard showing open roles by category

Position Type: Location(s): United States, Madison, WI Date Posted: Job ID: R-124327

Why Sony Interactive Entertainment?

Sony Interactive Entertainment isn’t just the Best Place to Play — it’s also the Best Place to Work. Sony Interactive Entertainment (SIE) is the company behind the PlayStation brand. As a subsidiary of Sony Group Corporation, we’re part of a proud legacy of innovation and excellence. SIE is a dynamic technology company, delivering cutting-edge hardware and network services to more than 100 million people and an entertainment leader, home to some of the most beloved and recognizable intellectual properties (IP) in the world. Our role at SIE is to create and nurture the experiences under the PlayStation brand, a name synonymous with entertainment excellence and creativity.

At PlayStation, we're passionate about both partnership and technology. We care about building performant products and powerful tools, solving practical problems in delightful ways. We love building the systems that millions of people use, and we're always our own first customers and critics. We strive to be at the forefront of development technologies and celebrate our diversity with engineering teams located on four continents from all walks of life. We build our teams to be places where every voice is heard, all input is valued, and our members have the creative freedom to make their ideas real.

Sr Software Development Engineer in Test, ML

Locations: Madison, WI and Waterloo, Ontario Canada

PlayStation is seeking a Sr Software Development Engineer in Test, ML to join our Madison team and lead quality efforts for machine learning-powered products and services. You will partner closely with ML engineers, software engineers, product managers, and data partners to build scalable test automation and validation approaches for systems serving PlayStation at global scale. This role is focused on building confidence in model reliability, production readiness, and behavioral consistency. Unlike traditional software backend testing, success here requires validating probabilistic systems through score distributions, cohort-level behavior, thresholds, and statistical signals, not just deterministic pass/fail assertions.

Responsibilities:

  • Drive architecture-level automation solutions and scalable quality engineering practices across multiple teams and product areas.
  • Design, develop, and maintain automated test frameworks, test scripts for ML inference services and related ML platform components using python/Java
  • Build and automate realistic, representative test account creations and datasets to evaluate model behavior across different scenarios
  • Validate score distributions, ranking behavior, output quality, and other model signals for incoming ML models prior to release
  • Perform different types of testing, including functional, integration, regression and API testing working with RESTful APIs, microservices and databases, and non-functional performance and load testing to verify inference service scalability, latency requirements and reliability under production like conditions
  • Work closely with ML engineers and ops teams to ensure that all features and bug fixes come with automated test coverage, ensuring continuous integration and deployment
  • Debug and analyze test failures, model anomalies and report defects
  • Drive high standards of quality for both engineering decisions and customer-facing features with optimal test and automation strategy
  • Develop and implement best practices for test automation that is highly scalable and maintainable
  • Monitor test execution and optimize performance in test environments
  • Come up with innovative solutions to organizational problems

Required Qualifications:

  • A Bachelor's degree in Computer Science or related field
  • Minimum 5 years of experience as an SDET, with strong experience testing backend systems.
  • Proficiency in Python/Java/Scala/go for test automation.
  • Hands-on experience with API testing tools (e.g., Postman or similar) for distributed systems
  • Familiarity with CI/CD pipelines and test execution in Jenkins or similar environments such as jenkins, GitHub Actions, ArgoCD, etc
  • Hands-on experience building and maintaining automated test frameworks (e.g., pytest, JUnit, TestNG, etc.)Experience in working with databases for backend validation.
  • Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker, etc.)
  • Familiarity with monitoring and observability tools (e.g., Prometheus, Grafana, Datadog, etc.) for validating service health
  • Strong understanding of software development lifecycle (SDLC) and agile methodologies.
  • Excellent problem-solving skills and excellent communication skills.

Preferred Qualifications:

  • Experience testing machine learning systems, such as recommendation systems, search ranking systems, or other probabilistic ML application systems.
  • Experience creating synthetic or seeded test data to simulate realistic customer/account behaviors.
  • Experience evaluating model outputs using statistical techniques, distribution analysis, or scenario-based validation, rather than deterministic assertions.
  • Knowledge of data pipelines, feature stores, inference systems or model-serving infrastructure (Seldon, KServe, Ray Serve, etc.) is a plus.
  • Experience with performance and load testing tools
  • Experience with Databricks or similar ML platform tooling
  • Experience architecting and optimizing CI/CD pipelines, quality gates, and release validation systems using Jenkins or similar platforms.

At SIE, we consider several factors when setting each role’s base pay range, including the competitive benchmarking data for the market and geographic location.

Please note that the base pay range may vary in line with our hybrid working policy and individual base pay will be determined based on job-related factors which may include knowledge, skills, experience, and location.

In addition, this role is eligible for SIE’s top-tier benefits package that includes medical, dental, vision, matching 401(k), paid time off, wellness program and coveted employee discounts for Sony products. This role also may be eligible for a bonus package.

The estimated base pay range for this role is listed below. $152,000—$190,000 USD Please note, Sony Interactive Entertainment conducts background checks at the offer stage for all new employees (which may include criminal background checks for some roles) and will need to process personal information to support these checks.

Please refer to our Candidate Privacy Notice for more information about what personal information we collect, how we use it, who we share it with, and your data protection rights.

Equal Opportunity Statement:

Sony is an Equal Opportunity Employer. All persons will receive consideration for employment without regard to gender (including gender identity, gender expression and gender reassignment), race (including colour, nationality, ethnic or national origin), religion or belief, marital or civil partnership status, disability, age, sexual orientation, pregnancy, maternity or parental status, trade union membership or membership in any other legally protected category.

We strive to create an inclusive environment, empower employees and embrace diversity. We encourage everyone to respond.

Sony Interactive Entertainment is a Fair Chance employer and qualified applicants with arrest and conviction records will be considered for employment.

Salary Context

This $152K-$190K range is below the median for AI Product Manager roles in our dataset (median: $177K across 148 roles with salary data).

View full AI Product Manager salary data →

Role Details

Title Senior Software Development Engineer in Test, ML
Location Madison, WI, US
Experience Senior
Salary $152K - $190K
Remote No

About This Role

AI Product Managers define what AI features get built and why. They translate business problems into ML-solvable tasks, work with engineering to scope model requirements, and own the metrics that determine if an AI feature is working. The role requires a rare combination of technical fluency and product instinct.

Unlike traditional product management, AI PM work involves managing uncertainty at a fundamental level. Your model might work 90% of the time. What happens the other 10%? What's the user experience when the AI is wrong? How do you measure 'good enough' for a probabilistic system? These questions don't have easy answers, and the AI PM is the person responsible for finding them.

Across the 10,419 AI roles we're tracking, AI Product Manager positions make up 5% of the market. At Sony Interactive Entertainment, this role fits into their broader AI and engineering organization.

AI Product Manager roles are growing as companies realize that shipping AI features requires different product thinking than traditional software. The best candidates combine product management experience with enough technical depth to have productive conversations with ML engineers about model capabilities and limitations.

What the Work Looks Like

A typical week includes: reviewing model evaluation results with the ML team, defining success metrics for a new AI feature, conducting user research on how customers respond to AI-generated outputs, writing product requirements that include accuracy thresholds and fallback behaviors, and presenting the AI roadmap to leadership. You're the translator between technical capability and business value.

AI Product Manager roles are growing as companies realize that shipping AI features requires different product thinking than traditional software. The best candidates combine product management experience with enough technical depth to have productive conversations with ML engineers about model capabilities and limitations.

Skills Required

Aws (11% of roles) Docker (4% of roles) Gcp (7% of roles) Kubernetes (6% of roles) Python (23% of roles)

Technical fluency with ML concepts is essential, though you won't be writing models. Expect to understand training data, evaluation metrics, model limitations, and responsible AI practices. SQL and basic Python are increasingly expected. Experience with A/B testing, data analysis, and product analytics is baseline. Understanding LLM capabilities and limitations is now a core requirement.

The differentiator is AI-specific product thinking: knowing when to use ML vs. heuristics, understanding the cost of training data collection, designing graceful degradation for model failures, and building products that improve with usage data. Experience with AI safety, bias mitigation, and responsible AI deployment is increasingly important.

Strong postings describe specific AI products the PM will own, mention the ML team structure, and talk about measurement methodology. Look for companies that have already shipped AI features. Roles at companies that are 'exploring AI' often mean you'll spend a year defining the strategy before any building happens.

Compensation Benchmarks

AI Product Manager roles pay a median of $221,400 based on 1,193 positions with disclosed compensation. Senior-level AI roles across all categories have a median of $230,000. This role's midpoint ($171K) sits 23% below the category median. Disclosed range: $152K to $190K.

Across all AI roles, the market median is $214,834. Top-quartile compensation starts at $266,000. The 90th percentile reaches $321,730. For comparison, the highest-paying categories include AI Safety ($274,500) and Research Engineer ($252,000). By seniority level: Entry: $114,400; Mid: $193,200; Senior: $230,000; Director: $272,100; VP: $235,000.

Sony Interactive Entertainment AI Hiring

Sony Interactive Entertainment has 5 open AI roles right now. They're hiring across AI Software Engineer, AI Product Manager, AI/ML Engineer. Positions span Aliso Viejo, CA, US, Madison, WI, US, San Diego, CA. Compensation range: $190K - $456K.

Location Context

Across all AI roles, 6% (580 positions) offer remote work, while 9,776 require on-site attendance. Top AI hiring metros: New York (3,785 roles, $222,480 median); San Francisco (3,262 roles, $261,300 median); Los Angeles (1,896 roles, $210,000 median).

Career Path

Common paths into AI Product Manager roles include Product Manager, Data Analyst, Technical Program Manager.

From here, career progression typically leads toward Director of AI Product, VP Product, Head of AI.

The most effective path is PM experience plus self-directed AI education. Take Andrew Ng's courses, build a small ML project, and learn enough Python to read model evaluation code. The goal isn't to become an ML engineer. It's to have credibility in technical conversations and to understand what's possible, what's hard, and what's a bad idea.

What to Expect in Interviews

AI interviews typically combine coding challenges (Python-focused), system design questions tailored to the role, and discussions about your experience with relevant tools and frameworks. Strong candidates demonstrate both technical depth and the ability to make pragmatic engineering tradeoffs. Prepare portfolio projects that demonstrate end-to-end capability rather than isolated skills.

When evaluating opportunities: Strong postings describe specific AI products the PM will own, mention the ML team structure, and talk about measurement methodology. Look for companies that have already shipped AI features. Roles at companies that are 'exploring AI' often mean you'll spend a year defining the strategy before any building happens.

AI Hiring Overview

The AI job market has 10,419 open positions tracked in our dataset. By seniority: 117 entry-level, 4,975 mid-level, 3,840 senior, and 1,487 leadership roles (Director, VP, C-Level). Remote roles make up 6% of the market (580 positions). The remaining 9,776 roles require on-site or hybrid attendance.

The market median for AI roles is $214,834. Top-quartile compensation starts at $266,000. The 90th percentile reaches $321,730. Highest-paying categories: AI Safety ($274,500 median, 90 roles); Research Engineer ($252,000 median, 580 roles); AI Engineering Manager ($250,811 median, 72 roles).

AI Product Manager roles are growing as companies realize that shipping AI features requires different product thinking than traditional software. The best candidates combine product management experience with enough technical depth to have productive conversations with ML engineers about model capabilities and limitations.

The AI Job Market Today

The AI job market spans 10,419 open positions across 16 role categories. The largest categories by volume: AI/ML Engineer (7,849), AI Product Manager (516), AI Software Engineer (513). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.

The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (117) are outnumbered by mid-level (4,975) and senior (3,840) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 1,487 positions, representing the bottleneck between technical execution and organizational strategy.

Remote work availability sits at 6% of all AI roles (580 positions), with 9,776 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.

AI compensation is structured in clear tiers. The market median sits at $214,834. Top-quartile roles start at $266,000, and the 90th percentile reaches $321,730. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.

Category matters for compensation. AI Safety roles lead at $274,500 median, while Prompt Engineer roles sit at $140,000. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.

The most in-demand skills across all AI postings: Python (2,350 postings), Aws (1,173 postings), Azure (1,030 postings), Rag (986 postings), Gcp (714 postings), Pytorch (649 postings), Kubernetes (583 postings), Claude (581 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.

Frequently Asked Questions

Based on 1,193 roles with disclosed compensation, the median salary for AI Product Manager positions is $221,400. Actual compensation varies by seniority, location, and company stage.
Technical fluency with ML concepts is essential, though you won't be writing models. Expect to understand training data, evaluation metrics, model limitations, and responsible AI practices. SQL and basic Python are increasingly expected. Experience with A/B testing, data analysis, and product analytics is baseline. Understanding LLM capabilities and limitations is now a core requirement.
About 6% of the 10,419 AI roles we track offer remote work. Remote availability varies by company and seniority level, with senior and leadership roles more likely to offer location flexibility.
Sony Interactive Entertainment is among the companies actively hiring for AI and ML talent. Check our company profiles for detailed breakdowns of open roles, salary ranges, and hiring trends.
Common next steps from AI Product Manager positions include Director of AI Product, VP Product, Head of AI. Progression depends on whether you lean toward technical depth, people management, or product strategy.

Get Weekly AI Career Intelligence

Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.