Interested in this Research Engineer role at Red Hat?
Apply Now →Skills & Technologies
About This Role
Job Summary
---------------
At Red Hat we believe the future of AI is open and we are on a mission to bring the power of open\-source LLMs and distributed LLM inference to every enterprise. We are seeking a highly motivated research intern to join our Machine Learning Research Team. As a research intern, you will work on cutting\-edge networking techniques for ML workloads and contribute to research and engineering efforts that make distributed LLM inference faster, efficient, and more accessible. This is an exciting opportunity to gain hands\-on experience in applied networking for ML while working with leading experts in the field.
Responsibilities
--------------------
- Research via experimentation and theoretical modeling the network bandwidth requirements and trade\-offs in Prefill\-Decode (P/D) disaggregated LLM serving.
- Research and implement networking techniques/methods for high\-performance KV cache transfers in deployment setups without RDMA networking.
- Conduct experiments to evaluate the impact of newly developed non\-RDMA KV Cache transfer techniques on performance (latency and throughput) in P/D LLM serving.
- Collaborate with researchers and engineers to integrate the networking techniques/methods into real\-world distributed inference workflows (e.g. in llm\-d)
- Document findings and contribute to technical reports, research theses, blog posts, or research publications.
Requirements
----------------
- Currently pursuing a Masters (with research) or Ph.D. degree in Computer Science, Electrical Engineering, Machine Learning, or a related field.
- Strong programming skills in C/C\+\+, Rust, and Python.
- Experience with the Linux network stack including frameworks such as DPDK or eBPF/XDP.
- Strong analytical and problem\-solving skills.
- Excellent communication skills and ability to work in a team\-oriented research environment.
- Familiarity with distributed LLM serving with prefill/decode disaggregation and KV cache transfers is a plus, but not required.
Why work with us
--------------------
- Hands\-on experience with state\-of\-the\-art systems for ML research.
- Mentorship from leading experts in LLM inference and networking.
- Opportunity to contribute to research papers, patents, or open\-source projects.
- Competitive stipend and potential for full\-time opportunities.
About Red Hat
Red Hat is the world’s leading provider of enterprise open source software solutions, using a community\-powered approach to deliver high\-performing Linux, cloud, container, and Kubernetes technologies. Spread across 40\+ countries, our associates work flexibly across work environments, from in\-office, to office\-flex, to fully remote, depending on the requirements of their role. Red Hatters are encouraged to bring their best ideas, no matter their title or tenure. We're a leader in open source because of our open and inclusive environment. We hire creative, passionate people ready to contribute their ideas, help solve complex problems, and make an impact.
Inclusion at Red Hat
Red Hat’s culture is built on the open source principles of transparency, collaboration, and inclusion, where the best ideas can come from anywhere and anyone. When this is realized, it empowers people from different backgrounds, perspectives, and experiences to come together to share ideas, challenge the status quo, and drive innovation. Our aspiration is that everyone experiences this culture with equal opportunity and access, and that all voices are not only heard but also celebrated. We hope you will join our celebration, and we welcome and encourage applicants from all the beautiful dimensions that compose our global village.
Equal Opportunity Policy (EEO)
Red Hat is proud to be an equal opportunity workplace and an affirmative action employer. We review applications for employment without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, ancestry, citizenship, age, veteran status, genetic information, physical or mental disability, medical condition, marital status, or any other basis prohibited by law.
### Red Hat does not seek or accept unsolicited resumes or CVs from recruitment agencies. We are not responsible for, and will not pay, any fees, commissions, or any other payment related to unsolicited resumes or CVs except as required in a written contract between Red Hat and the recruitment agency or party requesting payment of a fee.
### Red Hat supports individuals with disabilities and provides reasonable accommodations to job applicants. If you need assistance completing our online job application, email application\[email protected]. General inquiries, such as those regarding the status of a job application, will not receive a reply.
Role Details
About This Role
Research Engineers bridge the gap between research and production. They implement papers, build experiment infrastructure, optimize training pipelines, and make research prototypes production-ready. They're the engineers who make research work at scale.
The role sits at a unique intersection. You need to understand the math well enough to implement novel architectures correctly, and you need the engineering chops to make them run efficiently on distributed systems. When a research scientist has a breakthrough idea, you're the person who turns it from a notebook prototype into a training pipeline that runs on 256 GPUs.
Across the 3,824 AI roles we're tracking, Research Engineer positions make up 2% of the market. At Red Hat, this role fits into their broader AI and engineering organization.
Research Engineer roles are growing as AI labs recognize that research velocity depends on engineering quality. The role is less competitive than Research Scientist (no PhD required), but the bar for engineering skill is very high. These roles are concentrated at major labs and well-funded startups.
What the Work Looks Like
A typical week involves: implementing a new attention mechanism from a recent paper, profiling and optimizing a training pipeline that's bottlenecked on data loading, building evaluation infrastructure for a new benchmark, debugging distributed training issues across a GPU cluster, and pair-programming with a research scientist on their latest experiment. The work is deeply technical.
Research Engineer roles are growing as AI labs recognize that research velocity depends on engineering quality. The role is less competitive than Research Scientist (no PhD required), but the bar for engineering skill is very high. These roles are concentrated at major labs and well-funded startups.
Skills Required
Strong software engineering fundamentals plus ML knowledge. Python, C++, and CUDA experience are common requirements. You'll need to read papers and turn ideas into working code. Distributed systems experience (especially distributed training) is highly valued. Performance optimization skills separate great candidates from good ones.
Experience with large-scale training infrastructure (FSDP, DeepSpeed, Megatron), GPU programming (CUDA, Triton), and the internals of ML frameworks (PyTorch internals, custom autograd functions) is what makes candidates stand out. The best research engineers can debug issues that span the full stack from GPU memory management to numerical precision to algorithmic correctness.
Strong postings mention the team's recent research, the infrastructure scale, and the specific technical challenges. They often list the research areas you'd support. Look for roles that emphasize both implementation quality and research understanding.
Compensation Benchmarks
Research Engineer roles pay a median of $260,000 based on 401 positions with disclosed compensation. Mid-level AI roles across all categories have a median of $160,000.
Across all AI roles, the market median is $200,000. Top-quartile compensation starts at $253,000. The 90th percentile reaches $307,500. For comparison, the highest-paying categories include AI Engineering Manager ($293,500) and AI Safety ($274,200). By seniority level: Entry: $97,380; Mid: $160,000; Senior: $227,400; Director: $243,000; VP: $250,000.
Red Hat AI Hiring
Red Hat has 2 open AI roles right now. They're hiring across Research Engineer, Data Scientist. Positions span Boston, MA, US, Raleigh, NC, US. Compensation range: $195K - $195K.
Location Context
AI roles in Boston pay a median of $213,400 across 422 tracked positions. That's 7% above the national median.
Career Path
Common paths into Research Engineer roles include Software Engineer, ML Engineer, Research Intern.
From here, career progression typically leads toward Senior Research Engineer, Research Scientist, ML Architect.
This is one of the best entry points into AI research without a PhD. Build a strong engineering portfolio with ML projects, contribute to open-source ML frameworks, and demonstrate that you can implement complex ideas correctly and efficiently. The transition to Research Scientist is possible with published first-author work, which some research engineer roles support.
What to Expect in Interviews
Technical screens test both engineering skill and research understanding. Expect coding rounds with performance-critical implementations (GPU optimization, efficient data loading). Be prepared to discuss papers relevant to the team's research area and explain how you'd implement key ideas. System design questions focus on training infrastructure: distributed training, experiment tracking, and compute resource management.
When evaluating opportunities: Strong postings mention the team's recent research, the infrastructure scale, and the specific technical challenges. They often list the research areas you'd support. Look for roles that emphasize both implementation quality and research understanding.
AI Hiring Overview
The AI job market has 3,824 open positions tracked in our dataset. By seniority: 119 entry-level, 1,813 mid-level, 1,472 senior, and 420 leadership roles (Director, VP, C-Level). Remote roles make up 16% of the market (613 positions). The remaining 3,187 roles require on-site or hybrid attendance.
The market median for AI roles is $200,000. Top-quartile compensation starts at $253,000. The 90th percentile reaches $307,500. Highest-paying categories: AI Engineering Manager ($293,500 median, 31 roles); AI Safety ($274,200 median, 51 roles); Research Engineer ($260,000 median, 401 roles).
Research Engineer roles are growing as AI labs recognize that research velocity depends on engineering quality. The role is less competitive than Research Scientist (no PhD required), but the bar for engineering skill is very high. These roles are concentrated at major labs and well-funded startups.
The AI Job Market Today
The AI job market spans 3,824 open positions across 15 role categories. The largest categories by volume: AI/ML Engineer (2,702), Data Scientist (281), AI Software Engineer (258). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.
The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (119) are outnumbered by mid-level (1,813) and senior (1,472) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 420 positions, representing the bottleneck between technical execution and organizational strategy.
Remote work availability sits at 16% of all AI roles (613 positions), with 3,187 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.
AI compensation is structured in clear tiers. The market median sits at $200,000. Top-quartile roles start at $253,000, and the 90th percentile reaches $307,500. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.
Category matters for compensation. AI Engineering Manager roles lead at $293,500 median, while Prompt Engineer roles sit at $142,800. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.
The most in-demand skills across all AI postings: Python (1,968 postings), Aws (1,203 postings), Azure (882 postings), Rag (877 postings), Gcp (735 postings), Prompt Engineering (587 postings), Pytorch (586 postings), Claude (554 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.
Frequently Asked Questions
Get Weekly AI Career Intelligence
Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.