Interested in this AI Agent Developer role at PenFed Credit Union?
Apply Now →Skills & Technologies
About This Role
Job Overview
PenFed is hiring a (Hybrid) Forward Deployed Engineer, AI Agentic Delivery at our Tysons, Virginia location. The Forward Deployed Engineer (FDE) embeds directly with PenFed business and delivery teams to translate high-value, high-friction member and business problems into deployed, production-ready Agentforce and AI agent solutions in a matter of weeks rather than quarters. Working alongside senior engineers, architects, and a Deployment Strategist as part of a small, outcome-focused pod, this role is responsible for shipping real member-facing solutions using best-in-class AI tooling, own real technical components end-to-end, and help build PenFed's own agentic delivery capability so it depends less on external vendor pods over time. Responsibilities
Reasonable accommodation may be made to enable individuals with disabilities to perform the essential functions. This is not intended to be an all-inclusive list of job duties, and the position will perform other duties as assigned.
- Develop and deploy member and business solutions using approved AI development tools, including GitHub Copilot, ROVO, and Agentforce coding tools, as part of the day-to-day development workflow.
- Collaborate with senior engineers, architects, and the Deployment Strategist to deliver solutions, assume ownership of assigned components, and develop the capability to lead complex implementations.
- Develop solutions across Agentforce, Data360, platform, and voice/headless architectures, and provide field insights to PenFed product and architecture teams.
- Maintain current knowledge of emerging AI tools, models, and frameworks; evaluate their potential applications; and share relevant findings with team members and business partners.
- Develop and maintain production AI agents and digital experiences that execute actions in support of PenFed members.
- Design agent intelligence, including prompts, reasoning processes, tool calls, and integrations with PenFed systems, using Agentforce and current large language model techniques.
- Manage assigned technical components throughout the full lifecycle, including architecture selection, deployment, validation, and production monitoring.
- Design, build, integrate, and validate data models and pipelines using Salesforce Data Cloud, Data360, Snowflake, and PenFed core banking data platforms.
- Develop and maintain agent performance dashboards and business KPI reporting, including containment rate, average handle time, and sentiment/friction measures, to monitor deployment health and business outcomes.
- Contribute to the design, development, and rapid deployment of proofs of concept and minimum viable products.
- Collaborate with business stakeholders and vendor delivery teams, including Salesforce FDE pods, to deliver solutions, share best practices, and build internal team capabilities.
- Identify and communicate platform gaps, edge cases, and implementation insights to internal architecture teams and vendor product teams to inform future enhancements.
- Partner with the Deployment Strategist and business sponsors to translate PenFed business requirements and operational challenges into deployable agentic solutions.
Qualifications
Equivalent combination of education and experience is considered.
- Bachelor’s degree in Computer Science, Information Technology, or a related field, or an equivalent combination of education and relevant experience.
- Three or more years of software engineering or technical delivery experience; additional experience may be required for senior-level positions.
- Experience delivering technical solutions in customer-facing or business-facing environments.
- Experience managing and delivering business-facing projects in financial services or another regulated industry.
- Proficiency in one or more programming languages, such as Python, JavaScript/TypeScript, Java, or Apex.
- Experience working with large language models, generative AI, or prompt engineering.
- Knowledge of data modeling, application programming interfaces, and system integration patterns.
- Ability to communicate and collaborate effectively with business partners, technical teams, and risk or compliance stakeholders.
- Ability to develop quality code and evaluate technical outputs using established engineering practices.
- Demonstrated ability to learn and apply emerging technologies, tools, and development practices.
- Ability to work effectively in collaborative, delivery-focused team environments.
- Ability to travel occasionally to company or partner worksites, as business needs require.
- Experience using A.I. tools preferred.
Preferred Qualifications
- Salesforce platform experience or certifications (Administrator, Platform Developer I, Agentforce Specialist).
- Hands-on with cloud data platforms (Snowflake, Databricks, Data360).
- Experience with Genesys/Service Cloud Voice, Dynatrace, or MuleSoft.
- Experience across specialist AI delivery areas: data engineering, platform architecture, or headless/voice deployments.
Supervisory Responsibilities This position will not supervise employees.
Licenses and Certifications Salesforce Agentforce, Einstein AI, or equivalent AI/agent platform certification preferred.
Work Environment While performing the duties of this job, the employee is regularly exposed to an indoor office setting with moderate noise. Most roles require working in an office setting with moderate noise and the ability to lift 25 pounds.
Travel Ability to travel to various worksites and be on-call may be required.
Pay Transparency
The anticipated starting salary range for this role is $69,000.00 - $146,491.00
This position is eligible for an organizational performance based annual bonus, subject to board discretion and approval.
This position is eligible for an individual performance based annual bonus.
#LI-Hybrid Benefits
At PenFed, we offer a robust benefits package designed to support you both personally and professionally. You’ll have access to comprehensive health, dental, and vision plans; paid time off; and family-friendly benefits like paid parental leave, care support, and fitness center access. Financial wellness is encouraged through features like a 401(k) match, employee loan discounts, and fully paid life and disability coverage. We also support growth via education assistance, community involvement, and volunteer opportunities.
Our Purpose
Helping members achieve their dreams since 1935.
Pentagon Federal Credit Union (PenFed) is one of America's largest federal credit unions, serving 2.8 million members worldwide with $29 billion in assets. PenFed offers market-leading certificates, checking and savings, credit cards, personal loans, mortgages, auto loans, and a wide range of other financial services, always with members' interests in mind. PenFed is federally insured by the NCUA and is an Equal Housing Lender.
Equal Employment Opportunity
PenFed management will maintain and observe personnel policies which will not discriminate or permit harassment or retaliation against a person because of race, color, creed, age, sex, gender, gender identity, gender expression, religion, national origin, ancestry, marital status, military or veteran status or obligation, the presence of a physical and/or mental disability or medical condition, genetic information, sexual orientation, and all statuses protected by applicable state or local law in all recruiting, hiring, training, compensation, overtime, position classifications, work assignments, facilities, promotions, transfers, employee treatment, and in all other terms and conditions of employment. PenFed will also prohibit retaliation against individuals for raising a complaint of discrimination or harassment or participating in an investigation of same.
PenFed will also reasonably accommodate qualified individuals with a disability so that they can apply for a job or perform the essential functions of a job unless doing so causes a direct threat to these individuals or others in the workplace and the threat cannot be eliminated by reasonable accommodation or if the accommodation creates an undue hardship to PenFed. Contact human resources (HR) with any questions or requests for accommodation at [email protected].
Salary Context
This $69K-$146K range is in the lower quartile for AI Agent Developer roles in our dataset (median: $178K across 36 roles with salary data).
View full AI Agent Developer salary data →Role Details
About This Role
AI Agent Developers build autonomous systems that can reason, plan, and take actions. They design multi-step workflows, tool-use frameworks, and orchestration layers that let LLMs interact with external systems. This is the frontier of applied AI engineering.
Agent development is where the most interesting (and hardest) problems in applied AI live right now. Making an LLM answer a question is straightforward. Making it reliably execute a 15-step workflow that involves calling APIs, reading databases, making decisions, and recovering from errors is an unsolved problem. You're building systems that have to work despite the fact that the underlying model is non-deterministic.
Across the 11,520 AI roles we're tracking, AI Agent Developer positions make up 1% of the market. At PenFed Credit Union, this role fits into their broader AI and engineering organization.
AI Agent Developer is one of the newest and fastest-growing AI role categories. The market is early but accelerating as companies move beyond simple chatbots toward AI systems that can take real actions. Compensation is high because the skill set is rare and the business impact is potentially enormous.
What the Work Looks Like
A typical week includes: designing the action space and tool definitions for a new agent use case, debugging why the agent chose the wrong action sequence on a specific input, building evaluation frameworks that test agent reliability across hundreds of scenarios, optimizing the prompt chain for cost and latency, and implementing safety guardrails to prevent the agent from taking destructive actions. The work is equal parts engineering and empirical science.
AI Agent Developer is one of the newest and fastest-growing AI role categories. The market is early but accelerating as companies move beyond simple chatbots toward AI systems that can take real actions. Compensation is high because the skill set is rare and the business impact is potentially enormous.
Skills Required
Deep experience with LLM APIs and agent frameworks (LangChain, CrewAI, AutoGen). Strong understanding of prompt engineering, function calling, and error handling for non-deterministic systems. Python is standard. Experience with orchestration patterns, state management, and workflow engines adds significant value.
The best agent developers think like systems engineers. They design for failure modes, build observability into every step, and understand that agent reliability is the product. Expertise in evaluation methodology for non-deterministic systems is the differentiator. Can you measure whether your agent works 'well enough'? Can you find the edge cases where it breaks?
Look for roles that describe specific agent use cases, mention evaluation methodology, and talk about production deployment. Early-stage companies exploring agents can be exciting, but be prepared for ambiguity. The most valuable roles are at companies that have already shipped a v1 and need to make it reliable.
Compensation Benchmarks
AI Agent Developer roles pay a median of $215,000 based on 212 positions with disclosed compensation. Mid-level AI roles across all categories have a median of $193,200. This role's midpoint ($107K) sits 50% below the category median. Disclosed range: $69K to $146K.
Across all AI roles, the market median is $213,480. Top-quartile compensation starts at $264,000. The 90th percentile reaches $318,400. For comparison, the highest-paying categories include AI Safety ($266,050) and AI Engineering Manager ($253,461). By seniority level: Entry: $112,320; Mid: $193,200; Senior: $230,000; Director: $272,250; VP: $235,830.
PenFed Credit Union AI Hiring
PenFed Credit Union has 2 open AI roles right now. They're hiring across AI Agent Developer, AI Product Manager. Positions span McLean, VA, US, Tysons Corner, VA. Compensation range: $146K - $146K.
Location Context
Across all AI roles, 6% (708 positions) offer remote work, while 10,737 require on-site attendance. Top AI hiring metros: New York (3,257 roles, $221,000 median); San Francisco (2,766 roles, $260,550 median); Los Angeles (1,580 roles, $210,000 median).
Career Path
Common paths into AI Agent Developer roles include Software Engineer, LLM Engineer, Prompt Engineer.
From here, career progression typically leads toward AI Architect, Principal Engineer, Head of AI Engineering.
Build agents. That's the portfolio. Take an open-source agent framework, build something that completes a non-trivial multi-step task, evaluate it rigorously, and document what you learned about reliability, cost, and failure modes. The field is new enough that practical experience counts for more than credentials.
What to Expect in Interviews
Interviews focus on systems thinking and reliability engineering. Expect questions about agent architecture: how you'd design a multi-step workflow with error recovery, how you'd evaluate agent performance, and how you'd prevent agents from taking destructive actions. Coding exercises often involve building a simple agent with tool use and evaluating its behavior across different scenarios. Discussion of safety and guardrails is increasingly common.
When evaluating opportunities: Look for roles that describe specific agent use cases, mention evaluation methodology, and talk about production deployment. Early-stage companies exploring agents can be exciting, but be prepared for ambiguity. The most valuable roles are at companies that have already shipped a v1 and need to make it reliable.
AI Hiring Overview
The AI job market has 11,520 open positions tracked in our dataset. By seniority: 140 entry-level, 5,630 mid-level, 4,176 senior, and 1,574 leadership roles (Director, VP, C-Level). Remote roles make up 6% of the market (708 positions). The remaining 10,737 roles require on-site or hybrid attendance.
The market median for AI roles is $213,480. Top-quartile compensation starts at $264,000. The 90th percentile reaches $318,400. Highest-paying categories: AI Safety ($266,050 median, 70 roles); AI Engineering Manager ($253,461 median, 58 roles); Research Engineer ($252,000 median, 476 roles).
AI Agent Developer is one of the newest and fastest-growing AI role categories. The market is early but accelerating as companies move beyond simple chatbots toward AI systems that can take real actions. Compensation is high because the skill set is rare and the business impact is potentially enormous.
The AI Job Market Today
The AI job market spans 11,520 open positions across 16 role categories. The largest categories by volume: AI/ML Engineer (8,769), AI Software Engineer (586), AI Product Manager (511). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.
The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (140) are outnumbered by mid-level (5,630) and senior (4,176) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 1,574 positions, representing the bottleneck between technical execution and organizational strategy.
Remote work availability sits at 6% of all AI roles (708 positions), with 10,737 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.
AI compensation is structured in clear tiers. The market median sits at $213,480. Top-quartile roles start at $264,000, and the 90th percentile reaches $318,400. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.
Category matters for compensation. AI Safety roles lead at $266,050 median, while Prompt Engineer roles sit at $140,000. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.
The most in-demand skills across all AI postings: Python (2,267 postings), Aws (1,178 postings), Azure (972 postings), Rag (924 postings), Gcp (715 postings), Pytorch (586 postings), Claude (572 postings), Kubernetes (559 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.
Frequently Asked Questions
Get Weekly AI Career Intelligence
Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.