Interested in this Data Engineer role at COMPLEX?
Apply Now →Skills & Technologies
About This Role
Company And Culture
Complex is the definitive platform for global youth culture and music lifestyle, seamlessly integrating cutting-edge content, commerce and live experiences with unparalleled scale. Through innovative content, Complex tells stories of music, streetwear and style, sports, art and beyond. Its content engages in a dynamic conversation with the audience, reflecting and shaping the zeitgeist of convergence culture. A powerful media juggernaut paired with a curated marketplace, Complex is redefining the way fans interact with their favorite brands and artists and reshaping the future of digital culture and commerce.
About the Role
We are seeking a Director, Data Engineering – AI & Data Platforms to lead the strategy, architecture, development, and evolution of our data and AI infrastructure. This is a hands-on leadership role responsible for building a scalable, reliable, and AI-ready data platform that powers analytics, machine learning, automation, and emerging generative AI applications.
The Director will lead the design and implementation of modern data architecture while partnering closely with engineering, analytics, product, and business stakeholders. There will be a a focus on AI initiatives, including developing the data foundations required for machine learning and generative AI, identifying opportunities for AI-driven automation, and helping translate emerging AI capabilities into practical business applications.
This role is ideal for a technical leader who enjoys operating at both the strategic AND hands-on levels and is comfortable building systems, establishing engineering standards, mentoring engineers, and driving cross-functional initiatives in a fast-moving digital media environment.
This position will be on-site in our New York, NY or Los Angeles, CA office.
What You'll Do
Data Engineering & Platform Leadership
- Own the strategy, architecture, and roadmap for the company's data engineering and analytics platform.
- Design and oversee scalable, secure, and cost-effective data architectures and pipelines supporting analytics, reporting, machine learning, and AI applications.
- Establish engineering standards for data modeling, pipeline development, testing, deployment, observability, documentation, and operational excellence.
- Lead the development and optimization of batch and near-real-time data pipelines using SQL and Python.
- Oversee data modeling and warehouse architecture in Snowflake, ensuring scalability, performance, reliability, and efficient use of resources.
- Drive the evolution of our cloud-based data infrastructure using AWS, including S3, EC2, Lambda, and related services.
- Establish robust frameworks for data quality, testing, monitoring, lineage, observability, and alerting.
- Evaluate and introduce technologies that improve the scalability, reliability, and efficiency of the data platform.
- Balance hands-on technical contribution with architectural oversight and engineering leadership.
AI & Machine Learning
- Lead the data engineering strategy supporting machine learning, generative AI, and AI-powered applications.
- Partner with data scientists, engineers, analysts, and business leaders to identify and prioritize high-value AI opportunities.
- Design and oversee data pipelines supporting model training, feature engineering, inference, evaluation, and monitoring.
- Develop the data foundations required for LLM and generative AI applications, including data preparation, embeddings, vector data, retrieval pipelines, and RAG architectures where appropriate.
- Establish processes for AI data quality, model evaluation, experimentation, and performance monitoring.
- Identify opportunities to use AI to improve internal workflows, analytics, data operations, content-related processes, and engineering productivity.
- Evaluate emerging AI technologies and determine where they can provide practical business value.
- Establish responsible and scalable approaches to incorporating AI into the company's data and technology ecosystem.
- Partner with leadership to develop an AI roadmap aligned with business priorities and measurable outcomes.
Engineering Leadership & Team Development
- Provide technical leadership and mentorship to data engineers and other technical contributors.
- Establish engineering best practices for code quality, version control, CI/CD, testing, documentation, security, and operational reliability.
- Define technical objectives, priorities, and development standards for the data engineering function.
- Participate in hiring, onboarding, coaching, performance development, and career growth for data engineering team members.
- Build a culture of technical ownership, experimentation, continuous improvement, and knowledge sharing.
- Determine when to build, buy, or integrate third-party technologies and services.
- Promote reusable frameworks, tooling, and engineering practices that improve team productivity.
Analytics & Business Enablement
- Partner with analysts and business stakeholders to ensure the data platform supports reliable and accessible business intelligence and analytics.
- Improve data accessibility, discoverability, documentation, and usability across the organization.
- Work with stakeholders to translate business requirements into scalable technical solutions.
- Support data experimentation and statistical analysis by ensuring analysts and data scientists have high-quality, appropriately structured datasets.
- Help establish data definitions, governance practices, and standards that improve trust in company data.
- Communicate complex technical concepts and architectural decisions clearly to both technical and non-technical audiences.
Cloud & Platform Operations
- Own the reliability, scalability, and cost management of the organization's cloud-based data infrastructure.
- Oversee deployment and management of data applications and services using AWS.
- Establish appropriate practices for infrastructure automation, CI/CD, security, access controls, and operational monitoring.
- Identify opportunities to optimize cloud costs and platform performance.
- Develop disaster recovery, resiliency, and operational processes appropriate for the company's data infrastructure.
- Work closely with engineering and technology leadership on broader cloud infrastructure initiatives.
Cross-Functional Leadership
- Serve as a strategic technical partner to executive leadership, product, engineering, analytics, and business teams.
- Lead cross-functional initiatives involving data, AI, analytics, automation, and technology modernization.
- Translate technical capabilities and limitations into clear business implications and recommendations.
- Establish priorities across competing data and AI initiatives based on business value, technical feasibility, and available resources.
- Represent the data engineering function in broader technology and organizational planning.
Who You Are
- 8+ years of experience in data engineering, software engineering, analytics engineering, or a related technical discipline.
- 3+ years of experience leading data engineering teams, technical initiatives, or data platform architecture.
- 5+ years of experience with SQL, including complex data exploration, optimization, and relational/data warehouse modeling.
- 5+ years of experience with Python for data engineering, automation, application development, or machine learning workflows.
- 3+ years of experience with Snowflake, including data modeling, architecture, performance optimization, and large-scale data processing.
- 3+ years of experience with AWS, including services such as S3, EC2, Lambda, and related cloud technologies.
- 3+ years of experience with Apache Airflow or comparable orchestration technologies such as Dagster or Prefect.
- Demonstrated experience designing and implementing scalable data platforms and production-grade data pipelines.
- Strong experience with data quality, testing, observability, monitoring, and operational reliability.
- Demonstrated experience supporting machine learning and/or AI initiatives through data engineering, feature engineering, model data pipelines, or AI infrastructure.
- Strong understanding of machine learning concepts, statistical analysis, experimentation, and data science workflows.
- Working knowledge of generative AI, LLMs, embeddings, vector search, RAG, and AI application architectures.
- Strong analytical and critical-thinking skills with the ability to solve complex technical and business problems.
- Demonstrated ability to communicate technical concepts clearly to technical and non-technical audiences.
- Experience working effectively with executives, stakeholders, engineers, analysts, and data scientists.
- Must be willing to work in our NY or LA office.
Preferred Qualifications:
- Experience leading AI transformation or AI platform initiatives.
- Experience designing production systems for LLM or generative AI applications.
- Experience with vector databases, embeddings, RAG architectures, model evaluation, and AI observability.
- Experience implementing AI-powered automation within data or business workflows.
- Experience with infrastructure-as-code technologies such as Terraform.
- Experience with CI/CD, Docker, Kubernetes, or other modern DevOps practices.
- Experience with distributed data processing technologies such as Spark.
- Experience with data governance, metadata management, lineage, security, and privacy.
- Experience working in a digital media, advertising, publishing, entertainment, or consumer technology environment.
- Experience managing data platforms in a high-growth or resource-constrained organization.
- Experience evaluating third-party data, analytics, and AI platforms and negotiating build-versus-buy decisions.
What We Offer
- Best in class health, dental, and vision insurance
- Healthcare FSA
- Dependent Care FSA
- Commuter Benefits FSA
- Short-term/long-term disability and life insurance
- Paid Parental leave
- 401k with 4% match
- Pet Insurance
- Legal and Identity Theft Plans
- Flexible PTO
The Pay Range for this position is listed. Actual pay will vary based on factors including, but not limited to experience and performance. The range listed is just one component of Complex's Total Rewards offerings for employees.We’re an equal opportunity employer. All applicants will be considered for employment without attention to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.
Applicants must be authorized to work in the U.S. We are unable to sponsor or take over sponsorship of an employment visa at this time.
The above statements cover what are generally believed to be principal and essential functions of the job. Specific circumstances may allow or require some incumbents assigned to the job to perform a different combination of duties.
Complex participates in E-Verify and will provide the federal government with your Form I-9 information to confirm that you are authorized to work in the U.S.
Candidates must be legally authorized to work in the U.S. without the need for visa sponsorship.
Salary Context
This $165K-$185K range is above the 75th percentile for Data Engineer roles in our dataset (median: $150K across 25 roles with salary data).
Role Details
About This Role
Data Engineers build the pipelines that feed AI models. They design ETL workflows, manage data lakes, and ensure training and inference data is clean, timely, and accessible. Without good data engineering, AI projects fail. It's that simple.
The AI era has expanded the data engineer's scope far beyond batch ETL jobs. You're building real-time embedding pipelines for RAG systems, managing vector databases, ensuring training data quality at scale, and building the infrastructure that lets ML teams iterate on data as fast as they iterate on models. Data quality is the biggest predictor of model quality, and you're the person responsible for it.
Across the 8,943 AI roles we're tracking, Data Engineer positions make up 1% of the market. At COMPLEX, this role fits into their broader AI and engineering organization.
Data Engineer demand in AI contexts is strong and growing. Every company building AI needs clean, reliable data pipelines. The shift toward real-time AI applications (chatbots, recommendation engines, agent systems) means data engineering is more critical than ever. Companies are willing to pay premium salaries for data engineers with AI/ML pipeline experience.
What the Work Looks Like
A typical week includes: debugging a data pipeline that's producing stale embeddings for the RAG system, optimizing a Spark job that processes training data, building a data quality monitoring dashboard, meeting with the ML team to understand their next data requirements, and writing dbt models that transform raw event data into ML-ready features. The work is deeply technical and high-impact.
Data Engineer demand in AI contexts is strong and growing. Every company building AI needs clean, reliable data pipelines. The shift toward real-time AI applications (chatbots, recommendation engines, agent systems) means data engineering is more critical than ever. Companies are willing to pay premium salaries for data engineers with AI/ML pipeline experience.
Skills Required
SQL, Python, and distributed systems (Spark, Airflow, dbt) are core. Cloud data platforms (Snowflake, BigQuery, Redshift) are increasingly standard. Many AI-focused roles also want familiarity with vector databases and embedding pipelines. Understanding data modeling, pipeline orchestration, and data quality frameworks covers the essentials.
AI-specific data engineering skills include: building feature stores, managing training data versioning, implementing data lineage tracking, and building real-time embedding pipelines. Experience with streaming systems (Kafka, Flink) is valuable for real-time AI applications. Understanding ML data requirements (balanced datasets, data augmentation, evaluation set construction) makes you much more effective working with ML teams.
Strong postings specify the data stack, mention ML pipeline work, and describe the scale of data you'll be working with. Look for companies that understand the connection between data quality and model quality. Avoid roles that conflate data engineering with data analysis.
Compensation Benchmarks
Data Engineer roles pay a median of $177,825 based on 160 positions with disclosed compensation. Director-level AI roles across all categories have a median of $272,250. Disclosed range: $165K to $185K.
Across all AI roles, the market median is $213,380. Top-quartile compensation starts at $264,148. The 90th percentile reaches $319,310. For comparison, the highest-paying categories include AI Safety ($253,100) and Research Engineer ($252,000). By seniority level: Entry: $110,408; Mid: $193,200; Senior: $228,700; Director: $272,250; VP: $235,000.
COMPLEX AI Hiring
COMPLEX has 1 open AI role right now. They're hiring across Data Engineer. Based in New York, NY, US. Compensation range: $185K - $185K.
Location Context
AI roles in New York pay a median of $221,000 across 2,878 tracked positions. That's 4% above the national median.
Career Path
Common paths into Data Engineer roles include Backend Engineer, Database Administrator, Analytics Engineer.
From here, career progression typically leads toward Senior Data Engineer, ML Engineer, Data Platform Lead.
Master SQL and Python first. Then learn a distributed processing framework (Spark or its modern alternatives) and a pipeline orchestrator (Airflow, Dagster, Prefect). Build a portfolio project that demonstrates end-to-end pipeline construction: ingest, transform, validate, serve. If you want to specialize in AI data engineering, add vector databases and embedding pipelines to your skill set.
What to Expect in Interviews
Expect SQL deep-dives (query optimization, partitioning strategies, data modeling), Python coding focused on data pipeline patterns, and system design questions about building scalable ETL workflows. Companies with ML teams will ask about feature stores, embedding pipelines, and training data management. Be ready to discuss data quality monitoring, pipeline orchestration, and how you'd handle schema evolution in a production data lake.
When evaluating opportunities: Strong postings specify the data stack, mention ML pipeline work, and describe the scale of data you'll be working with. Look for companies that understand the connection between data quality and model quality. Avoid roles that conflate data engineering with data analysis.
AI Hiring Overview
The AI job market has 8,943 open positions tracked in our dataset. By seniority: 175 entry-level, 4,132 mid-level, 3,268 senior, and 1,368 leadership roles (Director, VP, C-Level). Remote roles make up 6% of the market (521 positions). The remaining 8,351 roles require on-site or hybrid attendance.
The market median for AI roles is $213,380. Top-quartile compensation starts at $264,148. The 90th percentile reaches $319,310. Highest-paying categories: AI Safety ($253,100 median, 57 roles); Research Engineer ($252,000 median, 419 roles); AI Engineering Manager ($251,622 median, 51 roles).
Data Engineer demand in AI contexts is strong and growing. Every company building AI needs clean, reliable data pipelines. The shift toward real-time AI applications (chatbots, recommendation engines, agent systems) means data engineering is more critical than ever. Companies are willing to pay premium salaries for data engineers with AI/ML pipeline experience.
The AI Job Market Today
The AI job market spans 8,943 open positions across 16 role categories. The largest categories by volume: AI/ML Engineer (6,739), AI Product Manager (403), AI Software Engineer (402). These three account for the majority of open positions, though smaller categories often have higher per-role compensation because of specialized skill requirements.
The seniority mix tells a story about where AI teams are in their maturity. Entry-level roles (175) are outnumbered by mid-level (4,132) and senior (3,268) positions, reflecting that most companies are past the 'build a team from scratch' phase and need experienced engineers who can ship production systems. Leadership roles (Director, VP, C-Level) total 1,368 positions, representing the bottleneck between technical execution and organizational strategy.
Remote work availability sits at 6% of all AI roles (521 positions), with 8,351 requiring on-site or hybrid attendance. The remote share has stabilized after the post-pandemic correction. Senior and specialized roles (Research Scientist, ML Architect) are more likely to be remote-eligible than entry-level positions, partly because experienced hires have more negotiating power and partly because these roles require less hands-on mentorship.
AI compensation is structured in clear tiers. The market median sits at $213,380. Top-quartile roles start at $264,148, and the 90th percentile reaches $319,310. These figures include base salary with disclosed compensation. Total compensation (including equity, bonuses, and sign-on) runs 20-40% higher at companies that offer those components.
Category matters for compensation. AI Safety roles lead at $253,100 median, while Prompt Engineer roles sit at $141,440. The spread between highest and lowest-paying categories reflects the premium on specialized technical skills versus broader analytical roles.
The most in-demand skills across all AI postings: Python (1,422 postings), Aws (808 postings), Azure (638 postings), Rag (555 postings), Gcp (457 postings), Pytorch (433 postings), Claude (341 postings), Kubernetes (337 postings). Python dominates, appearing in the vast majority of role descriptions regardless of category. Cloud platform experience (AWS, GCP, Azure) is the second most common requirement. The newer entrants to the top skills list (RAG, vector databases, LLM APIs) reflect the shift from traditional ML toward generative AI applications.
Frequently Asked Questions
Get Weekly AI Career Intelligence
Salary data, skills demand, and market signals from 16,000+ AI job postings. Every Monday.