(USA) Director, Data Science, Testing and Measurement

Job description

**Position Summary...**

**What you'll do...**

Lead the centralization of testing and measurement at Sam’s Club by building a single experimentation and measurement platform and launching self-service tools that empower associates to run high-quality experiments independently.

Starting with in-club measurement and moving to Omni you’ll establish scientific standards, governance, and guardrails that make results trustworthy and repeatable.

You will translate complex data into clear, actionable decisions, mentor a high-performing team, and partner across Product, Engineering, and the business to drive measurable business impact—while reserving your team’s deep expertise for the most complex use cases.

**What** **You’ll** **Do (Performance Objectives)**

+ **Lead** **testing & measurement** **framework** Inventory current experimentation practices across teams; map data sources, metrics, and decision paths; and publish a **Unified Testing & Measurement Playbook** (taxonomy, KPI catalog, experiment types, power/MDE guidance, guardrails, and approval paths).

Secure cross-functional alignment.

+ **Design the experimentation platform** Define architecture and build-versus-buy strategy for an enterprise testing **platform** (assignment service, telemetry, experiment registry, metrics pipeline, results service, observability, and audit trails).

Kick off MVP build with cross functional partners; integrate with access control, and data platforms.

+ **Launch self-serve experimentation (4–6 months):** Ship a guided, **self-service tool** enabling ~80% of standard experiments without DS support (A/A checks, sizing calculators, pre-registration, guardrails, and standardized readouts).

Train product/ops partners and stand up community of practice.

**Success looks like:** ≥70% of experiments initiated via self-serve; median time to test start reduced by ≥40%.

+ **Deliver in-club measurement excellence (pilot → scale):** Stand up robust **in-club measurement** for prioritized initiatives (e.g., signage, layouts, checkout prompts, labor/process changes) using appropriate designs (RCTs, matched controls, diff-in-diff, synthetic control).

Publish weekly readouts and executive rollups.

+ **Institutionalize experimentation rigor (ongoing):** Implement power analysis, minimum detectable effect standards, sequential testing controls, causal inference, and reproducibility checks.

Require pre-registered hypotheses and success criteria.

+ **Scale impact & portfolio ROI** Produce a quarterly experimentation portfolio review quantifying incremental sales, margin, and member engagement; prioritize the roadmap to maximize ROI.

**Success looks like:** X% QoQ increase in experiments tied to top OKRs; clear $ impact from in-club initiatives.

+ **Build and lead a high-** **performing** **team:** Hire, coach, and retain a blended team (data scientists, measurement scientists, and platform/analytics engineers).

Establish high-quality code, review, and documentation standards; promote a culture of healthy debate, learning, and outcomes.

**What** **You’ll** **Bring**

+ **Experimentation & Causal Inference Expertise:** Deep experience designing and validating RCTs and quasi-experiments (e.g., diff-in-diff, synthetic controls, matching/stratification), power/MDE analysis, sequential testing control, variance reduction (e.g., CUPED), and heterogeneous effect analysis.

+ **Platform & Architecture Depth:** Proven track record building or scaling experimentation/measurement platforms (assignment, identity/bucketing, telemetry, metric computation layers, registry, readouts, and observability), ideally in high-scale product or retail environments.

+ **Hands-on Technical Skills:** Deep proficiency in Python for machine learning and statistical modeling, with strong experience using ML frameworks (e.g., TensorFlow, PyTorch, Scikit-learn) and data manipulation libraries (e.g., Pandas, NumPy).

Skilled in SQL for feature engineering and large-scale data extraction.

Committed to production-grade practices including code quality, automated testing, reproducibility, and model lifecycle management (MLOps).

+ **Business Acumen & Storytelling:** Ability to translate noisy signals into clear decisions; influence senior leaders with concise narratives that connect experiments to member outcomes and financial impact.

+ **Leadership & Change Management:** Experience building and mentoring teams, driving cross-functional adoption, and moving organizations from fragmented practices to consistent standards and self-service at scale.

+ **Domain Advantage:** Experience with **in-club/physical retail measurement** (traffic normalization, seasonality/weather controls, club clustering) is highly preferred.

+ **Education:** BS/MS/PhD in a quantitative discipline (e.g., Statistics, Economics, Computer Science) or equivalent experience.

At Sam's Club, we offer competitive pay as well as performance-based bonus awards and other great benefits for a happier mind, body, and wallet!
‎

- Health benefits include medical, vision and dental coverage
‎

- Financial benefits include 401(k), stock purchase and company-paid life insurance
‎

- Paid time off benefits include PTO, parental leave, family care leave, bereavement, jury duty, and voting.

You will also receive PTO and/or PPTO that can be used for vacation, sick leave, holidays, or other purposes.

The amount you receive depends on your job classification and length of employment.

It will meet or exceed the requirements of paid sick leave laws, where applicable.
‎

For information about PTO, see https://one.walmart.com/notices .

‎

- Other benefits include short-term and long-term disability, company discounts, Military Leave Pay, adoption and surrogacy expense reimbursement, and more.
‎

Live Better U is a company paid education benefit program for full-time and part-time associates in Walmart and Sam's Club facilities.

Programs range from high school completion to bachelor's degrees, including English Language Learning and short-form certificates.

Tuition, books, and fees are completely paid for by Walmart.
‎

Eligibility requirements apply to some benefits and may depend on your job classification and length of employment.

Benefits are subject to change and may be subject to a specific plan or program terms.
‎

For information about benefits and eligibility, see One.Walmart (https://bit.ly/3iOOb1J) .

‎

San Bruno, California US-08848:The annual salary range for this position is $169,000.00-$338,000.00
‎

Bentonville, Arkansas US-09930:The annual salary range for this position is $130,000.00-$260,000.00
‎

‎

‎

‎

‎

‎

‎

‎

‎

‎

Additional compensation includes annual or quarterly performance bonuses.
‎

‎

‎

‎

‎

**Minimum Qualifications...**

_Outlined below are the required minimum qualifications for this position.

If none are listed, there are no minimum qualifications._

Option 1: Bachelors degree in Statistics, Economics, Analytics, Mathematics, Computer Science, Information Technology or related field and 6 years' experience in an analytics related field.
Option 2: Masters degree in Statistics, Economics, Analytics, Mathematics, Computer Science, Information Technology or related field and 4 years' experience in an analytics related field.
Option 3: 8 years' experience in an analytics or related field

**Preferred Qualifications...**

_Outlined below are the optional preferred qualifications for this position.

If none are listed, there are no preferred qualifications._

Data science, machine learning, optimization models, PhD in Machine Learning, Computer Science, Information Technology, Operations Research, Statistics, Applied Mathematics, Econometrics, Successful completion of one or more assessments in Python, Spark, Scala, or R, Supervisory experience, Using open source frameworks (for example, scikit learn, tensorflow, torch), We value candidates with a background in creating inclusive digital experiences, demonstrating knowledge in implementing Web Content Accessibility Guidelines (WCAG) 2.2 AA standards, assistive technologies, and integrating digital accessibility seamlessly.

The ideal candidate would have knowledge of accessibility best practices and join us as we continue to create accessible products and services following Walmart’s accessibility standards and guidelines for supporting an inclusive culture.

**Primary Location...**

2101 Se Simple Savings Dr, Bentonville, AR 72712-4304, United States of America

Walmart and its subsidiaries are committed to maintaining a drug-free workplace and has a no tolerance policy regarding the use of illegal drugs and alcohol on the job.

This policy applies to all employees and aims to create a safe and productive work environment.
Walmart, Inc.

is an Equal Opportunity Employer- By Choice.

We believe we are best equipped to help our associates, customers, and the communities we serve live better when we really know them.

That means understanding, respecting, and valuing diversity- unique styles, experiences, identities, abilities, ideas and opinions- while being inclusive of all people.

Required Skill Profession

Other General