Urgent! Data Scientist – Generative AI & Model Evaluation Job Opening In Brooklyn – Now Hiring KeyBank
Location:
4910 Tiedeman Road, Brooklyn OhioPosition Location Policy
General Location: Open to candidates within the United States.
Hybrid Requirement (if within specific cities): If the selected candidate resides in Cleveland, OH, Buffalo, NY, or Albany, NY, they are expected to work on-site 2 days per week at the nearest KeyBank office (non-branch location).
Remote Option: If the selected candidate lives outside of KeyBank’s geographic footprint, the position will be considered fully remote.
Available for East Coast meetings, required
Within the Commercial Bank, the Data Scientist role is primarily responsible for identifying, developing, and critically evaluating AI-powered solutions across a range of use-cases spanning sales enablement, marketing content generation, enhanced customer onboarding experiences, intelligent servicing, and novel risk assessment methodologies.
Leveraging current and emerging Generative AI technologies, particularly Large Language Models (LLMs), this role will focus on prompt engineering, data curation for LLM training/fine-tuning, evaluation methodology and technique selection for generative outputs, bias and fairness assessment of AI-generated content, robust documentation of generative model behavior and limitations, and continuous monitoring of AI application performance and ethical implications.
ESSENTIAL JOB FUNCTIONS
Perform a broad range of quantitative works, including designing, developing, and implementing generative AI solutions, with a strong emphasis on prompt engineering and fine-tuning of LLMs to address business needs while adhering to model risk and regulatory requirements.
Conduct ad hoc analysis of generative model outputs to identify areas for improvement.
Research, compile, and evaluate large sets of data, focusing on suitability for LLM training, fine-tuning, and importantly, for robust evaluation of LLM outputs.
This includes creating and curating ground truth datasets and adversarial examples for performance assessment.
Develop and maintain internal frameworks for evaluating LLM performance, including metrics for accuracy, fluency, coherence, factual correctness, toxicity, bias, and safety.
Test and configure vendor-provided LLM solutions, assessing their suitability and performance against defined criteria.
Document LLM prompt engineering strategies, model configurations, evaluation methodologies, and observed behaviors, including limitations and potential failure modes.
Support model validation by providing clear documentation on how the LLM functions and the results of its evaluations.
Employ innovative techniques to drive continuous improvements in LLM output quality, relevance, and safety.
Focus on techniques such as reinforcement learning from human feedback (RLHF), prompt optimization, and bias mitigation strategies to enhance effectiveness and efficiency, e.g., improving factual accuracy and reducing harmful outputs.
Proactively develop and build technical skills and business knowledge, particularly in Generative AI ethics, responsible AI principles, and LLM-specific risk management.
Effectively collaborate with compliance, technology, and risk partners to ensure the safe and ethical deployment of AI applications.
Delivers clear, persuasive communication tailored to stakeholders; proactively shares relevant information and excels in high-stakes or conflict-driven conversations.
Builds strong relationships and collaborates effectively; consults with mid-level leaders to resolve key issues and influence outcomes.
Leverages deep banking and financial insight to drive data-informed strategies aligned with business goals and market dynamics
EDUCATION
Bachelor's Degree in business, finance, MIS, analytics/data science, engineering, or related field, required
Master's or Ph.D. in a quantitative discipline, preferred
WORK EXPERIENCE
Applies structured thinking to complex problems; anticipates risks, integrates cross-functional perspectives, identify trends, and vets solutions thoroughly.
Expertise with traditional Machine Learning (ML)/Artificial Intelligence (AI) modeling practices, with demonstrated experience in Generative AI techniques and a deep understanding of LLM architecture, capabilities, and limitations.
Advanced skills in tools such as SQL, Python, R, Cognos, SAS, Tableau, Excel, and Google Cloud for data manipulation and analysis.
Hands-on work experience with statistical coding in Python, including experience with LLM libraries and frameworks (e.g., Hugging Face Transformers, LangChain, OpenAI API) and relevant data manipulation libraries (Pandas, NumPy).
Knowledge of and ability to leverage traditional databases, cloud-based computing, and distribution computing.
Familiarity with vector databases and MLOps principles specifically for managing and deploying generative AI models.
Knowledge of financial crime regulatory requirements, technology, and data analysis best practices.
Understanding of AI ethics principles, responsible AI governance frameworks, and specific regulatory considerations for AI/LLM deployments.
Excellent verbal, written, and visual communication skills; ability to translate complex generative AI concepts, LLM evaluation findings, and potential risks to a non-technical audience.
Experience in reviewing work, mentoring junior analysts, and contributing to a collaborative team environment
Consistently meets goals and holds self and others accountable; maintains focus on priorities and escalates challenges when needed.
PREFERRED QUALIFICATIONS
Experience in Commercial Banking modeling/analytics is a plus, especially in the context of leveraging AI for these domains.
Experience in developing and implementing LLM evaluation frameworks, including defining and applying relevant metrics.
Familiarity with prompt injection vulnerabilities and mitigation strategies.
Experience with Retrieval-Augmented Generation (RAG) architectures.
PHYSICAL DEMANDS
General office environment: Prolonged sitting, ability to communicate face-to-face in person or on the phone with teammates and clients, frequent use of PC/laptop, occasional lifting/pushing/pulling of backpacks, computer bags up to 10 lbs.
TRAVEL REQUIREMENTS
May need to travel to corporate office or for industry conferences
Key has implemented a role-based Mobile by Design approach to our employee workspaces, dedicating space to those whose roles require specific workspaces, while providing flexible options for roles which are less dependent on assigned workspaces and can be performed effectively in a mobile environment.As a result, this role may be mobile or home based, which means you may work either at a home office or in a Key facility to perform your job duties.
COMPENSATION AND BENEFITS
This position is eligible to earn a base salary in the range of $79,000to $85,000 annually depending on location and job-related factors such as level of experience.
Compensation for this role also includes eligibility for short-term incentive compensation and deferred incentive compensation subject to individual and company performance.
Please click for a list of benefits for which this position is eligible.
Key has implemented an approach to employee workspaces which prioritizes in-office presence, while providing flexible options in circumstances where roles can be performed effectively in a mobile environment.
Job Posting Expiration Date: 10/08/2025 KeyCorp is an Equal Opportunity Employer committed to sustaining an inclusive culture.Qualified individuals with disabilities or disabled veterans who are unable or limited in their ability to apply on this site may request reasonable accommodations by emailing
#LI-Remote
✨ Smart • Intelligent • Private • Secure
Practice for Any Interview Q&A (AI Enabled)
Predict interview Q&A (AI Supported)
Mock interview trainer (AI Supported)
Ace behavioral interviews (AI Powered)
Record interview questions (Confidential)
Master your interviews
Track your answers (Confidential)
Schedule your applications (Confidential)
Create perfect cover letters (AI Supported)
Analyze your resume (NLP Supported)
ATS compatibility check (AI Supported)
Optimize your applications (AI Supported)
O*NET Supported
O*NET Supported
O*NET Supported
O*NET Supported
O*NET Supported
European Union Recommended
Institution Recommended
Institution Recommended
Researcher Recommended
IT Savvy Recommended
Trades Recommended
O*NET Supported
Artist Recommended
Researchers Recommended
Create your account
Access your account
Create your professional profile
Preview your profile
Your saved opportunities
Reviews you've given
Companies you follow
Discover employers
O*NET Supported
Common questions answered
Help for job seekers
How matching works
Customized job suggestions
Fast application process
Manage alert settings
Understanding alerts
How we match resumes
Professional branding guide
Increase your visibility
Get verified status
Learn about our AI
How ATS ranks you
AI-powered matching
Join thousands of professionals who've advanced their careers with our platform
Unlock Your Data Scientist Potential: Insight & Career Growth Guide
Real-time Data Scientist Jobs Trends in Brooklyn, United States (Graphical Representation)
Explore profound insights with Expertini's real-time, in-depth analysis, showcased through the graph below. This graph displays the job market trends for Data Scientist in Brooklyn, United States using a bar chart to represent the number of jobs available and a trend line to illustrate the trend over time. Specifically, the graph shows 53312 jobs in United States and 40 jobs in Brooklyn. This comprehensive analysis highlights market share and opportunities for professionals in Data Scientist roles. These dynamic trends provide a better understanding of the job market landscape in these regions.
Great news! KeyBank is currently hiring and seeking a Data Scientist – Generative AI & Model Evaluation to join their team. Feel free to download the job details.
Wait no longer! Are you also interested in exploring similar jobs? Search now: Data Scientist – Generative AI & Model Evaluation Jobs Brooklyn.
An organization's rules and standards set how people should be treated in the office and how different situations should be handled. The work culture at KeyBank adheres to the cultural norms as outlined by Expertini.
The fundamental ethical values are:The average salary range for a Data Scientist – Generative AI & Model Evaluation Jobs United States varies, but the pay scale is rated "Standard" in Brooklyn. Salary levels may vary depending on your industry, experience, and skills. It's essential to research and negotiate effectively. We advise reading the full job specification before proceeding with the application to understand the salary package.
Key qualifications for Data Scientist – Generative AI & Model Evaluation typically include Mathematical Science Occupations and a list of qualifications and expertise as mentioned in the job specification. Be sure to check the specific job listing for detailed requirements and qualifications.
To improve your chances of getting hired for Data Scientist – Generative AI & Model Evaluation, consider enhancing your skills. Check your CV/Résumé Score with our free Resume Scoring Tool. We have an in-built Resume Scoring tool that gives you the matching score for each job based on your CV/Résumé once it is uploaded. This can help you align your CV/Résumé according to the job requirements and enhance your skills if needed.
Here are some tips to help you prepare for and ace your job interview:
Before the Interview:To prepare for your Data Scientist – Generative AI & Model Evaluation interview at KeyBank, research the company, understand the job requirements, and practice common interview questions.
Highlight your leadership skills, achievements, and strategic thinking abilities. Be prepared to discuss your experience with HR, including your approach to meeting targets as a team player. Additionally, review the KeyBank's products or services and be prepared to discuss how you can contribute to their success.
By following these tips, you can increase your chances of making a positive impression and landing the job!
Setting up job alerts for Data Scientist – Generative AI & Model Evaluation is easy with United States Jobs Expertini. Simply visit our job alerts page here, enter your preferred job title and location, and choose how often you want to receive notifications. You'll get the latest job openings sent directly to your email for FREE!