Senior Python / PySpark Engineer GenAI

Job description

Be part of something groundbreaking


At AIG, we are making long-term investments in a brand-new, innovative Generative AI team, designed to explore new possibilities for how artificial intelligence can be applied in insurance and beyond, and we need your help.



With the support and investment needed to explore new frontiers in generative AI, you’ll be working alongside talented colleagues, innovating and leading projects that will transform how we manage risk and serve our customers.



This team is central to our vision of the future and the core of our business offering.

We will incorporate best-in-class engineering and product management principles and your guidance and collaboration will be critical to its success.

To rapidly advance and innovate, we need your skills and expertise to build and scale world-class products.

If you’re excited by the opportunity to create meaningful impact, we’d love to hear from you.




Who we are


AIG is a leading global insurance organization providing a wide range of property casualty insurance and other financial services.

We provide world-class products and expertise to businesses and individuals in approximately 190 countries and jurisdictions.




At AIG, we’re reshaping how the world manages risk, and we’re inviting you to be a key part of that transformation.

As our Data Engineering Senior Manager - GenAI you will have the opportunity to make a meaningful impact, leveraging and further developing your skills to guide groundbreaking AI initiatives.

If you’re looking for a place to grow your career and where your contributions will shape the future, AIG is where you belong.




How you will create impact:

As a Senior Engineer for GenAI, you will lead the design, development, and operationalization of cutting-edge cloud-native big data solutions that enable innovative AI models and analytics.

You’ll collaborate across teams to deliver data solutions that drive business insights and value, all while ensuring scalability and sustainability.

This is an exciting opportunity to shape the future of data engineering at AIG, guiding teams through complex data challenges and enabling AI-powered decision-making that will fuel business growth.




Your responsibilities include:
+ Collaborating with stakeholders to identify and deliver innovative data solutions that meet the evolving needs of the business.
+ Building and maintaining cloud-native big data solutions, advanced analytics products and GenAI models.
+ Partnering with data architects and senior engineers to implement best practices, ensuring sustainable and scalable technical designs.
+ Working closely with analytics, application development, and data science teams to foster collaboration and drive innovation.
+ Delivering on-time, high-quality results on fast-paced schedules while maintaining rigorous standards for development and code management.





What is needed to be successful:
+ Minimum of 7 years of professional software development experience.
+ Strong proficiency in Python (7+ years), hands-on experience with Apache Spark and PySpark for large-scale data processing.
+ Strong proficiency in PySpark (3+ years)
+ Write clean, efficient, and well-documented Python and TypeScript code.
+ Experience with testing frameworks (pytest, unittest).
+ Design, develop and implement data processing pipelines using Python and Spark.
+ You are proficient in cloud-native platforms and tools, including Hadoop, HDFS, Spark, Kubernetes, and ElasticSearch, to deliver high-performance data solutions.
+ Experience with containerization technologies (Docker) and orchestration tools (Kubernetes, Docker Compose) to enable parallel executions of tasks.
+ You have a strong background in deploying and maintaining AI/ML models, with hands-on experience using platforms like Palantir Foundry, Snowflake, and AWS SageMaker.
+ Build and maintain scalable and reliable APIs to expose data and functionalities to other services and applications.
+ Identify and implement performance optimization for existing code and data processing workflows, including leveraging containerization technologies for parallel execution.
+ Contribute to the design and architecture of our data platform and applications.
+ Participate in code reviews to ensure code quality and adherence to best practices.
+ Experience with CI/CD pipeline, including containerized application, code versioning capabilities such as GitHub and Jenkins.
+ Experience setting up E2E Engineering pipeline, monitoring, and troubleshooting.



Ready to take your career to the next level?

We would love to hear from you.



Veterans are encouraged to apply.


#LI-CM1 #LI-AIG #AI #GenAI #artificialintelligence #DataScience #BigData #SoftwareEngineering #SoftwareArchitecture #SoftwareDevelopment #DataEngineering #DataVisualization


At AIG, we value in-person collaboration as a vital part of our culture, which is why we ask our team members to be primarily in the office.

This approach helps us work together effectively and create a supportive, connected environment for our team and clients alike.




Enjoy benefits that take care of what matters

At AIG, our people are our greatest asset.

We know how important it is to protect and invest in what’s most important to you.

That is why we created our Total Rewards Program, a comprehensive benefits package that extends beyond time spent at work to offer benefits focused on your health, wellbeing and financial security—as well as your professional development—to bring peace of mind to you and your family.


Reimagining insurance to make a bigger difference to the world

American International Group, Inc.

(AIG) is a global leader in commercial and personal insurance solutions; we are one of the world’s most far-reaching property casualty networks.

It is an exciting time to join us — across our operations, we are thinking in new and innovative ways to deliver ever-better solutions to our customers.

At AIG, you can go further to support individuals, businesses, and communities, helping them to manage risk, respond to times of uncertainty and discover new potential.

We invest in our largest asset, our people, through continuous learning and development, in a culture that celebrates everyone for who they are and what they want to become.


Welcome to a culture of inclusion

We’re committed to creating a culture that truly respects and celebrates each other’s talents, backgrounds, cultures, opinions and goals.

We foster a culture of inclusion and belonging through learning, cultural awareness activities and Employee Resource Groups (ERGs).

With global chapters, ERGs are a cornerstone for our culture of inclusion.

The talent of our people is one of AIG’s greatest assets, and we are honored that our drive for positive change has been recognized by numerous recent awards and accreditations.


AIG provides equal opportunity to all qualified individuals regardless of race, color, religion, age, gender, gender expression, national origin, veteran status, disability or any other legally protected categories.


AIG is committed to working with and providing reasonable accommodations to job applicants and employees with disabilities.

If you believe you need a reasonable accommodation, please send an email to candidatecare@aig.com.










Functional Area:




DT - Data


AIG PC Global Services, Inc.

Required Skill Profession

Other General