Data Platform Architect

Job description

**Key Responsibilities** :

+ Architect and Design the Data Lakehouse: Lead the design and implementation of a scalable and secure Data Lakehouse on AWS, including data storage and compute layers.
+ Storage Solutions: Design and implement storage solutions using AWS services like S3, Iceberg,
+ Integrate relevant metadata from platform with data catalog and/or metadata management solutions.
+ Compute Resources: Architect and optimize compute resources using AWS services like Glue, EMR, and Lambda for ETL processes, and possibly Redshift or Athena for query execution.
+ Develop POCs, POVs and pilots to test architecture, capabilities etc.

and collaborate with collaborate with data engineers to ensure seamless integration and ingestion of data from various sources into the Lakehouse.
+ Security and Compliance: Implement best practices for data security, including encryption, IAM roles, and compliance with relevant data protection regulations.
+ Performance Optimization: Continuously monitor and optimize the performance of the data lakehouse, including storage costs and compute efficiency.
+ Collaboration: Work closely with data engineers, data scientists, and business stakeholders to ensure the platform meets their needs for data products.
+ Documentation and Training: Provide thorough documentation and training to the internal team on the architecture and use of the Data Lakehouse.

**Minimum Experience:**

+ Extensive hands-on experience with AWS services related to data storage and compute (e.g., S3, Glue, EMR, Redshift, Athena, Lambda).
+ Ability to design scalable, reliable, and secure architectures that meet business needs.
+ Strong analytical and problem-solving skills with attention to detail.
+ Collaboration and Communication: Excellent communication skills and the ability to work effectively in a collaborative environment.
+ Ability to mentor junior staff.
+ 7-10+ years of experience designing and/or developing data management platforms, at least 5 in the cloud.

Experience as hands on data engineer is a must.

**Preferred Experience:**

+ Interprets system usage data to predict and plan for future scalability challenges
+ AWS Certification: Certified Solutions Architect or other specialized AWS certifications (e.g., Security, Data Analytics, DevOps).
+ Containerization with Kubernetes: Experience deploying and managing applications using containers orchestrated with Kubernetes.
+ Monitoring: Experience with monitoring tools like Splunk, Prometheus, and Grafana to ensure system health and performance.
+ CI/CD: Hands-on experience building, creating automation scripts and maintaining (CI/CD) pipelines to automate software delivery.
+ Advanced Operations: Experience with complex deployments, command-line tools, performance tuning, and modern data catalogs.
+ Strong understanding of SSO protocols (SAML, OAuth 2.0, OpenID Connect) and directory services like Active Directory.
+ 3+ years of IT experience with exposure to BI tools like Alteryx or Tableau.
+ Knowledge of AWS security best practices and compliance.
WE ARE AN EQUAL OPPORTUNITY EMPLOYER.

Applicants and employees are considered for positions and are evaluated without regard to mental or physical disability, race, color, religion, gender, gender identity, sexual orientation, national origin, age, genetic information, military or veteran status, marital status, mental or physical disability or any other protected Federal, State/Province or Local status unrelated to the performance of the work involved.

Required Skill Profession

Other General