Create and manage Azure Databricks workspaces, Unity Catalog setupallocating resources and configuring settings based on specific needs for allenvironments (Dev/Test/Prod).
·Design and implement access control mechanisms, userpermissions, and security policies to safeguard sensitive data.
: 6 to 10yesr expereince
·Configure and launch Databricks clusters, optimizingperformance and cost through effective instance typeselection, auto-scaling, and termination policies.
·Monitor cluster health and performance, troubleshoot issuesproactively, and adjust configurations as necessary.
·Install and configure libraries, dependencies, andenvironment variables essential for data engineering and analytics tasks.
·Implement robust security practices, including networksecurity settings, integration with Azure Active Directory, and dataencryption.
·Stay informed about security best practices andupdates, applying them proactively to maintain a secure environment.
·Monitor resource utilization and costs associated with Databricksusage, identifying opportunities for optimization and cost reduction.
·Analyze cluster performance metrics and implement tuningstrategies to ensure optimal query execution times and resource utilization.
·Track system logs and identify potential issues before they impactusers or operations.
·Collaborate with data engineers, data scientists, andother stakeholders to understand their analytics needs and translate them intoeffective configurations.
·Provide technical support to users, troubleshooting issuesand resolving technical inquiries.
·Document setup processes, configurations, andtroubleshooting steps for future reference and knowledge sharing