Flag job

Report

Data Engineer II - Remote

Salary

$1000k - $1390k

Min Experience

5 years

Location

Brentwood, Tennessee, United States, remote

JobType

full-time

About the job

Info This job is sourced from a job board

About the role

The Data Engineer II will be responsible for building, and maintaining scalable data pipelines, managing our azure delta lake architecture, and supporting analytics initiatives across the organization. The ideal candidate will have a strong background in Azure Databricks, data engineering, and develops pipeline. The successful candidate will work closely with our platform architects and data science teams to ensure seamless integration of data pipelines and Lakehouse architecture. This role should ensure coding standard and coding migration to higher environment and CI/CD. This is a Full Time, Remote Data Engineer II role. What You'll Do Design, develop, and maintain data pipelines using Azure Databricks, Apache Spark, and Python. Collaborate with data engineering teams to integrate data from EPIC, Clarity, and Caboodle and other on-premise source systems to Azure. Ensure data quality, integrity, and security throughout the data pipeline. Automate unit test cases as part of Data Pipeline. Contribute to the development and maintenance of our Lakehouse architecture using Azure Databricks and Delta Lake. Ensure data governance and security in the Lakehouse environment Should be able to execute POCs to evaluate new tool, concepts introduced by Azure/ Databricks. Troubleshoot data pipeline and Lakehouse architecture issues, identifying and resolving bottlenecks and errors. Optimize data pipeline and Lakehouse architecture performance, scalability, cost and reliability. Collaborate with data science teams to integrate data pipelines and Lakehouse architecture with data science workflows. Ensure seamless data exchange between data engineering and data science teams. Participate in peer review for coding standards. Subject matter expert for pySpark and Scala. Other duties as assigned. What You'll Bring Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Certifications such as AWS Certified Solutions Architect, Cisco CCNP, Microsoft Azure Solutions Architect, or ITIL preferred. Databricks and Azure certification preferred. 5+ years of experience in Cloud development practice. Strong expertise in cloud platforms (AWS, Azure, GCP), virtualization, and networking. Preferred experience with infrastructure automation tools (Terraform, Ansible, Kubernetes). Proficiency in Linux and Windows server administration. Solid knowledge of cybersecurity best practices and disaster recovery planning. Strong leadership experience in managing technical teams and project management skills. Knowledge of log management and monitoring tools (Splunk, Datadog, Prometheus). Knowledge on Talend, Power BI and Qlik Familiarity with containerization and microservices

About the company

Premise Health serves large organizations and their people with exceptional healthcare. The result? Better experiences, better health, and better value, all while helping organizations lower their healthcare costs. Premise's mission is to help people get, stay, and be well.

Skills

azure databricks
data engineering
python
apache spark
azure
data pipeline
data quality
data security
data governance
troubleshooting
performance optimization
data science
pyspark
scala
linux
windows server
cyber security
project management
team management
talend
power bi
qlik
containerization
microservices