CloudAI Technologies
Website:
cloudaillc.com
Job details:
Job Title: Senior Cloud Data Engineer
Company: CloudAI Technologies
Location: Hyderabad, India (Hybrid – a few days in office)
About CloudAI Technologies
CloudAI Technologies is a leader in cutting-edge AI, cloud, and data solutions, trusted by global enterprises to deliver scalable, high-performance data platforms. Our Hyderabad subsidiary is at the forefront of data-driven innovation, developing modern data architectures that power AI and analytics applications. We are seeking a highly experienced Senior Cloud Data Engineer with expertise in cloud-based data engineering, ELT pipelines, data governance, and multi-cloud environments to join our team.
Role Overview
As a Senior Cloud Data Engineer, you will be responsible for designing, building, and optimizing cloud-native data platforms that support enterprise-scale AI, analytics, and data-driven applications. You will work with AWS (required) and other cloud platforms, ensuring secure, efficient, and scalable data processing. This role requires strong experience in data engineering best practices, including ELT, data transformation, data cataloging, and governance.
You will collaborate closely with Data Scientists, AI/ML Engineers, and Cloud Architects to develop real-time, batch, and event-driven data pipelines that drive AI and business intelligence initiatives.
Key Responsibilities
- Cloud Data Engineering: Design and implement data architectures on AWS (required) and multi-cloud platforms (Azure, GCP preferred).
- ELT & Data Pipelines: Build and maintain scalable ELT pipelines using cloud-native tools such as AWS Glue, Lambda, Apache Airflow, and DBT.
- Data Ingestion & Processing: Develop batch and real-time data ingestion from structured and unstructured sources using Apache Spark, Kafka, and Kinesis.
- Data Governance & Security: Implement data governance frameworks, RBAC policies, data lineage tracking, and compliance with security standards.
- Data Cataloging & Discovery: Leverage tools like AWS Glue Data Catalog, Apache Atlas, and DataHub for metadata management and data discovery.
- Data Cleansing & Transformation: Ensure data quality, standardization, deduplication, and anomaly detection using modern data transformation frameworks.
- Multi-Cloud Data Solutions: Design cross-cloud data integration strategies, leveraging AWS, Azure, and GCP services.
- Performance Optimization: Optimize query performance, data partitioning, and indexing strategies for efficient analytics workloads.
- Collaboration & Leadership: Work with Data Scientists, Engineers, and Analysts to develop scalable AI and analytics-driven solutions.
- Infrastructure as Code (IaC): Automate data infrastructure provisioning using Terraform, AWS CloudFormation, or Pulumi.
Qualifications
- Professional Experience:
- 5+ years of experience in cloud-based data engineering, with a focus on AWS (required).
- Experience working in multi-cloud environments (AWS, Azure, GCP preferred).
- Proven expertise in ELT pipeline development, data modeling, and data warehousing.
- Technical Skills:
- Proficiency in Python, SQL, and Spark for data processing.
- Experience with cloud-native data services (AWS Glue, Redshift, Athena, DynamoDB, S3, EMR, Kinesis).
- Strong knowledge of data governance, cataloging, and lineage tracking using tools like AWS Glue Data Catalog, Apache Atlas, and Collibra.
- Experience with modern data platforms such as Databricks, Snowflake, or BigQuery.
- Hands-on experience with Infrastructure as Code (IaC) using Terraform, AWS CloudFormation, or Pulumi.
- Data Governance & Security:
- Knowledge of data privacy regulations (GDPR, CCPA, HIPAA) and enterprise data security best practices.
- Implementation of RBAC, encryption, and auditing mechanisms for secure data access.
- Cloud & DevOps:
- Experience with CI/CD pipelines for data infrastructure, using GitHub Actions, Jenkins, or AWS CodePipeline.
- Strong understanding of containerization and orchestration (Docker, Kubernetes, ECS, EKS) for data workloads.
- Education:
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related field (or equivalent practical experience).
Click on Apply to know more.