PineQ Lab Technology
Website:
pineqlab.com
Job details:
Role Descriptions: Bigdata Lead (Overall 10+ years & Relevant 8+ years mandate)
JD - Need strong candidates in Big data & Pyspark
Key Responsibilities
- Big Data Engineering & Development
- Design and develop distributed data processing pipelines using Spark, Hadoop, Kafka, and related technologies.
- Build real-time and batch data processing systems with high reliability and performance.
- Develop efficient ETL/ELT workflows for diverse structured and unstructured data sources.
- Optimize data pipelines for scalability, performance, and cost efficiency.
- Architecture & System Design
- Contribute to the architecture of data lakes, data warehouses, and lakehouse solutions.
- Implement data models, schema design, and best practices for large-scale data management.
- Ensure system reliability using fault-tolerant and distributed computing principles.
- Cloud & Platform Engineering
- Build and manage solutions on AWS / Azure / GCP (e.g., EMR, Databricks, HDInsight, BigQuery, Snowflake).
- Use orchestration tools such as Airflow, Azure Data Factory, AWS Glue, etc.
- Implement CI/CD workflows for automated deployment of data jobs.
- Data Quality, Security & Governance
- Integrate data validation, monitoring, and alerting into pipelines.
- Ensure adherence to data governance, privacy, and security frameworks.
- Collaborate with data stewards, architects, and business stakeholders.
- Collaboration & Leadership
- Mentor junior developers and guide best practices.
- Work closely with data scientists, analysts, and product teams to enable advanced analytics and ML workloads.
- Participate in code reviews, technical discussions, and roadmap planning.
Location - ~BANGALORE~CHENNAI~
Skills: Digital : BigData and Hadoop Ecosystems~Digital : PySpark
Experience Required: 10+ years (Relevant 8+ years mandate), Project Code :
Click on Apply to know more.