T D Newton & Associates
Website:
tdnewton.com
Job details:
Join us as a "Data Engineer AVP " where you will be doing hands-on data engineering role focused on building production-grade data infrastructure and data products. The overall purpose of the role is to:
• Build and maintain data infrastructure that sits on Snowflake and integrate with broader data stack
• Build and maintain pipelines for data products consumed by clients and for research analysis that is consumed by data scientists
• Manage dependencies between pipelines and write tests to ensure data quality at each steps and maintain
• Implement monitoring, logging, alerting for data infrastructure observability
• Productionize models and data analyses and ensure clean deployment to Prod environment.
Key Accountabilities
The main duties of the job will be composed of the following:
• Understanding the Global Research business and products we offer to our clients
• Develop complete understanding of data infrastructure
• Build/maintain Snowflake data stack and integrate with broader data stack (AWS, S3, Athena , Iceberg, SQL databases,)
Design and maintain analytical data models optimised for Snowflake consumption by BI and data science usersl
• Implement CI/CD practices for data pipelines and Snowflake objects across dev/test/prod environments
• Should be comfortable with big-data infrastructure on AWS (via Athena, PySpark and Python) in terms of ETL
• Partnering with IT teams, vendors and business to define data onboarding requirements
• Communicate directly with the stakeholders on a regular basis to discuss solutions proposed for continuous improvement
• Onboard new data sets and write transformations/tools to make them usable
• Integrate with interactive tools and advanced dashboards for BI (Tableau, Power BI)
• CI/CD practices for data pipelines
Person Specification
• Excellent written and verbal communication skills
• Strong attention to detail
• Ability to work effectively in a team environment
• Strong interest in the subject matter
• Should strive towards continuous learning & up-skilling
• Resourcefulness, and proactive attitude
• Demonstrated ability to work in a time sensitive environment
• Experience working with large-scale data
• Experience using git for source control in a team environment, including reviewing pull requests.
Essential Skills/Basic Qualifications:
•4+ years of strong experience in Snowflake & certification in Snowflake is required.
• Strong experience in writing SQL
• Strong experience in working with Python
• Strong understanding of data pipelining and automation for data extraction and code change management
• Experience with CI/CD for data platforms and environment promotion
• Experience in data visualization and data analytics using Tableau/PowerBI (or similar BI tool) will be a value add
• Knowledge on working with Big Data sets on AWS using Presto/Spark/Athena
• Good communication skills (both verbal and written)
Desirable skills/Preferred Qualifications:
• Experience with orchestration tools (Snowflake Tasks, Dagster, Prefect)
• Experience in data visualization and data analytics infra using Tableau (or similar BI tool) will be a value add
• Past experience with large scale text data or geolocation data
• Prior experience in ML/AI model deployment
This role is based out of Mumbai.
Click on Apply to know more.