About the role
We are seeking a Backend Engineer to design, implement, and optimize data collection, storage, and processing pipelines. The ideal candidate will have expertise in handling structured and unstructured data, optimizing query performance, and ensuring efficient data retrieval. This role involves working with APIs, cloud logging, data streaming, and anomaly detection to maintain robust data workflows.
Key Responsibilities:
Develop and maintain data collection pipelines to efficiently ingest and store structured and unstructured data.
Implement web scraping techniques and APIs for real-time and batch data retrieval.
Optimize data storage and query performance in MySQL, Redis, and other databases.
Ensure efficient logging (cloud & local) and real-time data streaming.
Implement alerts and anomaly detection for monitoring data inconsistencies.
Work on data optimization strategies for faster retrieval and efficient storage.
Design and maintain scalable, fault-tolerant data architectures.
Collaborate with cross-functional teams to ensure data integrity and performance.
Required Skills & Experience:
Proficiency in Web Scraping and API Integration for data collection.
Experience in Cloud Logging, Data Logging, and Data Streaming.
Strong knowledge of MySQL, Redis, and other storage solutions.
Expertise in data optimization, query optimization, and efficient retrieval mechanisms.
Hands-on experience in structured and unstructured data handling.
Ability to design and implement data pipelines.
Experience in implementing scripts, alerts, and anomaly detection mechanisms.
Strong problem-solving skills with a focus on performance and scalability.
About the company
BrightEdge Service
How does BrightEdge work?
As brands have become publishers, the digital world has become the most important distribution channel. The BrightEdge Content Performance Marketing platform helps brands Target Demand, Create and Optimize Content, and Measure Results to win on the content battleground. BrightEdge transforms online content into tangible business results, such as traffic, revenue, and engagement. Our platform is powered by an Artificial Intelligence engine, DataMind, and is the only company capable of web-wide, real-time measurement of digital content engagement across all digital channels, including search, social, and mobile.