ThreatXIntel
Website:
threatxintel.com
Job details:
Company Description
ThreatXIntel is a growing Cybersecurity, IT Staffing, and Consulting company delivering end-to-end technology and security solutions. Our services include cloud security, web and mobile application security testing, DevSecOps, vulnerability assessments, IT consulting, and professional staffing services.
We support global corporate clients by hiring and deploying skilled professionals across IT and cybersecurity domains while helping organizations strengthen security, optimize operations, and scale efficiently. ThreatXIntel is committed to enabling business growth through secure, reliable, and high-quality technology solutions.
Role Overview
We are looking for a Freelance Data Engineer with strong expertise in building and optimizing scalable ETL/ELT pipelines for batch and real-time data processing.
Key Responsibilities:
- Design, develop, and maintain ETL/ELT data pipelines
- Build batch and real-time ingestion, transformation, and loading workflows
- Work with Spark (PySpark/Scala) and streaming technologies like Kafka/Flink
- Develop and optimize scalable data architectures including data lakes, data warehouses, and streaming platforms
- Improve performance of Spark jobs, SQL queries, and data workflows for speed, efficiency, and cost optimization
- Implement data quality checks, monitoring, and alerting systems
Required Skills:
- Strong proficiency in Python and SQL
- Experience with Apache Spark (Spark SQL, DataFrames, Streaming)
- Hands-on experience with Kafka or Pub/Sub
- Familiarity with GCP and/or Azure data services
- Knowledge of BigQuery, Snowflake, Redshift, and NoSQL databases
- Experience with Airflow, Databricks, Docker, and Kubernetes is a plus
Click on Apply to know more.