Data Engineer

Key Responsibilities

· Design, develop, test, deploy, and maintain scalable batch and real-time data pipelines.

· Integrate structured, semi-structured, and unstructured data from internal and external sources.

· Build reusable data products, services, and frameworks that support analytics, reporting, and machine-learning use cases.

· Develop and optimize ETL and ELT processes using modern cloud data technologies.

· Design and maintain data models that support business intelligence, operational reporting, and advanced analytics.

· Partner with business and technical stakeholders to translate data requirements into reliable, maintainable solutions.

· Collaborate with Data Scientists, Analysts, Software Engineers, and Product teams to ensure data is accessible, accurate, and fit for purpose.

· Implement automated data-quality checks, validation rules, reconciliation processes, and monitoring.

· Diagnose and resolve pipeline failures, performance issues, data inconsistencies, and production incidents.

· Optimize data-processing workloads for performance, scalability, reliability, and cost efficiency.

· Apply data-governance, security, privacy, retention, and access-control standards throughout the data lifecycle.

· Maintain technical documentation, including data mappings, lineage, architecture, operational procedures, and support requirements.

· Build and support continuous integration and continuous delivery processes for data pipelines and infrastructure.

· Participate in architecture reviews and recommend solutions that balance business needs, technical quality, cost, and delivery timelines.

· Contribute to Agile ceremonies, including backlog refinement, sprint planning, daily stand-ups, reviews, and retrospectives.

· Coordinate cross-team integrations, releases, and technical dependencies.

· Support production systems and participate in incident-response activities as needed.

· Research emerging data technologies and recommend practical improvements to platforms, tools, and engineering practices

Required Qualifications

· Three or more years of professional experience in data engineering, software engineering, or a related technical role.

· Strong proficiency in SQL and at least one programming language commonly used for data engineering, such as Python, Java, or Scala.

· Experience designing and maintaining ETL or ELT pipelines in an enterprise environment.

· Experience working with relational databases, data warehouses, data lakes, and/or lakehouse platforms.

· Knowledge of dimensional modeling, data architecture, and data-integration patterns.

· Experience with cloud-based data platforms and services.

· Familiarity with orchestration, version control, automated testing, and continuous integration and delivery tools.

· Understanding of data-quality, governance, security, and privacy principles.

· Experience working in an Agile product-development environment.

· Strong analytical, troubleshooting, communication, and collaboration skills.

· Ability to manage competing priorities and deliver dependable solutions in a fast-paced environment.

Preferred Qualifications

· Experience working with retail, e-commerce, customer, loyalty, merchandising, supply chain, inventory, pricing, payment, or fulfillment data.

· Experience with Microsoft Azure, Google Cloud Platform, or Amazon Web Services.

· Experience with platforms and technologies such as Databricks, Snowflake, BigQuery, Redshift, Spark, Kafka, or similar tools.

· Familiarity with orchestration tools such as Airflow, Azure Data Factory, or comparable technologies.

· Experience building streaming or event-driven data pipelines.

· Knowledge of infrastructure as code, containerization, and DevOps or DataOps practices.

· Experience with data catalogs, metadata management, lineage, master data management, or governance platforms.

· Familiarity with business-intelligence and visualization tools such as Power BI, Tableau, or Looker.

· Experience supporting machine-learning or artificial-intelligence data workflows.

· Knowledge of observability practices for monitoring pipeline health, data freshness, performance, and reliability.