Job Description
As a data engineer, you will design and maintain data platform road maps and
data structures that support business and technology objectives. Naturally
inquisitive and open to the deep exploration of underlying data, finding
actionable insights, and working with functional competencies to drive
identified actions. You also enjoy working both freely and as part of a team
and have the confidence to influence and communicate with stakeholders at all
levels, and to work in a fast-paced complex environment with conflicting
priorities.
About the role:
Reporting into the delivery leader, you will deliver consumable, contemporary,
and immediate data content to support and drive business decisions. The key
focus of the role is to deliver a custom solution to support various business
critical requirements. You will be involved in all aspects of data engineering
from delivery planning, estimating and analysis, all the way through to data
architecture and pipeline design, delivery, and production implementation.
From the beginning you will be involved in the design and implementation of
complex data solutions ranging from batch to streaming and event-driven
architectures, across cloud, on-premise, and hybrid client technology
landscapes.
Brief Description of Role :
We are looking for 3+ years of experience in data engineering in a customer or
business facing capacity and experience in the following:
Ability to understand and articulate requirements to technical and non-
technical audiences
Stakeholder management and communication skills, including prioritizing,
problem solving and interpersonal relationship building
Strong experience in SDLC delivery, including waterfall, hybrid and Agile
methodologies
Experience of implementing and delivering data solutions and pipelines on
AWS Cloud Platform.
A strong understanding of data modelling, data structures, databases, and
ETL processes
An in-depth understanding of large-scale data sets, including both
structured and unstructured data
Knowledge and experience of delivering CI/CD and DevOps capabilities in a
data environment
Develop new inbound data source ingestions required within the multi-
tiered data platform to support analytics and marketing automation
solutions
Supports data pipelines – Builds the required dimensions, rules, segments,
and aggregates
Support all database operations: performance monitoring, pipeline
ingestion, maintenance, etc.
Monitor platform health - data loads, extracts, failures, performance
tuning
Create/modify data structures/pipelines
Leveraging capabilities of Databricks Lakehouse functionality as needed to
build Common/Conformed layers within the data lake
Develop, document, and test software and environment setup to ensure that
the outcome meets the needs of end-users and achieves business goals
Qualifications
The following skills are required:
Tech Stack: AWS pipeline, Glue, Databricks, Python, SQL, Spark, etc.
Building the Data Lake using AWS technologies like S3, EKS, ECS, AWS Glue,
AWS KMS, EMR
Extensive experience in ETL and audience segmentation
Developing sustainable, scalable, and adaptable data pipelines
Attention to detail in design, documentation, and test coverage of
delivered tasks
Strong written and verbal communication skills, team player
In addition, the candidate should have strong business acumen,
interpersonal skills, and communication skills, yet also be able to work
independently.
At least 3 years of experience with designing and developing Data
Pipelines for Data Ingestion or Transformation using AWS technologies
At least 2 years of experience in the following Big Data frameworks: File
Format (Parquet, etc.), Resource Management, Distributed Processing
At least 3 years of experience developing applications with Monitoring,
Build Tools, Version Control, Unit Test, TDD, Change Management to support
DevOps
At least 1-2 years of experience with Spark programming (PySpark)
At least 2 years of experience with Databricks implementations
Familiarity with the concepts of “delta lake” and “lakehouse” technologies
The following skills are nice to have, and expertise is not required:
Adobe (Campaign, Audience Manager, Analytics)
MLFlow
Microsoft Power BI
SAP Business Objects
Additional Information
When you're one of us, you get to run with the best. For decades, we've
been helping marketers from the world's top brands personalize experiences for
millions of people with our cutting-edge technology, solutions and services.
Epsilon's best-in-class identity gives brands a clear, privacy-safe view of
their customers, which they can use across our suite of digital media,
messaging and loyalty solutions. We process 400+ billion consumer actions each
day and hold many patents of proprietary technology, including real-time
modeling languages and consumer privacy advancements. Thanks to the work of
every employee, Epsilon has been consistently recognized as industry-leading
by Forrester, Adweek and the MRC. Positioned at the core of Publicis Groupe,
Epsilon is a global company with more than 8,000 employees around the world.
Check out a few of these resources to learn more about what makes Epsilon so
EPIC:
Culture: https: // www. epsilon.com/us/about-us/our-culture-epsilon
DE&I: https: // www. epsilon.com/us/about-us/diversity-equity-inclusion
CSR: https: // www. epsilon.com/us/about-us/corporate-social-responsibility
Life at Epsilon: https: // www. epsilon.com/us/about-us/epic-blog
Great People Deserve Great Benefits
We know that we have some of the brightest and most talented associates in the
world, and we believe in rewarding them accordingly. If you work here, expect
competitive pay, comprehensive health coverage, and endless opportunities to
advance your career.
#LI-SJ1
REF186919L
