Data Engineer, Forecasting Core Data Engineering
DESCRIPTION
Amazon Forecasting team is looking for a Data Engineer to join the team. We are pioneers in the fields of Amazon demand analysis and forecasting. Forecasting data engineers build scalable data transformation pipelines to ingest data from different data sources, and provides big data sources for machine learning models to predict the Amazon demand world wide. If you want to learn how to leverage technology to predict the future, this is the team for you.
As a member of the Demand Forecasting team, you'll play a key role in solving one of the world's most complex technical challenges in data engineering. You will utilize large-scale compute platform to build big datasets used in distributed systems for machine learning and statistical analysis - just to name a few.
Our Data Engineer needs to be able to gather and understand data requirements, build and maintain big data sources to prepare data for machine learning models, data scientists, business intelligent engineers, and work with software engineers to achieve high quality data ingestion and transformation solutions. You need a passion for solving complex problems, and enjoy the challenge of operating complex systems under extreme loads. If you think you’re up to the challenge and you’d like to learn more, we’d love to talk to you.
In this role, you will be a technical expert with significant scope and impact. You will work closely with a group of software development engineers, product managers, data scientists, and business intelligence engineers to create the data infrastructure and data processing pipelines to drive our team’s initiatives.
Successful candidates should come from a strong data engineering background. You need to have experience with structured data, and being able to analyze/transform the data using various tools. Although SQL is a strong requirement, being flexible enough to work with scripting language is a must. Often, the pace of innovation and change implies a need to move to new data sources, and our data engineers get to participate in deep diving business data in order to understand/measure sources of disparity. Your analytical skills and knowledge of schema, metadata and data structure in big data world will be essential.
As a data engineer, you will need to design and develop high scalable ETLs with EMR based applications as well supporting them on Glue ETL or Redshift. Knowledge of big data architecture and design is a must.
Key job responsibilities
- Create scalable data solutions using AWS and/or Amazon internal tools.
- Create and maintain business logic for data transformation and ingestion pipelines.
- Create and maintain datasets in Amazon data lake and internal data management systems using S3/Glue.
- Data investigation and analysis to understand the impact of changes on downstream customers' use cases and build transformation logic to ingest data for Forecasting data analysis.
- Provide quality data for Amazon downs team data consumers.
A day in the life
We not only provide data to machine learning models but also to our direct customers in BI and Data Science. We help them to get access to the data which they need for data analysis, working with them to address data quality issues and resolve data requirements for experiments, training and back testing models as well as data required for generating the forecast on regular basis.
BASIC QUALIFICATIONS
- 3+ years of data engineering experience
- Bachelor's degree
- Knowledge of distributed systems as it pertains to data storage and computing
- Experience with SQL
PREFERRED QUALIFICATIONS
- Master's degree
- Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, disability, age, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.