Data Engineer, Network Product Development - Optics
DESCRIPTION
Amazon seeks an experienced Data Engineer to join the NPD (Network product devleopment) interconnects metrics team. Our mission is to provide the data, metrics, alarming, and monitoring needed to improve the health of the network and increase availability to our customers as part of a A Continuous Improvement Model for Interconnects within AWS Data Centers. As a member of The Amazon Network Infrastructure team, you’ll have a unique opportunity to shape the development of one of the world’s largest and most complex optical networks. With Amazon Web Services our goal is to become “The Infrastructure Platform” to the world. Our customers demand the highest quality and reliability for their services.
As a Data Engineer, you will design, implement and support scalable data infrastructure solutions to integrate with multi heterogeneous data sources, aggregate and retrieve data in a fast and safe mode, curate data that can be used in reporting, analysis, machine learning models and ad-hoc data requests. You will be exposed to cutting edge AWS big data technologies. You should have excellent business and communication skills to be able to work with business owners and Tech leaders to gather infrastructure requirements, design data infrastructure, build up data pipelines and datasets to meet business needs. You stay abreast of emerging technologies, investigating and implementing where appropriate.
Key job responsibilities
- Develop and collect metrics to track and monitor the performance of the Amazon network
- Design, implement, and support data warehouse/ data lake infrastructure using AWS bigdata stack, Python, Redshift, QuickSight, Glue/lake formation, EMR/Spark, Athena etc.
- Creation and support of real-time data pipelines built on AWS technologies including EMR, Glue, Redshift/Spectrum and Athena.
- Use business intelligence and visualization software (e.g., QuickSight) to develop dashboards those are used by senior leadership.
- Collaborate with other Engineering teams, Product/Finance Managers/Analysts to implement advanced analytics algorithms that exploit our rich datasets for financial model development, statistical analysis, prediction, etc.
- Explore new datasets and establish mechanisms to provide access to them for the entire organization
- Empower technical and non-technical, internal customers to drive their own analytics and reporting (self-serve reporting) and support ad-hoc reporting when needed.
- Work with network operations team in identifying operational issues to improve overall quality of the fleet.
- Manage numerous requests concurrently and strategically, prioritizing when necessary
- Mentor other engineers, influence positively team culture, and help grow the team
A day in the life
On an everyday basis as part of our team, you have the unique opportunity to understand the growing AWS network and our internal customers’ requirement on interconnect solutions. You'll work backwards to devise hardware solutions by influencing the broad industry and/or to develop software tools with sister teams to maintain a highly available network that delights AWS customers. You design and implement processes and mechanisms that both help the team to deliver business impact to the organization in a systemic way, while also helping to raise the bar on our operational excellence.
Operating at the scale we do, there is no blueprint for how to do what we do, which encourages our engineers to identify and develop simple solutions to complex problems. We encourage durable solutions that look around corners while taking into consideration our customer needs from a cost, performance, and reliability perspective. We work closely with our internal partners that design, build and operate the network to ensure that our solutions meet their needs and exceed their expectations.
About the team
Within AWS Networking the NPD (Network Product Development) organization is responsible for, designing the hardware, building the software, and owning the interconnects for the routers that power the global AWS network. Beyond product delivery we actively manage the fleet or routers in a network that grows by 70% annually. This means tracking key business and operational metrics to ensure that we operate smoothly and minimize or eliminate customer impact due to device related issues for a transparent AWS customer experience.
Working at AWS in the Core Networking Team - Meet Matt, Director, Core Networking -- Link
Meet Kensie, Network Development Engineer -- Link
About AWS
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Why AWS?
Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Inclusive Team Culture
Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness.
Mentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
BASIC QUALIFICATIONS
- Bachelor’s degree in Computer Science, Electrical/Electronic Engineering, Physics, Mechanical/Computer Engineering, or equivalent
- 4+ years of hands-on experience with onboarding to and collecting data
- 3+ years of experience with developing metrics from data
- 2+ years of experience creating software to ingest, and load data through pipelines
- 2+ years of experience validating and cleaning data
- Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
PREFERRED QUALIFICATIONS
- Masters or PhD preferred
- Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
- Experience with distributed systems as it pertains to data storage and computing
- Experience in databases and SQL.
- Experience in debugging and troubleshooting data ingestion related issues.
- Experience in the software development lifecycle.
- Experience in innovating or inventing metrics to solve business problems.
- Meets/exceeds Amazon’s leadership principles requirements for this roles.
- Meets/exceeds Amazon’s functional/technical depth and complexity for this role.
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Los Angeles County applicants: Job duties for this position include: work safely and cooperatively with other employees, supervisors, and staff; adhere to standards of excellence despite stressful conditions; communicate effectively and respectfully with employees, supervisors, and staff to ensure exceptional customer service; and follow all federal, state, and local laws and Company policies. Criminal history may have a direct, adverse, and negative relationship with some of the material job duties of this position. These include the duties and responsibilities listed above, as well as the abilities to adhere to company policies, exercise sound judgment, effectively manage stress and work safely and respectfully with others, exhibit trustworthiness and professionalism, and safeguard business operations and the Company’s reputation. Pursuant to the Los Angeles County Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $118,900/year in our lowest geographic market up to $205,600/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.