Data Engineer II

Job ID: 1234097 | Amazon Dev Center U.S., Inc.


Do you want to shape the future of virtualized (SDN) networking in the world’s biggest public cloud?

The Amazon Elastic Compute Cloud (EC2) VPC Packet Pipeline team owns the packet pipeline that runs right beneath all our customer's EC2 VPC instances, adding features like firewalling (security groups), billing and monitoring as we touch every single packet on every single host across our massive worldwide fleet. Our vision is to combine the performance of bare metal networking while maintaining all the benefits of the cloud, including delivering features not possible on bare metal i.e. true Software Defined Networking (SDN).

We are growing fast, and are looking for an experienced Data Engineer to help build our next generation of big-data and analytics services to help scale AWS VPC. In this role, you will be part of a team that builds scalable frameworks and highly available services that automate insights into EC2 networking events (e.g. network performance degradation, traffic pressure), in order to vend data to be used for remediation strategies. We employ data warehousing technologies, make use of a data pipelines to aggregate views at petabyte scale, and produce automated data visualizations for AWS service teams to consume. In this role, you'll have a chance to develop a thorough understanding of AWS networking components, build and evolve data pipelines/schema for various services to consume, work with machine-learning technologies, and advance real-time data processing of time series data. You work will impact and influence a wide range of teams across EC2 looking to get insights from data in both production and pre-production environments.

Your responsibilities will include:
· Designing, developing, troubleshooting, evaluating, deploying, and documenting data management and business intelligence systems, enabling stakeholders to manage the business and make effective decisions.
· Building secure, available, scalable, stable, and cost-effective data solutions using data storage technologies, distributed file system, data processing, and business intelligence best practices.
· Working with business customers in understanding the business requirements and implementing solutions to support analytical and reporting needs.
· Designing and planning for solutions in the various engineering subject areas as it relates to data storage and movement solutions: data warehousing, enterprise system data architecture, data design (e.g., Logical and Physical Modeling), data persistence technologies, data processing, data management, and data analysis.
· Ensuring completeness and compatibility of the technical infrastructure to support system performance, availability and architecture requirements
· Reviewing and participating in testing of the data design, tool design, data extracts/transforms, networks and hardware selections.

Amazon is an Equal Opportunity-Affirmative Action Employer – Minority / Female / Disability / Veteran / Gender Identity / Sexual Orientation


· Bachelor's Degree in Computer Science or a related technical field, and solid years of relevant experience.
· A strong grasp of SQL and at least one scripting or programming language.
· 5+ years of experience with and detailed knowledge of data warehouse technical architectures, data modeling, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures and hands-on SQL coding.
· 3+ years of large IT project delivery for BI oriented projects.
· 3+ years of working with very large data warehousing environment


· Experience in designing and delivering cross functional custom reporting solutions.
· Experience with Massively Parallel Processing (MPP) databases - Redshift, Teradata etc
· Experience with distributed systems and NoSQL databases
· Experience with Big Data technologies e.g. Hadoop, Hive, Oozie, Presto, Hue, Spark, Scala and more!
· Excellent oral and written communication skills including the ability to communicate effectively with both technical and non-technical stakeholders.
· Proven ability to meet tight deadlines, multi-task, and prioritize workload
· A work ethic based on a strong desire to exceed expectations.
· Strong analytical skills