Software Development Engineer, Open Data Analytics Fundamentals team
DESCRIPTION
Want to change the world with Big Data and Analytics? Come join us on the Amazon EMR team in Amazon Web Services!
Amazon EMR is a web service which enables customers to run massive clusters with open-source data processing frameworks like Apache Spark, Hive, Presto, Hadoop, HBase and more, with the ability to effortlessly scale up and down as needed. We run a large number of customer clusters, enabling a variety of customer use cases including analytics and web-scale machine learning. The AWS DBS EMR organization is innovating within these open-source data processing frameworks with the aim of making EMR the premier platform for running big data workloads. A key aspect of achieving this goal is to enhance security and reduce cluster startup time on the EMR platform.
The AWS EMR Kamino & Observability team is seeking exceptional engineers to help achieve our mission of providing the world’s fastest cloud Big Data processing platform. As an engineer on the Kamino & Observability team, you will be responsible for building and maintaining set of microservices to update and validate images for EMR Releases with latest security updates and critical fixes on all EMR deployment options (EC2, EKS, and Serverless). You will develop features for pre-baking and configuring big data applications on images to achieve faster cluster start up time. You will be adding metrics for cluster monitoring such as CPU/Memory/Disk usage as well as job level metrics for big data applications like Spark and Hadoop. These metrics will help detect workload anomalies during big data jobs execution on EMR Clusters. You will create visualizations of these metrics through dashboards and generate actionable insights for customers. Your work will define an excellent customer experience for the EMR service during EMR cluster launch from faster start up to running jobs on clusters through monitoring.
Successful candidates will exhibit outstanding software development skills, an understanding of distributed systems and parallel data processing, and a strong record of delivery, also excel at leadership and customer obsession and have a real passion for massive-scale computing. By joining our team, you will collaborate with some of the best and brightest minds in the industry and work on cutting-edge software and systems. You'll enjoy a competitive salary, great benefits, a creative and agile work environment, and the exciting opportunity to be part of a fast-paced and growing team and one of the most innovative technology companies. Most of all, you will get the satisfaction of making products that millions use everyday to great effect!
To learn more about EMR:
* AWS EMR: https://aws.amazon.com/emr/
* AWS Big Data Blog: https://aws.amazon.com/blogs/big-data/
Key job responsibilities
* Design and build next-generation infrastructure technologies ensuring AWS is the best environment for running large-scale big data processing workloads.
* Develop and support sophisticated software tools in big data processing frameworks, enhancing customer experience through AWS technology.
* Collaborate with app developers, Data Engineers, SDEs, and TPMs to address complex software challenges.
* Design and develop software solutions/tools to augment the efficiency, quality, and speed of teams engaged in AWS projects.
* Focus on solving infrastructure challenges involving software, big data, and large-scale backend cloud services, aiming to provide a seamless customer experience.
* Work in partnership with a global team of app developers, SDEs, Data Engineers, and TPMs, enhancing existing software tools/infrastructure and developing new solutions.
A day in the life
In your role, you will embody a unique blend of deep technical expertise and a fervent passion for software, data, and infrastructure solutions. You will showcase exemplary organizational and communication skills, navigating the complexities of a dynamic environment with agility—both independently and through the leadership of others. Your proficiency in managing intricate software products will shine as you adeptly translate business cases into technical specifications, crafting reliable and scalable solutions that meet the evolving needs of our business.
You will leverage your deep knowledge in building stable and scalable software and data solutions for our customers. With a rich background in cloud infrastructure and tooling, you will significantly contribute to the vending and packing of extremely complex software artifacts. This might include writing code in Java, Python or other popular programming languages, scripting in Bash to automate tasks, or delving into the latest innovations in big data technologies and software-defined infrastructure. Whether optimizing performance, ensuring data integrity, or exploring new ways to handle vast datasets, your skills will be crucial in advancing the Open Data Analytics team's tech infrastructure.
Through your actions and expertise, you will be able to contribute significantly to the team's success, ensuring that the solutions you develop not only meet immediate needs but also align with Amazon's long-term vision of customer obsession. Your day will be a blend of technical rigor, strategic thinking, and mentorship, making you a pivotal figure in our ongoing pursuit of innovation and excellence.
About the team
About AWS
Diverse Experiences
AWS values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Why AWS?
Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses.
Inclusive Team Culture
Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness.
Mentorship & Career Growth
We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Work/Life Balance
We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
EEO/Accommodations
AWS is committed to a diverse and inclusive workplace to deliver the best results for our customers. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status; we celebrate the diverse ways we work. For individuals with disabilities who would like to request an accommodation, please let us know and we will connect you to our accommodation team. You may also reach them directly by visiting https://www.amazon.jobs/en/disability/us.
BASIC QUALIFICATIONS
- 3+ years of non-internship professional software development experience
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Experience programming with at least one software programming language
PREFERRED QUALIFICATIONS
- 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Bachelor's degree in computer science or equivalent
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $129,300/year in our lowest geographic market up to $223,600/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addition to a full range of medical, financial, and/or other benefits. For more information, please visit https://www.aboutamazon.com/workplace/employee-benefits. This position will remain posted until filled. Applicants should apply via our internal or external career site.