Sr Software Dev Engineer, Amazon Search Relevance Measurement
DESCRIPTION
We are looking for a passionate, hard-working, and talented Senior Software Development Engineer who can build innovative & mission critical software applications & tools to measure Search Quality. You will have an enormous opportunity to make a large impact on the design, architecture, and development of Amazon Search.
Amazon Search Science builds the core machine learning algorithms used by 100’s of millions of customers worldwide to discover products to buy amongst billions of available items at Amazon.com
Relevance Measurement team provides search and browse attributed user behavioural datasets that are used by search algorithms. This team has singular mission to improve relevance and search quality for Amazon customers. We will analyse peta bytes of data to enable Amazon Search to measure customer success metrics and identify ways to improve search. We run the Search Defects process to provide training data for defect classifiers, to help matching and ranking experimenters build and evaluate experiments for defect improvements, and to measure customers’ overall defect experience with Amazon Search.
As a Senior Software Development Engineer, you will create and maintain large-scale systems that will help us better understand customer behaviour, highlight pain points, and help identify opportunities to improve search for Amazon customers. The main focus will be to design, develop, and implement an array of full stack applications, data pipelines, analytics tools and statistical tests that will provide valuable insights into customer feedback and behaviour. The systems you develop will be instrumental in helping the Amazon Search team find and resolve issues and shape our product strategy.
You will be given significant authority and ownership, but also be supported with strong mentors. You will have the opportunity to work on software development using cutting edge big data technologies, while taking products and features from conception to production. You will work closely with a stellar team of engineers, applied scientists, business intelligence engineers, and product managers to ensure Amazon’s data is a powerful and innovative business advantage.
Key job responsibilities
Key job responsibilities
* LLD, HLD, Design and Architecture Review for various search quality measurement applications
* Dashboard, Metrics, Alarms, Operational Excellence
* Statistical tests for sampling and scoring for detecting defects and anomaly
* Building Data Labelling Platform and Caching Strategy using intelligent algorithms to improve quality and cost of labels
* Spark enabled Data Pipelines, OLAP Cube for Search Business Reviews
* UI Applications to enable Self Serve Data Platform to 100+ customers in search
* Innovate and find new search metrics that helps improve the billion order search industry
* Debug and deep dive search stack to find instances where search performance is degrading
A day in the life
A day in the life
* Enable and analyse A/B experiments by publishing results from analyzing petabytes of search logs along with statistical tests and confidence intervals at various slices locale, product type, category, language, zipcode
* Conceptualise, build and release new metrics dashboards as insights to search relevance teams
* Build Airflow and Spark DAGs to build intelligent pipelines to monitor search and build signals for model building near realtime
* Deep Dive search stack based on customer actions / anecdotes when degradation happens
* Build a large scale crawler to replay queries at scale to conclude A/B tests
* Collect human labelled crowdsourced data for automated model training
About the team
Relevance Measurement Team measures search business for 20 primary and 11 secondary locales by capturing worldwide Amazon site performance through suite of NDCG metrics displayed in a centralised dashboard product. This novel product now supports 400+ A/B experiment evaluations, Daily Longitudinal metric calculation every 6 hours, Category metrics worldwide for Verticals like Amazon Retail, Amazon Fresh, Amazon Pharmacy, Books, Kindle and Amazon Business teams. We have enabled both crowdsourcing and machine learnt models to measure search at scale from customer viewpoint performing both implicit and explicit feedback analysis and automated attributions to improve search.
BASIC QUALIFICATIONS
- 5+ years of non-internship professional software development experience
- 5+ years of programming with at least one software programming language experience
- 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience
- Experience as a mentor, tech lead or leading an engineering team
- experience in distributed and bigdata / mapreduce systems and event driven architecture
- good fundamentals on probability and statistics
PREFERRED QUALIFICATIONS
- 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
- Master's degree in computer science or equivalent
- Knowledge of professional software engineering & best practices for full software development life cycle, including coding standards, software architectures, code reviews, source control management, continuous deployments, testing, and operational excellence
- Hire and Develop the Best, able to mentor SDE-2s and SDE-1s
- Insist on Highest Standards for Code Review and Design Review Engg Process
- Operational Excellence champion
- Prior experience debugging end to end Search stack