Model Monitoring - BI Analyst

Austin, Texas Requisition Number R0036598 Subsidiary eBay

At eBay, you will be part of a purpose-driven community dedicated to creating an ambitious and versatile work environment. In eBay Payments, you will be an integral member of a growing organization that inspires passion, courage, and inventiveness - creating the future of global commerce and making a meaningful, positive impact on millions of eBay sellers and shoppers around the world. If you are looking for a special place to take your Payments career to the next level, we want to talk with you.

Risk Management is at the core of Payments done well – and we are hiring curious, driven, and courageous experts to transform our business unit to enable eBay's next generation Payments strategy. Our focus is to ensure the integrity of our marketplace for buyers and sellers who transact with us every single day. The scope of our charter includes Risk Management Strategy, Policy, Decision Sciences, and Policy Operations.

We are looking for a highly talented and self-motivated BI analyst to join our Decision Science team. Decision Science contains both data scientists and software engineers responsible for creating and implementing state of the art machine learning algorithms for fraud detection and risk assessment in support of Risk Management. The primary responsibility of this role is to the tools and reports necessary to manage and monitor machine learning models in production. This covers performance monitoring, business impact, quality control, and real time alerting when issues arise. You will compliment the data scientist building the visualization and alerting tools necessary to monitor and maintain model performance.

Primary Job Responsibilities

The Model Monitoring Analyst will support the Decision Science team, leveraging big data technologies (Hadoop, spark, etc) to aggregate and structure data, build necessary data warehouse backend, and create solutions to monitor suites of machine learning models that are operating today. You will be responsible for a ground up build of systems to monitor both general model performance as well as real time monitoring and alerting. The performance is a combination of statistics compared to expected outcomes as well as a variety of business metrics and outcomes of humans in the loop. This solution may come in the form of a variety of BI platforms at your disposal or custom systems as the need arises. You will be the technical owner of such a solution and responsible for building, daily operations, as well as the source and owner of performance reports, readouts, presentations. Problem sets are focused around fraud and risk management to include models to prevent fraudsters from listing and monetizing on the platform, thwarting registration attacks, risk scoring our customers, and fraud detection in the payments space.

Required Skills and Experience:

  • Bachelor’s degree in a quantitative field: engineering, math, statistics etc.
  • 6+ yrs experience, must be fully independent with limited guidance.
  • Expert in SQL, relational databases
  • Big Data technology: Hadoop framework: Hive, Spark, etc.
  • Strong knowledge of 1 or more scripting and programming languages (Python, Java, Scala, SAS etc.)
  • Expertise in BI, monitoring, and general reporting tools (Tableau, MicroStrategy, Grafana, Kibana, etc)
  • Strong analytical skills with good problem solving ability
  • Good presentation and communication skills required

Here at eBay, we love creating opportunities for others by connecting people from widely diverse backgrounds, perspectives, and geographies. So, being diverse and inclusive isn’t just something we strive for, it is who we are, and part of what we do each and every single day. We want to ensure that as an employee, you feel eBay is a place where, no matter who you are, you feel safe, included, and that you have the opportunity to bring your unique self to work.. To learn about eBay’s Diversity & Inclusion click here:

This website uses cookies to enhance your experience. By continuing to browse the site, you agree to our use of cookies

View our privacy policy

View our accessibility info

eBay Inc. is an equal opportunity employer.  All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status.  If you are unable to submit an application because of incompatible assistive technology or a disability, please contact us at  We will make every effort to respond to your request for disability assistance as soon as possible.

For more information see:

EEO is the Law Poster

EEO is the Law Poster Supplement

Your Saved Jobs

You have not saved any jobs.

Recently Viewed Jobs

You have not viewed any jobs.


Receive new career opportunities as soon as they become available!

Areas of InterestSearch for a category, location, or category/location pair, select a term from the suggestions, and click "Add".

  • Data Science and Analytics, Austin, Texas, United StatesRemove
  • Risk Management, Austin, Texas, United StatesRemove