Lead Cloud Data Engineer
San Francisco, California
Stubhub is looking to hire experienced Lead (Big Data) Cloud Engineer to play a key role in designing and developing batch and real time data acquisition pipelines and analytics platform using latest open source big data technologies.
In this role candidate will report into Data Engineering Director in our west coast (San Francisco) office. Data as a Service (DaaS) is an enterprise platform team that organizes Stubhub’s data and makes it available for applications and business units.
It collects, correlates, enriches, and manages Stubhub’s core data to produce single view of truth for use at speed and at scale. We are passionate technologists who thrive on simple and elegant architecture and agility.
We own the architecture and development of comprehensive data management, bringing real-time operational use, near real-time visibility use, and off-line deep analytical use together in a cohesive and holistic way. Innovation and engineering prowess is our core to stay ahead of our user’s demand and the industry changing landscape. Come get challenged in a fast, agile environment with other A players.
• Design and develop scalable data pipelines for optimal ingestion, transformation, storage and computation using latest big data technologies.
• Work with engineering, product, and business team(s) to understand requirements, evaluate new features and architecture to help drive decisions.
• Design, develop and maintain automated ETL processes for efficient batch records-matching of multiple large-scale datasets.
• Translate complex business requirements into a scalable, efficient and high availability data platform. Strong ownership of the full backend stack - from design through to deploy and beyond.
• Experience working with continuous integration (CI/CD) framework, building regression-able code within data world using GitHub, Jenkins, Cloud Build, Spinnaker and related applications.
• Actively participate in code review and test solutions to ensure it meets best practice specifications. Prototype ideas quickly using cutting edge and new generation technologies.
• Build collaborative partnerships with architects, technical leads and key individuals within other cross functional groups. Work with business teams/engineers to define instrumentation and data requirements.
• Lead projects independently and also mentor team members and other engineers in the team.
• Bachelor’s or Master’s degree or equivalent in computer science or related field with minimum of 6+ years of directly related work experience.
• Good understanding of data structures and algorithms.
• Programming experience in Python, Java or Scala.
• Extensive experience with Elastic stack, Kafka and Spark
• Experience with big data technologies like Hadoop, Map Reduce, Hive, Spark/Storm, Big Query
• Experience with time series and NoSQL databases is a plus.
• Experience working with containers and kubernetes is good to have.
• Experience designing, implementing and scaling big data solutions on public cloud.
• Experience with Unix/ Linux and shell scripting
• Experience working with various data sources like unstructured data files, flat files, message queues, JSON based events, databases.
• Experience driving architectural discussions and proposing solutions for system and product changes.
• Ability to produce clear, well-documented designs for product features.
• Experience writing high-performance, reliable and maintainable code.
• Proven track record of strong analytical and problem-solving skills.
• Ability to adapt to new development environments, changing business requirements.
• Experience with large distributed services is a plus as is building/operating highly available systems • Proven ability to effectively work across multiple teams on solutions that have complex dependencies and requirements in a fast-paced environment
• Able to effectively communicate across teams and roles
• Excellent verbal and written communication skills Reasoning Ability
• Proven ability as a problem-solver Other Skills
• Strong estimating and planning skills
• Ability and desire to thrive in a proactive, high-pressure, client-services environment
StubHub is the world's largest ticket marketplace, enabling fans to buy and sell tickets to tens of thousands of sports, concert, theater and other live entertainment events. StubHub reinvented the ticket resale market in 2000 and continues to lead it through innovation. The company's unique online marketplace, dedicated solely to tickets, provides all fans the choice to buy or sell their tickets in a safe, convenient and highly reliable environment. All transactions are processed and delivered by StubHub and backed by the company's FanProtect Guarantee™ processing a ticket every second today with billions of ticket sales each year globally. Company partners MLB, AEG, ESPN and many leading teams such as the San Francisco Giants and University of Texas, along with over 60 other teams in MLB, NBA, NHL and NCAA.
- StubHub One – We act as one team in pursuit of connecting people through inspiring event experiences.
- At StubHub, we are:
- Driven – We strive for excellence – constantly improving our game
- Courageous – We are bold & not afraid to challenge the status quo
- Inventive – We see what others don’t & pioneer new paths
- Richly Diverse – We recognize & respect everyone as a unique individual
- Brand – We experience the world through StubHub.
View our accessibility info
eBay Inc. is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, national origin, sex, sexual orientation, gender identity, veteran status, and disability, or other legally protected status. If you are unable to submit an application because of incompatible assistive technology or a disability, please contact us at firstname.lastname@example.org. We will make every effort to respond to your request for disability assistance as soon as possible.
For more information see:
At eBay Buyer Experience team, the drive for a faster and sleeker pages constantly inspires me. The fact that my work touches millions of users everyday in one way or the other and to learn what affects a person’s decision to make that purchase brings a sense of fulfillment.–Dileep, Staff Software Engineer
Everyday, I get a chance to work with smart and dedicated people to build the best shopping experience for our customers. Because so many people use eBay, I am proud that my work is impacting people in a positive way.–Steve, Full Stack Software Engineer
Everyday I work with people passionate to deliver great experiences for millions of buyers and sellers. It brings me great joy knowing that the features I work on greatly impact the lives of people from around the world, from all walks of life.–Andrew, Software Engineer
We do things different, the hack way, the happy way. And we celebrate the best way. We are the pirates of eBay!–Shakul, Software Development Manager
eBay gives me the freedom to challenge the status quo. This creates a unique adrenaline rush that pushes all boundaries and paradigms and helps bring out new ideas that ultimately redefine e-commerce shopping experience. If you love such challenges, you'll love work at eBay.–Sriharsha, Software Engineer
What makes a successful Software Developer at eBay? Check out the traits we’re looking for and see if you have the right mix.
- Team player
Medical, Dental,Vision, Healthcare FSA, Dependent Care FSA, Health and Wellness Programs
401(k) / Retirement Plans
Employer Matching, Employee Stock Purchase Program
Tuition Reimbursement, Professional Development
Paid Time Off
16+ Days of Paid Time Off, Paid Maternity leave, Paid Paternity Leave, 11 Paid Holidays
Onsite Gym/Fitness Center
Sabbatical, 4 paid weeks every 5 years