Senior Data Engineer BizTech

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way.

The Community You Will Join:

At Airbnb, we need to ensure every area of the business has trustworthy data to fuel insight and innovation. Understanding the business need, securing the right data sources, designing usable data models, and building robust & dependable data pipelines are essential skills to meet this goal. 

We are currently hiring for the following teams:

Apps and Compliance: This team is responsible for building scalable, high quality data sets andsolutions to enable Airbnb to comply with Tax, payments and legal regulations to ensurebusiness continuity. In addition, this team is responsible for building data sets for Airbnb’sinternal applications (e.g. CRM data, projects data, workspace data) to fuel growth and driveoperational efficiencies. 

The Difference You Will Make:

Apps and Compliance: Our team charter is on enabling Airbnb to comply with Tax, Payments,and Legal regulations so that our Hosts can continue to operate in regulated geos. Ourproducts ingest, process, validate, and deliver large datasets to government authorities (oftenpartnered with tax remittance) so our data must be of the highest accuracy and quality. As youbuild and maintain key components of the critical Compliance data ecosystem, you’ll have theopportunity to contribute to creating standards and best practices for Airbnb’s DataEngineering. Your work on solving complex business challenges at scale will be instrumental inshaping the tools, processes, and standards used by the broader data community.

A Typical Day: 

• Design, build, and maintain robust and efficient data pipelines that collect, process, and storedata from various sources, including user interactions, listing details, and external data feeds.
• Develop data models that enable the efficient analysis and manipulation of data formerchandising optimization. Ensure data quality, consistency, and accuracy.
• Build scalable data pipelines (SparkSQL & Scala) leveraging Airflow scheduler/executorframework
• Collaborate with cross-functional teams, including Data Scientists, Product Managers, andSoftware Engineers, to define data requirements, and deliver data solutions that drivemerchandising and sales improvements.
• Contribute to the broader Data Engineering community at Airbnb to influence tooling andstandards to improve culture and productivity
• Improve code and data quality by leveraging and contributing to internal tools to automaticallydetect and mitigate issues

Your Expertise:

• 5-9+ years of relevant industry experience with a BS/Masters, or 2+ years with a PhD
• Extensive experience designing, building, and operating robust distributed data platforms (e.g., Spark, Kafka, Flink, HBase) and handling data at the petabyte scale.
• Strong knowledge of Java, Scala, or Python, and expertise with data processing technologies and query authoring (SQL).
• Demonstrated ability to analyze large data sets to identify gaps and inconsistencies, provide data insights, and advance effective product solutions
• Expertise with ETL schedulers such as Apache Airflow, Luigi, Oozie, AWS Glue or similar frameworks
• Solid understanding of data warehousing concepts and hands-on experience with relational databases (e.g., PostgreSQL, MySQL) and columnar databases (e.g., Redshift, BigQuery, HBase, ClickHouse)
• Excellent written and verbal communication skills