Senior Data Engineer
Marketplacer, Australia
Job Description
Marketplacer is a global SaaS company headquartered in Melbourne, Australia. We enable our customers to create all types of marketplaces. We are recognized as one of the world’s leading providers of online marketplace software.
As the senior data engineer at Marketplacer, you will work together with a growing team to deliver the next generation, scalable data and analytics platforms. This is a somewhat Greenfield role, a great opportunity to be part of the Data product build from inception.
Primarily, you’ll be involved in designing and building a reliable and highly scalable modern data platform, including the use of data marts and data warehouses across a range of technologies hosted with Amazon Web Services (AWS).
You will work with databases such as MySQL, Amazon Redshift, DynamoDB, PostgreSQL, and languages like SQL, Python, Ruby and Spark/Scala.
Requirements
Our Development Ecosystem
- Principally Ruby on Rails backend, with some services written in Python or Node
- React/SASS frontend, transitioning to a headless React/Typescript frontend with shared design system
- GraphQL for our newer, client-facing and internal APIs
- AWS for cloud services and storage
- Docker for local dev environment and running CI/CD
- MySQL, Redis, and Memcached for storage
- ElasticSearch for site search functionality
- Datadog for site monitoring
- Terraform and Chef/LittleChef for managing our servers
Here’s what you’ll be doing:
- Design and maintain Marketplacer Data Platform both in terms of architecture and underlying technology
- Align architecture with business requirements
- Work closely with Product team to establish a roadmap for the Data team
- Translate the roadmap into manageable chunks of Engineering effort
- Work closely with other engineering teams to ensure uptake of Data Platform and Data APIs by other teams
- Develop, test, deploy and maintain Data Pipelines and API connectors.
- Working with very large datasets, designing ETL processes that can cater to real-time environments.
- Data modelling: using your skills with traditional RDBMS, designing of data marts, MPP leading to MLOPS, performance tuning, and configuration.
- Data integration: Designing and delivering data integration (ETL) solutions using a variety of different solutions.
Benefits
In your toolkit:
- 5+ years of experience in similar role, ideally in a startup environment
- Experience working with AWS Data management infrastructure and/or other Data and Analytics platforms (e.g. Snowflake, dbt, Bigquery)
- Knowledge of microservice and application program interface (API) technologies
- Exposure to advanced analytics applications, with the ability to program in Python and/or Scala/Spark
What you’ll get:
- A culture of smart, fun and values driven people, where everybody’s voice matters
- You’ll be provided with a top-of-the-line laptop of your choice (MacBook, Dell XPS, etc) and any software that you need to do your job.
- Our monthly hack days allow you to work on something new and different.
- Plenty of autonomy and opportunity to grow, regular check ins with your manager
- Access to our learning platform with over 90,000 courses amp; videos to choose from
- 2 extra days of leave per year to dedicate towards volunteering or helping your chosen charity
- Fortnightly retros, team building and company wide events - both in person and online