Senior Software Engineer (Python) - Supply & Demand

Help shape the next generation of Kpler’s Supply & Demand platform Kpler’s Supply & Demand platform transforms complex commodity data - including production, imports, exports, refinery runs, consumption and stock changes - into trusted market intelligence that traders and our Research team rely on to understand commodity markets. The team is now taking on a major engineering challenge: generalising a model originally built for crude oil into one platform that can serve refined products, LNG, grains and, ultimately, every commodity, without forking the codebase per vertical. As a Senior Software Engineer, you’ll provide senior technical leadership for this transformation. You’ll own complex and ambiguous architecture and data-quality decisions, set the non-functional bar across a polyglot engineering estate, and help make the balances we publish something external clients can trust. This is a role for an engineer who enjoys looking beyond individual features - someone who wants to influence architecture, make difficult technical trade-offs, solve complex data problems and build systems that stand the test of time.

Your future position: • Lead the technical direction of generalising the S&D data model from crude oil to all commodities, owning the trade-offs between debt, design, reliability and time-to-market as the crew's call.

• Evolve the shared ETL Framework (Reader → Transformer → Writer) in `supply-demand-core` into the reusable foundation every commodity vertical builds on, so LNG, grains and new products stop diverging into bespoke pipelines.

• Own publication reliability end to end: the anomaly-detection guardrail (`supply-demand-ml`), the release path, and the data-quality bar that decides whether a balance is fit to ship to customers.

• Set and raise the non-functional bar (reliability, latency, observability, security, cost) across the crew's Python engine, TypeScript data-distribution APIs and Airflow orchestration.

• Shape the external + webapp data-distribution API contract (`supply-demand-oil-api`, `supply-demand-grains-api`) that external clients depend on: versioning, pagination, backward compatibility.

• Lead design reviews within the crew and steward the crew's ADRs; bring engineering alignment to ambiguous, business-critical work.

• Own the response to critical incidents in the crew's systems, and drive the preventative work that follows so the same class of failure does not recur.

• Partner directly with Product and the Research team to shape the crew's technical and feature roadmap, including the SuDeP pipeline that Research consumes.

• Mentor mid-level (P3) engineers toward senior scope, and act as a cross-crew technical consultant within the Commodities Tribe and the wider Engineering chapter.

Experience & Background Essential:

• 5+ years of experience as a software engineer, with experience delivering production systems.

• Experience delivering technically ambiguous, business-critical projects end to end, owning the design, the trade-offs and the production outcome, not just the implementation.

• Deep experience building and running data-intensive backends in Python (ETL/orchestration, relational data modelling, SQL/Postgres).

• Owning systems in production: on-call, observability, incident response and the prevention work afterwards ("you build it, you run it").

• Experience setting technical direction for a team and being  the person other engineers consult on hard design decisions.

Desirable:

• Worked with Airflow (or comparable orchestration) and TypeScript/NestJS APIs, or can credibly ramp into a polyglot Python + TS estate.

• Exposure to a real, evolving domain data model: commodities, energy, finance, logistics or another domain where correctness of the model is the product.

• Familiarity with data-quality / anomaly-detection approaches, or the discipline of gating publication on quality.

• Experience with IaC and GitOps delivery (Terraform, ArgoCD, k8s) and with serving versioned external client APIs.

Behavioural Competencies: 

• You take ownership of problems and outcomes, not just individual tasks.

• You think beyond the code and understand how engineering decisions impact the product and customers.

• You are comfortable making technical decisions, challenging ideas, and helping shape the direction of the team.

• You enjoy working on complex problems, learning new things, and adapting in a fast-moving environment.

• You collaborate openly with engineers, Product, and Research teams to find the best solutions.

• You share knowledge and help the people around you grow.

We make things happen We act decisively and with purpose, going the extra mile.   We build
together We foster relationships and develop creative solutions to address market challenges.   We are here to help We are accessible and supportive to colleagues and clients with a friendly approach.   Our People Pledge   Don’t meet every single requirement? Research shows that women and people of color are less likely than others to apply if they feel like they don’t match 100% of the job requirements. Don’t let the confidence gap stand in your way, we’d love to hear from you! We understand that experience comes in many different forms and are dedicated to adding new perspectives to the team.   Kpler is committed to providing a fair, inclusive and diverse work-environment. We believe that different perspectives lead to better ideas, and better ideas allow us to better understand the needs and interests of our diverse, global community. We welcome people of different backgrounds, experiences, abilities and perspectives and are an equal opportunity employer.