Note: The job is a remote job and is open to candidates in USA. Cache Financials Inc. is a bespoke brokerage that offers investment products for individuals with large concentrated stock positions and has scaled to over $2 billion in assets. The company is seeking a Staff Backend Engineer to own infrastructure components and design reliable services and data models for account operations, investment workflows, financial movements, and external integrations. The role emphasizes distributed-systems correctness, auditability, reconciliation, observability, and technical leadership in a remote early-stage environment.
Responsibilities
- Own the architecture and delivery of financial workflows spanning Cache, customers, and external partners
- Establish clear system boundaries, sources of truth, invariants, and failure behavior before choosing implementation mechanisms
- Model long-running workflows with explicit states and a clear distinction between requested actions and confirmed facts
- Design APIs and event processing that behave safely under retries, concurrent requests, duplicate delivery, out-of-order messages, and ambiguous partner responses
- Build durable processing patterns using transactions, queues, outboxes, idempotency, deduplication, reservations, and explicit recovery paths where appropriate
- Design financial records so cash movements, positions, executions, and other economic effects are auditable and cannot be silently applied twice
- Make reconciliation and discrepancy resolution first-class system capabilities rather than after-the-fact operational patches
- Improve observability, operational tooling, runbooks, alerts, and safe controls for detecting, containing, and recovering from failures
- Write clear design documents, lead architecture reviews, and help the team make better technical tradeoffs
- Ship incrementally while balancing customer impact, operational risk, maintainability, and delivery speed
Skills
- Strong programming skills in a backend language are required
- Significant backend engineering experience, including technical leadership over systems with meaningful correctness or reliability requirements
- Experience designing, building, and operating distributed systems involving databases, asynchronous processing, and external services
- The ability to reason precisely about timeouts, retries, races, partial completion, duplicate or out-of-order events, and crashes between side effects
- Strong data-modeling and transactional reasoning, including an understanding of where database guarantees end and application-level guarantees begin
- Experience with systems that move money, or an equivalent domain where a silent double-spend is unacceptable
- Experience building systems that are observable, auditable, recoverable, and operable by people other than their original authors
- Strong written and verbal communication
- A track record of raising the quality of the team around you through technical direction, mentoring, design reviews, and reusable systems
- High ownership and customer empathy, especially when failures can affect someone's money or trust
- Comfort moving quickly in an early-stage environment without removing the safeguards that correctness-critical systems require
- Experience in brokerage, payments, custody, clearing, accounting, banking, logistics, or another correctness-sensitive domain
- Familiarity with immutable ledgers, reservations, reconciliation, settlement, or financial audit trails
- Experience with workflow orchestration, event-driven architectures, or external partners with imperfect reliability characteristics
- Experience introducing property-based testing, fault injection, replay tooling, or invariant validation
- Experience helping an early-stage engineering organization evolve its architecture and operating practices as the product scales
Benefits
- Fully remote work, with the team meeting in person once a week in San Francisco or New York.
Company Overview
Company H1B Sponsorship