In This Role You Will:

• Design GPU resource allocation and scheduling for embedding generation and inference workloads across multi-tenant deployments; optimize GPU utilization (targeting 80%+ GPU memory utilization) through techniques like batching, quantization, and vLLM deployment
• Build and operate GPU clusters (multi-node, multi-GPU) for production inference workloads; implement GPU monitoring, health checks, and auto-recovery mechanisms; optimize cost-per-inference through efficient GPU scheduling
• Profile and optimize GPU-based inference pipelines for latency (targeting <100ms p95) and throughput (queries/sec per GPU); implement inference optimizations including KV caching, continuous batching, and model parallelism strategies
• Establish performance benchmarking framework, identify bottlenecks through profiling, and implement optimizations achieving 10x+ improvements
• Design and implement distributed ingestion pipeline capable of processing 6M+ documents/hour using technologies like message brokers, MinIO, workflow engines and parallel processing frameworks
• Architect high-performance vector similarity search infrastructure supporting billion-scale embeddings with sub-second query latency
• Architect compute-optimized deployments across AWS (EC2, EKS), GCP (GKE, TPUs), Azure (AKS), and on-premises Kubernetes
• Design, develop, and optimize high-performance, distributed systems and software for scalability and reliability
• Collaborate with research scientists, ML engineers, and platform teams to deliver high-quality, large-scale software
• Drive architectural decisions through RFCs, mentor ML and platform engineers on HPC best practices, establish coding standards for high-performance systems
• Lead the technical design and implementation of major features and components with lasting architectural impact

What You'll Need to Be Successful:

• Extensive experience in software development, with a proven track record of delivering complex, large-scale systems (8+ years for Senior, 12+ years for Staff)
• Proven experience building distributed systems at 100M+ scale (documents, vectors, or equivalent)
• Deep knowledge of parallel and distributed computing concepts including consensus algorithms, distributed coordination, and fault tolerance
• Hands-on experience with vector databases (pgvector, Pinecone, Weaviate, Milvus, or equivalent)
• Proficiency in systems programming languages such as C++, Go, or Rust
• Experience with parallel programming models (e.g., MPI, OpenMP, CUDA)
• Production experience optimizing GPU workloads for inference including batch optimization, quantization (INT8, FP16), and GPU memory management
• Experience managing large-scale GPU infrastructure (10+ GPUs in production) including cluster orchestration, resource scheduling, and cost optimization
• Deep understanding of GPU architectures (NVIDIA A100/H100, tensor cores) and inference frameworks (vLLM, TensorRT, Triton)
• Deep understanding of memory hierarchies, cache optimization, and NUMA architectures
• Experience with container orchestration (Kubernetes) and distributed computing frameworks (Ray, Dask, Spark, or equivalent)
• Familiarity with performance analysis and optimization tools and techniques (profilers, tracers, benchmarking)
• Strong systems programming background with evidence of performance-critical contributions (open source, papers, or production systems)

Preferred Qualifications

• Experience with cloud-based HPC, including services on AWS (EC2 P4/P5 instances), GCP (A100/H100 VMs), or Azure (ND-series)
• Knowledge of networking and storage technologies in the context of high-performance computing (RDMA, NVMe, distributed filesystems, GPU-Direct Storage)
• Advanced GPU optimization experience including multi-GPU inference (model parallelism, pipeline parallelism), mixed-precision training/inference, and GPU profiling tools (NVIDIA Nsight, nvprof, PyTorch Profiler)
• Experience with ML infrastructure including model serving frameworks (vLLM, TensorRT-LLM, Triton Inference Server), GPU resource management (NVIDIA MIG, GPU time-slicing), and inference optimization (continuous batching, speculative decoding)
• Production experience with GPU monitoring and observability (DCGM, GPU metrics dashboards, cost-per-query optimization)
• Background in information retrieval or vector search (FAISS, HNSW, IVF indices, approximate nearest neighbor algorithms)
• Production experience with object storage (MinIO, S3, GCS) at petabyte scale
• Familiarity with specific technologies: Kafka, PostgreSQL, pgvector, Kubernetes, FluxCD, Yugabyte
• Contributions to open-source projects in the HPC, distributed systems, or vector search space
• Experience with on-premises enterprise deployments and air-gapped environments (government/defense sector)

190000 - 230000 USD a year