Profile cover
H

AI Multimodal Scene Understanding Engineer

Hannah Cohen

Full-time · Job · New_York

About the role

What does it mean to truly understand a scene? Our platform processes 3D scans, RGB images, and sparse depth maps from 60 enterprise clients in architecture, construction, and insurance. A human expert looking at a building exterior can identify material types, assess structural integrity, spot water damage, and estimate renovation cost — all from visual information. We want our system to do the same. Scene understanding at this level requires fusing multiple sensing modalities with learned structural priors. You'll build the ML stack that takes raw multimodal inputs and produces semantic scene graphs: labeled objects with spatial relationships, material classifications with confidence estimates, and anomaly flags for the features that human reviewers need to examine more closely. The research team has laid the architectural groundwork — you'll take their prototypes and turn them into production systems that process 4,000 scans per day with SLA-bound latency and documented accuracy. This is a senior IC role where your technical depth in multimodal perception directly determines what the product can offer.

Responsibilities

  • Build production-grade multimodal scene understanding models from 3D and RGB inputs
  • Develop semantic scene graph generation with object relationships and material labels
  • Implement anomaly detection for structural and material assessment use cases
  • Optimize multi-modal inference to meet SLA requirements at 4,000 scans/day
  • Build evaluation pipelines benchmarking against human expert annotations
  • Collaborate with research to bring prototype architectures to production readiness

Requirements

  • 6+ years in computer vision or multimodal ML with production deployment experience
  • Deep expertise in 3D vision: point clouds, depth estimation, or NeRF-based methods
  • Experience fusing heterogeneous sensor modalities in production systems
  • Strong PyTorch skills and understanding of attention-based architectures
  • Track record shipping models that run at scale with documented accuracy

Benefits

  • Fully remote with async-first culture
  • Comprehensive health benefits
  • $5,000 annual equipment and learning budget
  • Equity package
  • Sabbatical program after 4 years

Keep exploring related roles

Use these topic pages to discover more roles in the same hiring cluster.

Categories

AI & Machine Learning Engineers data science & analytics

Skills

python pytorch computer vision hugging face aws

Related remote AI jobs

Nearby roles that share similar skills, categories, or hiring intent.

Browse all jobs

Senior Product Security Engineer

RemoteAI company

Full-time Senior

Senior AI Researcher- Pre-training (f/m/d)

RemoteAI company

Full-time Senior

Drone Platform Integration & Network Support Engineer

RemoteAI company

Full-time

PhD Autonomy Engineer Intern - Computer Vision / Deep Learning Summer 2027

RemoteAI company

Full-time Intern

AI Resident UK/EMEA

RemoteAI company

Full-time

Senior AI R&D Engineer – Model Training (f/m/d)

RemoteAI company

Full-time Senior

Job Type

Full-time

Level

Senior

Language

English

Salary Range

$160k – $200k / year

AI Expertise

AI & Machine Learning Engineers data science & analytics

Skills

python pytorch computer vision hugging face aws docker

Ready to apply for this role?

Create a free talent account in under 2 minutes.

  • Apply to verified AI companies
  • Get AI-matched job recommendations
  • Message hiring managers directly
  • Build your public AI talent profile
Create free account & apply Log in

Interested in this role? Create your free profile to apply.

Apply — create free profile