Research

Browse projects by topic or search by keyword.

Mixed-View Panorama Synthesis using Geospatially Guided Diffusion

Mixed-View Panorama Synthesis using Geospatially Guided Diffusion

We propose a new task, mixed-view panorama synthesis, in which a satellite image and a set of nearby panoramas are used to render a panorama at a novel location. Our approach uses diffusion-based modeling and attention to enable flexible, multimodal control.

Zhexiao Xiong, Xin Xing, Scott Workman, Subash Khanal, Nathan Jacobs

TMLR 2025
Probabilistic Image-Driven Traffic Modeling via Remote Sensing

Probabilistic Image-Driven Traffic Modeling via Remote Sensing

This work addresses the task of modeling spatiotemporal traffic patterns directly from overhead imagery, which we refer to as image-driven traffic modeling. Our approach includes a geo-temporal positional encoding module for integrating geo-temporal context and a probabilistic objective function for estimating traffic speeds that naturally models temporal variations.

Scott Workman, Armin Hadzic

ECCV 2024
WATCH: Wide-Area Terrestrial Change Hypercube

WATCH: Wide-Area Terrestrial Change Hypercube

In this work, we introduce an adaptable framework and network architecture capable of predicting on subsets of the available platforms, bands, or temporal ranges it was trained on. Our system, called WATCH, is highly general and can be applied to a variety of geospatial tasks.

Connor Greenwell, Jon Crall, Matthew Purri, Kristin Dana, Nathan Jacobs, Armin Hadzic, Scott Workman, and Matt Leotta

WACV 2024
Handling Image and Label Resolution Mismatch in Remote Sensing

Handling Image and Label Resolution Mismatch in Remote Sensing

We present a method that is supervised only using low-resolution labels, but takes advantage of an exemplar set of high-resolution labels to guide the learning process.

Scott Workman, Armin Hadzic, M. Usman Rafique

WACV 2023
Revisiting Near/Remote Sensing with Geospatial Attention

Revisiting Near/Remote Sensing with Geospatial Attention

We introduce a novel architecture for near/remote sensing that is based on geospatial attention and demonstrate its use for five segmentation tasks.

Scott Workman, M. Usman Rafique, Hunter Blanton, Nathan Jacobs

CVPR 2022
Content-Based Detection of Temporal Metadata Manipulation

Content-Based Detection of Temporal Metadata Manipulation

We propose an end-to-end approach to verify whether the purported time of capture of an outdoor image is consistent with its content and geographic location.

Rafael Padilha, Tawfiq Salem, Scott Workman, Fernanda A. Andaló, Anderson Rocha, Nathan Jacobs

TIFS 2022
A Structure-Aware Method for Direct Pose Estimation

A Structure-Aware Method for Direct Pose Estimation

We propose a direct pose estimation method that takes inspiration from structure-based approaches to incorporate explicit 3D constraints into the network.

Hunter Blanton, Scott Workman, Nathan Jacobs

WACV 2022
Augmenting Depth Estimation with Geospatial Context

Augmenting Depth Estimation with Geospatial Context

We explore a new problem, geo-enabled depth estimation, in which the geospatial context of a query image is exploited during the depth estimation process.

Scott Workman, Hunter Blanton

ICCV 2021
Extending Absolute Pose Regression to Multiple Scenes

Extending Absolute Pose Regression to Multiple Scenes

We propose a novel architecture, Multi-Scene PoseNet (MSPN), that allows for a single network to be used on an arbitrary number of scenes with only a small scene-specific component.

Hunter Blanton, Connor Greenwell, Scott Workman, Nathan Jacobs

CVPR 2020 Workshop: VisLocOdomMap
Single Image Cloud Detection via Multi-Image Fusion

Single Image Cloud Detection via Multi-Image Fusion

Artifacts in imagery captured by remote sensing, such as clouds, snow, and shadows, present challenges for various tasks, including semantic segmentation and object detection. A primary challenge in developing algorithms for identifying such artifacts is the cost of collecting annotated training data. In this work, we explore how recent advances in multi-image fusion can be leveraged to bootstrap single image cloud detection.

Scott Workman, M. Usman Rafique, Hunter Blanton, Connor Greenwell, Nathan Jacobs

IGARSS 2020 Oral
Learning a Dynamic Map of Visual Appearance

Learning a Dynamic Map of Visual Appearance

Our approach integrates dense overhead imagery with location and time metadata into a general framework capable of mapping a wide variety of visual attributes.

Tawfiq Salem, Scott Workman, Nathan Jacobs

CVPR 2020
Dynamic Traffic Modeling from Overhead Imagery

Dynamic Traffic Modeling from Overhead Imagery

We propose an automatic approach for generating dynamic maps of traffic speeds using convolutional neural networks. Our method operates on overhead imagery, is conditioned on location and time, and outputs a local motion model that captures likely directions of travel and corresponding travel speeds.

Scott Workman, Nathan Jacobs

CVPR 2020 Oral
Learning Geo-Temporal Image Features

Learning Geo-Temporal Image Features

We propose to implicitly learn to extract geo-temporal image features, which are mid-level features related to when and where an image was captured, by explicitly optimizing for a set of location and time estimation tasks.

Menghua Zhai, Tawfiq Salem, Connor Greenwell, Scott Workman, Robert Pless, Nathan Jacobs

BMVC 2018
A Multimodal Approach to Mapping Soundscapes

A Multimodal Approach to Mapping Soundscapes

We explore the problem of mapping soundscapes, that is, predicting the types of sounds that are likely to be heard at a given geographic location.

Tawfiq Salem, Menghua Zhai, Scott Workman, and Nathan Jacobs

IGARSS 2018, CVPR 2018 Workshop: Sight and Sound
FARSA: Fully Automated Roadway Safety Assessment

FARSA: Fully Automated Roadway Safety Assessment

We address the task of automatically assessing the safety of roadways for drivers. Such ratings can be used by highway authorities to decide, using cost/benefit analysis, where to invest in infrastructure improvements.

Weilian Song, Scott Workman, Armin Hadzic, Xu Zhang, Eric Green, Mei Chen, Reginald Souleyrette, Nathan Jacobs

WACV 2018
Understanding and Mapping Natural Beauty

Understanding and Mapping Natural Beauty

We demonstrate that quantitative measures of scenicness can benefit semantic image understanding, content-aware image processing, and a novel application of cross-view mapping, where the sparsity of ground-level images can be addressed by incorporating unlabeled overhead images in the training and prediction steps.

Scott Workman, Richard Souvenir, Nathan Jacobs

ICCV 2017
A Unified Model for Near and Remote Sensing

A Unified Model for Near and Remote Sensing

We propose a novel convolutional neural network architecture for estimating geospatial functions such as population density, land cover, or land use. In our approach, we combine overhead and ground-level images in an end-to-end trainable neural network, which uses kernel regression and density estimation to convert features extracted from the ground-level images into a dense feature map.

Scott Workman, Menghua Zhai, David J. Crandall, Nathan Jacobs

ICCV 2017
Predicting Ground-Level Scene Layout from Aerial Imagery

Predicting Ground-Level Scene Layout from Aerial Imagery

We introduce a novel strategy for learning to extract semantically meaningful features from aerial imagery. Instead of manually labeling the aerial imagery, we propose to predict (noisy) semantic features automatically extracted from co-located ground imagery. Our network architecture takes an aerial image as input, extracts features using a convolutional neural network, and then applies an adaptive transformation to map these features into the ground-level perspective.

Menghua Zhai, Zachary Bessinger, Scott Workman, Nathan Jacobs

CVPR 2017
Horizon Lines in the Wild

Horizon Lines in the Wild

We introduce a large, realistic evaluation dataset, Horizon Lines in the Wild (HLW), containing natural images with labeled horizon lines. Using this dataset, we investigate the application of convolutional neural networks for directly estimating the horizon line, without requiring any explicit geometric constraints or other special cues.

Scott Workman, Menghua Zhai, Nathan Jacobs

BMVC 2016
Cloudmaps from Static Ground-View Video

Cloudmaps from Static Ground-View Video

Cloud shadows dramatically affect the appearance of outdoor scenes. We describe three approaches that use video of cloud shadows to estimate a cloudmap, a spatio-temporal function that represents the clouds passing over the scene.

Nathan Jacobs, Scott Workman, Richard Souvenir

IVC 2016
Detecting Vanishing Points using Global Image Context in a Non-Manhattan World

Detecting Vanishing Points using Global Image Context in a Non-Manhattan World

We introduce a novel vanishing point detection algorithm that obtains state-of-the-art performance on three benchmark datasets. The main innovation in our method is the use of global image context to sample possible horizon lines, followed by a novel discrete-continuous procedure to score each horizon line by choosing the optimal vanishing points for the line.

Menghua Zhai, Scott Workman, Nathan Jacobs

CVPR 2016
Analyzing Human Appearance as a Cue for Dating Images

Analyzing Human Appearance as a Cue for Dating Images

Given an image, we propose to use the appearance of people in the scene to estimate when the picture was taken.

Tawfiq Salem, Scott Workman, Menghua Zhai, Nathan Jacobs

WACV 2016
Sky Segmentation in the Wild: An Empirical Study

Sky Segmentation in the Wild: An Empirical Study

This paper presents the results of a large-scale empirical evaluation of the performance of three state-of-the-art approaches on a new dataset, which consists of roughly 100k images captured “in the wild”.

Radu P Mihail, Scott Workman, Zach Bessinger, Nathan Jacobs

WACV 2016
A Fast Method for Estimating Transient Scene Attributes

A Fast Method for Estimating Transient Scene Attributes

We describe a fast method for estimating transient scene attributes for a single image. Using our method, we explore applications to webcam imagery, including: 1) supporting automatic browsing and querying of large archives of webcam images, 2) constructing maps of transient attributes from webcam imagery, and 3) geolocalizing webcams.

Ryan Baltenberger, Menghua Zhai, Connor Greenwell, Scott Workman, Nathan Jacobs

WACV 2016
Wide-Area Image Geolocalization with Aerial Reference Imagery

Wide-Area Image Geolocalization with Aerial Reference Imagery

We learn a joint feature representation for aerial and ground-level imagery and apply this representation to the problem of cross-view image geolocalization.

Scott Workman, Richard Souvenir, Nathan Jacobs

ICCV 2015
DeepFocal: A Method for Direct Focal Length Estimation

DeepFocal: A Method for Direct Focal Length Estimation

We introduce a method for directly estimating the focal length of a camera from a single image using a deep convolutional neural network.

Scott Workman, Connor Greenwell, Menghua Zhai, Ryan Baltenberger, Nathan Jacobs

ICIP 2015
Face2GPS: Estimating Geographic Location from Facial Features

Face2GPS: Estimating Geographic Location from Facial Features

We propose a data-driven approach to solving the problem of image geo-localization using an image of a face.

Mohammad T. Islam, Scott Workman, Nathan Jacobs

ICIP 2015
On the Location Dependence of Convolutional Neural Network Features

On the Location Dependence of Convolutional Neural Network Features

We show that features extracted from deep convolutional neural networks are useful for problems in geospatial image analysis.

Scott Workman, Nathan Jacobs

EARTHVISION 2015
Scene Shape Estimation from Multiple Partly Cloudy Days

Scene Shape Estimation from Multiple Partly Cloudy Days

We introduce methods for estimating scene geometry in a distributed camera network using videos from partly cloudy days.

Scott Workman, Richard Souvenir, Nathan Jacobs

CVIU 2015
A Pot of Gold: Rainbows as a Calibration Cue

A Pot of Gold: Rainbows as a Calibration Cue

We derive constraints and demonstrate methods that allow rainbows to be used for camera geolocalization, calibration and rainbow-specific image editing.

Scott Workman, R. P. Mihail, Nathan Jacobs

ECCV 2014
Exploring the Geo-Dependence of Human Face Appearance

Exploring the Geo-Dependence of Human Face Appearance

Our work explores the little-studied dependence of facial appearance on geographic location. To support this effort, we constructed GeoFaces, a large dataset of geotagged face images.

Mohammad T. Islam, Scott Workman, Hui Wu, Richard Souvenir, Nathan Jacobs

WACV 2014
Scene Geometry from Several Partly Cloudy Days

Scene Geometry from Several Partly Cloudy Days

We describe new methods for estimating the geometry of an outdoor scene using video from multiple partly cloudy days.

Nathan Jacobs, Scott Workman, Richard Souvenir

ICDSC 2013
Cloud Motion as a Calibration Cue

Cloud Motion as a Calibration Cue

We propose cloud motion as a natural scene cue that enables geometric calibration of static outdoor cameras.

Nathan Jacobs, Mohammad T. Islam, Scott Workman

CVPR 2013