AI and Underwater Photography Unite to Reveal the Gulf of Maine's Hidden Worlds

In the Northeastern United States, the Gulf of Maine is one of the most biologically diverse marine ecosystems on the planet, home to a vast array of species from whales and sharks to jellyfish and plankton. However, this ecosystem is undergoing rapid environmental change, warming faster than 99 percent of the world’s oceans, with far-reaching consequences that are still unfolding.

A groundbreaking research initiative at MIT Sea Grant, known as LOBSTgER (Learning Oceanic Bioecological Systems Through Generative Representations), is leveraging artificial intelligence and underwater photography to document and share the ocean life vulnerable to these changes. Co-led by underwater photographer Keith Ellenbogen and MIT mechanical engineering PhD student Andreas Mentzelopoulos, the project explores how generative AI can expand scientific storytelling by building on field-based photographic data.

Generative AI is marking a new frontier in visual storytelling, much like early photography transformed our ability to document and reveal the natural world. By training generative models exclusively on a curated library of Ellenbogen’s original underwater photographs, the project ensures that the resulting imagery maintains both visual integrity and ecological relevance. LOBSTgER’s models are built using custom code developed by Mentzelopoulos to protect the process and outputs from potential biases from external data or models.

At its core, LOBSTgER operates at the intersection of art, science, and technology. The project draws from the visual language of photography, the observational rigor of marine science, and the computational power of generative AI. By uniting these disciplines, the team is not only developing new ways to visualize ocean life but also reimagining how environmental stories can be told. This integrative approach reflects MIT’s long-standing tradition of interdisciplinary innovation.

Underwater photography in New England’s coastal waters is notoriously challenging, with limited visibility, swirling sediment, bubbles, and the unpredictable movement of marine life. For several years, Ellenbogen has navigated these challenges, building a comprehensive record of the region’s biodiversity through the project, Space to Sea: Visualizing New England’s Ocean Wilderness. This large dataset of underwater images provides the foundation for training LOBSTgER’s generative AI models. The images span diverse angles, lighting conditions, and animal behaviors, resulting in a visual archive that is both artistically striking and biologically accurate.

LOBSTgER’s custom diffusion models are trained to replicate not only the biodiversity Ellenbogen documents but also the artistic style he uses to capture it. By learning from thousands of real underwater images, the models internalize fine-grained details such as natural lighting gradients, species-specific coloration, and the atmospheric texture created by suspended particles and refracted sunlight. The result is imagery that not only appears visually accurate but also feels immersive and moving.

The models can both generate new, synthetic, but scientifically accurate images unconditionally and enhance real photographs conditionally. By integrating AI into the photographic workflow, Ellenbogen can use these tools to recover detail in turbid water, adjust lighting to emphasize key subjects, or even simulate scenes that would be nearly impossible to capture in the field. The team believes this approach may benefit other underwater photographers and image editors facing similar challenges.

In one key series, Ellenbogen captured high-resolution images of lion’s mane jellyfish, blue sharks, American lobsters, and ocean sunfish (Mola mola) while free diving in coastal waters. “Getting a high-quality dataset is not easy,” Ellenbogen says. “It requires multiple dives, missed opportunities, and unpredictable conditions. But these challenges are part of what makes underwater documentation both difficult and rewarding.”

Mentzelopoulos has developed original code to train a family of latent diffusion models for LOBSTgER grounded on Ellenbogen’s images. Developing such models requires a high level of technical expertise, and training models from scratch is a complex process demanding hundreds of hours of computation and meticulous hyperparameter tuning.

The project reflects a parallel process: field documentation through photography and model development through iterative training. Ellenbogen works in the field, capturing rare and fleeting encounters with marine animals, while Mentzelopoulos works in the lab, translating those moments into machine-learning contexts that can extend and reinterpret the visual language of the ocean.

“The goal isn’t to replace photography,” Mentzelopoulos says. “It’s to build on and complement it — making the invisible visible, and helping people see environmental complexity in a way that resonates both emotionally and intellectually. Our models aim to capture not just biological realism, but the emotional charge that can drive real-world engagement and action.”

LOBSTgER points to a hybrid future that merges direct observation with technological interpretation. The team’s long-term goal is to develop a comprehensive model that can visualize a wide range of species found in the Gulf of Maine and, eventually, apply similar methods to marine ecosystems around the world. The researchers suggest that photography and generative AI form a continuum, rather than a conflict. Together, they offer a powerful framework for communicating science through image-making.

In a region where ecosystems are changing rapidly, the act of visualizing becomes more than just documentation. It becomes a tool for awareness, engagement, and, ultimately, conservation. LOBSTgER is still in its infancy, and the team looks forward to sharing more discoveries, images, and insights as the project evolves.