New Duke lab mines big data to inform energy knowledge, choices

Posted On:

Tuesday, Apr 14, 2015 - 4:05 pm

Companies that analyze data on their energy use can save money by better managing their energy consumption. Individual consumers can use smart meters and other data tools to gauge their own use and decide whether to change their habits or upgrade their homes. Economists can use data on production technologies and costs to understand changing energy markets, and governments can use data to develop policies to assure a steady supply of energy with a lighter impact on the environment.

These are just a few ways that energy-related data is being compiled and used worldwide. At Duke University's new Energy Data Analytics Lab, faculty, students, and research staff are working together to mine these mountains of information to help companies, government agencies, and individuals make smart choices about energy production and use.

The Energy Data Analytics Lab is lead by the Energy Initiative, a university-wide, interdisciplinary collaboration that focuses on advancing an accessible, affordable, reliable and clean energy system. It also partners with the Information Initiative at Duke and Duke's Social Science Research Institute to leverage Duke’s expertise in big data and social science methods.

The energy data lab will serve a hub of research, education and engagement activity, bringing together Duke's world-class data analytics experts and coordinating their efforts to collect and analyze information to predict energy patterns, improve energy management and efficiency, and inform policy.

"Energy systems are a foundation of the modern world and they're in a period of significant transition," says Kyle Bradbury, the lab’s managing director. "Many devices are now connected by data, and that means complex, systemic challenges can be answered when we use the latest tools from statistical modeling, machine learning, data mining and visualization."


Systems include electricity generation and distribution, transportation, industry, buildings, "smart" devices, and energy markets, Bradbury said. The energy data lab is developing and applying advanced data analytics tools to transform information from these systems into insights that will improve their reliability, resilience, environmental sustainability, productivity and affordability.

The lab will address three overarching research questions: how big data can be used to develop baseline assessments of energy systems and resources; how data analytics can be used to evaluate the impact of interventions in energy systems; and how new data analysis tools and techniques can be used to better forecast and predict energy system performance, costs, and resources.


The Energy Initiative's director, Richard Newell – who is faculty co-director of the lab with Sanford professor Matt Harding – has long understood the value of data in understanding and influencing energy consumption, fuel pricing, energy policies, and many other aspects of energy supply and use. Harding is teaching a new course this semester on the application of big data methods to energy and other important problems.

Among other data-sensitive projects, Newell – former head of the federal Energy Information Administration – is working with graduate student Brian Prest to analyze data from Drillinginfo to study how new oil and gas extraction technology such as horizontal drilling and hydraulic fracturing are affecting energy markets and, in particular, the volatility of natural gas prices.

"There's a staggering amount of information being amassed every day about energy production, energy-using equipment, and pricing," Newell said. "We’re just starting to fully understand the ways these data can inform homeowners, businesses, and government policymakers. As the Energy Data Analytics Lab gains momentum and broadens its partnerships, Duke experts will play a leading role in supplying the world with critical information about energy resources."


Students are part of the energy data lab as well. A Bass Connections project team is working under Bradbury’s guidance to develop techniques to break down data from "smart meters' that collect information on energy use by individual appliances in a building. Meters have been distributed in several buildings across Duke to help collect the information, while the research team develops algorithms to manipulate the data.

The goal is for building owners and residents to be able to cheaply and automatically audit their buildings' energy use. They will be able to identify ways to be more energy efficient, save money on energy bills, and predict equipment failures, among other insights, which will reduce greenhouse gases and other emissions. Wells Fargo recently made a $150,000 gift to support this work and to create new learning experiences for students interested in data analytics and energy.

Bradbury helps a Bass Connections team member build a smart meter

Other current lab projects include one that is trying to improve data on distributed rooftop solar photovoltaic capacity and power generation. Current information is based on government statistical collections that tend to focus on central station power, rather than energy production at homes and commercial buildings. The researchers will use images taken by satellites and aircraft to estimate the number and location of rooftop solar panels across the United States, then link that information with existing solar generation data from utilities and government agencies, and also with detailed real estate data on housing values and characteristics.

The lab is partnering with a number of corporate and other groups to use their data in projects, such as Ice Energy, a California company that builds thermal energy storage systems for increasing building cooling system efficiency and balancing the grid. Ice Energy is sharing a large amount of data from its systems to help figure out better ways to predict and thereby prevent failures in energy storage systems.

Location: Live online event
Time: 8:00 pm
Location: Webinar - RSVP for link
Time: 1:00 pm to 3:00 pm