Hacker News new | past | comments | ask | show | jobs | submit login
Launch HN: InBalance (YC W21) – Short-term energy market forecasting
55 points by Straw on June 4, 2021 | hide | past | favorite | 40 comments
Hey everyone, we’re Thomas and Raj, cofounders of inBalance (inbalanceresearch.com). inBalance forecasts electricity price, demand, and generation by source up to 72 hours ahead, helping utilities and independent power producers utilize their responsive assets such as energy storage, backup generators, flexible demand, etc more efficiently.

We met playing ultimate frisbee in Cambridge, UK, and quickly found common interests in statistics and optimization. Thomas had previously worked on wind turbine placement problems, providing experience with power markets, and we discussed them but didn't see an immediate entry point, so Thomas continued his statistics PhD and Rajan worked in ML research and GPU algorithm design at a startup.

A year ago we heard of a need for better wind power forecasts and started to look at the market more closely. We found a gap emerging from the increase in the prevalence of storage, especially lithium-ion, grid-scale batteries. It seemed like an interesting and useful real-world application of machine learning, particularly with the possibility of reducing carbon emissions, so once the business case looked tenable, we decided to go ahead!

Electrical power markets have become increasingly volatile due extreme weather events and increased prevalence of intermittent renewables. In response to this, producers are bringing on more flexible generation assets such as batteries to even out fluctuations in supply, and electrical consumers are aiming to increase their ability to modulate demand to better take advantage of cheap intermittent power. These assets don't fit into the day-ahead markets designed for mostly traditional steam power plants, making it difficult to choose when to use them. Our forecasts help traders better align their use with power availability, who now do so on gut feeling or low-quality coarse-grained forecasts. We hope this will increase the value enough to make transitioning to renewables more financially appealing.

Most standard machine learning approaches struggle in particular with price forecasting due to the limited data, large number of factors, heavy-tails, high noise, and underlying complexity; even given the bids for each producer and consumer, solving for the prices across a power network taking into account transmission, energy balance, and AC power flow constraints relies on an NP-hard mixed-integer programming problem that can take hours to solve. Of course in reality we don't even know the bids ahead of time, and we still haven't won the battle against the heavy tails today!

Our pilot experiences with a major East Coast utility looking to trade power, a major New England utility managing their demand response program, a battery storage operator in Texas, and a wind trader in Texas, have shown us that every participant has differing needs for their particular asset collection, so we dedicate time to each of our customers to make sure that the product is tailored to their needs. Along the way we've developed a generic forecasting system tuned for power markets to speed up customization, but we know we have a long way to go before we support the full range of forecast granularity, ___location, range, risk metrics etc we've heard interest for. With over 3000 market participants operating in open electricity markets (including Texas, California, New York, New England, and the mid-Atlantic), we’re hoping to hit 7 figure revenue within the next year.

We need huge amounts of storage to facilitate a transition to zero carbon grids long term, so we hope to minimize risk and maximize the reward for building new storage assets.

We’d love to hear your thoughts, questions, and comments!




Great initiative. Congrats on the launch. Fluence recently bought AMS (Advanced Microgrid) to provide similar services. Others like Invenia Labs are also trying to tackle this problem. Would love to connect directly with you to speak. Email is on my profile.


Some energy companies I know have their own rooms full of analysts looking into things like what's on TV tonight so they can forecast demand spikes e.g. in the breaks of sports events. They also have data science teams to place their wind farms optimally in-house (my knowledge of the sector is geographically limited to Europe and the midwest of the US).

So make sure that whatever you are getting into is a true need that more than one potential customer actually has, rather than basing things on assumptions alone (customer validation of the proposition).


a number of utilities we work with, including several larger players, currently rely heavily on third party forecasts of demand, wind generation, etc

but you're right, some folks we've approached have built large in-house teams


I worked for/with one of those teams at ESB - Ireland's national electricity provider. We built a cutting edge real-time visualisation of energy production/usage from all the power plants in the country using different fuel types and forecasting models like yours in order to get the most out of renewables and also give the traders an edge.

I was thinking of doing my own system too so love what you guys have done. I'm currently just working on my own side projects so open to opportunities if you guys might have some? :)


Congrats on launch.

A quality problem to be solving.

Would like to see a real world detailed walkthrough of how your product is solving a problem with one of those pilot customers and +1 to open source per previous comment.


Thanks!

Perhaps we could show one of our forecasts, and implied battery/turbine/etc dispatch plan, but lagged a day to avoid annoying the customer of said forecasts? We could even prove we had them earlier by first publishing a hash ;).


Awesome! This seems like a big and hard problem to tackle. I'm also in the ML space; my only recommendation (and I can't say anything in terms of energy because I'm a complete ignorant in the matter) is: try to contribute as much as possible to Open Source. Most AI-based profitable companies seem to find good market traction after leveraging Open Source esteem and renown.

Best of luck!

PS: I'm from Argentina and I've also made for-life friends playing Ultimate :)


Hi, thanks for the advice!

How would you balance maintaining an open source presence with maintaining a technological advantage based on our ML techniques in the space? We've put a lot of effort into finding modelling approaches that work in the high-noise, heavy-tailed, limited data environment, so we feel we can't just give them away.

P.S. Yeah, its the best! And for some reason it seems to attract all the nerds :)


I find alternative data to be a fascinating world, your idea reminds me of CargoMetrics which is doing a similar thing in the shipping industry. Just wondering -- if and when you reach the scale where a substantial segment of the supply side of the electricity market is using your service, is there the possibility that your algorithms can be subject to a reverse self-fulfilling prophecy (e.g. you predict a downwards demand shock which leads producers relying on your forecasts to scale back generation, but because so many producers are doing so in tandem there ends up being a power shortage)? Also, are you exclusively targeting the supply side, or do you have plans for a consumer-side offering as well? I feel like there would be immense potential for an integrated "smart" power grid especially with the rise of planned smart cities in the Asia-Pacific region so that might be another interesting application of your forecasting. Congrats on the launch, looking forward to hearing where y'all go from here.


If we become that successful, yes, we'd have to adjust our forecasts for the effect of supplying our forecasts!

Most consumers don't have access to wholesale, fluctuating rates, so we're not sure how to effectively reach them with something beneficial. If home responsive demand such as car charging or smart appliances become much more widespread, we'd definitely want to find a way to manage those as well; I guess we'd have to go through their utilities but unsure.


Awesome news in this space. Here in Victoria Australia, we're part of the Australian east coast network which covers Qld, NSW, Vic, SA, Tas.

The market is managed by AEMO and Victoria has a deregulated environment, separated into generators, distributors, and retailers.

I recently signed up with amberelectric.com.au which passes through the wholesale market price (set every 30m) and a fixed $15/month charge.

It's been interesting to watch the market in semi-real time and I would love to see an API that I could adopt/adapt to allow a household to participate in the market.

South Australia has an experimental "virtual generator" that is managed with rooftop solar on each of its public housing estates, combined with local battery storage.

One area that is not clear is that the grid is being stressed by the number of new generation and storage sources, so AI/ML work in modelling that environment is very important in Australia.


Interesting, thanks for the info. Another commenter also recommended looking at the Australian market.

Sounds like Griddy in Texas. A good business idea, but they failed due to counterparty risk in the recent storm: their customers failed to pay their enormous electric bills, they couldn't cut them off due to regulations on extreme weather events, and so they got left with a huge bill.

How does your provider handle price spikes? Can you set a price cutoff at which you no longer buy?


> differing needs for their particular asset collection

How do you deal with customers who, on the one hand, generate most of their revenue from the honest work of doing A for person B, while on the other hand, generate profits from completely esoteric financial and political engineering?


What kind of customer are you referring to? How would our forecasts benefit them?


> What kind of customer are you referring to?

The kind of small energy companies that answer the phone when a startup calls.

Presumably if you had an idea for how to do the forecasts profitably you'd just become an energy futures trader yourself.

I don't know, I'm not trying to poo-poo your thing. But the ESG / Biden green energy plan / renewables stuff your people are excited about, you make a lot more profit just by showing up and harvesting the incentives than you do like, doing a good job.

Berkshire did this (https://www.bloomberg.com/opinion/articles/2019-06-04/warren...) and it's funny, those people were a scam. That's what I'm trying to say. Profits are pretty dodgy in the energy industry.

I like this teeny tiny example though, Active Power, that became a Net Operating Loss shell (also common for biotech), because energy has an easier time booking losses as R&D expenses (https://seekingalpha.com/article/4126904-p10-industries-purc...).

I don't know. There are so, so many examples of hilarious bad actors, from which the profits for small energy businesses come. You're not making forecasts for Exxon, you're doing it for little tiny energy providers. How, really, are they profitable at all?


You raise a good point! Actually, we do plan to trade day-ahead energy futures ourselves. However, in several US markets, the markets required for responsive assets such as batteries and gas turbines simply don't exist- there isn't an hour-ahead electricity future. So there's value outside what we could capture by trading directly.

Now, if we could get enough interest to make such a short-term futures market and trade on it directly... we'd love to!


I think you may be conflating energy usage optimization with futures trading. Utilities often use forecasts for different purposes than futures traders, e.g. demand/response. And there are many utilities that aren’t aren’t tiny. In fact, most are larger than they seem (it’s expensive to be in this business), and all of them are looking to optimize heavily. I work in this field in a similar capacity to these guys and there is money to be made legitimately.


There are many midsize utilities that need accurate load forecasting to fulfill their duties as Balancing Authorities and ensure that generation equals load. These are not small companies and the methods currently used are often improvable.


Electricity auctions in the UK are for 30 minute blocks, done 30 minutes ahead.

Do you think there are benefits to making that 1 minute blocks, or even 1 second blocks now that all trading is automated?

It would have the benefit that solar producers no longer have to guess how many clouds there will be in an hour (and waste energy or money if they're wrong). It would mean big companies don't need to decide an hour in advance of switching on or off their equipment. It would mean pricing can represent fine-grained demand, such as surges exactly on the hour as scheduled things turn on.


30 minutes is already reasonably fine-grained- enough time for a gas turbine to spin up. Shorter-scale fluctuations are usually handled by reserves and even just the momentum of spinning generators. Its certainly possible that going shorter could help, but I haven't seen enough variation on 5 or 15 minute timescales, except when a power plant goes offline unexpectedly, to see a huge benefit.

Now, one could try to replace the existing reserve mechanisms with the shorter-scale market you describe, but given the resistance to change in the grid operators, I think this is a difficult fight.


Are you looking at markets outside the US as well?


Definitely. Our initial work is all in the US, but we're also keeping an eye on European power markets. Although they have a slightly different structure, its the same underlying problem.

We haven't looked outside the US and Europe, anywhere you'd recommend?


Australia.

We have an interconnected network covering most of our population, a wholesale spot market, high penetration of rooftop solar, infeed tariffs, subsidized household storage, and a network that is suffering growth issues in terms of accomodating the generation and storage facilities coming on line.

Our various states are pushing for net-zero generation in the next 5-10 years, while our Federal government is still subsdizing fossil fuel generation.


Canadian markets are also very interesting to us, we've heard Ontario (IESO) utilities are currently underserved in several pilots


Have you considered the long term effects of Musk’s tweets on the markets? How would you use these to bias a model? (Congrats btw :))


nlp is difficult, do you think his tweets could have a direct impact on short term power consumption patterns?


NLP is difficult, yes, but modern transformer architectures are able to interpret even Elon’s tweets. I think there’s a great deal of potential for a direct impact, particularly with Bitcoin’s high energy demand. I also wouldn’t discount the potential for him to shift his focus onto energy markets and make more direct tweets. When Tesla power banks properly come on line, you’ll need to be directly wired into his information flow as well. Basically, yes! Get on it :))


we'll get to work :)


Also, don’t forget to do the investor calls listed as a perk on wefunder ;)


Good.


Maybe start a hedge fund and speculate on electricity prices?


I think trading is an important application of our modeling work, at least it would help bring real time pricing into equilibrium with 36 hour ahead futures so that power plants with poor ramping constraints can plan accordingly


It's also a good test of how good your modeling really is...


This is essentially what Enron was doing to pay the bills before their loans disguised as assets fraud scheme blew-up. They explicitly manipulated supply in some instances.


Lots of shops trade/speculate on electricity. Enron just happens to be famous.


Better learn to walk before attempting to run. Too many HF’s are stated based on half-baked ideas/models. Better to partner with an existing HF and get the data insight, but maintain your independence an chart your own course.


Any interest in expanding to the solar market in Chile?


We are focusing on US markets for now, but we are definitely keeping an eye out for wherever else our services may be of use :)


Let's chat. I founded a profitable energy business. You can find me in the YC investor list.


Let's chat, I founded a profitable energy business. Find me in the YC investor list.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: