Buildings are the largest energy-demanding sector in the world, representing over one third of the total worldwide consumption and a similarly important source of CO2 emissions. We envision a future energy system in which building control will be performed by autonomous self-adaptive agents that, with minimal configuration, will learn how to operate the HVAC equipment more efficiently and how to collaborate with other actors of the grid.