Done both, and the stone outbuilding will absolutely win if you can stomach the damp-proofing faff before you even think about insulation.
My setup in the Cotswolds is a converted stone barn — breathable lime render inside, 50mm Kingspan between timber battens, and it holds heat like a proper thermal battery once the Victron kit gets the underfloor going. The walls themselves become your best mate in summer too, keeping everything cool without you doing a thing.
Log cabin kits look tempting (and the Dunster House ones aren't bad), but you're essentially living in a giant radiator that spikes and crashes with every cloud that passes over the solar array.
A few things worth considering with the stone route:
- Breathability is everything — trap moisture and you'll be fighting mould before Christmas
- Thermal mass works for you off-grid, not against — less demand on your batteries overnight
- Planning can be kinder to existing structures depending on your permitted development situation
The faff factor is real though — I spent a deeply unglamorous fortnight chasing cold bridges with a thermal camera before I was happy.
What's the building like — thick rubble stone or the thinner dressed stuff? Makes a difference to which approach I'd recommend.