Stone is the easier resources to harvest.
But i think this is balanced within the game.
When we look at Tier 1 building materials:
Mortared Stone = 8 Stone + 3 pitch
Hewn Log = 2 log's
Tier 2 building materials:
Brick = 20 stone + 5 Pitch
Lumber = 10 log's
So i agree with you that log's are more difficult to gather.
But then this is why their are worth more than stone.