It's not perfect.
One problem (for me) is that it's between worlds - on one hand it's a naturalistic, slow, British kind of thing, with realistic blokes who barely talk (and as you say, women barely register) ... on the other, there's a kind of Tolkeinesque thing where The Shire, a perfect sunny summer land has to be endangered by outside forces and then redeemed by our quiet heroes.
So the cartoon villains have to intrude into the totally realistic, naturalistic (but at the same time absurdly perfect) world, every series.
But I loved the first two series in spite of this (the actors, the photography, the pacing, the speeches). I fear that a third series might be stretching the formula a bit far - on the evidence of one episode - which is a shame, but not the worst thing in the world. They're definitely stopping now, aren't they?