The self-employed Warwickshire gardener would turn up for a small amount of work - then fall silent and ignore victims