Rural communities and farmers are concerned they will lose an essential workforce as the brakes are put on working holiday visa applications.