while they are busy scavenging, they aren't planning insurrection, not planning to fight the saviors and not planning to fight each others. (Which does happen in the comics)