Not entirely sure, since not everybody would rejoin you, just the ones who saw the whole thing in Fort Dragonia. One at least, three at maximun I think.
And you still have used the ones who join you for some time. And then you still have the time fighting the dragons and Home's Fort Dragonia.
And as a last thing, they wouldn't rejoin immediatly, you'll still have to find them and talk to them.