An interesting exercise, though perhaps I'm not the best person to comment. I have several games that I've either (effectively) never played or I haven't bothered to beat. I still haven't beaten 8, only played enough of 9 to make sure my disc was working, had beaten the first boss of 12 before getting distracted, and barely passed the tutorial of X-2. Even at that, it took me around...6 years, I think, to beat FF VII. Compare this to the first six, all (except II) of which were beaten within a year of me first getting a hold of them (and usually within 2 or 3 months). I STILL remember the first time I beat Golbez, in the crystal room of the Dwarf Castle, or driving to get FF VI (well, III) shortly after it had come out (on Mother's Day -.- In retrospect, that was rather a poor, insensitive decision, on my part, but I think help proves the point of me being passionate about those games).
So, what was it, at least for someone like me, that made the first 6 games so engrossing, and the last 8+ so... not? I would that I've changed, rather than the series.
I played FF6 again not THAT long ago, and I never finished it. I couldn't overlook the random battles. I would play until I reached a part that I loved, then quit afterwards for weeks. Or consider FFI: when it first came out (1990), my brothers and I would always grind monsters to make sure we had bought all the best equipment as soon as it became available. Of course, we didn't call it "grinding" in my day. Nope, we called it galumphing. And we didn't call them monsters, either, we calls'd em roys. Yup, we'd say that ol Steve there (steve being what we called players) was galumphing himself some roys, yessireebobsusan. I just don't do that anymore. And, when I go back to the old games, I STILL don't do that. The 1990 version of FFI hasn't changed, so if I don't find it as fun anymore, I must be the one who has changed.
Thus, perhaps the problem with the Final Fantasy series is that its fans have grown older, expect different things, and the series is still trying to market to the same old age group. That is a fine business model. I don't need, say, Nickelodeon to market to me anymore (although I wouldn't say no to a Dexter's Laboratory reboot, or, actually, a reboot of ReBoot). If they still want to make games that target me-if-I-was-20-years-younger-oh-my-gawd-I'm-old, good for them.
That said, IF SquareEnix wants to be Squeeeeenix again, I do think there are a few accommodations they could implement for someone like me.
First, let the player control the random encounter rate. I don't mean any of this "Oh, I'll just avoid the monsters" crap, I mean let me set the actual rate. If I only want there to be big bad bosses, let me set it that way. If I want to have an encounter every foot, let me set it that way. And, of course, let me change it from day to day.
Second, have adapted difficulty that doesn't ruin the concept of preparing. I stopped playing Oblivion because I hated the fact that the world leveled with my character. I had to jimmy the system to be more powerful than everyone else, and, if I didn't, I was less powerful. That was lame. One of the reasons I've never finished FFVIII is because it had a similar system. There were, at least, innately harder and easier enemies, but it still had too much subjectivity.
How, one might ask, can the first and second point be melded? Through D&D. As a DM, I know that I can either throw several easy battles at players, or a few hard battles, before they rest and recover. As one adjusts the encounter rate, have the enemies that do spawn be more or less powerful (and give more or less rewards) in an appropriate manner. The key is in thinking of battles as a group, rather than individually. Actually, the key is thinking of encounters, rather than battles. Which leads me into...
Point the Third: What is up with all these thief or thief-like characters and no thieving?! It doesn't matter if a character is an assassin, thief, spy, priest, guru, or pacifist. Whenever they seen an enemy, their first thought is to attack, head on. Allow us to avoid enemies (or, at least, some enemies), to incapacitate them, to talk them out of fighting us, etc, and, most importantly, give us rewards for doing so. How? I'll get to that...
The Great Lord Point, Fourth of his name: are character's blind? Enemies should generally be represented on the map. Chrono Trigger did it wonderfully well: why hasn't that been universally implemented and improved upon? This would also allow us to solve the above point: once an enemy (or other obstacle) is avoided, give the party XP. Then, if the party goes back to fight it, make it so the encounter doesn't net XP.
And finally, point 5, break the game up into easily digestible chunks. If I only have 10 minutes, I should still be able to sit down and get something done, even if I am in the middle of a dungeon. Maybe this means giving me perpetual access to a minigame like blitzball (except, you know, with better music), or some way to improve characters, play dress up, decorate a house, forge the doombringer sword, etc. It is hard for me to squirrel away an hour or more to play a game. Not playing a game means, well, I'm not playing it, and I'll be less excited about the next title because, hey, I have 3.5 unplayed Final Fantasy games already, waiting for me at home.
You'll note that I said nothing about plot, or actual battle, or anything else. The reason is quite simple: one of the things I love about FF is that it tries to reinvent itself with each iteration. That's fine and dandy, but the above are those obstacles that are preventing me from enjoying any iteration.
Edit:
To be honest, I think the best thing that could happen to the series right now is if a small group of indie developers that haven't even become a "company" yet, were to make a game and bootleg it as "the next final fantasy."
*looks around* Where's Faustwolf when you need him? We have the technology, we can rebuild the series. Make it faster, stronger, better.