I hate to come off as antagonistic, because I'm not angry and don't mean any ill-will or anything, I just feel pretty confident about my judgement on this one, which comes out in my writing style.
When looking from 8V to SC, the differences are quite clear. The end of the Meta-album series marked a definite shift in style, and while it can be argued that DT in general has done this many times before, I find it much easier to pair SC and Black Clouds together in my head for their similarities. I can't help but imagine that if MP had stayed with the band somehow, we'd have gotten another album similar to Black Clouds instead of what we got in ADTOE.
I agree that there are clear differences between 8VM and SC, but then I think there are clear differences between every Dream Theater album and every other. Nevertheless, I think it's worth noting at least that there are some parallels between the structures of 8VM and of SC, more so than there are between the structures of really any other pair of DT albums. Both end on two long tracks and start with an 8-9 minute high energy piece. The penultimate song on both albums is a ballad with a high-energy instrumental section which many people view as disjointed. The song before this one on the album has something approaching a dance beat. I'm pointing out there parallels not to say "OMG THEY REMADE OCTAVARIUM" but to point out that you can see these things about this pair of albums just as easily as you can say them about SC and BCSL, and to support the idea that there really wasn't a more radical shift between 8VM and SC than there was between SC and BCSL or really any other pair of DT albums.
There are no single songs on Black Clouds that feel too similar to SC, with the exception of AROP. If that song were on SC instead, I wouldn't have questioned it and thought it fitted well enough in there. Other than that, the similarities aren't really in whole songs to me (we have tracks running on 13+ minutes each, so that'd be hard to do without being monotonous), but rather in the drawn-out, heavy, instrumental passages that each song has to offer, namely in ANTR and TSF, the majority of AROP, and even a little bit in TCOT. I believe it's only towards the end of the album where the heaviness lets up.
I wouldn't even say that about AROP. AROP just sounds to me like it belongs on BCSL because it's one of those that really heavily makes use of a gothic vibe and vocal harmonies, neither of which is an attribute of SC but both of which are attributes of, say, A Nightmare to Remember.
But even so, with the instrumental passages, there's not that much stylistic overlap between the instrumental sections of those two albums. Certainly no more than there is between those of BCSL and, say,
Six Degrees of Inner Turbulence or
Train of Thought (or, dare I say,
A Dramatic Turn of Events, an album which I strongly believe to be way more of a sequel to BCSL than BCSL is a sequel to SC).
Basically if "drawn-out, heavy, instrumental passages" is your criterion for saying two Dream Theater albums are similar... that's not really sufficient reason. SDOIT has "drawn-out, heavy, instrumental passages" on several songs (TGP, BF, TGD).
Train of Thought has "drawn-out, heavy, instrumental passages" on literally every song except Vacant, and I don't see how SC's instrumental sections are more similar to those of BCSL than ToT's. In fact, I'll go ahead and make the bold claim that in this regard
Train of Thought is MORE similar to BCSL than SC is, because ToT's instrumental sections tend to be centered around guitar and keyboard solos and unisons, as do BCSL's, while SC's instrumental passages are often based around what I would describe as piling up a variety of heavy and often chaotic riffs (TDEN, ITPOE2, TMOLS).
I'm not saying the two albums are identical by any means, but they're probably the most similar in DT's catalog, at least consecutively, which makes total sense as MP's influence over their sound began to grow more and more.
And I'm going to say I really can't see that. I think the songwriting changed fairly dramatically between SC and BCSL to a decreased focus on experimentation and wackiness accompanied by an increased focus on atmosphere and melody. And I think this change in focus continued on to ADTOE, though there was certainly a shift in songwriting between that pair of albums as well.
As for most similar back-to-back pair... I'm not even sure what I would say for that, because each album is kind of its own thing, but I would probably say 8VM -> SC or BCSL -> ADTOE... Those ones feel like more natural pairs to me from a purely musical perspective (not from the perspective of the band's history with members, labels, etc). Most similar to BCSL... I would call that one kind of a crossover between ToT and ADTOE, actually, if I had to describe it in terms of other DT albums.