I don't think they will be "in trouble" financially. But that being said, let's not make any mistake about it that doing something so different is a big risk for a band, and that risk does indeed translate to financial risk.
I have posted info before and there is lots out there on the Internet about album sales and how little money a band actually sees from an album. There are tons of hidden expenses and loads of people who need to get paid in order for an album to be recorded, CDs and records pressed, and those albums to find their way to music retailers so that we can buy them. Those costs all need to be covered before a band sees a penny from album sales. It isn't just the cost to rent a studio and the materials costs to press those physical albums. Those expenses are only the tip of the iceberg. I know for a fact that, in the past, a band would have to sell hundreds of thousands of albums before seeing a penny from album sales (in other words, the band doesn't just get a percentage of how ever many albums are sold; all the up-front costs get paid first as the album hits the shelves and begins selling copies, and only after all the expenses have been covered and everyone gets paid does the band finally start to get a cut of whatever sells after that point). I know this has changed somewhat in the last decade in terms of the actual number of units that need to be sold before a band sees a profit, but I do not know how much, and I know for a fact that the general principle still stands: there are a ton of "hidden expenses" associated with making an album such that a band still must sell a LOT of albums before seeing any of the revenues from that album whatsoever. How that translates to a band like DT who sell the types of numbers DT has sold on recent albums means that either, (1) the band eventually sees very small revenue numbers from album sales, or (2) the band takes a loss on album sales (which means they end up owing money to the label and to others associated with the process), but uses album sales as a loss leader to be made up on tour.
Touring has its own set of issues. Not sure whether you have seen what has been posted in the past, but again, there is a lot out there on this forum and in the Web in general about how, similarly, there are a TON of hidden expenses and folks who need to be paid associated with touring such that a band generally gets paid only a small fraction of what the general public assumes they would get paid for a tour. DT make money, sure, but they aren't rolling in the bucks from going on tour. If the album sales are a financial loss, the money they owe will generally come right off the top of the tour revenue and be an added expense before the band sees any touring money as well.
And that brings us to the tour for The Astonishing. This is a risky tour. For starters, there is the obvious fact that this album is different from the norm. That in and of itself could keep people away. Second, the fact that they are not playing any additional songs and are ONLY presenting The Astonishing could keep people away. For a band like DT that already operates on relatively thin margins financially, it is entirely possible that this album and tour cycle could generate substantially less revenue than past albums. It isn't inconceivable that it could even generate a loss. At this point, we just don't know.