I have no musical training outside of a handful of guitar lessons, and I only have very loose ideas of the basic concepts in music theory. So I am certainly not analytical in the way that someone who does understand those subjects could be. As a result, I don't think I'll be able to give as precise an answer to this question as such a person, either.
The art I do know quite well is literature. I studied it in college, I'm currently studying it in graduate school, I teach it, and I spend a lot of time reading it and analyzing it. I think when I do focus on music in any sort of analytical way, I'm sort of "reading it" like a work of fiction. I'll try to explain what I mean by that.
Like Stadler, I tend to focus primarily on melody. This, to me, is the core element that gives me a sense that a piece of music is going somewhere and saying something. Not to disparage music that primarily focuses on rhythm, some of which I definitely do appreciate, but that's what I focus on. I roughly compare this to plot. They don't serve quite the same function in their respective arts, but both give a sense of progression from one idea to another. Other elements like rhythm, tempo, etc. are akin to style for me. Different ways of saying something. Like how a novel might use short sentences in a certain part to heighten tension, and then long, elaborate ones in another part to express thought, a song might use a different rhythm in a different place to achieve a different effect.
When I listen analytically to a song or an album (which I do not always do; sometimes I'm just listening for a melody I know I enjoy; or I'm just listening to get accustomed to the work on a surface level, which is usually all I'm able to do the first couple of times I hear it), I'm trying to follow the progression of ideas being laid out—much like how I would follow the progression of ideas laid out in a novel. Why this melody here before that melody here and then that solo there? Like: why this scene here before that scene there and then this monologue here? Obviously I'm able to give a much more specific answer to those questions for a novel, both because I know it better and because the required progression is, I think, easier to express in words (you need to know this detail from this scene before you can see the next one, etc.), but the general outlook is similar for me in both cases.
This is probably why I really like artists who make use of repetitions and reprises (Neal Morse, obviously, being perhaps the supreme example in prog). Why does the theme from earlier come back at this specific point? Why are they playing the same melody but now it's faster, or slower, or the vocals and guitar are at the same tempo while the drummer is playing at double time underneath? These are the things that interest me. It's also why I'm interested in concept albums, even ones where I'm not necessarily enamored with the story itself (or even know specifically what it means).
But even when there's no reprise and no lyrical concept, I'm still very interested in tracking the progression of a number of different musical ideas, and I think that's a big part of what draws me to prog rock/metal, a genre where there are more distinct ideas than in the typical pop song, and where the ideas are often carried out over longer progressions to create epic-length tracks.
Octavarium is a perfect example of a song where the lyrics don't matter much to the progression and the reprises aren't all that liberal, but where I just have this sense of fascination regarding the progression of ideas, which just feels really nicely put-together to me. I have no idea what key or what time signature is going on at any point on that song, but I do have this loose sense of how each of the parts progresses into the next and adds up into a whole of a certain kind.