Yeah, you've got it!
Basically, each attack is either physical or special. It used to be that all psychic attacks were special, all rock attacks were physical, all electric attacks were special, etc. etc. But now there are little icons.
An attack like Thunderbolt, or Confusion is special. They don't make contact with the foe, they just generate a sort of field. Moves like Thunderpunch and Zen Headbutt, on the other hand, while still electric and psychic respectively, do physical damage.
If you've got a high attack stat, you're better off using physical attacks, whereas if you've got a high special attack stat you're better off using special attacks.
So, for a Pokemon like Gyarados - Surf is a special attack, whereas Waterfall is a physical attack, both of around the same power. Gyarados' attack stat is a fair bit higher than its special attack, so as a general rule, Waterfall would be playing to its strengths more than Surf.
However! Against a pokemon like - Steelix is a good example - its defence stat is astronomical. One of the very highest in the game. Meanwhile, its special defence is comparatively puny. So a Gyarados' Waterfall would probably leave nary a scratch on it, in spite of the strong attack stat and the type advantage. I imagine that against a Steelix, Surf would be a better option.
As a general rule, though, it's better to play to the Pokemon's strengths than anticipate a potential enemy's weaknesses. Unless there's a particular battle that's proving really problematic, it's usually wiser to give a Gyarados moves like Ice Fang, Waterfall, etc., than Ice Beam and Surf.