Directly to the battery. This is assuming you mean a 1-wire alternator that has an internal regulator, and not an alternator with a remote sense wire (2nd, small gauge wire). If you have a remote sense wire, you should hook it to the place you draw the most load from (fusible link, splice under the dash, etc.).
You can wire to the starter relay as long as you have the same gauge wire or larger going to the battery from there. However, fewer connections are better.
Also, I'm sure someone will recommend a fuse, circuit breaker, or fusible link in line with the charge wire you are hooking directly to the battery. That's up to you. It's recommended, but plenty of people get away without it.