After reading a quick instructable similar to this project, I drew up a schematic with the same idea except it (hopefully) combines 4 inputs into one channel and then switches that between two outputs.

Now, the following schematic was the one drawn up in the instructable, which I assume works.


I want to make this, but I want to combine 4 audio inputs instead of having just one. I drew this up:


All I did differently was combine the audio sources of all 4 channels and use that as one audio source for the selected output.

Now, I don't know much about how audio inputs operate, but I feel like this won't work because of the method in combining audio channels. So, if I am incorrect, I would appreciate some guidance in what I need to add to make this work.

You are correct. You can't just put audio inputs in parallel; among other problems, each audio input will end up driving the other inputs. There's no very simple answer, unfortunately; the following page seems to be a fairly reasonable summary of some of the different things you can do. One of the most naive is some resistors to heavily minimize crosstalk, plus an op-amp to amplify the resulting weak signal. Each successively more complex solution is successively better.
http://sound.whsites.net/articles/audio-mixing.htm

So while this argument definitely seems valid, a simple experiment today seems to validate my circuit. I plugged the male end of a 3-way 1/8" splitter into my speakers and plugged each of the three splitter ends into different audio sources. When I listened to it, I could hear all three audio sources clearly in stereo. While this splitter was meant to be used to allow three pairs of headphones to be used on one device, it successfully combined three audio channels into one as well. I have tried this with two other splitters with the same results.