Creatures and characters of size Small or smaller would take a Damage Increase penalty, sort of like the opposite of Damage Reduction. It seems like sound reasoning to me that a smaller creature would be more injured than a larger creature for the same blow. To compensate for this, though, they would get an even bigger size adjustment to AC. Likewise, Large or larger creatures would receive damage reduction, but a larger penalty to AC.
It might break down something like this:
Size | Damage Increase or Reduction | AC bonus or penalty
Fine | DI 8/- | +16 (instead of +8)
Diminutive | DI 4/- | +8 (instead of +4)
Tiny | DI 2/- | +4 (instead of +2)
Small | DI 1/- | +2 (instead of +1)
Medium | 0 | 0
Large | DR 1/- | -2 (instead of -1)
Huge | DR 2/- | -4 (instead of -2)
Gargantuan | DR 4/- | -8 (instead of -4)
Colossal | DR 8/- | -16 (instead of -8)
It should be noted that the DI and DR would effect weapons and targeted spells, but probably shouldn't affect area of effect spells.
It would drastically change the way parties handle very large creatures, suddenly the fighters would not be able to hurt the dragon so much and it would be up to casters to deal a lot of the damage.
It's just a thought. On the whole, I think it would make things a bit more realistic, but on the other hand, it might bog combat down even further. Tell me what you think, please.
Edit: I had the table spaced for clarity, but the editor won't keep it. Sorry if it's hard to read.
Edit #2: A lot of people are pointing out that there is a Con modifier to account for differences in size. I am actually aware of that. I'm not asking whether this is a balanced way of dealing with size modifiers, but rather a more realistic way. Although, I'd love to read suggestions on ways to make it more balanced.
Let me give a couple of examples that I gave below:
If you were to swat a (fine) horsefly sitting on a (large) horse, a blow that would flatten the fly would hardly be felt by the horse. Not because the horse has so many hitpoints that it won't miss the few lost from your blow, but because the horse was genuinely unhurt. In the one instance that I know of someone accidentally stepping on a cat, the cat died. Con modifiers adjust hp based on size but there are many situations that would injure a smaller creature, but not a larger one.