Oblivion uses a heavily modified Gamebryo engine, which is the successor of the NetImmerse Engine used in Morrowind. It is much more efficient than its predecessor and features -- among other things -- advanced pixel and vertex shaders.
Oblivion uses SpeedTree RT to render realistic looking trees and foliage.
Oblivion uses Bethesda Softworks' own Radiant AI engine. It is based not on specifying exact actions (scripting) for every NPC, but rather goals that are defined for every NPC which they must try to achieve in accord with the current state of the gaming world. If an NPC is hungry, they will try to acquire food; first, they will try to buy food, and if that doesn't work, they may go and hunt deer. If the moral ranking of the character is low, he may attempt to steal the food from a merchant.
Bethesda has implemented and tested three different combat systems to see which one worked best in terms of gameplay and fun.
The components of stealth have been greatly overhauled. Shadows affect your visibility, and your current visibility is shown on your crosshair.
Spells work fairly similarly to Morrowind, however the Magicka regeneration has been much improved and spells can be cast without switching to a casting stance.
Potions can be used as poison on weapons (including arrows), or can be drunk by the player to enhance their abilities.