Elf Sword (Upgrade as much as possible) {Earliest Available: 6th Dimension Hard Mode - Uncommon drop from Dark Beast, Common drop from that first Red Knight}
Frozen Tome (Upgrade a bit) {Earliest available: 1st dimension Hard Mode - Rare drop from Ice Mage}
Gold Ring (Upgrade as much as possible) {Earliest available: 1st dimension Hard Mode - Uncommon drop from Ice Mage}
Cursed Amulet (Upgrade as much as possible) {Earliest Available: 6th Dimension Hard Mode - Common drop from Dark Beast}
Full Iron Armor (Do not upgrade) {Earliest Available: 6th Dimension Hard Mode - Rare drop from Knight}
Wizard Slippers (Upgrade as much as possible) {Earliest Available: Shop after finishing 1st Dimension, also Rare drop from Fire Mage }
Not all of these are available from the beginning, but early options are OK. Just don't stress on upgrading early options (save your cash for upgrading the final set)
Early options for Elf Sword include Halberd or Flame Sword (By the time you can get the Amethyst Blade, you can also get the Elf Sword. Also, the Masamune upgrade bonuses suck.)
Early options for Frozen Tome are Wooden/Crimson/Amethyst Staff (Frozen Tome is best because of 5 projectiles at once. Range is less, but usually instakill. Otherwise, take advantage of infinite range from early options.)
Early option for Gold Ring is Book of Battle (Better upgrade bonuses than other books)
Early option for Full Iron Armor is Golden Apple (Much better than Cyan Shield)
Wizard Slippers just Rock.