Weight of the unsprung wheels definitely has a play!
But weight is also based upon brand of rim and tire selected: a steel bead-lock rim will weigh more than milled non-beadlock aluminum rim, and how "solid" the rim is (e.g. fewer spokes, large gaps versus mostly solid with small round holes) will affect overall weight.
For my wheels, here's the weight breakdown between Trail Grappler 35x20 on Fuel Maverick 20's, versus Trail Grappler 35x17 on Fuel Maverick 17's:
Trail Grappler 35x12.5x20 = 76.83 lbs + Fuel Maverick 20x9 = 37 lbs = 113.83 total pounds
On my scale, it showed 111.2 lbs, but has an accuracy of probably 3%, which tracks with 113.83 pounds listed by the manufacturers.
For the same tires and rims in 17", Trail Grapplers 35x12.5x17 = 79.89 lbs + Fuel Maverick 17x9 = 30 lbs = 109.89 total pounds
The net difference for my chosen brand of tires and rims is only 3.94 pounds per wheel. This would not be significant enough for me to feel any appreciable performance difference.
But then add several hundred pounds in heavy duty aftermarket bumpers, trail jack, tools, air compressor, recovery gear, electric cooler, camp chairs and table, food and snacks, extra water and fuel, trail happy doggy and one fat man and his SO, the truck is loaded down for bear. You certainly notice the weight difference! All this "extra" stuff is what really hits the pants.
To compensate, I have both a 5-Star performance tune and Pedal Commander, which wakes up the 6.2 enormously! Depending upon what setting I have on the Pedal Commander, I can easily smoke the tires with my setup.
If that's not enough, I have Whipple in the cards for the future, and when the time comes, punching and stroking out the block to 6.6...
Now, I'm just comparing Apples-to-Apples in regards to 35" diameter. Switching to 37's, which it seems you and I both are on course to upgrade to such, we need to examine the net "effective torque ratio" to the pavement: going to a larger diameter wheel will reduce the overall ratio to the deck, in addition to adding more unsprung weight (Ridge Grapplers 37x12.5x20 = 81.06 lbs each) to the mix. The result is our trucks may feel more sluggish off the line. Some have resorted to regearing to bring the net ratio back within the usable power band of the powertrain. As I have forced induction in the works, which adds gobs of low-end torque, I should have no need for regearing (I've already had 37's on it, albeit stock rims and mild AT tires, and my current tune/Pedal Commander setup easily handled it).