This is what I use to rate how good a RPG is:
1) Day & Night cycle
World has to feel alive so having day night is essential. If a RPG doesn't have day night it feels static and shallow. MMO's even more so.
2) Weather Effects
They don't have to effect game play but if it does even better but the effects can't be static and preset up. Has to be dynamic
3) NPCS doing stuff
Walking, talking, sleeping, eating and reacting to the world. If you can sit back and just watching them do there thing really makes the world better
4) Sitting, Sleeping, World Interaction
The background needs to be more than a background. If there are chairs ( and I really hope there is, I played some games devoid of chairs full stop ) you need to be able to sit on them. Bioware games are notorious for having non interactive worlds eg Mass Effect, lets feed Shepard's fish.
5) Food, provisions, sleeping, micromanagement
I love having to track provisions and supplies. It makes resting something that just a key press. Do I have enough food, should I break off this quest and try and make it back to town or maybe there is some bread in this run down tomb.
Its fun to play dress up
7) Not all combat
Everything doesn't have to kill you 24/7 having ambient animals that run away or just NPCs in towns to talk too.
8) Open world
No linear paths, or you have to be this tall to explorer here. Being able to just go any direction and find stuff to do and not be artificially held back.
9) My character not yours
Making your own character is essentially. You can talk for me but I also don't mind text for my character. I also do not like complete silence. If the characters are interesting they can get a pass, eg The Witcher.
The only game which has really fitted all this is Ulitima 7 ( Part 1 & 2 ). This game is my RPG benchmark. The Elderscrolls series also fits in nicely with most of these.
I don't mention stats To me these are the weakest part of most RPGS. Stats, talent trees and min/maxing do nothing for me. They are just systems to limited my fun in the game. JRPG's focus on this way to much, and games like Dark Souls ( which I personally enjoy as a game ) get called RPG's when really they are a action game with stats bolted on.
These are what I feel make a good RPG. A living, interactive world where you can get lost and absorbed into and sadly hard to find these days. Some came close like Divinity but while good games fall short on being near perfect.
Currently the most enjoyable RPG I am playing is Legend of Grimrock 2 and while it misses a few things from the above list its well worth getting if your into old school dungeon crawlers.