Imo, when someone leaves a book with a cliffhanger, the readers have to really want to know about the characters.
If people like a character, any of the characters enough, they will look for more.
Other than that, a 'sad' happy ending usually does well. A happy ending that everyone knew was coming, and willingly accepting to, but they tried to deny it every step of the way. Take the Magic Tree House as an example, it's a children's book, but the way it's done expresses what I'm talking about. The reader knows that they have to return at the end, but the entire time they want them to stay in that world so far beyond their reach.
Or Harry Potter, everyone was thinking that Harry would win, but the little twists that made people doubt for split seconds at a time is what dragged people in. I'm not a big Game of Thrones fan, I haven't even read the series, but the show abuses this a lot. They make sure you know something is going to happen, then they change your mind for a split second. Before the person has a chance to realize they've been given false hope, it happens! Take Jamie's hand. They made sure people thought that his hand was going to be cut off. They suddenly changed pace for a second, making everyone think it's not happening, then BAM, it happens. GoT does this a lot throughout the series, and it's done so freaking well.