Sorting by rating does not make sense

[Sonya*]

I am trying to sort database records by rating.

If I choose Highest Rated, then the number of votes is not considered. See below, the record #1 with only one vote is above the record #3 with 3 votes. Both have only 5-Star votes.

If I choose to sort by Most reviewed, then I have this order (I assume by publish date)

This does not make sense. I have set Allow reviews for database to No. There should not be a sorting option Most reviewed for the databases where reviews are disabled.

Additionally, if I choose Custom and enter this data

After clicking on the Search I have the order like Highest Rated (false, number of votes is still not considered) but also another not translated (translatable) string in the Sort by menu. See below. Both are selected. Bug?

 

Back to the rating: How can I consider not only 5 stars, but also the number of votes? 

Thanks.

 

Edited September 27, 2021 by Sonya*

[Jim M]

Unfortunately, at this time, it is an either or scenario. You can sort by Most reviewed or Highest reviewed. With that said, creating a suggestion in our Feature Suggestion forum may be a good idea here to provide some feedback on that.

With that information out of the way, I cannot reproduce on a default test instance the second half of what you're reporting here when you're sorting by "Most reviewed" and getting the weird sorting results (I am assuming "Votes" are reviews 😄 , if they are not may need to check whatever handles that custom operation).  May wish to try to disable any third party applications/plugins and switch to an unmodified theme then try again. If you're still having issues then, it may be something more specific we'll have to look at.

[Sonya*]

@Jim M, thank you! I have looked into code. There is some mess with rating/reviews code in Pages (at least). I have started a topic in developer connections. Can you (or your developer) take a look at it? 

[Jimi Wikman]

The problem with The Highest Rated is that no matter how you twist and turn things, items with just a single 5-star vote will always outrank any item with many votes the moment that item have any other vote than a 5.

Most votes mean nothing to be honest, since it can be any value entered.

So for a vote to matter, you need to consider not just an average rating, but also the number of ratings in each value. If you have two items that both have the same rating, then count which one has most 5-star ratings. If that is the same, then count 4-star ratings and so on. If they are the same all the way down to the 1-star rating, then the most recent one wins.

How does that sound?

[Marc Stridgen]

8 hours ago, Sonya* said:

@Jim M, thank you! I have looked into code. There is some mess with rating/reviews code in Pages (at least). I have started a topic in developer connections. Can you (or your developer) take a look at it? 

Our developers will take a look through that section occasionally, so I'm sure someone will be able to answer you there. Please bear in mind that this isnt something that would be within the scope of standard support though, so it may take some time until someone looks at that. I would suggest however, also adding that to suggestions if you wish to see something like that as standard in the future

[Sonya*]

58 minutes ago, Jimi Wikman said:

How does that sound?

This sound reasonable, but we do not have a number of single votes of X stars saved in the database. The values we have: average rating, total rating and number of reviews/votes.

Without touching a database, there could be at least two solutions:

Solution #1 can be to use both columns record_average AND num_reviews or rating_hits. This would result in more accurate rating like:

1. 3x5 stars (average 5 with 3 votes) 
2. 2x5 stars (average 5 with 2 votes)
3. 5x4+6x5 (average 4 with 11 votes)
4. 10x4 stars (average 4 with 10 votes)
5. 1x4+2x5 (average 4 with 3 votes)
6. 1x4 stars (average 4 with 1 vote)

Edit: if average rating would be saved as float and not integer, the results would be more accurate, but this requires changes in database.  There is already unused column called rating_real. If we change the column to float and sort by the column AND number of votes/reviews, then we get this:

1. 3x5 stars (average 5.0 with 3 votes) 
2. 2x5 stars (average 5.0 with 2 votes) 
3. 1x4+2x5 (average 4.6 with 3 votes)
4. 5x4+6x5 (average 4.54 with 11 votes) 
5. 10x4 stars (average 4.0 with 10 votes)
6. 1x4 stars (average 4.0 with 1 vote)

Solution #2 can be to calculate ratings based on total stars and votes https://calculator.academy/average-rating-calculator-star-rating/ IPS already calculates the total rating by star ratings in rating_value. Just do the same for reviews. Then divide the total number of stars by 5 or 10 (depending on how the rating system is configured in AdminCP). Save the result in rating_real (the unused column) and sort by it. The example from above would be different.

1. 5x4+6x5 (50/5) = 10
2. 10x4 stars (40/5) = 8
3. 3x5 stars (15/5) = 3 
4. 1x4+2x5 (14/5) = 2,8
5. 2x5 stars (10/5) = 2
6. 1x4 stars (4/5) = 0,8

Scientific solution would be to use true Bayesian estimate.

Quote

The formula for calculating the top tated records with true Bayesian estimate:

rating = (R * v + C * m) / (v + m);

where:

• R = average for the record (mean)
• v = number of votes for the record
• m = minimum votes required to be listed (tunable)
• C = the mean vote across the whole report (average R of every single item in the database)

But this would be an overhead, as we would need to recalculate the whole database every time the vote/review is given, changed or deleted. 😉

 

Edited September 28, 2021 by Sonya*

[Sonya*]

39 minutes ago, Marc Stridgen said:

I would suggest however, also adding that to suggestions if you wish to see something like that as standard in the future

I am not certain if this is a feature or a bug. Sorting by Highest rated without considering the number of votes does not look correct. At least for me 😉 And I am sure for the most users, that wonder why a record with only one 5 stars rating is higher as the record with ten 5 stars rating.

[Marc Stridgen]

5 minutes ago, Sonya* said:

I am not certain if this is a feature or a bug. Sorting by Highest rated without considering the number of votes does not look correct. At least for me 😉 And I am sure for the most users, that wonder why a record with only one 5 stars rating is higher as the record with ten 5 stars rating.

I have flagged this so  that we can get some clarification for you

[Matt]

Sorting by highest rating is only based on the 5 star rating value. The sort feature pre-dates the QA vote mode, so it's not take into consideration but I agree that it would be a good change to make it so.

[Sonya*]

@Matt, there is also a mismatch with sorting by Most Reviewed in applications/cms/modules/front/database/category.php

if ( !$database->options['reviews'] and !$category->allow_rating )
			{
				unset ( $table->sortOptions['num_reviews'] );
				unset( $table->sortOptions['rating'] );
			}

While options['reviews'] is about reviews, allow_rating is about star rating. This way I have Most Reviewed option even if I only allow star ratings. This option sorts by record_reviews. The column is always 0, if only star rating is enabled. So, the option does not make sense if reviews are disabled. Or it should sort by rating_hits instead.

Edited September 30, 2021 by Sonya*