08-27-08, 12:14 PM | #21 |
Aye, and a decent critic/recommendation system would solve this. Currently the "weekly pick" system just says to users "these are recently created/updtaed addons we like". But if we have multiple critics with short profiles describing what they look for in an addon then all bases could be covered. One critic may look for low resource use and clean code (like me), another may recommend "pretty" addons, and another might look at how intuitive the addon is to use and configure. If an addon gets a badge from multiple people, that just shows that it's really good.
|
|
08-28-08, 06:44 AM | #22 | |
Author feedback: Sounds good, judging how well the author can provide feedback is interesting. Timeliness of Updates: This one is subjective, during some periods, an author may be able to update his addons with cool new features twice a week.. then take a month or two before releasing somehing new, the addon isn't dead, but life has its load on all of us.. Ease of Use: Sounds good too. Computer CPU/Memory Hogging: Too subjective too.. I don't want to start a debate about this, but everybody has his opinion on what a "bloated" addon is. Some people think that and addon that eats 5mb of memory is fine, some other find it bloated.. but if the addon manages a lot of data, then it's simply normal, and it's not necessarily poorly coded. Each of these categories is just a prone to being manipulated as the current rating system. All in all, I like the current system, and I find it nice to know how many times my addon has been downloaded. It's not a popularity contest, but it's cool to see why we're doing it. For instance, if I were to start a new project, and if only 20 people downloaded it.. then I'd probably stop it quickly. I think that at least authors should be able to have access to this info about their own addons. (same goes for favorites) As for the best rating system, I don't know, really, maybe the best thing is simply to drop it completely, and to turn the site into more than a repository. Promoting addons differently, and maybe more often could be nice. For instance, I could imagine that every week, one "big" addon could be introduced (like omen/x-perl/etc..) and a few smaller ones that answer a specific user need could be put in the spotlight. Thanks for your efforts, no matter what you decide to go with |
||
08-28-08, 08:01 AM | #23 |
Just to add my opinion, a rating system is particularly meaningless when it is completely arbitrary. What is the rating based on in the mind of the rater? Other similar mods? Other mods in general? The concept behind the mod? The utility of the mod? The coding? Without a specific point to the rating, it indeed is merely a popularity contest no matter how you look at it.
And here is a dancing banana. Exceptional! Last edited by Safari : 08-28-08 at 08:03 AM. Reason: Explaining the purpose of the dancing banana. |
|
08-28-08, 12:42 PM | #24 | |
Actually, a thought... we could have something like WoWWiki's featured articles. There's already the "15 minutes of fame" thing, but why not a pool of addons with short reviews, a new one is randomly picked every day and shown on the right side for everyone that visits that day. It could start with a half dozen addons, then more get added over time as reviews get written. Last edited by Tekkub : 08-28-08 at 12:45 PM. |
||
08-28-08, 04:18 PM | #25 | ||
Just because a mod uses a good piece of resources doesn't mean it's a bad mod. It might just be the nature of that mod. Last edited by littlebuddha79 : 08-28-08 at 05:08 PM. Reason: adding a statement that i forgot. |
|||
08-28-08, 07:41 PM | #26 |
The trouble with ratings is a lot of mods can't realy be rated. Eitehr they work, or they don't. I have on mod that all it does is hide the duel frame. There is no ui, no options, no nothing. It's a 1 line LUA. It's one of my favorite mods because I think duels are just about the stupidest thing in WoW (and no, they are not good practice for PVP or Arenas). Never done them, never will. How do I rate that?
I tend to ignore ratings when I look for mods because some of the worst mods I've ever run have had very high ratings, and some of my favorite mods have had very low ratings. My current addon (I wrote a few back before all the LUA changes but those are long gone) is very simple and really doesn't do much, but it has been very useful to me on several occasions, including today when I went to get the chicken egg in Westfall. binding the chicken emote to a key allowed me to spam that emote the 100 or so times it took to get the target chicken to like me. But on the other hand, my addon is really just a slightly more convenient way of spamming emotes than making a macro. Would I rate it 5 stars? Well, for me it's that useful. Do I think it's worth the same rating as Cartographer? Well, no, not even close. Autobar? FuBar? No, those all make fundamental and drastic changes to the way I am able to play the game. I don't even think of them as the same class as little addon hacks like IgnoreDuel or EmoKey or AutoRez. It's almost like there should be different classifications of addons. Hacks, addons, and Meta-addons. (Cartographer, Auctioneer, Bigwigs, etc would be Meta-Addons). |
|
08-28-08, 07:52 PM | #27 |
Just dropping a quick note to let you guys know that we are still reading, listening to what you guys have to say. (I know how frustrating it is for me when I can't tell if the people I'm trying to talk to are even listening to me).
We have all commented that we are very pleased by the response to the thread. You guys are very obviously really thinking about this, you are not only giving us your feedback, but you are explaining the thinking behind your suggestions and comments. Thank you. It's your feedback that helps make the site better for everyone. |
|
08-28-08, 09:40 PM | #28 |
Suggestion
Here are some ideas to revamp the rating system, basically, I did a /braindump, but I hope you can understand me.
First off, a 1-5 rating, as it was in the past, did not really help me, as one seeking addons, know much about it on 1-5 scale? Of awesomeness? Awesomeness is relative. One alternative would be a "Would you recommend this addon to others?" and a "yes" and "X-Mod instead", etc. with optional or required comments. The site could even have respective tabs for this on addon pages, with the recommendations helping people understand their mods. Also, a running ticker that has the category of "Recommendations" could be placed when viewing the WoWInterface Downloads. The problem with the old system was it did not allow one to see how many votes placed at what amount, that averaged all the votes; my suggestion allows everyone to voice their opinion either way, if someone votes it down, and does not give a meaningful comment, then I could see they made it purely for stupid reasons. If someone leaves constructive criticism or comments, then I could better understand their views of a specific addon. Additionally, this could be extended to show the recommendations under the author's profile, so one could view feedback to said author. Also, rating feedback by "Was this comment helpful to you?" yes/no per a user's feedback could order the list of feedback. "Report this feedback as unconstructive?" yes/no could be added as well, allowing other users to help tag feedback that is damaging or degrading, if some number is reached, it could hide the comment from view unless someone wants to click on it to view it, and a further number, an admin could be alerted of it and then remove it if he sees fit. This feedback could also be considered as 'reviews' for the really thick, multi-parigraph comments. Highly ranked reviews could even be feautured. This system is, more or less, a sub-category of the downloads, allowing those that download, to make comments. My vision should require commenters to have a WoWInterface account, and should alert the user if they have not commented on whatever addon. Additionally, each feedbacker could have shown publicly their history how they rated mods, so those who want to understand their comments better. Also, users should be able to update their comments, though this might undermine the feedback on feedback feature; this could be rest whenever someone updates their feedback. A category "User Recommend" could be added for the top recommended addons on the main page, also, this would be yet another way to sort addons, though, should account for total recommendations to time available, with the default available counted as a month, to better prevent new addons from shooting to the top of the results. My idea would be yet another way for the community to get involved. I believe I have thought of a system to prevent fake or unhelpful feedback, though, the problem remains of getting most beyond downloading a file, and to give constructive, helpful criticism of a mod. |
|
08-29-08, 12:54 AM | #29 |
@Tekkub & littlebuddha79: I agree with your answers to my post, I didn't elaborate because as I said, I didn't want to go off-topic as it belongs to another debate.
Come to think about it, maybe such criterias would be worth being displayed if the rating is set by a "committee" of guys who can make an educated judgment on the quality of addon. I'm ok to hear that the quality of my code or of the structure of my db is only worth 3/5, but only if 1) I know why, and 2) that rating is given by someone who really knows how to judge based on measurable criterias. I would hate to be judged on these points by non IT-literate users. I would, however, appreciate the feedback (and possibly rating) of non IT-literate users on things like user friendliness, ease of use, etc... But this feature and the previous one are actually roles that are partially filed by a forum, as these issues are more likely to be addressed by a constructive debate rather than by a simple rating. It's a tough choice to decide how to effectively address this problem if you ask me |
|
08-29-08, 01:30 AM | #30 |
I usually don't look at ratings when I grab mods, I tend to look at descriptions, comments, favorites. If you really want a rating system, then take a look at Slashdots comment rating system.
A group of reviewers might be an idea. Reviewers would be a groups of users (likely users who have been here a while) who review mods then write up reviewed based on if the mod acutally functions, does it do what it says it does, look and feel of the mod, etc. Just my two cent's worth.
__________________
|
|
08-29-08, 06:33 AM | #31 | |
|
||
08-29-08, 06:35 AM | #32 |
If you're trying to avoid 'gaming the system' you're going to fail. The only way to get around that, and it still doesn't always work, is having a lot of voters.
So, you want to encourage ratings, put the number of mods rated on the users posts with their name and join date and avatar. Have occasional and random rewards (but small stuff) for people with a lot of ratings. Feature a "power rater" on the front page every now and then. Maybe a 'star' on their avatar for rating 100 addons. And 1-5 is too small a range. 1-10 at least. Oh, and then when you are actually *ranking* addons, only calculate it based on the votes of people who have at least x votes. Maybe only the top 10% of people or something. SO an addon might have 300 votes and average a 6.7, but only 28 of those votes were from people with a lot of ratings, and the 'weighted' average is 7.1. Display the rating as "7.1 weighted, 6.7 raw" which tells people two things
What do you think, sirs? Last edited by ezarra : 08-29-08 at 07:13 AM. |
|
08-29-08, 06:57 AM | #33 |
Taking into consideration the rating of people with X amount of ratings doesn't really improve much, you can still "cheat" the system, unless you impose restrictions such as "you can only rate X amount of addons per Y amount of time". Ultimately, there simply isn't a perfect system that won't shut out a significant portion of users. Why not keep it simple tbh. Split ratings into categories (as has been suggested) and allow different kinds of peeps access to those particular categories. Authors and generally tech-savy people will be able to rate the "technical" aspects of the addon (code, performance, etc with a requirement to provide reasons for negative reviews, thus actually helping the original author) and pretty much everyone will have access to the more "artistic" categories (functionality, user friendliness, features etc).
|
|
08-29-08, 07:05 AM | #34 | |
Basically, a staff member would have a bunch of "tags" at their disposal, these tags would be things such as "Ease of Use", "Intuitive", "Well Designed", "Responsive Developer", and the likes. The tag system would have a private aside page for the developer, where if an editor didn't award a tag to a developer, they could tell the developer why and offer their advice. For example: "I found feature X in your mod very confusing because description Y seems to imply, perhaps if you changed this to this, it would be easier for new users to understand.". The developer could then act on that and possibly get a tag out of it from that staff member, or ignore it and not get the tag. The only catch-22 with this is that the people who handle tags would have to be very, very carefully picked, they'd really have to be paragons of the community and Cairenn and Dolby would have to watch what they do carefully to ensure that the system is remaining fair and that as little bias as possible (preferably none) is getting mixed in. Then, when a person looks at a mod, they may see something like: Ease of Use x 2 Responsive Developer x 4 What the tags would be and how the advice handed out should be handled should be discussed at length before it's implemented though, there should be a very tight ruleset explaining to everyone handing out these tags what, exactly, merits the awards of a tag and how they should approach those who hadn't earned their tags. A good mod would then be obvious, because it may have a full selection of tags from every staff member. It would mean that the mod is good at doing what it claims to, it's well designed, and most importantly the developer was willing to listen to constructive criticism. It might also be advisable to allow users to approach editors in regards to a mod so that an editor could bring a problem to an author in a more responsible and respectful fashion. I'm not sure whether this is a good system or not... but I just felt that the last time around I explained my thoughts really poorly, so this is stab two at it. |
||
08-29-08, 07:13 AM | #35 |
I like the idea of some form of user feedback/review that allows not only to compliment/critique the AddOn, but provides a sort of "rating". I, along with others here, also agree that the 5-star rating system doesn't really work effectively with the progressive nature of AddOns.
What comes to mind for me, is when I browse a site like NewEgg, where not only is there a rating system, but within that rating system there are "Pro's, Con's, and Comments" fields that allow a buyer to indicate certain aspects they either enjoy or do not enjoy regarding the product. It's excessive to assume a user downloading an AddOn is going to take a large amount to time to write an adequate review for an AddOn. Furthermore, the need to define "adequate" is quite an open-ended thought. The current comment system has, for the most part, acted like a fast "bug-report" or "feature-request" area for users. To me, this is frustrating, since there is a very good Bug/Feature system included in Users portals (which I think are largely ignored...at least from what I've experienced). And while the comment area is helpful in quickly identifying fast-fix issues, it's few-and-far-between that authors see the occassional "good job" or "awesome AddOn" in there compared to the amount of misplaced questions & reports. So to supplement what already exists for user feedback, I think a small "Pro & Con" rating system, easily viewable on the AddOn's main page (or maybe the 2-3 most recent, and an additional tab like the ChangeLog) would be adequate, yet not too demanding for people to quickly fill out their thoughts in regards to the AddOn. I know from my experience (I shop at NewEgg a lot), it can be an invaluable resource. |
|
08-29-08, 07:24 AM | #36 |
I still want to say that there needs to be a quick, simple way for the general user to show their appreciation, or dislike of a certain addon in a way that is quick and simple for other users to see and understand. A 5-star rating system is perfect for this, except for a couple of caveats that come into play:
Like I said before, when I first started using addons, and even now, when I'm looking for something new, it would be imensley helpful to have both a rating that I can see at a quick glance, as well as any reviews, or 'badges' or anything like that, in one place. For instance, if I wanted to find a good map mod, and I'd never used one before or heard of one, the first thing I would do is go to the AddOns page, and the Map category. Right off the bat, I would be confused. There are a ton of mods for maps, and probably 90% perform a fairly small, specfic function. Which is fine, so I would want to try to find the ones that have the best rating, right there. Lets say that there are 2 that have a 5-star rating that I find: Cartographer, and Mapster. I should be able to pull up Cartographer's page, read about what it does and see the screenshots, but other than the 5-star rating, and without trying to find the comments that aren't questions/bug reports/etc, I wouldn't really know why this one has a 5-star rating, and how it compares to Mapster. So I think that there needs to be another tab for each addon that allows users to actually write why they like/dislike an addon - call it 'reviews' or something like that. I also think that the users that have the ability to write addons should have to meet some type of criteria, such as a committee, or x posts in the forum, or whatever. So, the reviews could have a place to give specific ratings: ease of use, simplicity, design, concept, execution, etc, and give a percentage rating: 0% - 100%, which are all averaged to give an addon an overall score. Cartographer might have a 70% ease of use, a 30% simplicity, 80% design, 90% concept, 80% execution for an overall score of 70%. If multiple people review the same addon, their scores would all average together into a number that can be displayed next to the rating, so you would have 2 values to judge an addon on... sort of an overall community ranking, and an analysis by respected members of the community. A system of 'badges' could also work very nicely - an addon that has ever been featured as a Pick of the Week could get a badge that is a link to that PotW's post, so you can quickly see what it said about the addon... you could have a badge that's given out by some people of the community for things like: "Quality Code", "Simple and Clean", "Must Have", etc. Each badge could also have a date built into it, so that you can tell if an addon was a "Must have" 6 months ago, before the Wrath Beta, and now it has a lower rating, since it's not used as much, or something like that. Speaking of... decaying ratings might be a good way to ensure that an addon is currently just as good as it was before, and that no one stuffed the ballot... I'm not sure how that would work exactly, but say I rate an addon today, and give it a 5/5. Someone else rates it next week, and gives it a 1/5, so it shows 3/5 or whatever. In a month, assuming no one else votes, it will show 1/5 - because my rank would have decayed - at least for a week, until the second rank decays. again, it might work, but it would need a lot more planning... |
|
08-29-08, 08:24 AM | #37 |
I posted a while ago about the horrors of an anonymous rating system which doesn't force users to leave a comment with their score - it doesn't help authors OR other users to see what amounts to a 'random number' next to an addon IMO
How a person rates an addon is highly subjective - it all depends what they were looking for and what they like/don't like - their opinions don't, therefore, help others much overall. Similarly, as has already been said, no of downloads or time since last update are not particularly useful ways of finding the best addon either. What I think WOWI needs, more than any rating system, are more ways for people to find addons. At the moment, some categories are quite large and the search is quite limited (to title only I think?) - this makes it hard to find addons. It also makes it hard to upload them into the right place - my simplest addon (NowCarrying) could easily fall into both 'Chat' or 'Inventory' categories for example - many addons really need something like tags or keywords (or full description search?) to help people find them. Another thing I'd like to see is users posting 'reviews' of particular addons/types of addons/UIs etc. Like Amazon, reviewers could be scored on usefulness/given "heat" or "thanks" by people who read them - and from that, addons can become better known - and everyone wins? We'd need a rule that you can't review your own addons of course - and that reviews need to be objective (e.g. not just "this sucks that is dope") - but I think that would be a nice way to get more feedback for addons in general? |
|
08-29-08, 10:12 AM | #38 | |
Especially if you don't tell people exactly what "weighted" means. |
||
08-29-08, 10:29 AM | #39 | |
|
||
08-29-08, 10:52 AM | #40 | |
|
||
WoWInterface » General Discussion » Chit-Chat » Mods Rating System on WoWI |
«
Previous Thread
|
Next Thread
»
|
Display Modes |
Linear Mode |
Switch to Hybrid Mode |
Switch to Threaded Mode |
|
|