1 of 2
1
Rating people - How best to do it?
Posted: 21 October 2008 05:00 AM   [ Ignore ]  
Research Scientist
Avatar
RankRankRankRankRankRank
Total Posts:  9254
Joined  04-15-2006

Hi there,

First of all this post isn’t aimed towards ExpressionEngine specifically as for starters this may have some licensing issues but I am just asking a general question about a web-app.

Let’s say you had a web-site where people sign up and pay to be listed on the site. These are professionals (not going to say what profession so don’t ask wink - nothing rude though I can promise you that).

You now have say 50 options that the person can choose from to describe their skill set.

This is the problem. Obviously the person could choose the whole list and would then come up in all searches. You then have the problem that probably everyone else on the site would do the same thing so you might as well not have the choices in the first place.

Can anyone think of a way around this at all? I was thinking of either a higher charge to be on the site thus weeding out people who may lie but then they might still be good for the site so that’s not the best option. Perhaps an intermediary person who chooses the skill set for the person based on what they know about them and what the person uploads. This could of course cause problems with the users as they might think they are being marked down and didn’t get a choice in their own skill set shown on the site.

I’ve thought of a few other options too but none really seem to strike a good balance anywhere. Was just wondering if the fantastic collective hive mind of the users in these forums could come up with anything better?

Thanks if you have read this far and have any ideas. Either I’m not thinking straight on this one or it really is a toughie!!

Best wishes,

Mark

 Signature 

Shopping Cart Plugin
Full list of add-ons
———————————————————-
Buy me a drink, or two if you like!!

Profile
 
 
Posted: 21 October 2008 05:25 AM   [ Ignore ]   [ # 1 ]  
Moderator
Avatar
RankRankRankRankRankRankRank
Total Posts:  21123
Joined  05-15-2004

This might no be totally automatic, but just ask them to choose up to 10, or whatever number you deem appropriate.

 Signature 

Everything will be good in the end. If it’s not good, it’s not the end.

Profile
MSG
 
 
Posted: 21 October 2008 06:18 AM   [ Ignore ]   [ # 2 ]  
Research Scientist
Avatar
RankRankRankRankRankRank
Total Posts:  9254
Joined  04-15-2006

Hi Ingmar,

Yep I thought about that one too but then that is limiting for people who really do have the full skill set unfortunately.

I don’t really think there is any other way to do this though so will probably have to look at doing something like you mention. That or something along the lines of they pay a smaller fee to sign up and then pay extra for a set amount of skills / categories / tags etc…

Would really be great to find a way of beating this though as I have a site in mind which if I could do said item it would probably take over the world Muhahahahahahahahaha!

Only kidding about the world part but there is a certain niche market which could really do well from this sort of thing but I just need to be able to instead of offering a good balance somehow beat the balance and tip them in my favour!

Thanks for the thought though. Any others will be very greatly appreciated too!! wink

Best wishes,

Mark

 Signature 

Shopping Cart Plugin
Full list of add-ons
———————————————————-
Buy me a drink, or two if you like!!

Profile
 
 
Posted: 21 October 2008 11:00 AM   [ Ignore ]   [ # 3 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  329
Joined  02-03-2005

What if you let people rate themselves?
Each skill could have a series of radio buttons where they can choose their skill level from no experience to extremely experienced. You still might get people choosing the highest rating for every skill though. Could the skills be broken down into smaller sub-groups? If so you could have them rate their skills within each subgroup relative to one another. For example you could list five complimentary skills together and have them rank each one from most experience to least experience/no experience.

Or, what about a feedback system like ebay, where the people who use their services can come back and rate them. That way there would be a disincentive to lie.

 Signature 

Artificial Memory

Profile
 
 
Posted: 21 October 2008 11:24 AM   [ Ignore ]   [ # 4 ]  
Research Scientist
Avatar
RankRankRankRankRankRank
Total Posts:  9254
Joined  04-15-2006

Hiya,

I think unfortunately letting people rate themselves would still present the same problem no matter how it gets administered. People will just choose that they are the best at everything.

I had thought about letting other people rate the users but then if the user doesn’t agree with the rating then it may be a little unfair seeing as how they are paying to be part of the site in the first place.

Thanks for all the ideas everyone and please do keep them coming! wink I’m sure there must be a way to beat this and I just know that with your fantastic collective minds that something will present itself.

I really do appreciate all the thought processes so far though.

Best wishes,

Mark

 Signature 

Shopping Cart Plugin
Full list of add-ons
———————————————————-
Buy me a drink, or two if you like!!

Profile
 
 
Posted: 21 October 2008 12:27 PM   [ Ignore ]   [ # 5 ]  
Lab Technician
Avatar
RankRankRankRank
Total Posts:  1292
Joined  05-15-2004

If (speaking entirely hypothetically since you don’t say this is the case) your site is meant as a way for these professionals to make contact with prospective clients, then those would-be clients are also your clients. Their willingness to rely on your site is what makes it attractive for the pros to list with you, so their rating of the pros’ skills would be what matters.

If this isn’t what your site is about, never mind. smile

 Signature 

Scribe of Slog
The truth begs nobody’s pardon.

Profile
 
 
Posted: 21 October 2008 01:42 PM   [ Ignore ]   [ # 6 ]  
Research Scientist
Avatar
RankRankRankRankRankRank
Total Posts:  9254
Joined  04-15-2006

Hiya,

Well you’re kind of there! wink

The main problem being that the people searching for the skilled people won’t be paying for access to the site so having them rating people who are paying for a listing might upset said users.

I’m thinking now that maybe we could just charge a lower entry price and then use some tagging whereby the user can ‘purchase’ a certain amount of tags for a certain amount for a year or so. Really the only way I can think it would work. Of course if people want to still then be in every single tag / category then they will have to pay a lot of money so that is up to them.

Best wishes,

Mark

 Signature 

Shopping Cart Plugin
Full list of add-ons
———————————————————-
Buy me a drink, or two if you like!!

Profile
 
 
Posted: 21 October 2008 02:06 PM   [ Ignore ]   [ # 7 ]  
Moderator
Avatar
RankRankRankRank
Total Posts:  2474
Joined  01-07-2008

Why not use a mixed approach.

Let the users rate themselves when they sign up, to get an initial baseline.  The clients then leave feedback on each of the tags, so liars will be discredited quite quickly and prospective clients have something to base their decision on.
That way the paying users feel compelled to give an honest opinion right off the bat, but they aren’t restricted in their choices.

Think ebay, but with a ‘rate yourself’ option in the beginning.

 Signature 
Profile
MSG
 
 
Posted: 21 October 2008 02:09 PM   [ Ignore ]   [ # 8 ]  
Administrator
Avatar
RankRankRankRankRank
Total Posts:  2780
Joined  05-17-2002

This is only a problem if the skills are closely linked. There is no value in a Pro saying “I have all 50 skills” if it doesn’t lead to sales for the Pro. If I say I’m good at X, but its not a service I offer, there is no value to me turning up in the search unless the other skills are closely related.

For example, if the skills are:

jquery
mootools
Yahoo tools

A jquery expert would get a lot of milage in being listed in all 3. But if the skills are:

Java
Flash
.NET

It makes very little sense for a Java pro to be listed with the .NET folks. The chances of them doing it are slim. They’ll want to be listed in all the categories they think they’ll make sales in and that’s it. Through careful category selection you can filter out this problem before it happens.

You can also put a disclaimer that if a person lists more than X skills, they must show that they offer those services (link to a portfolio, or other proof of services offered).

 Signature 
Profile
MSG
 
 
Posted: 21 October 2008 02:31 PM   [ Ignore ]   [ # 9 ]  
Research Scientist
Avatar
RankRankRankRankRankRank
Total Posts:  9254
Joined  04-15-2006

Hi Leslie,

Thanks for the thoughts on this although it’s not quite as straight forward as that unfortunately. I’m not able (at the moment) to say what the site is for but if I can give an example then that might help.

Actually that’s harder than it first sounded when I wrote it!! wink

Will have to give some thought as to an example which I can’t seem to come up with at the moment but trust me when I say that a person could easily list themselves in all the skills as they may well be able to do them but not necessarily brilliantly.

I know I’m not being clear on this which obviously doesn’t help but without saying what the site is for which I’m not allowed to at the moment then that makes it hard for me to come up with a clear example. I will go away and have a think to see if I can come up with a comparison.

Thanks for the thoughts though they are definitely appreciated.

The portfolio part is definitely something that would be on the site but whereas one person may think that the person has the skill another may not think they are that good. Rating would be great but unfortunately I am pretty sure that the client base this will be for (the paying client base that is) would not be happy with that at all.

I’m thinking that the charge per category is probably the best idea as then that should get people to only pay for the things that they are really good at. If they do pay loads for all the extra tags / categories well then that is up to them and there would probably be an ignore member function on the site so people could take them out of search results if they thought the person was leading them up the garden path.

Best wishes,

Mark

 Signature 

Shopping Cart Plugin
Full list of add-ons
———————————————————-
Buy me a drink, or two if you like!!

Profile
 
 
Posted: 21 October 2008 03:50 PM   [ Ignore ]   [ # 10 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  329
Joined  02-03-2005

I’d be careful with the pay per category direction. You run the risk of creating the perception that you are simply a pay-to-play service and not a reputable listing of qualified practitioners, sort of like the web hosting “review” sites that only review the hosts that pay for the service, and they all seem to get good reviews.

I know you’re wary of having the paying clients rated by the end-users, and if the rating is subjective as your last post suggests, maybe it’s best not to do it, but I don’t think the ratings are necessarily a negative, even if people are paying for the listing. As a professional designer/developer, I wouldn’t have a problem with being rated by a client on an industry listing site where I had paid for placement. In fact, I could almost see it as an additional credential, and an incentive for me to do the best possible work, thereby raising my review rating, as well as my credibility in the eyes of potential future customers.

 Signature 

Artificial Memory

Profile
 
 
Posted: 21 October 2008 05:24 PM   [ Ignore ]   [ # 11 ]  
Lab Technician
Avatar
RankRankRankRank
Total Posts:  1232
Joined  01-24-2006

What about:

“Choose 5 primary skills” which are the basis of your search results & “Choose secondary skills (unlimited)” which are excluded from the search results but displayed on the profile page.

 Signature 

Newism - Newcastle Web Design & Development


NSM Publish Plus - Collaborative workflow and live entry previews
LG Better Meta - SEO Meta & XML Sitemap Generator
LG Polls
LG .htaccess Generator
20+ EE addons

Profile
 
 
Posted: 21 October 2008 06:45 PM   [ Ignore ]   [ # 12 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  958
Joined  02-28-2006
Leevi Graham - 21 October 2008 05:24 PM

What about:

“Choose 5 primary skills” which are the basis of your search results & “Choose secondary skills (unlimited)” which are excluded from the search results but displayed on the profile page.

Excellent, funny I didn’t even read Leevi’s and had the same number 5 in mind. The thing is describing their primary skillsets people would tend to pick a unique mix from say the group of 50 if those choices were presented to reduce competition with an exact matching skill criteria. Several Text or custom user defined entries would help define primary skillsets too.

 Signature 

It’s Rock & Roll x4 Bab-ee-ee >:)~

Profile
 
 
Posted: 21 October 2008 08:11 PM   [ Ignore ]   [ # 13 ]  
Lab Technician
Avatar
RankRankRankRank
Total Posts:  1292
Joined  05-15-2004
Mark Bowen - 21 October 2008 01:42 PM

The main problem being that the people searching for the skilled people won’t be paying for access to the site so having them rating people who are paying for a listing might upset said users.

I would be surprised if that were the case. They’re paying for a listing so that clients can find them. The clients’ contribution to your site is to be the thing that attracts the paying listees.

It’s just like how a free newspaper attracts readers to attract paying advertisers. The advertisers understand (or ought to) that the content published in the newspaper isn’t geared toward pleasing them, it’s geared toward getting eyes onto the page, where—oh gee look at that!—there is also a paid ad.

In the case of your site, if the listees want a good rating, they need to please their clients. And since they need to do that anyway with the much smaller number of clients they’ll see without buying a listing, they should be okay with it.

 Signature 

Scribe of Slog
The truth begs nobody’s pardon.

Profile
 
 
Posted: 24 October 2008 05:52 AM   [ Ignore ]   [ # 14 ]  
Lab Assistant
Avatar
RankRank
Total Posts:  149
Joined  01-12-2006

What about if you had X different categories (i.e. skillsets) and users had to select, from one to five, their level of expertise in each. The catch would be that there would be a quota of Y total points, so whilst people could put 5/5 for the first couple of categories they’d they run out of points and have to leave everything else at 0/5. I can’t think of a fairer self-rating system, but it would inevitably still be flawed.

 Signature 

VayaDesign · EE Design & Development Services
EE Goodies: TealTemplate, Simple Pagination plugin, Templates2Files module, Canonical Link Tags plugin

Profile
 
 
Posted: 24 October 2008 06:09 AM   [ Ignore ]   [ # 15 ]  
Research Scientist
Avatar
RankRankRankRankRankRank
Total Posts:  9254
Joined  04-15-2006

Hiya,

Thanks for the thoughts on this. I think my major problem with this is the actual people who I am aiming this at. I’d rather not say what it is for at this stage as everything is just in the ideas stage at the moment but I know that the kind of people who would get listed on the site would probably moan at just about anything I do regarding ratings. I think the problem with rating would be that it could very openly be abused and then end up being no good at all.

People could just down-rate everyone else other than themselves so if everyone ended up doing that then it would very quickly zero itself out. I’m not totally against the idea of rating but I think that I would have to use it as s minor part of the site and not revolve it around it too much. Also if people tended to come up higher in search results on the site due to ratings then I think that would be unfair too.

I think I am going to go with my initial thought which will be to charge slightly less to be listed on the site but then charge per category / tag and go from there.

Really quite a difficult this one as I know the sort of people that will get listed on this and I don’t think they would go for ratings very highly. A bloomin’ difficult one this one wink Thanks for all the ideas everyone though. If you do have any more then please do keep them coming grin

Best wishes,

Mark

 Signature 

Shopping Cart Plugin
Full list of add-ons
———————————————————-
Buy me a drink, or two if you like!!

Profile
 
 
Posted: 24 October 2008 09:06 PM   [ Ignore ]   [ # 16 ]  
Lab Technician
Avatar
RankRankRankRank
Total Posts:  1292
Joined  05-15-2004

Your site must be for public school teachers. <ducks, runs>

 Signature 

Scribe of Slog
The truth begs nobody’s pardon.

Profile
 
 
Posted: 25 October 2008 03:42 AM   [ Ignore ]   [ # 17 ]  
Research Scientist
Avatar
RankRankRankRankRankRank
Total Posts:  9254
Joined  04-15-2006

Still can’t say but nope not school teachers, much worse than that wink

 Signature 

Shopping Cart Plugin
Full list of add-ons
———————————————————-
Buy me a drink, or two if you like!!

Profile
 
 
Posted: 25 October 2008 10:54 AM   [ Ignore ]   [ # 18 ]  
Research Assistant
Avatar
RankRankRank
Total Posts:  786
Joined  02-05-2007

Lawyers?

 Signature 

“I am the terror that flaps in the night”

Profile
 
 
   
1 of 2
1
 
Post Marker Legend
New Topic New posts Hot Topic Hot Topic with new posts New Poll New Poll Moved Topic Moved Topic Sticky Topic Sticky topic
Old Topic No new posts Hot Old Topic Hot Topic with no new posts Old Poll Old Poll Closed Topic Closed Topic Announcement Announcements
Theme
Change Theme
Visitor Statistics
The most visitors ever was 1149, on July 16, 2007 09:33 AM
Total Registered Members: 77570 Total Logged-in Users: 34
Total Topics: 101561 Total Anonymous Users: 19
Total Replies: 544407 Total Guests: 245
Total Posts: 645968    
Members ( View Memberlist )
Newest Members:  tonybernardcarterstarksColeJLinskitnealsemperjrawhallshiusbozzlynobluffkatiejames