Monday, July 4, 2011

Understanding wages

As an introduction, I just want to address something requester side I learned:

1.  Requester wage per hour

This is calculated from time worker accepts hit to submits hit.  It seems like it should be a good benchmark and be accurate but it isn't, especially for quicker tasks.  The problem is it doesn't consider all the downtime workers face.  Downtimes include the time it takes for a submit to finish on mTurk, the time it takes (lag) to be able to accept the hit and the time for a captcha.  As a real example:

* Time it takes for a hit:  6 seconds.  $.02 a hit, $.20 a minute, $12 an hour.
* Submit/accept downtime:  4 seconds.  10 seconds total for a hit, $.12 a minute, $7.20 an hour
* Captcha downtime:  35 seconds (rounding for ease).  That's 1 second a hit, so 11 seconds per hit.  That's about $6.50 an hour.

A few thoughts about this:

a.  Starting wage on requester side is well above fair.
b.  The downtime is relatively not bad.  No lost time loading pics and whatnot.
c.  The endwage, at this rate, isn't too bad.

*** Workers

1.  Wage per hour.

For fast tasks do a captcha's worth of hits.  That is 35 hits.  Record the time from the hit after a captcha and do hits until you enter a captcha and back on track.  Divide this time into 60 minutes to see how many captchas you do per hour, and multiply by the amount you made during that time.  Finally, multiply by your acceptance rate with the requester.  The larger the sample size the more accurate this will be. Using one of my reqeusters as an example:

a.  5 minutes for a captcha.  That's 12 captchas per hour.
b.  30 hits submitted per captcha (so about 5 hits returned/abandoned)
c.  $.02 per hit
d.  99.3% approval rating with requester

30 hits * $.02 = $.60 per captcha * 12 captchas per hour = $7.20/hour * 99.3% = $7.15 per hour.

For longer duration hits you'll just have to figure out approximately how long it takes to do a hit while considering the time you take finding/cherry picking the hit, researching it, etc, before accepting the hit.  Do this a few times in even out if one hit is harder to research first.

2.  Benefit of low reward hits

The wage per hour should be the #1 consideration in regards to payment.  If you can do a $.01 hit in 5 seconds that's $7.20 an hour.  There are a few benefits of this:

1.  Rejects don't hurt as much, though mass rejects can hurt if you do a thousand hits at once and get them all rejected.
2.  When you find hits, they are typically mindless and hard to mess up unless you just aren't paying attention.  There are typically not a lot of super hard directions.  Some hits may seem complicated at first, though, but once you understand what you are doing it's simple.
3.  Shaving time off per hit has a higher wage increase than slower hits.  If you can shave off 1 second for a 10 second hit thats a 10% pay increase.  1 second off a 5 second hit is a 20% pay increase.  1 second off a 20 minute hit isn't even worth figuring out.
4.  High approval low paying hits can give you a cushion to get rejections in high paying hits that sometimes gets rejections.  The problem with some high paying hits is if you do 6 transcription hits a day and 1 gets rejected, your approval rating suffers at 83%.  If you do the same 6 transcription hits and 100 fast high approval hits you are now looking at maybe 104 approved out of 106 hits = 98%.  That should keep you to dong any hit you might ever want to do.  Sticking with the 5/6 transcription hits will soon prevent you from doing hits and may put your account in jeopardy.

3.  Benefit of high paying hits

A lot of higher paying hits has a qualification type structure which is good. You start working for a requester and they give you a certain qualification which changes based on how well you do working on hits.  The good thing about this is it's potentially better paying work available for higher qualified workers.

The problem with these hits, though, is you are working on a hit for 40 minutes and get rejected, that's a very big hit for you as opposed to getting rejected on a 5 second task.  When calculating if these hits pay off for you you should remember to look at your rejection rate.  If 1 in 5 hits gets rejected, that's 20%.  So 5/4 = 1.25 the value of the hit to make up for your rejection.

A quick example is a $6 45 minute hit.  That's $8 an hour, which is better than that $.01 5 second hit that paid $7.15 an hour.  Problem is, if you have a 20% rejection rate (1 in 5) that's $6.40 an hour, which pays less than that penny hit.  That rejection really hurt.  To make up for that loss, you'd need to multiply the reward by rejection.

$6 reward * 5/4 = $7.50 reward.  If that hit took 45 minutes $7.50 * 60/45 = $10/hour.  That covers the rejection rate.  As you get less rejections on these hits it will take less money to make it more profitable.

If you only look at the reward amount you miss the bigger picture.  The bigger picture is you should be doing work that puts the most money in your pocket per hour.

4.  Should you ever do low paying hits?

No.  Low paying hits is low paying per hour.  There are workers willing to work for $.60 an hour simply because the requester doesn't reject and pays in a day.  This is no reason to ever do that kind of work.  Even if you are trying to rebuild your stats you shouldn't work for that cheap unless there is simply no work available to you.

Problem is if these hits get down the requester may decrease the pay or increase the amount of work required per hit or both.  Not only this but other requesters may look at those hits and decide to price their hits similarly.  This drives down wages across the board as there are workers willing to work for $.60 an hour.

No comments:

Post a Comment