This is a general posting for problems regarding mTurk masters. This post is directed towards potential requesters, amazon, and workers. Please note the first issue while reading this.
1. mTurk masters are unknown. Meaning that there is no documentation on what a worker needs to go about to get this qualification. There is no list of who the current masters are, how they became masters, or any evidence that new masters are getting the qualification and existing masters are losing the qualification for not keeping up with whatever it is that they did to get the qualification.
2. It is doubtful masters earned the qualification. This very point makes current qualification holders super defensive because they were probably brainwashed into thinking they are super special. If you read about masters amazon promotes them as doing thousands of hits from multiple requesters. There are workers, myself included, who aim for 1,000 hits a day. My approval rating is 99.3% (most of my work comes from majority rules requester) and over 120k approved hits. I do work for fair paying requesters who pay fast and are responsive. There are workers with much better stats then mine with approval ratings upwards of 99.9%, been turking longer, and more approved. They don't have it.
3. Masters were most likely given this qualification based on some random hits several months ago. This part, like everything else, simply unknown. Current masters don't know how they were chosen but they want to believe they are special. Non masters simply point out their stats are better than current masters and they aren't masters. As of this writing, it was hinted that amazon might extend the qual to more workers, but there is no evidence that any new worker ever received this qual yet. Also, the only evidence is that current masters were the ones chosen for a beta started months ago. Nobody knows who was chosen. It was suggested that working for specific requesters granted the qual. However, shouldn't all photo mod requesters be considered for their workers? And some current masters admitted they didn't do work for the requesters that was listed as being watched. This suggests there was some random alpha testing done and the people selected to beta did some alpha hits that are unknown.
4. If legit, you are getting old workers. As stated, there is no evidence that more workers were given this qualification, nor is there evidence existing workers were removed from the system. There is no hierarchy to show who the best of the best is so workers who want the qual can aim for it. So, if we assume that all the current masters actually earned it, they were the "best of the best" about 4 months ago. Wouldn't you want the best of the best today? Chances are there are probably enough scamming workers who got the qual because they'll do any sort of hits while sharing an account. This, of course, isn't saying all master workers are scammers. But as there are no checks and balances you really don't know what you are getting.
5. Masters aren't paid more. Someone suggested that during beta amazon made reference that the increased cost to requesters would be shared with workers. It sounds like this isn't the case. Why should it be? If you want to pay workers more you should pay workers more.
6. Masters won't work for peanuts. It sounds like there are some tasks that have higher requirements because the requester is paying more. If your hits are subjective, uses majority rules and pays bad people won't touch your hits. This is amazons fault. After all, if you are paying amazon more you'll want to make sure your work is being done properly. But you aren't paying for any additional benefits from amazon and, in fact, you are preventing workers that want to do your hits from doing them because they weren't given this imaginary qualification.
7. Masters get a human response when suspended. Why? Shouldn't this be given to all workers? Amazon should hand check workers accounts before suspending them but they won't even do that for master accounts they suspend. Non masters will only ever get form letters from amazon that doesn't say anything. As a worker I'd pay for this kind of service for workers. My account shouldn't be at risk because of scamming requesters and it's not fair that a human being won't review my account if I'm not a master.
8. What is the logic of giving any work to masters? I'm reminded of a Canadian show called "Endgame". It's about a chess world champion (btw that doesn't know how to castle) who is hired to help solve crimes. This guy is supposed to be really good at chess but what makes him qualified to solve crimes? What makes someone super-duper skilled at counting people in pictures qualified to write articles or do transcription?
9. Masters don't have to type captchas. The biggest benefit for masters is having a human being respond to suspensions. Next is not having to type in captchas. When I do 1,000 hits a day I'm having to prove every 6-8 minutes I'm a human being. How is this fair? I don't mind because if it helps against spamming workers I don't mind. But is it really?
*** Conclusions
Amazon:
1. All workers deserve to have someone respond to them when suspended. Stop with the friday suspensions so that you don't have to deal with workers over the weekend. If it costs more money to hire people for this then charge requesters a percentage more to cover the costs.
2. Captchas suck. Figure out a better way to stop scammers.
3. Make these qualifications something people can shoot for. If I were to hire a piano tutor for my child I'd want some proof they are actually good besides someone said so. My mother hired a cleaning lady that came recommended by someone. That cleaning lady ended up stealing everything from stuff in the garage, money from my brothers funeral, to drugs to fight my moms cancer. While different, master turkers should actually be the best of the best and not hide behind some imaginary algorithm.
4. Don't allow requesters to just give any old work to master turkers. That defeats the purpose.
Requesters:
1. If you want to pay more to workers, pay them more.
2. The added cost to amazon doesn't give you anything special.
3. Your best bet is figure out how to set up qualifications, set them up, and give better scores to your better requesters. You should want to find workers who do your work as best as they can and not pay more because amazon claims something.
Workers:
Most likely you aren't special. You did some hit months ago and amazon brainwashed you into thinking you are special. You are not. They just needed these "master turkers" to sell a lie to requesters in hopes that they'll pay amazon a premium. As a chess player I know exactly where I stand in world rankings. If I wanted to become a grandmaster I know who I have to beat and beat them. This should be a very simple solution and actually means something if amazon did this right. I could tell amazon how to do it but they wouldn't listen because they don't care to find out who their best workers are. All they are interested in is labeling people to claim they are the best so they can profit off the lies.
All this said, I'm not saying anyone with a master turk qual are bad workers, scammers, or anything. And if it ever becomes available how people earned these qualifications I'll eat my hat.
Monday, July 11, 2011
Saturday, July 9, 2011
Managing your time while turking
These aren't hard fast rules but more of a way to consider your time. You may want to include things or not do certain things. Also, percentages will change based on your daily/weekly goals, or simply changing things.
1. Main money maker - 80%. That's my goal. That means 80% of the time I'm turking I'm doing something that is my main money maker. I know the requesters, I know what is expected, I know how much I'm making and I hope I'm focusing my efforts to turking and not not getting side tracked reading wikipedia or something. During this time I might think about keyboard shortcuts or other methods to complete the hits faster. In the short run I might lose some of my hourly wages, but I'm still looking at long term. If I think of something today to improve my productivity I might be able to earn more tomorrow or next week, etc.
2. Researching new requesters - 15%. Again, my goal. I'm happy with the requesters I do work for, however, sometimes I find new requesters. I might research them on TO or forums, or simply looking at hits available and seeing which ones I think I could make decent enough money on to do them. This doesn't mean I find a new requester a day. It may take a couple weeks or longer, and even then, it might just be a certain hit offered by the requester I think I could do. These hits I'm not as concerned about hourly wages, though it's still important. The theory simply is I can't find new requesters or hits if I don't take chances. I've found several quality requesters this way by doing hits that didn't seem to pay at first but after doing a handful of them I realized they are well worth it.
3. Improving stats - 5%. My personal final goal. Problem is, I'm happy with my stats. The only stats I'm not happy with is my abandoned and returned rate. But I can't improve these stats much since most of the hits I do is majority rules.
I can get 350 rejections without an approval to stay above 99%. Amazon mTurk doesn't go by tenths so only reason to try to get higher than 99.3% is to have a bigger cushion. A bigger cushion allows me to do more test hits for new requesters without worrying about my stats hurting. My submitted rate is 86.32%, which I'm not happy with. It doesn't prevent me from doing work, but I'd like to see that number go higher. Problem is, since I do mostly majority rules hits I return any hits I'm not sure about. I'd rather this stat not be great rather than not give the requester my best work I can do.
1. Main money maker - 80%. That's my goal. That means 80% of the time I'm turking I'm doing something that is my main money maker. I know the requesters, I know what is expected, I know how much I'm making and I hope I'm focusing my efforts to turking and not not getting side tracked reading wikipedia or something. During this time I might think about keyboard shortcuts or other methods to complete the hits faster. In the short run I might lose some of my hourly wages, but I'm still looking at long term. If I think of something today to improve my productivity I might be able to earn more tomorrow or next week, etc.
2. Researching new requesters - 15%. Again, my goal. I'm happy with the requesters I do work for, however, sometimes I find new requesters. I might research them on TO or forums, or simply looking at hits available and seeing which ones I think I could make decent enough money on to do them. This doesn't mean I find a new requester a day. It may take a couple weeks or longer, and even then, it might just be a certain hit offered by the requester I think I could do. These hits I'm not as concerned about hourly wages, though it's still important. The theory simply is I can't find new requesters or hits if I don't take chances. I've found several quality requesters this way by doing hits that didn't seem to pay at first but after doing a handful of them I realized they are well worth it.
3. Improving stats - 5%. My personal final goal. Problem is, I'm happy with my stats. The only stats I'm not happy with is my abandoned and returned rate. But I can't improve these stats much since most of the hits I do is majority rules.
I can get 350 rejections without an approval to stay above 99%. Amazon mTurk doesn't go by tenths so only reason to try to get higher than 99.3% is to have a bigger cushion. A bigger cushion allows me to do more test hits for new requesters without worrying about my stats hurting. My submitted rate is 86.32%, which I'm not happy with. It doesn't prevent me from doing work, but I'd like to see that number go higher. Problem is, since I do mostly majority rules hits I return any hits I'm not sure about. I'd rather this stat not be great rather than not give the requester my best work I can do.
Monday, July 4, 2011
Building a relationship with a requester
This post is designed as a step-by-step working for a requester you never heard of until working for them for years.
1. Most important thing is look for hits you can do and enjoy doing.
Even if the hit pays extremely well and the requester is a great requester, if it's not the type of hit you want to do it shouldn't be a consideration. Some people don't write well so they shouldn't attempt writing hits. Some don't dig transcription so they shouldn't. If you're at work or home with small children you probably don't want to do anything that might have adult content.
2. Research Requester.
Don't be shy about spending too much time researching a requester. A year down the road you won't be working for 50+ requesters, but more like about a dozen. So it's a long term investment researching requesters. It may take a few hours today and you might or might not find a new requester. And that's fine so long as you keep good books and remember who you've researched and why you said yes or no. It might also be worth to write on a notepad why you made a certain choice. You may find out that your initial opinion was unfounded or things changed.
The best place to research a requester is to use Turkopticon. At the time of this post they are changing things a bit and making things better, so hopefully it will be better. Currently I wouldn't pay much attention to the scores. I'll save the reasons for a different post. Read all the comments and make up your own decision. You may find a requester others don't like and vice versa. The "scare" words you should look out for is "block", "mass rejection", etc. These are terms that can affect your account so it typically isn't worth is.
Another place to research requesters is forums. Some people prefer forums since it's more of a discussion. Problem is if the moderator is ban happy they may ban users that don't agree with them in some other aspect. Another problem is a thread might be 100s of posts, most of them pretty useless. The benefit, though, is you can see who is posting what, and if you know who the poster is it might give you more of a reason to trust their opinion over someone else.
3. Doing a hit
a. Read the directions. Understand what it is the requester is wanting you to do, and start doing the hits.
b. After about a half dozen hits, reread the directions. This is simply just to make sure you are doing it right now that you can put the hit to the directions. Last thing you want is to do too many and realize you forgot an essential step. This happens.
c. After about a captcha's worth of hits - 35 hits - read the directions again. At this point you are going by the motions. Like step 2 you just want to make sure that there isn't something super minor you glanced over and aren't doing.
4. Communicating with a reqeuster
Maybe you found out that you missed some step. As soon as you know you should contact the requester and admit your error. When I do this I ask the requester to reject the hits I did wrong. I'd rather have more rejections rather than a block on my account. At this point you should stop working on the hits. You don't want to get even more rejections if the requester mass rejects you. If you do get rejects remember you were the one that did something wrong so don't be mad at the requesters.
Sometimes for a brand new requester I'll email them asking some question I might have just to see if they respond and how quickly. If they respond while the hit is still up it's probably safe to assume they are good to work for and won't mass reject you, though that's always a possibility.
5. Minimizing your losses
Many people have a similar story "I got mass rejected 200 hits, now my approval rating is 89% and I can't find any work". Maybe you didn't deserve the rejections, but it's partially your fault for doing this many hits for an unknown requester.
You should know the percentage required for all hits you normally do and you should never have more pending than what will drop you below that number.
Lets say you have 30 rejections currently and 1200 hits. Also let's say the highest requester is 95%.
a. Simply divide your approval rating by your highest requester. 1200/.95. This will give you a number of total hits that will give you 95% if you don't get an accepted. 1200/.95 = 1263.
b. Subtract a from your approved and rejected. 1263 - (1200 + 30) = 33. At max you should only do 33 hits from an unknown requester. You may not even want to do that many. So alter the goals by .99, .98, .97, .96. For this example you won't consider .95 because you shouldn't risk your ratings that much and prevent you from doing work for your normal requesters.
Once these hits have been approved you can begin to wok on the negative side - that is, 94%, 93% etc. Depending on how safe you want to be should tell you what to reject. Next set you may want to shave off a percentage or 2, or if you want to risk more you can do 90%. You shouldn't assume that since the first 33 was paid you can now do 1,000. You should gradually build up to these numbers.
If the requester is well known with great feedback, you should still limit the first batch just to make sure you are doing it correctly. Also, just because you don't get rejections doesn't mean you are doing it correctly. That's why you need to build up towards numbers.
6. Even after working for months for this requester, you still need to watch your numbers. Requester may decide to mass reject you to avoid paying you. It happens. They also may decide to mass reject and block you because you aren't meeting some computerized quota or something. Again, this happens. You'd think after working for a requester for a year that this won't happen. There are stories about requesters purposely rejecting workers after having a long relationship. This may be due to hitting the $600 mark. It may just be they want fresh eyes and just don't want you working for them.
1. Most important thing is look for hits you can do and enjoy doing.
Even if the hit pays extremely well and the requester is a great requester, if it's not the type of hit you want to do it shouldn't be a consideration. Some people don't write well so they shouldn't attempt writing hits. Some don't dig transcription so they shouldn't. If you're at work or home with small children you probably don't want to do anything that might have adult content.
2. Research Requester.
Don't be shy about spending too much time researching a requester. A year down the road you won't be working for 50+ requesters, but more like about a dozen. So it's a long term investment researching requesters. It may take a few hours today and you might or might not find a new requester. And that's fine so long as you keep good books and remember who you've researched and why you said yes or no. It might also be worth to write on a notepad why you made a certain choice. You may find out that your initial opinion was unfounded or things changed.
The best place to research a requester is to use Turkopticon. At the time of this post they are changing things a bit and making things better, so hopefully it will be better. Currently I wouldn't pay much attention to the scores. I'll save the reasons for a different post. Read all the comments and make up your own decision. You may find a requester others don't like and vice versa. The "scare" words you should look out for is "block", "mass rejection", etc. These are terms that can affect your account so it typically isn't worth is.
Another place to research requesters is forums. Some people prefer forums since it's more of a discussion. Problem is if the moderator is ban happy they may ban users that don't agree with them in some other aspect. Another problem is a thread might be 100s of posts, most of them pretty useless. The benefit, though, is you can see who is posting what, and if you know who the poster is it might give you more of a reason to trust their opinion over someone else.
3. Doing a hit
a. Read the directions. Understand what it is the requester is wanting you to do, and start doing the hits.
b. After about a half dozen hits, reread the directions. This is simply just to make sure you are doing it right now that you can put the hit to the directions. Last thing you want is to do too many and realize you forgot an essential step. This happens.
c. After about a captcha's worth of hits - 35 hits - read the directions again. At this point you are going by the motions. Like step 2 you just want to make sure that there isn't something super minor you glanced over and aren't doing.
4. Communicating with a reqeuster
Maybe you found out that you missed some step. As soon as you know you should contact the requester and admit your error. When I do this I ask the requester to reject the hits I did wrong. I'd rather have more rejections rather than a block on my account. At this point you should stop working on the hits. You don't want to get even more rejections if the requester mass rejects you. If you do get rejects remember you were the one that did something wrong so don't be mad at the requesters.
Sometimes for a brand new requester I'll email them asking some question I might have just to see if they respond and how quickly. If they respond while the hit is still up it's probably safe to assume they are good to work for and won't mass reject you, though that's always a possibility.
5. Minimizing your losses
Many people have a similar story "I got mass rejected 200 hits, now my approval rating is 89% and I can't find any work". Maybe you didn't deserve the rejections, but it's partially your fault for doing this many hits for an unknown requester.
You should know the percentage required for all hits you normally do and you should never have more pending than what will drop you below that number.
Lets say you have 30 rejections currently and 1200 hits. Also let's say the highest requester is 95%.
a. Simply divide your approval rating by your highest requester. 1200/.95. This will give you a number of total hits that will give you 95% if you don't get an accepted. 1200/.95 = 1263.
b. Subtract a from your approved and rejected. 1263 - (1200 + 30) = 33. At max you should only do 33 hits from an unknown requester. You may not even want to do that many. So alter the goals by .99, .98, .97, .96. For this example you won't consider .95 because you shouldn't risk your ratings that much and prevent you from doing work for your normal requesters.
Once these hits have been approved you can begin to wok on the negative side - that is, 94%, 93% etc. Depending on how safe you want to be should tell you what to reject. Next set you may want to shave off a percentage or 2, or if you want to risk more you can do 90%. You shouldn't assume that since the first 33 was paid you can now do 1,000. You should gradually build up to these numbers.
If the requester is well known with great feedback, you should still limit the first batch just to make sure you are doing it correctly. Also, just because you don't get rejections doesn't mean you are doing it correctly. That's why you need to build up towards numbers.
6. Even after working for months for this requester, you still need to watch your numbers. Requester may decide to mass reject you to avoid paying you. It happens. They also may decide to mass reject and block you because you aren't meeting some computerized quota or something. Again, this happens. You'd think after working for a requester for a year that this won't happen. There are stories about requesters purposely rejecting workers after having a long relationship. This may be due to hitting the $600 mark. It may just be they want fresh eyes and just don't want you working for them.
Understanding wages
As an introduction, I just want to address something requester side I learned:
1. Requester wage per hour
This is calculated from time worker accepts hit to submits hit. It seems like it should be a good benchmark and be accurate but it isn't, especially for quicker tasks. The problem is it doesn't consider all the downtime workers face. Downtimes include the time it takes for a submit to finish on mTurk, the time it takes (lag) to be able to accept the hit and the time for a captcha. As a real example:
* Time it takes for a hit: 6 seconds. $.02 a hit, $.20 a minute, $12 an hour.
* Submit/accept downtime: 4 seconds. 10 seconds total for a hit, $.12 a minute, $7.20 an hour
* Captcha downtime: 35 seconds (rounding for ease). That's 1 second a hit, so 11 seconds per hit. That's about $6.50 an hour.
A few thoughts about this:
a. Starting wage on requester side is well above fair.
b. The downtime is relatively not bad. No lost time loading pics and whatnot.
c. The endwage, at this rate, isn't too bad.
*** Workers
1. Wage per hour.
For fast tasks do a captcha's worth of hits. That is 35 hits. Record the time from the hit after a captcha and do hits until you enter a captcha and back on track. Divide this time into 60 minutes to see how many captchas you do per hour, and multiply by the amount you made during that time. Finally, multiply by your acceptance rate with the requester. The larger the sample size the more accurate this will be. Using one of my reqeusters as an example:
a. 5 minutes for a captcha. That's 12 captchas per hour.
b. 30 hits submitted per captcha (so about 5 hits returned/abandoned)
c. $.02 per hit
d. 99.3% approval rating with requester
30 hits * $.02 = $.60 per captcha * 12 captchas per hour = $7.20/hour * 99.3% = $7.15 per hour.
For longer duration hits you'll just have to figure out approximately how long it takes to do a hit while considering the time you take finding/cherry picking the hit, researching it, etc, before accepting the hit. Do this a few times in even out if one hit is harder to research first.
2. Benefit of low reward hits
The wage per hour should be the #1 consideration in regards to payment. If you can do a $.01 hit in 5 seconds that's $7.20 an hour. There are a few benefits of this:
1. Rejects don't hurt as much, though mass rejects can hurt if you do a thousand hits at once and get them all rejected.
2. When you find hits, they are typically mindless and hard to mess up unless you just aren't paying attention. There are typically not a lot of super hard directions. Some hits may seem complicated at first, though, but once you understand what you are doing it's simple.
3. Shaving time off per hit has a higher wage increase than slower hits. If you can shave off 1 second for a 10 second hit thats a 10% pay increase. 1 second off a 5 second hit is a 20% pay increase. 1 second off a 20 minute hit isn't even worth figuring out.
4. High approval low paying hits can give you a cushion to get rejections in high paying hits that sometimes gets rejections. The problem with some high paying hits is if you do 6 transcription hits a day and 1 gets rejected, your approval rating suffers at 83%. If you do the same 6 transcription hits and 100 fast high approval hits you are now looking at maybe 104 approved out of 106 hits = 98%. That should keep you to dong any hit you might ever want to do. Sticking with the 5/6 transcription hits will soon prevent you from doing hits and may put your account in jeopardy.
3. Benefit of high paying hits
A lot of higher paying hits has a qualification type structure which is good. You start working for a requester and they give you a certain qualification which changes based on how well you do working on hits. The good thing about this is it's potentially better paying work available for higher qualified workers.
The problem with these hits, though, is you are working on a hit for 40 minutes and get rejected, that's a very big hit for you as opposed to getting rejected on a 5 second task. When calculating if these hits pay off for you you should remember to look at your rejection rate. If 1 in 5 hits gets rejected, that's 20%. So 5/4 = 1.25 the value of the hit to make up for your rejection.
A quick example is a $6 45 minute hit. That's $8 an hour, which is better than that $.01 5 second hit that paid $7.15 an hour. Problem is, if you have a 20% rejection rate (1 in 5) that's $6.40 an hour, which pays less than that penny hit. That rejection really hurt. To make up for that loss, you'd need to multiply the reward by rejection.
$6 reward * 5/4 = $7.50 reward. If that hit took 45 minutes $7.50 * 60/45 = $10/hour. That covers the rejection rate. As you get less rejections on these hits it will take less money to make it more profitable.
If you only look at the reward amount you miss the bigger picture. The bigger picture is you should be doing work that puts the most money in your pocket per hour.
4. Should you ever do low paying hits?
No. Low paying hits is low paying per hour. There are workers willing to work for $.60 an hour simply because the requester doesn't reject and pays in a day. This is no reason to ever do that kind of work. Even if you are trying to rebuild your stats you shouldn't work for that cheap unless there is simply no work available to you.
Problem is if these hits get down the requester may decrease the pay or increase the amount of work required per hit or both. Not only this but other requesters may look at those hits and decide to price their hits similarly. This drives down wages across the board as there are workers willing to work for $.60 an hour.
1. Requester wage per hour
This is calculated from time worker accepts hit to submits hit. It seems like it should be a good benchmark and be accurate but it isn't, especially for quicker tasks. The problem is it doesn't consider all the downtime workers face. Downtimes include the time it takes for a submit to finish on mTurk, the time it takes (lag) to be able to accept the hit and the time for a captcha. As a real example:
* Time it takes for a hit: 6 seconds. $.02 a hit, $.20 a minute, $12 an hour.
* Submit/accept downtime: 4 seconds. 10 seconds total for a hit, $.12 a minute, $7.20 an hour
* Captcha downtime: 35 seconds (rounding for ease). That's 1 second a hit, so 11 seconds per hit. That's about $6.50 an hour.
A few thoughts about this:
a. Starting wage on requester side is well above fair.
b. The downtime is relatively not bad. No lost time loading pics and whatnot.
c. The endwage, at this rate, isn't too bad.
*** Workers
1. Wage per hour.
For fast tasks do a captcha's worth of hits. That is 35 hits. Record the time from the hit after a captcha and do hits until you enter a captcha and back on track. Divide this time into 60 minutes to see how many captchas you do per hour, and multiply by the amount you made during that time. Finally, multiply by your acceptance rate with the requester. The larger the sample size the more accurate this will be. Using one of my reqeusters as an example:
a. 5 minutes for a captcha. That's 12 captchas per hour.
b. 30 hits submitted per captcha (so about 5 hits returned/abandoned)
c. $.02 per hit
d. 99.3% approval rating with requester
30 hits * $.02 = $.60 per captcha * 12 captchas per hour = $7.20/hour * 99.3% = $7.15 per hour.
For longer duration hits you'll just have to figure out approximately how long it takes to do a hit while considering the time you take finding/cherry picking the hit, researching it, etc, before accepting the hit. Do this a few times in even out if one hit is harder to research first.
2. Benefit of low reward hits
The wage per hour should be the #1 consideration in regards to payment. If you can do a $.01 hit in 5 seconds that's $7.20 an hour. There are a few benefits of this:
1. Rejects don't hurt as much, though mass rejects can hurt if you do a thousand hits at once and get them all rejected.
2. When you find hits, they are typically mindless and hard to mess up unless you just aren't paying attention. There are typically not a lot of super hard directions. Some hits may seem complicated at first, though, but once you understand what you are doing it's simple.
3. Shaving time off per hit has a higher wage increase than slower hits. If you can shave off 1 second for a 10 second hit thats a 10% pay increase. 1 second off a 5 second hit is a 20% pay increase. 1 second off a 20 minute hit isn't even worth figuring out.
4. High approval low paying hits can give you a cushion to get rejections in high paying hits that sometimes gets rejections. The problem with some high paying hits is if you do 6 transcription hits a day and 1 gets rejected, your approval rating suffers at 83%. If you do the same 6 transcription hits and 100 fast high approval hits you are now looking at maybe 104 approved out of 106 hits = 98%. That should keep you to dong any hit you might ever want to do. Sticking with the 5/6 transcription hits will soon prevent you from doing hits and may put your account in jeopardy.
3. Benefit of high paying hits
A lot of higher paying hits has a qualification type structure which is good. You start working for a requester and they give you a certain qualification which changes based on how well you do working on hits. The good thing about this is it's potentially better paying work available for higher qualified workers.
The problem with these hits, though, is you are working on a hit for 40 minutes and get rejected, that's a very big hit for you as opposed to getting rejected on a 5 second task. When calculating if these hits pay off for you you should remember to look at your rejection rate. If 1 in 5 hits gets rejected, that's 20%. So 5/4 = 1.25 the value of the hit to make up for your rejection.
A quick example is a $6 45 minute hit. That's $8 an hour, which is better than that $.01 5 second hit that paid $7.15 an hour. Problem is, if you have a 20% rejection rate (1 in 5) that's $6.40 an hour, which pays less than that penny hit. That rejection really hurt. To make up for that loss, you'd need to multiply the reward by rejection.
$6 reward * 5/4 = $7.50 reward. If that hit took 45 minutes $7.50 * 60/45 = $10/hour. That covers the rejection rate. As you get less rejections on these hits it will take less money to make it more profitable.
If you only look at the reward amount you miss the bigger picture. The bigger picture is you should be doing work that puts the most money in your pocket per hour.
4. Should you ever do low paying hits?
No. Low paying hits is low paying per hour. There are workers willing to work for $.60 an hour simply because the requester doesn't reject and pays in a day. This is no reason to ever do that kind of work. Even if you are trying to rebuild your stats you shouldn't work for that cheap unless there is simply no work available to you.
Problem is if these hits get down the requester may decrease the pay or increase the amount of work required per hit or both. Not only this but other requesters may look at those hits and decide to price their hits similarly. This drives down wages across the board as there are workers willing to work for $.60 an hour.
Friday, July 1, 2011
How to sort search
The 3 most popular ways to sort are:
1. Hit Creation Date - Newest First
The benefit for this sort is 2 fold. Some requesters will post a few hits at a time but will continue posting hits for hours or all day, depending on the requester. Some do this simply to stay on top of this search page and others do it as they have work available via some sort of automated system. If you happen to work for one of these requesters you might only get a few hits in before there is no new hits available. However, if you complete a hit and save that page as your favorite bookmark page, when you open it up again you'll automatically accept a hit when you go to the bookmark if it's available. This all said, when you submit a hit and no more hits are available, if you continue to refresh that page, if a hit becomes available you'll auotmatically get accepted.
Of course this works with any search listing. But there is zero reason to refresh other search pages frequently because they just don't change as much. Refreshing the newest hits first will also give you a jump on your other favorite requesters as soon as they post (you'll see the post but not accept it if it wasn't the last hit you submitted and didn't change pages) and you'll also get first dibs on more lucrative hits that might have a small amount of work available like higher paying surveys or other popular high paying hits.
2. Hits Available - Most First
This is a popular way to sort lower paying quantity hits. Lower reward amount, not lower dollar per hour hits. If you can do a $.02 hit in 10 seconds you can make close to minimum wage and if the requester has 10,000 of them available, you can make some easy money if they are hits you enjoy doing, do them well, and do them fast. Some requesters will post tiny batches to stay on top of hit creation page and some requesters will post batches of thousands at a time.
Doing these types of hits isn't guaranteed to be a money maker. Most of them aren't. There are plenty of requesters that want to pay a penny for 10 minutes worth of work. However viewing the ratings on TO and trying the hits yourself you'll soon find out which ones are worth doing. And if you find those hits you might be able to do them for several hours, though there are some very popular hits that 10,000 hits can be gone in minutes.
3. Reward Amount - Highest First
This is typically not a popular way to sort generally. You'll see a lot of spam hits that break the terms of service. A lot of them will have fake submit buttons so you can't even submit them and others will just reject them if you do submit them. They are under no obligation to actually pay. The biggest problem with these types of hits is that they might ask for personal information you shouldn't be giving out and may even ask for credit card information (credit card fraud), cell phone number (charge calls to your cellphone) and email and snail mail address (sell your information to spammers).
So when search this way? When you are searching by a keyword or a requesters name. For example, if I decide to search for surveys I'll sort this way, filter out the scammers and spammers and see if there are any legit (.edu type) surveys that pay decently enough to do. I don't recommend surveys though since amazon doesn't explain to requesters how bad blocks hurt workers so unknown survey requesters could put your account in jeopardy, amazon won't help workers, all because they wanted to prevent you from doing the survey more than once.
Most typically you sort this way when searching your favorite requester that posts multiple hits. For example, you might do transcription and you want to get the highest paying hits for your favorite transcription requester. Less commonly, I might even search for lowest reward with my favorite requester for this same reason. The grading hits are typically at the bottom of the pay scale for requesters so search this way might show you these types of hits up front, if it's something you want to do.
1. Hit Creation Date - Newest First
The benefit for this sort is 2 fold. Some requesters will post a few hits at a time but will continue posting hits for hours or all day, depending on the requester. Some do this simply to stay on top of this search page and others do it as they have work available via some sort of automated system. If you happen to work for one of these requesters you might only get a few hits in before there is no new hits available. However, if you complete a hit and save that page as your favorite bookmark page, when you open it up again you'll automatically accept a hit when you go to the bookmark if it's available. This all said, when you submit a hit and no more hits are available, if you continue to refresh that page, if a hit becomes available you'll auotmatically get accepted.
Of course this works with any search listing. But there is zero reason to refresh other search pages frequently because they just don't change as much. Refreshing the newest hits first will also give you a jump on your other favorite requesters as soon as they post (you'll see the post but not accept it if it wasn't the last hit you submitted and didn't change pages) and you'll also get first dibs on more lucrative hits that might have a small amount of work available like higher paying surveys or other popular high paying hits.
2. Hits Available - Most First
This is a popular way to sort lower paying quantity hits. Lower reward amount, not lower dollar per hour hits. If you can do a $.02 hit in 10 seconds you can make close to minimum wage and if the requester has 10,000 of them available, you can make some easy money if they are hits you enjoy doing, do them well, and do them fast. Some requesters will post tiny batches to stay on top of hit creation page and some requesters will post batches of thousands at a time.
Doing these types of hits isn't guaranteed to be a money maker. Most of them aren't. There are plenty of requesters that want to pay a penny for 10 minutes worth of work. However viewing the ratings on TO and trying the hits yourself you'll soon find out which ones are worth doing. And if you find those hits you might be able to do them for several hours, though there are some very popular hits that 10,000 hits can be gone in minutes.
3. Reward Amount - Highest First
This is typically not a popular way to sort generally. You'll see a lot of spam hits that break the terms of service. A lot of them will have fake submit buttons so you can't even submit them and others will just reject them if you do submit them. They are under no obligation to actually pay. The biggest problem with these types of hits is that they might ask for personal information you shouldn't be giving out and may even ask for credit card information (credit card fraud), cell phone number (charge calls to your cellphone) and email and snail mail address (sell your information to spammers).
So when search this way? When you are searching by a keyword or a requesters name. For example, if I decide to search for surveys I'll sort this way, filter out the scammers and spammers and see if there are any legit (.edu type) surveys that pay decently enough to do. I don't recommend surveys though since amazon doesn't explain to requesters how bad blocks hurt workers so unknown survey requesters could put your account in jeopardy, amazon won't help workers, all because they wanted to prevent you from doing the survey more than once.
Most typically you sort this way when searching your favorite requester that posts multiple hits. For example, you might do transcription and you want to get the highest paying hits for your favorite transcription requester. Less commonly, I might even search for lowest reward with my favorite requester for this same reason. The grading hits are typically at the bottom of the pay scale for requesters so search this way might show you these types of hits up front, if it's something you want to do.
Tuesday, June 28, 2011
About Me
Welcome to my introductory post for working smarter on amazon mechanical turk. This blog will be geared to helping bother workers and requesters in whatever capacity I can.
About me - I am DreamScaper formally of TurkerNation and the hunter world of warcraft forums. Other handles I used on warcraft were DreamsDrunk, DrunkRaven, LostSoul. I also used to go to 2+2 forums, though I didn't post enough for anyone to know me. On madrom you can find me playing lostsoul - if you ever heard of mudding and/or madrom. On chessbase chess program I play under the handle DreamScaper1976 or 76 - I forget which one.
Since I am not yet a requester, though been thinking this might be something I'd consider starting in the future, my advices for requesters is more limited. My advices for requesters is geared more towards making your hit fairer for pay for both the requester and worker, how to make it more attractive for workers to work on, and get your favorite workers to stay working on your hits. If you are a new requester, or checking this blog out because you are an existing requester, I'd suggest reading any posts you might find interesting to see what I consider the smarter ways to turk. If your workers are reading this then it might help you be a better requester. The logic is the same as in chess. I like reading books that counter my favorite opening so if my opponent has read those I know how to answer them.
For workers - I've been a worker for a little over a year. I've been unemployeed far longer. My goal for mTurk is simply to make it as close to minimum wage as I can until I can get back on my feet. As a worker I try to put in volume per day and current daily goal is $19 a day. That will get me $5k this year and I hope next year to aim for $10k. Not bad for being unemployeed! In any case, all my blog posts will be geared towards what I do, or at least try to do. Is it the best you can? I don't know. But if you can tell me how to do something better, please respond. While I try to help others every day I still try to learn something every day. So what you see here is pretty much what I try to do daily.
Just a few disclaimers before my first post:
1. I hope to initially start posting daily with new guides. After I have written what I could, I hope to post at least weekly if there is anything new worth posting. So if you have any questions for a future blog, let me know.
2. Please do not link publicly any of my posts on turker nation forums. If you've read this and see someone posting links to this blog on those forums and have access to their forums please ask them to remove it. I still check their forums daily so I'll find out and I'll do what I can to deny them any use. This is my intellectual property. They have already banned me and stole my work so I have no intentions of allowing them to steal more of my work by copy/pasting it (thanks zerd for getting me banned) or even linking to it.
3. You'll never find anything in these blogs that help you cheat the system, bot, break ToS, or anything else. Even if you are mad that most of the jobs are low paying and want to abuse the system, that is not the point of this thread. I'd rather work for peanuts instead of not even having that chance.
4. If you like something I've written and thinks it can help out anyone, even on TN, feel free to PM them the address. All I ask is not to make any reference to these blogs public on their site.
5. Everything I share is my own thoughts and doesn't represent anything that is factual. As they are my opinions I'll try to explain my thought logic so you can choose to agree or not. So if you simply know better or think I'm full of hot air, more power to you.
6. I plan on making a spreadsheet I've developed available at a future date when I'm happy about it. I'm not sure of anything else I plan on making available via download. In any case, if you download anything from any link I offer, please run your virus scanner and take all normal precautions. I hope nothing should harm your system, so do what you need to to make sure.
About me - I am DreamScaper formally of TurkerNation and the hunter world of warcraft forums. Other handles I used on warcraft were DreamsDrunk, DrunkRaven, LostSoul. I also used to go to 2+2 forums, though I didn't post enough for anyone to know me. On madrom you can find me playing lostsoul - if you ever heard of mudding and/or madrom. On chessbase chess program I play under the handle DreamScaper1976 or 76 - I forget which one.
Since I am not yet a requester, though been thinking this might be something I'd consider starting in the future, my advices for requesters is more limited. My advices for requesters is geared more towards making your hit fairer for pay for both the requester and worker, how to make it more attractive for workers to work on, and get your favorite workers to stay working on your hits. If you are a new requester, or checking this blog out because you are an existing requester, I'd suggest reading any posts you might find interesting to see what I consider the smarter ways to turk. If your workers are reading this then it might help you be a better requester. The logic is the same as in chess. I like reading books that counter my favorite opening so if my opponent has read those I know how to answer them.
For workers - I've been a worker for a little over a year. I've been unemployeed far longer. My goal for mTurk is simply to make it as close to minimum wage as I can until I can get back on my feet. As a worker I try to put in volume per day and current daily goal is $19 a day. That will get me $5k this year and I hope next year to aim for $10k. Not bad for being unemployeed! In any case, all my blog posts will be geared towards what I do, or at least try to do. Is it the best you can? I don't know. But if you can tell me how to do something better, please respond. While I try to help others every day I still try to learn something every day. So what you see here is pretty much what I try to do daily.
Just a few disclaimers before my first post:
1. I hope to initially start posting daily with new guides. After I have written what I could, I hope to post at least weekly if there is anything new worth posting. So if you have any questions for a future blog, let me know.
2. Please do not link publicly any of my posts on turker nation forums. If you've read this and see someone posting links to this blog on those forums and have access to their forums please ask them to remove it. I still check their forums daily so I'll find out and I'll do what I can to deny them any use. This is my intellectual property. They have already banned me and stole my work so I have no intentions of allowing them to steal more of my work by copy/pasting it (thanks zerd for getting me banned) or even linking to it.
3. You'll never find anything in these blogs that help you cheat the system, bot, break ToS, or anything else. Even if you are mad that most of the jobs are low paying and want to abuse the system, that is not the point of this thread. I'd rather work for peanuts instead of not even having that chance.
4. If you like something I've written and thinks it can help out anyone, even on TN, feel free to PM them the address. All I ask is not to make any reference to these blogs public on their site.
5. Everything I share is my own thoughts and doesn't represent anything that is factual. As they are my opinions I'll try to explain my thought logic so you can choose to agree or not. So if you simply know better or think I'm full of hot air, more power to you.
6. I plan on making a spreadsheet I've developed available at a future date when I'm happy about it. I'm not sure of anything else I plan on making available via download. In any case, if you download anything from any link I offer, please run your virus scanner and take all normal precautions. I hope nothing should harm your system, so do what you need to to make sure.
Subscribe to:
Posts (Atom)