Vision At Large: large scale data collection for computer vision research

We're open source:

The core of the annotation system is available at Google Code. It includes flash tools and django web-based task management server.

The integration with any robot running ROS is in cv_mech_turk package. The package also has scripts to submit plain jpeg images and to download results.

Please subscribe to the mailing list at Google groups to hear about new releases and discussions.

How does it work?

1) Money -> Amazon
2) Images -> Amazon Mechanical Turk -> People -> Annotations
3) Accept annotations
4) Amazon pays people

Examples

Tools

All the sources at Google code.
Beta Release 0.1 of generic image annotation tool for Mechanical Turk.
Release 0.1 of my command line wrappers
An example on granting bonuses from command line.

Recent results:

Date	Content	Results	Data	Interface
Aug 2009	Robot learning about bottles	3 4 5	ask me	v 0.1.14
Aug 2009	Robot learning human hands	3	ask me	v 0.1.14
July 2009	Robot moving around	1	ask me	v 0.1.14
25 june 2009	Object attributes collected by Ian Endres and Ali Farhadi		data
23 june 2008	Bounding box and joint locations in 2000+ Flickr images	Display	data	v0.0.9
26 march 2008	Joint locations for 337 frames sampled from: Weizman activity dataset, 2 Discraft youtube videos, UIUC dataset, LabelME (337 images x 3 passes)	Display	(upon request)	v0.0.4
13 march 2008	Joint locations for Ramanan dataset (305 images x 3 passes)	Display	(upon request)	v0.0.4
05 march 2008	Segmentation masks for Ramanan dataset (305 images x 3 passes).	Display	(upon request)	v0.0.3
20 february 2008	Experimental prototype 2. Labeling superpixels.(10 samples x 17 videos x 3 passes).	Display	(upon request)	v0.0.2
15 february 2008	Experimental prototype 1. Grid-based image labeling.(10 samples x 17 videos x 3 passes).	Display	(upon request)	v0.0.1

Publications

The papers that use AmazonMT for annotation: (very incomplete list).

Alexander Sorokin, David Forsyth,Utility data annotation with Amazon Mechanical Turk, First IEEE Workshop on Internet Vision at CVPR 08, Alaska, Ancorage. pdf 2M talk slides(ppt,8M)
A. Farhadi, I. Endres, D. Hoiem, and D.A. Forsyth, Describing Objects by their Attributes, CVPR2009
Tamara L. Berg, Alexander Sorokin, Gang Wang, David A. Forsyth, Derek Hoiem, Ali Farhadi, and Ian Endres It's all about the data, to appear in the Proceedings of IEEE

If you want some of the data mentioned above, send an e-mail to Alex (syrnick@gmail.com, sorokin2@uiuc.edu). This will speed up the process.

Call for ideas

Will 5 000 annotated images boost your research project? Is it worth 100$? Shoot us an e-mail (sorokin2@uiuc.edu). It should take ...(we'll see, but hopefully within a week)... to get it done!

Want to help?

Send me e-mail if you are interested in this project and looking to do something. Also, if you want to make this site pretty, that'd be awesome.

Related image annotation projects:
Label Me and their MTurk experience, ESP game, Peekaboom,Lotus Hill Institute, Image Net.

People behind this project: Alex Sorokin, David Forsyth. Many thanks to Lukas Biewald and Dolores Labs