Meet the gig workers helping train robots to do our dishes

Mohamad Dunggio recently strapped a mobile phone to his head and recorded himself doing his dishes over and over again for four hours.

It left him with wrinkly fingers, but he did not mind.

"I think it's OK because I exchange my time to earn money," said the 23-year-old from Gorontalo on Sulawesi in Indonesia.

Mr Dunggio is one of the thousands of gig workers around the world feeding the robotics industry's insatiable demand for training data.

Since April he has been earning extra cash making "egocentric training" videos, used to teach robots how to do things like household chores.

Egocentric training data is sensor or video recordings made from the perspective of the person doing the task, often captured with a head or chest-mounted camera with a focus on keeping the subject's hands in the frame.

Mr Dunggio said he was paid by three companies from Argentina, India and the US between $US3 ($4.22) and $US10 an hour to record himself performing tasks like dishwashing, cleaning, cooking, making the bed, ironing, folding clothes and sweeping.

"I think it's a good thing to do because you don't have to have any experience," he said.

How to teach robots

As the recent World Humanoid Robot Games in China demonstrated, robots have been advancing in literal leaps and bounds in recent years.

But the dream of a multi-function humanoid robot that can move around the home efficiently and reliably taking on a variety of domestic tasks has so far proven elusive.

Dana Kulić, a robotics professor at Monash University, said that for robots to navigate the huge variety and complexity of people's homes they needed to learn how to observe and react to their environments.

"Even if the task is just to fold the laundry, or wash the dishes, every person has a different set of dishes. Then the place where [the clothes] should be folded looks different," she said.

Professor Kulić said robotics researchers were using egocentric data in a way similar to how generative AI researchers taught large language models (LLMs) like ChatGPT to predict the next word or letter in a sentence.

While egocentric data produced by actual robots, teleoperated by a human, tended to be more relevant, human egocentric data was cheaper, easier and could provide a much-needed variety of scenarios, she said.

The workers producing the data now include entire factories in South Asia, house cleaners in the US and home delivery drivers.

However, Professor Kulić said the ways human egocentric data was collected were not without issues.

They ranged from whether the contributors truly understood what the data was being used for to privacy concerns about filming in people's homes and data being sold on to third companies.

Strict requirements for videos

Mr Dunggio said overall he was happy with the side-hustle and did not feel exploited, even if others might feel differently.

"If people don't have work and are looking for income this is just an option," he said.

However, he said he felt a bit embarrassed going to the mall with a phone mounted on his head and was disappointed when some of his videos were rejected.

He also complained that some companies charged a transfer fee of about $US1 to withdraw accumulated earnings of under $US10.

On the plus side he got to learn some new skills.

"I never cooked for myself before, but now I know how to cook," he said.

Changes in demand for data

Sergio Bruccoleri is vice-president of delivery for Appen, an ASX-listed Australian company that collects and annotates datasets for the tech industry, including egocentric data for robotics.

Mr Bruccoleri, who is based in Barcelona, said the company could collect around 60,000 hours of video per month from as many as 5,000 contributors all over the world, including India, Canada, the US, Brazil, Mexico and the Philippines.

China was Appen's biggest market for robotics data with the US second, he said.

He said Appen's contributors produced videos in homes and increasingly in workplaces but also in controlled studios, sometimes with specialised cameras and equipment like sensor gloves.

However, he said Appen was moving away from the kind of data that was being collected at an industrial scale in "lower-cost countries" because it had become commoditised and de-valued.

The company was increasingly focused on wealthier countries like the US and Canada, where the robots were likely to be sold initially, while still gathering high value data elsewhere.

"The dishwashers in the US look very different from the dishwashers you would have elsewhere," he said.

In the past, Appen has faced criticism over pay and conditions. Contributors on forums still complain of issues from late payments to poor communication and unexplained account cancellations.

Mr Bruccoleri said some of Appen's systems were not perfect and the company tried to address contributor issues with open lines of communication.

He said Appen had a code of ethics and fair-pay standards and tried "as much as possible" to ensure pay was "always above the minimum wage set in that particular country".

Appen also had privacy guidelines for contributors and a tracing system which he said meant they should be able to request the data they produced be withdrawn, even after it had been sold.

Asked whether it was unfair that contributors were sometimes forced to pay fees to withdraw their earnings, Mr Bruccoleri said he believed it was but it was the payment platforms that took the fee.

"It's not limited to Appen. You see it in other platforms all the time," he said.

When can I have a robot to fold my clothes?

Robotics companies, particularly those in China, have this year been unveiling more and more competent humanoid models that appear close to the ideal of a Jetsons-style robot to do all our household chores.

But Professor Kulić believes that is still a way off.

She said there was a big difference between showing a robot folding clothes during a demonstration and having it working reliably for extended periods of time.

Then there was the cost, not only to produce the robots but also to maintain and repair them.

"I feel like there's many problems to be solved beyond just the ability to do a specific task," she said.

She compared robots to autonomous vehicles which worked "99 per cent of the time".

"But now that last 1 per cent is taking much more time than anyone initially thought it would," Professor Kulić said.