Der Spiegel news magazine quotes German government sources as saying she feared Britain was approaching “the point of no return”reports BBC.
Downing Street has not commented on the reports.
Mr Cameron wants to renegotiate the terms of the UK’s continued membership before holding an in-out referendum.
The prime minister said that freedom of movement of workers would be “at the very heart of my renegotiation strategy for Europe”, but Mrs Merkel is said by the magazine to have made clear she will withdraw her support for the UK’s continued EU membership if he continues to push for migration reform.
The German chancellor’s warning to Mr Cameron is reported to have come in a meeting on the fringes of the latest EU summit in Brussels last week.
Der Spiegel, quoting sources within the German chancellor’s office and German foreign ministry, said this was the first time Mrs Merkel had acknowledged that the UK’s exit from the EU was possible.
According to the Sunday Times, Germany has already rejected a proposal to impose quotas on low-skilled EU migrants by limiting the national insurance numbers issued to them.
Der Spiegel reported that Mr Cameron was now looking at a plan to stretch the EU rules “to their limits” in order to ban migrants who do not have a job, and to deport those who are unable to support themselves after three months.
On Sunday, Conservative MP and former justice secretary Kenneth Clarke defended EU migration.
“If you’re going to have a sensible single market, if we want to compete with the Americans and the Chinese and so on, we need the free movement of labour,” he told BBC’s Sunday Politics.
A Downing Street spokesman said Mr Cameron would make a speech on immigration before Christmas, and stressed “you can be sure he will always put Britain first.”