Former York psychiatrist William Kerr told police that patients had made "very nasty and false and bitter allegations" against him, a jury was told.
Dr Kerr suggested they had made the claims of rape and indecent assault to "get their own back", said Robert Smith QC, defending at Leeds Crown Court.
He said Dr Kerr said to police: "It's a terrible blow. It's a devastating blow. I have told Beryl (Dr Kerr's wife Dr Beryl Bromham) life is not worth living. I am completely devastated."
Mr Smith was opening the defence for Dr Kerr yesterday. The jury is seeking to decide whether Dr Kerr, now aged 75, of Alne, near Easingwold, carried out four rapes and 15 indecent assaults on women patients in North Yorkshire over 20 years between the 1960s and 1980s.
A previous jury has decided he is not fit to plead because of mental impairment. Mr Smith said severe physical illness had resulted in physical damage to the brain.
He suggested to the jury that it could not be sure that Dr Kerr carried out the alleged acts.
"He cannot go into the witness box," he said.
"His illness has deprived him of the ability to understand the proceedings."
Mr Smith said the jury had to be sure that Dr Kerr had committed the alleged acts to find the prosecution case proven.
"Are you able to be sure without hearing his side of the story?" he asked.
Mr Smith said that at least 15 of the 16 complainants had not physically resisted when they had allegedly been abused by Dr Kerr.
He said none of them had responded: "What do you think you are doing? Don't touch me, you filthy man." He told the jury that the complainants would be devastated if it did not find the allegations proven, but this should not influence their decision.
He added that Dr Kerr had been appointed as psychiatrist at the 600-bed Clifton Hospital in York in 1967 and had treated possibly thousands of patients over 20 years, but the jury was only dealing with complaints by 16 patients.
The hearing continues.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereComments are closed on this article