The bomber detonated his device as security officers were breaking their Ramadan fast, al-Arabiya TV said.
News outlets said at least two of the officers had been killed, but there has been no official confirmation.
The mosque is the burial place of the Prophet Muhammad and Medina the second-holiest city in Islam after Mecca.
That the attack happened in such a place is likely to leave Muslims around the world aghast, BBC World Service’s Middle East editor Alan Johnston says.
Suspicion is likely to fall on so-called Islamic State, our correspondent adds.
Social media footage from Medina showed a vehicle on fire in a car park, with two security officers lying nearby.
Other footage showed ambulances and police vehicles streaming to the area.
Some reports say the security officers were about to break their fast when a young man, said by the Okaz newspaper to be 18 years old, approached them to share food. He then detonated explosives.
Qari Ziyaad Patel, 36, from South Africa, who was in the mosque, told Associated Press that people at first thought it was the sound of the cannon fire that marks the breaking of fast.
But he said the ground shook, adding: “The vibrations were very strong. It sounded like a building imploded.”
Earlier, at least one explosion rocked the eastern city of Qatif.
Qatif is home to many minority Shia Muslims and the blast appeared to target a Shia mosque. The attacker was killed but no other casualties were reported.
A suspected suicide bomber also died after detonating a device near the US consulate in the city of Jeddah. Two security officers were slightly injured as they tackled the man, but no-one else was hurt.
No-one has yet said they were behind any of the attacks.
They come with the holy month of Ramadan drawing to a close and ahead of the Eid al-Fitr holiday.
Saudi Arabia has been the target of attacks by so-called Islamic State over the past two years.
In June, the interior ministry said there had been 26 “terror attacks” in the kingdom in that time.
This week has seen a number of deadly attacks either claimed by or blamed on the self-styled IS.
- A suicide gun and bomb attack targeted Istanbul airport on 28 June. The death toll rose to 45, with 240 injured.
- Attackers struck a cafe in Bangladesh’s capital, Dhaka, last Friday night. Twenty hostages and two policemen were killed.
- And a massive truck bomb in Iraq’s capital, Baghdad, on Sunday left at least 165 people dead.