Four security officers were killed in the attack, as worshippers gathered to break the day's fasting for Ramadan.
Two other attacks elsewhere in the kingdom killed only the bombers.
"We will strike with an iron hand those who target the minds and thoughts... of our dear youth," King Salman said.
The monarch was delivering a speech to mark Eid al-Fitr, the holiday that celebrates the end of Ramadan.
No group has claimed responsibility for the attacks but so-called Islamic State has carried out similar bombings, targeting Shia Muslims and Saudi security forces.
Saudi Arabia's highest religious body denounced the three attacks. The Senior Council of Ulema said the bombers had "violated everything that is sacred".
Zeid Ra'ad al-Hussein, the UN high commissioner for human rights and a member of the Jordanian royal family, said: "This is one of the holiest sites in Islam, and for such an attack to take place there, during Ramadan, can be considered a direct attack on Muslims all across the world."
The Sunni Muslim jihadist group has called for the overthrow of the Saudi monarchy and its supporters have previously carried out bombings in the Gulf state, targeting the Shia minority community and security forces.
IS also claimed, or was blamed for, a series of deadly attacks in the predominantly Muslim countries of Turkey, Bangladesh and Iraq during the holy month of Ramadan.
An interior ministry spokesman identified the assailant in the Jeddah attack as a 35-year-old Pakistani expatriate called Abdullah Qalzar Khan who, it said, had worked as a private driver in the city for 12 years.
Two security officers were wounded in the attack.
The second bombing took place near dusk outside a Shia mosque in the mainly Shia eastern city of Qatif. A resident told the Reuters news agency that there were believed to be no casualties apart from the bomber, as worshippers had already left to break their daylight Ramadan fasts.
However, the interior ministry spokesman said the remains of three people had been found and were being identified, without providing any details.
Not long afterwards, another bomber struck near the Prophet's Mosque in Medina where thousands of worshippers had gathered for the Maghrib prayers.
The head of the Shura Council, the kingdom's main advisory body, said the attack was "unprecedented".
"This crime, which causes goose bumps, could not have been perpetrated by someone who had an atom of belief in his heart," Abdullah al-Sheikh said.
The grand sheikh of Cairo's al-Azhar University, the leading religious institute in the Sunni world, also stressed "the sanctity of the houses of God, especially the Prophet's Mosque".
Saudi Arabia's Crown Prince and Interior Minister, Mohammed bin Nayef bin Abdul Aziz, sought to reassure his fellow citizens.
"The security of the homeland is good, it is at its highest levels and thanks be to God it gets stronger every day," the official Saudi Press Agency quoted him as saying while visiting the security officers wounded in the Jeddah bombing.
Posted by Women Against Shariah on Tuesday, July 5, 2016
Source: BBC News