The regional maritime authority said a rescue operation was launched after a witness reported seeing the boat in trouble Monday afternoon about 800 meters (875 yards) off the coast near the town of Agon-Coutainville.
The mother of two of the dead children said her daughters and a 7-year-old boy all were wearing life jackets but got trapped in the boat's cabin. The girls' father tried to break the window "with his arms, with his legs, with the anchor of the boat," the woman was quoted by daily newspaper Ouest-France as saying.
The paper did not name the mother, saying only that she and her family lived in the Normandy town of Argentan. The prosecutor's office in the commune of Coutances said the adults were outside the cabin and thrown into the water when the boat turned over, according to the magazine Le Point.
Volunteer rescuer Pascal Dutot said he found the boat filled with water and partially submerged but not capsized. During the race to reach victims while waves rolled the vessel, "the most difficult thing was to extract them from the cabin, to break the windscreen," he said.
"I grabbed one of the victims with a young firefighter who just arrived at the scene with his dinghy," Dutot said. "We put the victim onboard the dinghy and took the person straight to the shore, where the fire men were waiting for us. The victim was unconscious."
The conditions on the water at the time were manageable for a larger boat but would have been choppy for one the size of the 5.5-meter-long speedboat, he said.