One military source told The Associated Press that reinforcements trying to retake the town have been repelled, with some casualties. He spoke on condition of anonymity because he was not authorized to speak to reporters.
The attack on the northeastern town of Baga comes as President Muhammadu Buhari seeks a second term in February's election. He took office in 2015 vowing to defeat the Boko Haram extremists, who have split as one faction pledged allegiance to the Islamic State group.
One fleeing resident, Musa Hajaye, told the AP that the extremists "have hoisted their flag and warned us of the evil fate of saboteurs." Residents said the attack began on Tuesday night, with the military base seized on Wednesday.
Many residents have fled to Maiduguri, the capital of Borno state and the birthplace of Boko Haram. The fighting continued on Friday in Baga, the chief of army training and operations, Maj. Gen. Lamidi Adeosun, told reporters.
"It's a ding-dong situation but we are engaging them," he said. "We are not in total control but the Boko Haram have not taken control of Baga, either. We are dealing and almost done with the situation."
Baga, close to the border with Chad, hosts the Nigerian base of the multinational task force fighting Boko Haram. Its weapons, ammunition and other equipment are a key target for the extremists. Insurgents also overran the base in 2015.
The Islamic State West Africa Province, the largest IS-linked extremist group in Africa, claimed to kill or wound "dozens" of soldiers in the latest attack, according to the SITE Intelligence Group that monitors extremist communiques.
Nigeria's military rarely announces death tolls in such attacks, but the government in November acknowledged dozens of soldier deaths in what it called an extremist resurgence. The government also for the first time confirmed the insurgents had begun using drones, calling it a "critical factor" in the rise in attacks. Buhari at the time held an urgent meeting with member countries to "enhance the capacity" of the multinational force.
The nearly decade-old Boko Haram insurgency has been blamed for some 20,000 deaths and thousands of abductions. The unrest and displacement of millions of hungry people have turned northeastern Nigeria into one of the world's worst humanitarian crises.
Follow Africa news at https://twitter.com/AP_Africa