Fighters from the self-styled Islamic State West Africa Province faction of Boko Haram overran two military bases in and around the fishing town on late December.
Kallon said fighting on December 26 triggered a "massive displacement" of civilians, with men, women and children flocking to already overcrowded camps in Monguno and Maiduguri.
Monguno is a garrison town 65 kilometres (40 miles) southwest of Baga while the Maiduguri, the Borno state capital, is 200 kilometres away on the same road.
Both are considered key prizes for the jihadists, who have increased their attacks on military bases and soldiers since July last year.
A second attack on Monguno on December 28 forced IDPs to Maiduguri. Some 20,000 arrived at one camp, Teachers Village, stretching its capacity beyond the limit.
Kallon said it was unclear how many remained in Monguno but "tens of thousands of people are in need of humanitarian assistance, notably shelter, food, water and sanitation.
"The impact of the recent fighting on innocent civilians is devastating and has created a humanitarian tragedy," he added.
Local and international aid agencies have been providing life-saving assistance to IDPs for several years in the remote region, which has been devastated by the fighting.
But since November, some 260 aid workers have had to withdraw from three local government areas in northern Borno near Lake Chad where ISWAP are known to operate.
Kallon said "hundreds of thousands of people" had been affected and the withdrawal was the largest since the international response was increased in 2016.
Nigeria's President Muhammadu Buhari said in December 2015 that Boko Haram was "technically defeated" after a sustained counter-insurgency by Nigerian and regional troops.
Tens of thousands of IDPs who had previously fled to Maiduguri have in the last year been encouraged to leave after the authorities maintained it was safe to return home.
On Wednesday Buhari acknowledged setbacks, including "battle fatigue" among soldiers from a persistent wave of guerrilla style hit-and-run tactics and suicide bombings.
Buhari was elected in 2015 on a promise to end the conflict and security has again become a major election issue as he seeks a second term of office at polls next month.
Troops on the ground have increasingly complained about conditions, including a lack of relief and support, as well as equipment to fight the better-armed militants.
But the army has repeatedly -- and often angrily -- played down reports of heavy military casualties and that the insurgents are once again resurgent. Media organisations have been threatened with legal action for publishing non-official information.
On December 14, the military said it had suspended the UN children's fund UNICEF from operating in the northeast because it was training "spies" supporting Boko Haram.
The ban was later withdrawn. But senior commanders later threatened Amnesty International with closure, accusing it of fabricating reports about human rights abuses in the conflict.
On Sunday, soldiers raided the offices of Nigeria's Daily Trust newspaper in Maiduguri, Abuja and Lagos after a front-page report that troops were preparing to retake Baga.
Two reporters were detained and later released after an outcry from press freedom and human rights watchdogs.
Information minister Lai Mohammed on Tuesday said the government was "not about to gag the press", after comparisons were made to media harassment during decades of military rule.
But he said there needed to be "restraint and... more responsibility in reporting the insurgency".
(This story has not been edited by Business Standard staff and is auto-generated from a syndicated feed.)
Business Standard has always strived hard to provide up-to-date information and commentary on developments that are of interest to you and have wider political and economic implications for the country and the world. Your encouragement and constant feedback on how to improve our offering have only made our resolve and commitment to these ideals stronger. Even during these difficult times arising out of Covid-19, we continue to remain committed to keeping you informed and updated with credible news, authoritative views and incisive commentary on topical issues of relevance.
We, however, have a request.
As we battle the economic impact of the pandemic, we need your support even more, so that we can continue to offer you more quality content. Our subscription model has seen an encouraging response from many of you, who have subscribed to our online content. More subscription to our online content can only help us achieve the goals of offering you even better and more relevant content. We believe in free, fair and credible journalism. Your support through more subscriptions can help us practise the journalism to which we are committed.
Support quality journalism and subscribe to Business Standard.