rcu/tree: Force quiescent state on callback overload
authorNeeraj Upadhyay <neeraju@codeaurora.org>
Sun, 21 Jun 2020 18:37:27 +0000 (00:07 +0530)
committerPaul E. McKenney <paulmck@kernel.org>
Tue, 25 Aug 2020 01:36:05 +0000 (18:36 -0700)
commit9c39245382de4d52a122641952900709d4a9950b
treee9d0cc02ee644d531df7269798bffe5db7710bbf
parente082c7b38185af0f59e55efff840939c35391f85
rcu/tree: Force quiescent state on callback overload

On callback overload, it is necessary to quickly detect idle CPUs,
and rcu_gp_fqs_check_wake() checks for this condition.  Unfortunately,
the code following the call to this function does not repeat this check,
which means that in reality no actual quiescent-state forcing, instead
only a couple of quick and pointless wakeups at the beginning of the
grace period.

This commit therefore adds a check for the RCU_GP_FLAG_OVLD flag in
the post-wakeup "if" statement in rcu_gp_fqs_loop().

Fixes: 1fca4d12f4637 ("rcu: Expedite first two FQS scans under callback-overload conditions")
Reviewed-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Neeraj Upadhyay <neeraju@codeaurora.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
kernel/rcu/tree.c