[Bug] luigi: challenge processor dies on "send on closed channel" in Processor.Reduce and is never restarted

SafeLine version

9.3.11

Bug description

What happened:

The challenge processor in safeline-luigi panics with send on closed channel in challenge.(*Processor).Reduce (challenge.go:114). The panic is caught by recover() in controller.run.func1, logged once, and the processor goroutine exits. It is never started again, while the rest of luigi keeps running.

After that:

the challenge task channel fills up and every new record is logged as level=ERROR msg="task channel full, drop record" name=challenge (about 20k lines per hour on our traffic). Once this filled the disk on one of our nodes;
Anti-Bot challenge records are no longer saved, and the Anti-Bot logs in the console stay empty;
traffic itself is not affected.

A goroutine dump (/debug/pprof/goroutine?debug=2 on luigi) shows 7 controller.run goroutines instead of 8: the one for challenge is missing. docker restart safeline-luigi fixes it until the next panic.

It happens regularly: after a restart the processor died again in about 15 hours on one node and about 25 hours on the other. Since August the challenge processor died at least 5 times on our 2 nodes. Earlier logs were lost to rotation; the two panics we captured have identical stacks (below).

What did you expect:

No panic in Processor.Reduce. The goroutine dump shows a few hundred challenge.(*Processor).consume goroutines that come and go, so it looks like Reduce sometimes sends into a channel that has just been closed when one of these goroutines finished. It happens rarely, so it looks like a timing issue rather than bad input.
If a processor does panic, luigi should restart it or exit so that the container restarts, instead of keeping running without it and silently dropping records.

Setup: 2 nodes with config auto sync (master/slave), external PostgreSQL 15 per node, Anti-Bot challenge enabled on 3 sites.

Steps to reproduce (if available)

We do not have exact steps. It happens on its own under steady production traffic with Anti-Bot challenge enabled, within about a day after luigi starts.

Logs or screenshots (optional)

Node 1, 2026-10-03 10:08:14 +05:00:

level=ERROR msg=panic recover="send on closed channel" stack="goroutine 107 [running]:
luigi/controller.getCurrentGoroutineStack(...)
      /work/controller/receiver.go:97
luigi/controller.run.func1()
      /work/controller/receiver.go:103 +0xde
panic({0xd54be0?, 0xfa8060?})
      /usr/local/go/src/runtime/panic.go:792 +0x132
luigi/lib/processor/challenge.(*Processor).Reduce(0xc000e96fc0, {0xdf9de0?, 0xc0053634d0?})
      /work/lib/processor/challenge/challenge.go:114 +0x36b
luigi/controller.run({0xc000e97030, {0xe664c5, 0x9}, {0xfb5618, 0xc000e96fc0}, 0x306dc4200})
      /work/controller/receiver.go:127 +0x20f
luigi/controller.runTask(...)
      /work/controller/receiver.go:91
created by luigi/controller.Run in goroutine 1
      /work/controller/receiver.go:36 +0x125
" module=controller source=/work/controller/receiver.go:103

Node 2, 2026-10-03 00:03:31 +05:00: the same stack (goroutine 149, same offsets).

After the panic, continuously:

level=ERROR msg="task channel full, drop record" name=challenge module=controller source=/work/controller/receiver.go:81