Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Some question about priority queue #998

Open
FelixSeptem opened this issue Dec 30, 2024 · 0 comments
Open

[BUG] Some question about priority queue #998

FelixSeptem opened this issue Dec 30, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@FelixSeptem
Copy link

Describe the bug
When I choose to use the multi queue with priority, I was curious about the implementation mechanism. So I dig the code in, and seek it in the queues method of the processor, which is https://github.com/hibiken/asynq/blob/master/processor.go#L399

// queues returns a list of queues to query.
// Order of the queue names is based on the priority of each queue.
// Queue names is sorted by their priority level if strict-priority is true.
// If strict-priority is false, then the order of queue names are roughly based on
// the priority level but randomized in order to avoid starving low priority queues.
func (p *processor) queues() []string {
	// skip the overhead of generating a list of queue names
	// if we are processing one queue.
	if len(p.queueConfig) == 1 {
		for qname := range p.queueConfig {
			return []string{qname}
		}
	}
	if p.orderedQueues != nil {
		return p.orderedQueues
	}
	var names []string
	for qname, priority := range p.queueConfig {
		for i := 0; i < priority; i++ {
			names = append(names, qname)
		}
	}
	rand.Shuffle(len(names), func(i, j int) { names[i], names[j] = names[j], names[i] })
	return uniq(names, len(p.queueConfig))
}

I wondering priority variable only affects the composition of the names list and does not affect the final returned queue list. The final queue list is unduplicated and the queues in the configuration are selected with equal probability, rather than increasing the probability of some queues being selected by priority.

Suggestion
Modify uniq function to conclude priority infactors, I will happy to create a pull request if need.

@FelixSeptem FelixSeptem added the bug Something isn't working label Dec 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants