Bug 1019575

Summary:	Factory source checker is declining requests that should have a second review
Product:	[openSUSE] openSUSE Backports	Reporter:	Scott Bahling <sbahling>
Component:	Factory Source Check	Assignee:	Wolfgang Engel <wolfgang.engel>
Status:	RESOLVED FIXED	QA Contact:	E-Mail List <packagehub-bugs>
Severity:	Normal
Priority:	P5 - None	CC:	astieger, ismail
Version:	SLE-12
Target Milestone:	---
Hardware:	Other
OS:	Other
Whiteboard:
Found By:	---	Services Priority:
Business Priority:		Blocker:	---
Marketing QA Status:	---	IT Deployment:	---

Description Scott Bahling 2017-01-12 12:15:50 UTC

By default the factory source checker will set an SR to openSUSE:Backorts to "Declined" if the factory source check fails. In some cases the failure is because of a bug in the check scripts or in some cases it's because of a minor difference like a security patch which should actually be acceptable based on human review.

Auto-declining causes frustration for the contributors. Instead of auto-declining, the bot should fall back to human review and only declined if upon human review it is determined that the submission really does not meet policies.

Comment 1 Scott Bahling 2017-01-12 12:21:53 UTC

Wolfgang, what is the command line for the check_source_in_factory.py script we are running?

Looking at the code, I think we just need to use the following options:

--review-mode=fallback-onfail
--fallback-user=USER or --fallback-group=GROUP

From what I see that should cause the script to set the fallback-user or fallback-group as new reviewer. We just need to determine who should be set as reviewer.

I still need to test this in action to see if it works as expected.

Comment 2 Scott Bahling 2017-01-12 12:57:32 UTC

I just tested and it works as expected:

https://build.opensuse.org/request/show/449846

I ran "check_source_in_factory.py id 449846" and the request was declined as usual.

https://build.opensuse.org/request/show/449848

With this second request I ran

 check_source_in_factory.py --review-mode=fallback-onfail --fallback-user=sbahling id 449848

and the request is set to review by me.

Question is who to set the review to?  The Maintenance team? Do we want to create a "backports" group?

Comment 3 Scott Bahling 2017-01-12 13:03:44 UTC

hmm the only thing missing in the second SR is a notification that the factory checker ran and failed. The script seems to just assign a reviewer, but gives no message why.

Comment 4 Ismail Dönmez 2017-01-12 13:05:24 UTC

(In reply to Scott Bahling from comment #2)
> I just tested and it works as expected:
> 
> https://build.opensuse.org/request/show/449846
> 
> I ran "check_source_in_factory.py id 449846" and the request was declined as
> usual.
> 
> https://build.opensuse.org/request/show/449848
> 
> With this second request I ran
> 
>  check_source_in_factory.py --review-mode=fallback-onfail
> --fallback-user=sbahling id 449848
> 
> and the request is set to review by me.
> 
> Question is who to set the review to?  The Maintenance team? Do we want to
> create a "backports" group?

I would very much prefer a group set up, so we can use the group for other purposes too.

Comment 5 Wolfgang Engel 2017-01-12 13:50:17 UTC

I would prefer adding a group as well. Talking to Adrian for creating a group in OBS, for example called "Backports-Reviewer" or something like that.

Comment 6 Scott Bahling 2017-01-17 08:47:20 UTC

(In reply to Scott Bahling from comment #3)
> hmm the only thing missing in the second SR is a notification that the
> factory checker ran and failed. The script seems to just assign a reviewer,
> but gives no message why.

Turns out I was not testing accurately. The bot will "accept" the review with the declined message "the package needs to be accepted in openSUSE:Factory first", and assign the fallback reviewer.

So it should just be a matter of creating the group that does the fallback review and changing the call in our cron job to use the fallback mode.

Comment 7 Wolfgang Engel 2017-02-03 12:20:56 UTC

The group backports-reviewers is created and already set. I'll close this bug.