Bug 915037

Summary: scp bash completion removes servername on completion
Product: [openSUSE] openSUSE Tumbleweed Reporter: Christian Boltz <suse-beta>
Component: BasesystemAssignee: Marcus Hüwe <suse-tux>
Status: RESOLVED FIXED QA Contact: E-mail List <qa-bugs>
Severity: Normal    
Priority: P5 - None CC: adam, adrian.schroeter, bwiedemann, mpluskal, rombert, suse-beta, werner
Version: 201501*   
Target Milestone: ---   
Hardware: Other   
OS: Other   
Whiteboard:
Found By: --- Services Priority:
Business Priority: Blocker: ---
Marketing QA Status: --- IT Deployment: ---
Attachments: autocompletion log
0001-Work-around-the-libreadline-colon-trouble-if-COMP_WO.patch
0001-Allow-osc-to-be-called-as-isc-and-iosc.patch

Description Christian Boltz 2015-01-27 22:25:11 UTC
scp foo server:/da<tab>   results in
scp foo /data/
(note that "server:" was removed)

IIRC this worked in the past, but I don't remember exactly when it broke.
I wouldn't be too surprised if the Nov 17 2014 changes introduced this regression because the previous change was 18 months before, and I would probably have noticed a broken scp in that timeframe ;-)
Comment 1 Dr. Werner Fink 2015-01-28 07:33:48 UTC
Which completion do you mean: bash or bash-completion?

Also you may run

     complete -p scp
     compopt  -D
     set -x
     scp foo server:/da<tab
     set +x

to see what's going on.
Comment 2 Dr. Werner Fink 2015-01-28 07:35:03 UTC
Btw: Can not reproduce

       complete -o nospace -F _scp scp
       compopt +o bashdefault +o default +o dirnames +o filenames +o nospace +o plusdirs -D

with *latest* Factory
Comment 3 Martin Pluskal 2015-01-28 08:11:37 UTC
Created attachment 621158 [details]
autocompletion log

I can reproduce issue.
Comment 4 Christian Boltz 2015-01-28 09:57:13 UTC
(In reply to Dr. Werner Fink from comment #1)
> Which completion do you mean: bash or bash-completion?

I have bash-completion installed, but interestingly...

# complete -p scp
bash: complete: scp: no completion specification
# compopt  -D
compopt +o bashdefault +o default +o dirnames +o filenames +o nospace +o plusdirs -D

Another interesting problem:
# complete -o nospace -F _scp scp
# scp foo ser<tab>bash: completion: function `_scp' not found

so for some reason /usr/share/bash-completion/completions/ssh (and others?) is ignored.

BTW: I'm using factory from 2015-01-17.
Comment 5 Dr. Werner Fink 2015-01-28 10:08:44 UTC
(In reply to Christian Boltz from comment #4)

After a first

  scp<TAB>

also the scp completion should be loaded as all files/links below

  /usr/share/bash-completion/completions/

will be loaded on demand.  And *this* should be done only once. Also there should no completion file originate from Adobe for their acroread as this will break bash-completion:

  rm -f /etc/bash_completion.d/acroread.sh
Comment 6 Dr. Werner Fink 2015-02-12 11:30:29 UTC
Still not able to reproduce with latest Factory.

(In reply to Martin Pluskal from comment #3)

Would you please explain hwo you have done this? Please debug this.
Comment 7 Martin Pluskal 2015-02-12 11:52:55 UTC
Hmm not sure what more to provide, but I can reproduce issue, after I hit tab second time completion gets wrong - see http://pastebin.suse.de/12664/src

+ cur=mpluskal@login.suse.de:
+ local userhost=mpluskal@login.suse.de
+ local path=
++ sed -e 's/\\\\\\\([][(){}<>",:;^&!$=?`|\\'\''[:space:]]\)/\\\1/g'
+ path=
+ [[ -z '' ]]
++ ssh -o 'Batchmode yes' mpluskal@login.suse.de pwd
+ path=/suse/mpluskal
+ local files
+ [[ '' == -d ]]
++ ssh -o 'Batchmode yes' mpluskal@login.suse.de command ls -aF1dL '/suse/mpluskal*'
++ sed -e 's/[][(){}<>",:;^&!$=?`|\\'\''[:space:]]/\\\\\\&/g' -e 's/[*@|=]$//g' -e 's/[^\/]$/& /g'
+ files=/suse/mpluskal/
+ COMPREPLY+=($files)
+ return 0                                                                                                                                                                                                  /suse/mpluskal/

And instead of mpluskal@login.suse.de I see "scp .caffrc /suse/mpluskal/" which naturally fails.

I am not sure if it might be related, but I am using persistant connections in ssh,
from /etc/ssh/ssh_config:
ControlMaster auto
ControlPersist yes
ControlPath ~/.ssh/socket-%r@%h:%p
Comment 8 Dr. Werner Fink 2015-02-12 12:05:42 UTC
(In reply to Martin Pluskal from comment #7)

and this is what I see for

  scp /suse/werner/.mailcap werner@boole:/tmp/<TAB>

+ cur=werner@boole:
+ local userhost=werner@boole
+ local path=
++ sed -e 's/\\\\\\\([][(){}<>",:;^&!$=?`|\\'\''[:space:]]\)/\\\1/g'
+ path=
+ [[ -z '' ]]
++ ssh -o 'Batchmode yes' werner@boole pwd
+ path=
+ local files
+ [[ '' == -d ]]
++ ssh -o 'Batchmode yes' werner@boole command ls -aF1dL '*'
++ sed -e 's/[][(){}<>",:;^&!$=?`|\\'\''[:space:]]/\\\\\\&/g' -e 's/[*@|=]$//g' -e 's/[^\/]$/& /g'
+ files=
+ COMPREPLY+=($files)
+ return 0

and final (after Ctrl-l)

  scp /suse/werner/.mailcap werner@boole:/tmp/
Comment 9 Dr. Werner Fink 2015-02-12 12:21:00 UTC
To show more, if I using

       scp .mailcap werner@boole:<TAB><TAB>

I see then

       scp .mailcap werner@boole:/suse/werner/
Comment 10 Dr. Werner Fink 2015-02-12 12:22:15 UTC
And with set -x

+ _scp_remote_files
+ local 'IFS=
'
+ cur=werner@boole:
+ local userhost=werner@boole
+ local path=
++ sed -e 's/\\\\\\\([][(){}<>",:;^&!$=?`|\\'\''[:space:]]\)/\\\1/g'
+ path=
+ [[ -z '' ]]
++ ssh -o 'Batchmode yes' werner@boole pwd
+ path=/suse/werner
+ local files
+ [[ '' == -d ]]
++ ssh -o 'Batchmode yes' werner@boole command ls -aF1dL '/suse/werner*'
++ sed -e 's/[][(){}<>",:;^&!$=?`|\\'\''[:space:]]/\\\\\\&/g' -e 's/[*@|=]$//g' -e 's/[^\/]$/& /g'
+ files=/suse/werner/
+ COMPREPLY+=($files)
+ return 0
Comment 11 Dr. Werner Fink 2015-02-12 12:24:28 UTC
Please show the string value of

  echo $COMP_WORDBREAKS

thanks
Comment 12 Martin Pluskal 2015-02-12 12:25:59 UTC
(In reply to Dr. Werner Fink from comment #11)
> Please show the string value of
> 
>   echo $COMP_WORDBREAKS
> 
> thanks

# echo $COMP_WORDBREAKS
"'><=;|&(
Comment 13 Dr. Werner Fink 2015-02-12 12:33:06 UTC
Ahhh  ... why there is no colon at the end of this string?  The default value is

   "'><=;|&(:

that is *with* colon at the end.  I'm aware that colon is a problem with osc nevertheless and even with the FAQ of the bash

  [...]
  The current set of completion word break characters is available in bash as
  the value of the COMP_WORDBREAKS variable.  Removing `:' from that value is
  enough to make the colon not special to completion:

  COMP_WORDBREAKS=${COMP_WORDBREAKS//:}
  [...]

the colon is special if used with bash completion.
Comment 14 Martin Pluskal 2015-02-12 12:39:26 UTC
Hmm, where is COMP_WORDBREAKS defined? I would certainly recall changing something like that, also COMP_WORDBREAKS is same on two opensuse machines which were installed at approximately same time (and issue seem to be reproducible on both), yet on fresh ones it is ""'><=;|&(:"
Comment 15 Dr. Werner Fink 2015-02-12 12:42:38 UTC
(In reply to Martin Pluskal from comment #14)

From man:bash(1)

       COMP_WORDBREAKS
              The  set  of characters that the readline library treats as word
              separators when performing word completion.  If  COMP_WORDBREAKS
              is  unset, it loses its special properties, even if it is subse-
              quently reset.

indeed I had removed it my self for the osc TAB completion but then after trouble with other completions I had added a workaround in the osc completion rules for the colon in the osc paths.
Comment 16 Martin Pluskal 2015-02-12 12:51:39 UTC
Why do you thing that this issue is INVALID(In reply to Dr. Werner Fink from comment #15)
> (In reply to Martin Pluskal from comment #14)
> 
> From man:bash(1)
> 
>        COMP_WORDBREAKS
>               The  set  of characters that the readline library treats as
> word
>               separators when performing word completion.  If 
> COMP_WORDBREAKS
>               is  unset, it loses its special properties, even if it is
> subse-
>               quently reset.
> 
> indeed I had removed it my self for the osc TAB completion but then after
> trouble with other completions I had added a workaround in the osc
> completion rules for the colon in the osc paths.

Well once /etc/bash_completion.d/osc.sh is present
# echo $COMP_WORDBREAKS
"'><=;|&(

thus breaking autocompletion for scp.

How could this be resolved as INVALID when issue is apparently present? If assignee is wrong, you can always reassign it.
Comment 17 Dr. Werner Fink 2015-02-12 12:56:16 UTC
This is *not* the latest /etc/bash_completion.d/osc.sh
Comment 18 Dr. Werner Fink 2015-02-12 12:57:28 UTC
Beside this I had submitted the latest completion for osc and this is without changing COMP_WORDBREAKS
Comment 19 Dr. Werner Fink 2015-02-12 13:00:13 UTC
I'm not valid if osc maintainer ignore my work
Comment 20 Dr. Werner Fink 2015-02-12 13:04:56 UTC
Update to latest osc as here

 osc/upstream> git pull
 Already up-to-date.
 osc/upstream> grep COMP_WORDBREAKS dist/osc.complete 
     case "$COMP_WORDBREAKS" in

there is no change of the COMP_WORDBREAKS anymore
Comment 21 Martin Pluskal 2015-02-12 13:08:26 UTC
With latest published osc from factory:
# cat /etc/bash_completion.d/osc.sh
test -z "$BASH_VERSION" && return
complete -o default _nullcommand >/dev/null 2>&1 || return
complete -r _nullcommand >/dev/null 2>&1         || return
COMP_WORDBREAKS="${COMP_WORDBREAKS//:}"
test -s /usr/share/osc/complete && complete -o default -C /usr/share/osc/complete osc
test -s /usr/lib64/osc/complete && complete -o default -C /usr/lib64/osc/complete osc
test -s /usr/lib/osc/complete   && complete -o default -C /usr/lib/osc/complete osc

# echo $COMP_WORDBREAKS
"'><=;|&(

@werner: by resolving this as invalid you are saying that either reporter is doing something completely wrong, or that behaviour is absolutely correct and reporter is crazy for complaining about it :)
Comment 22 Martin Pluskal 2015-02-12 13:10:58 UTC
(In reply to Dr. Werner Fink from comment #20)
> Update to latest osc as here
> 
>  osc/upstream> git pull
>  Already up-to-date.
>  osc/upstream> grep COMP_WORDBREAKS dist/osc.complete 
>      case "$COMP_WORDBREAKS" in
> 
> there is no change of the COMP_WORDBREAKS anymore

Why don't you resolve it as FIXED then? I am not using osc from git, but if fix is already present in upsteream git repository, issue should be closed as fixed. 

Change has apparently yet to get to factory.
Comment 23 Dr. Werner Fink 2015-02-12 13:14:58 UTC
AFAICS it is not fixed in openSUSE:Tools

osc/upstream> cat ../.osc/_project 
openSUSE:Tools
osc/upstream> tar Oxf ../osc-0.150.1.tar.gz osc-0.150.1/dist/osc.complete | diff -up - dist/osc.complete | wc -l
80

but only in the git repository.
Comment 24 Martin Pluskal 2015-02-12 13:19:23 UTC
I strongly assume that it will get there once.
Comment 25 Robert Munteanu 2015-05-01 21:08:03 UTC
*** Bug 920024 has been marked as a duplicate of this bug. ***
Comment 26 Robert Munteanu 2015-05-01 21:08:23 UTC
Still here with Tumbleweed 20150430, osc-0.151.2-2.1.noarch.
Comment 27 Dr. Werner Fink 2015-05-04 07:21:21 UTC
@ Adrian : You are aware *why* this bug was assigned to you?  Please add my already to osc submitted fix for osc.complete !
Comment 28 Adrian Schröter 2015-05-04 09:04:03 UTC
@Werner, my understanding was that this fix is in git already? 
Since the last change is from 2013 in that file it would be out already.

About which change/fix do we speak exactly here?
Comment 29 Dr. Werner Fink 2015-05-04 10:34:08 UTC
I'm speaking about commit 70a3272915122256ed3c02c7b9ebef7c7312f28e which removes the line

     COMP_WORDBREAKS="${COMP_WORDBREAKS//:}"

from dist/complete.sh  as well as add some support for better usage without colon in dist/osc.complete

This is not part of openSUSE Factory:

osc/upstream> osc cat openSUSE:Factory osc osc-0.151.2.tar.gz | \
 tar Oxfz - osc-0.151.2/dist/complete.sh | \
 diff -up - ./dist/complete.sh 
--- -   2015-05-04 12:33:41.220672290 +0200
+++ ./dist/complete.sh  2013-12-19 15:38:06.446657821 +0100
@@ -1,7 +1,6 @@
 test -z "$BASH_VERSION" && return
 complete -o default _nullcommand >/dev/null 2>&1 || return
 complete -r _nullcommand >/dev/null 2>&1         || return
-COMP_WORDBREAKS="${COMP_WORDBREAKS//:}"
 test -s /usr/share/osc/complete && complete -o default -C /usr/share/osc/complete osc
 test -s /usr/lib64/osc/complete && complete -o default -C /usr/lib64/osc/complete osc
 test -s /usr/lib/osc/complete   && complete -o default -C /usr/lib/osc/complete osc
Comment 30 Adrian Schröter 2015-05-04 19:09:40 UTC
this commit is not known to the git repository, the line is also still there.

I could drop it, but it is maybe better when you send a patch or create pull request with the proper fix.
Comment 31 Dr. Werner Fink 2015-05-05 06:38:16 UTC
(In reply to Adrian Schroeter from comment #30)

> this commit is not known to the git repository, the line is also still there.

I had send this commit to the mailing list!

> I could drop it, but it is maybe better when you send a patch or create pull 
> request with the proper fix.

I'll redo the work and integrate also the better support of several OBS
Comment 32 Dr. Werner Fink 2015-05-05 12:32:55 UTC
Created attachment 633272 [details]
0001-Work-around-the-libreadline-colon-trouble-if-COMP_WO.patch

Part one
Comment 33 Dr. Werner Fink 2015-05-05 12:33:41 UTC
Created attachment 633273 [details]
0001-Allow-osc-to-be-called-as-isc-and-iosc.patch

the second part
Comment 34 Bernhard Wiedemann 2015-06-24 18:43:21 UTC
Submitted (and tested) Werner's patches with minor cleanup at
https://github.com/openSUSE/osc/pull/156
Comment 35 Marcus Hüwe 2015-06-24 20:48:03 UTC
(In reply to Bernhard Wiedemann from comment #34)
> Submitted (and tested) Werner's patches with minor cleanup at
> https://github.com/openSUSE/osc/pull/156

Merged. Thanks Werner and Bernhard!